开发者

Max file size for File.ReadAllLines

开发者 https://www.devze.com 2022-12-25 01:19 出处:网络
I need to read and process a text file. My processing would be easier if I could use the File.ReadAllLines method but I\'m not sure what is the maximum size of the file that could be read with this me

I need to read and process a text file. My processing would be easier if I could use the File.ReadAllLines method but I'm not sure what is the maximum size of the file that could be read with this method with开发者_如何学Goout reading by chunks. I understand that the file size depends on the computer memory. But are still there any recommendations for an average machine?


On a 32-bit operating system, you'll get at most a contiguous chunk of memory around 550 Megabytes, allowing loading a file of half that size. That goes down hill quickly after your program has been running for a while and the virtual memory address space gets fragmented. 100 Megabytes is about all you can hope for.

This is not an issue on a 64-bit operating system.

Since reading a text file one line at a time is just as fast as reading all lines, this should never be a real problem.


I've done stuff like this with 1-2GB before, albeit in Python. I do not think .NET would have a problem, though. But I would only do this for one-off processing.

If you are doing this on a regular basis, you might want to go through the file line by line.


Its bad design unless you know the files sizes vs the computer memory that would be avaiable in the running app.

A better solution would be consider memory mapped files. They use themselvses as page fil storage,

0

精彩评论

暂无评论...
验证码 换一张
取 消