Reading a file by multiple threads

I have a 250Mb file to be read. And the application is multi threaded. If i allow all threads to read the file the memory starvation occurs. I get out of memory error.

To avoid it. I want to have only one copy of the String (which is read from stream) in memory and i want all the threads to use it.

while (true) {
    synchronized (buffer) {开发者_Python百科
        num = is.read(buffer);
            String str = new String(buffer, 0, num);

    }
    sendToPC(str);
}

Basically i want to have only one copy of string when all thread completed sending, i want to read second string and so on.

---

Why multiple threads? You only have one disk and it can only go so fast. Multithreading it won't help, almost certainly. And any software design that relies on having an entire file in memory is seriously flawed in the first place.

Suppose you define your problem?

---

I realize this is kind of late, but I think what you want here is to use the map function in the FileChannel class. Once you map a region of the file into memory, then all of your threads can read or write to that block of memory and the OS will synchronizes that memory region with the file periodically (or when you call MappedByteBuffer.load()), and if you want each thread to work with a different part of the file, then you can assign several maps each mapping a specific region of the file and just use one map per thread.

see the javadoc for FileChannel, RandomAccessFile, and MappedByteBuffer

---

Could you directly use streams instead of completely reading the file in to memory?

---

You can register all threads as callbacks in the File reading class. SO have something like an array or list of classes implementing an interface StringReaderThread which has the method processString(String input). After reading each line from the file, iterate over this array/list and call processString() on all the threads this way. Would this solve your problem?