Transferring data between process calls

I have a Linux process that is being called numerous times, and I need to make this process as fast as possible.

The problem is that I must maintain a state between calls (load data from previous call and store it for the开发者_开发问答 next one), without running another process / daemon.

Can you suggest fast ways to do so? I know I can use files for I/O, and would like to avoid it, for obvious performance reasons. Should (can?) I create a named pipe to read/write from and by that avoid real disk I/O?

---

Pipes aren't appropriate for this. Use posix shared memory or a posix message queue if you are absolutely sure files are too slow - which you should test first.

In the shared memory case your program creates the segment with shm_open() if it doesn't exist or opens it if it does. You mmap() the memory and make whatever changes and exit. You only shm_unlink() when you know your program won't be called anymore and no longer needs the shared memory.

With message queues, just set up the queue. Your program reads the queue, makes whatever changes, writes the queue and exits. Mq_unlink() when you no longer need the queue.

Both methods have kernel persistence so you lose the shared memory and the queue on a reboot.

---

It sounds like you have a process that is continuously executed by something.

Why not create a factory that spawns the worker threads? The factory could provide the workers with any information needed.

---

... I can use files for I/O, and would like to avoid it, for obvious performance reasons.

I wonder what are these reasons please...

Linux caches files in kernel memory in the page cache. Writes go to the page cash first, in other words, a write() syscall is a kernel call that only copies the data from the user space to the page cache (it is a bit more complicated when the system is under stress). Some time later pdflush writes data to disk asynchronously.

File read() first checks the page cache to see if the data is already available in memory to avoid a disk read. What it means is that if one program writes data to files and another program reads it, these two programs are effectively communicating via kernel memory as long as the page cache keeps those files.

If you want to avoid disk writes entirely, that is, the state does not need to be persisted across OS reboots, those files can be put in /dev/shm or in /tmp, which are normally the mount points of in-memory filesystems.