IPC communication between several identical processes

Context: Linux (Ubuntu), C

I have several identical minimalistic processes spawned on a single box. The processes are doing some networking, and spend most of their time stuck in poll (or select or whatever — I can chan开发者_StackOverflowge that as needed).

Processes lifetime is monitored by the runit with multiwatch (if that changes something).

From time to time a random one of the processes may decide that it wants to send a message (i.e. a bunch of data) to all other processes (including itself if it makes things simpler).

Also, from time to time, a human operator may want to send such a message from a commandline (assume that I can write a commandline tool as needed) — again, to each and every process.

Message delivery must be guaranteed. But it should not block sender if any (or all) recipients suddenly vanished (OTOH, if they are alive but busy, they must eventually receive the message.) If recipient process is idling in a poll, it must be awakened by the message.

What is the most robust way to organize this communication without adding a lot of complexity?

Note: For technical reasons I can not use POSIX signals to communicate between processes.

---

The terms broadcast and reliable are kind of opposed, I think you'll have to establish all to all reliable unicast connections and implement "broadcast" yourself.

I've written some clustering software where nodes on a LAN need to find each other and establish reliable communication - the scheme for this was as follows:

• Each process is configured with the same port number and lan broadcast address
• Each process binds a tcp listener to a random local port
• Each process binds to the specified local UDP port, listening for broadcasts
• Each process sends a message announcing its presence, ID (e.g. PID) and TCP endpoint every few seconds
• When a process discovers another process with a lower ID, it establishes a TCP connection to it, and optionally asks for a list of known peers

After this, every process is connected to every other process, and you can implement "reliable broadcast" manually.

---

I know this thread is quite old, thought the answer might help someone out there. I have used Kbus and found it to be great for broadcasting messages across multiple processes. It is a Linux Kernel module with a User library that processes needs to use to create descriptors. Uses the Publish/Subscribe design pattern which makes it a decoupled system.

Source: https://github.com/kynesim/kbus

Docs: https://kbus.readthedocs.io/en/latest/

Simith