Send Ctrl-C to remote processes started via subprocess.Popen and ssh

How do I send a Ctrl-C to multiple ssh -t processes in Popen() objects?

I have some Python code that kicks off a script on a remote host:

# kickoff.py

# i call 'ssh' w/ the '-t' flag so that when i press 'ctrl-c', it get's
# sent to the script on the remote host.  otherwise 'ctrol-c' would just
# kill things on this end, and the script would still be running on the
# remote server
a = subprocess.Popen(['ssh', '-t', 'remote-host', './script.sh', 'a'])
a.communicate()

That works great, but I need to kick off multiple scripts on the remote host:

# kickoff.py

a = subprocess.Popen(['ssh', '-t', 'remote-host', './script.sh', 'a'])
b = subprocess.Popen(['ssh', '-t', 'remote-host', './script.sh', 'b'])
a.communicate()
b.communicate()

The result of this is that Ctrl-C doesn't reliably kill everything, and my terminal always gets garbled afterwards (I have to run 'reset'). So how can I kill both remote scripts when the main one is killed?

Note: I'm trying to avoid logging into the remote-host, searching for 'script.sh' in the process list, and sending a SIGINT to both of the processes. I just want to be able to press Ctrl-C on the kickoff script, and have that kill both remote processes. A less optimal solution may involve deterministically finding the PID's of the remote scripts, but I don't know how to do that in my current set-up.

Update: the script tha开发者_如何学JAVAt gets kicked off on the remote server actually starts up several children processes, and while killing the ssh does kill the original remote script (probably b/c of SIGHUP), the children tasks are not killed.

---

The only way I was able to successfully kill all of my child processes was by using pexpect:

a = pexpect.spawn(['ssh', 'remote-host', './script.sh', 'a'])
a.expect('something')

b = pexpect.spawn(['ssh', 'remote-host', './script.sh', 'b'])
b.expect('something else')

# ...

# to kill ALL of the children
a.sendcontrol('c')
a.close()

b.sendcontrol('c')
b.close()

This is reliable enough. I believe someone else posted this answer earlier, but then deleted the answer, so I will post it in case someone else is curious.

---

When killed, ssh will send a SIGHUP to the remote processes. You could wrap the remote processes into a shell or python script that will kill them when that script receives a SIGHUP (see the trap command for bash, and the signal module in python)

It might even be possible to do it with a bloated command line instead of a remote wrapper script.

The problem is that killing the remote processes is not what you want, what you want is to have a working terminal after you do Ctrl+C. to do that, you will have to kill the remote processes AND see the remaining output, which will contain some terminal control sequences to reset the terminal to a proper state. For that you will need a mecanism to signal a wrapper script to kill the processes. This is not the same thing.

---

I haven't tried this, but maybe you can catch a KeyboardInterrupt and then kill the processes:

try
    a = subprocess.Popen(['ssh', '-t', 'remote-host', './script.sh', 'a'])
    b = subprocess.Popen(['ssh', '-t', 'remote-host', './script.sh', 'b'])
    a.communicate()
    b.communicate()
except KeyboardInterrupt:
    os.kill(a.pid, signal.SIGTERM)
    os.kill(b.pid, signal.SIGTERM)

---

I worked around a similar problem this problem by unmapping all of the signals I cared about. When Ctrl+C is pressed, it will still be passed through to the subprocess but Python will wait until the subprocess exits before handling the signal in the main script. This works fine for a signal subprocess as long as the subprocess responds to Ctrl+C.

class DelayedSignalHandler(object):
    def __init__(self, managed_signals):
        self.managed_signals = managed_signals
        self.managed_signals_queue = list()
        self.old_handlers = dict()

    def _handle_signal(self, caught_signal, frame):
        self.managed_signals_queue.append((caught_signal, frame))

    def __enter__(self):
        for managed_signal in self.managed_signals:
            old_handler = signal.signal(managed_signal, self._handle_signal)
            self.old_handlers[managed_signal] = old_handler

    def __exit__(self, *_):
        for managed_signal, old_handler in self.old_handlers.iteritems():
            signal.signal(managed_signal, old_handler)

        for managed_signal, frame in self.managed_signals_queue:
            self.old_handlers[managed_signal](managed_signal, frame)

Now, my subprocess code looks like this:

    with DelayedSignalHandler((signal.SIGINT, signal.SIGTERM, signal.SIGHUP)):
        exit_value = subprocess.call(command_and_arguments)

Whenever Ctrl+C is pressed, the application is allowed to exit before the signal is handled so you don't have to worry about the terminal getting garbled because the subprocess thread was not terminated at the same time as the main process thread.