How to launch a process that connects to a socket you're listening on?

I have a process which my code is launching from subprocess.Popen() that attempts to connect to a socket my code is also listening on. The problem is if the code starts listening on this socket first, it cannot launch the subprocess. It was blocked by sock.accept(), and when sock.accept() is times out, it is obviously not listening when subprocess.Popen() is ran. If the code launches the subprocess first, it attempts to connect to a socket but then fails before any code is able to listen for it.

Now.. any thoughts on how i can do this? It would seem i would need to start listening in a non-blocking fashion, and then launch the process, but i am a bit confused because even if i use select() to handle the queue, eventually sock.accept() is called and thus blocks the code... i think.

At any rate, some direction would be very handy! I would prefer not to, but if it makes life easier i am not apposed to using Twisted either.

Edit 1: I'll try and get something up in the way of code, i have to look at my old commits to find a working version. Basically though, i don't think my code is the problem. I think i am simply implementing it wrong.

For example, if i launch my socket listener and manually, in a shell, launch this subprocess.Popen() process it connects just fine. This is because the shell is already listening. I believe my problem is simply a chicken and the egg issue. In my code, in a single code path, if i launch the process first it fails instantly because there is no socket server listening yet. However if i launch the socket server first, it times out because it is blocking and no subprocess gets launched until it finishes blocking. My solution, i believe, lies in non-blocking code but i am very unfamiliar with how to achieve this properly. I see many mentions of select() but they look like they would block at the same point, sock.accept(). I say "look like" because i have yet to implement a select() version. I may be wrong here, and if i am please let me know.

Edit 2: Here is the socket part of the code.. Note that this is set to non-blocking currently..

 90         # Create our socket stream to listen on.
 91         serv = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
 92
 93         #serv.settimeout(5)
 94         serv.setblocking(0)
 95
 96         # Bind the address.
 97         serv.bind(('', self.PORT))
 98         serv开发者_如何学C.listen(5)
 99
100         try:
101             # Now start listening for a connection!
102             (self._sock, remote_address) = serv.accept()
103         except socket.timeout:
104             logger.debug('Socket connection failed!')
105             raise DBGPServerNotFoundError(
106                 'No connection was established coming from '
107                 '"%(address)s:%(port)i".' % {
108                     'address':self.ADDRESS,
109                     'port':self.PORT,
110                 })
111         else:
112             logger.debug('Socket connection established! The other end of '
113                          'the connection is at "%s:%i".' % remote_address)
114         finally:
115             serv.close()

And here is the error..

  File "/home/lee/projects/vim-debug/repo/vimbug/dbgp.py", line 100, in connect
    (self._sock, remote_address) = serv.accept()
  File "/usr/lib/python2.6/socket.py", line 197, in accept
    sock, addr = self._sock.accept()
error: [Errno 11] Resource temporarily unavailable

The subprocess launching code is in a different module (a unit test, specifically) but here that is for good measure. Note that con is a container object, and con.connect() is the function for the above code.

 56     con.connect()
 57     pydbgp_proc = subprocess.Popen(
 58         ('pydbgp.py', '-d', 'localhost:%i' % OPTIONS['pydbgp_port'],
 59         OPTIONS['debug_file']),
 60         stdout=subprocess.PIPE,
 61         stderr=subprocess.PIPE,)

Edit 3: Rewriting code a bit to try out connecting to a socket before sock.accept() is called. We'll see if it fails :)

Edit 4: Alright. Rewrote the code a bit.. still has the same error. Thoughts? (Also.. this editing junk is getting big.. is there some preferred way in stackoverflow to make these big updates/edits?

Code:

 77     def listen(self):
 78         # Create our socket stream to listen on.
 79         serv = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
 80
 81         #serv.settimeout(5)
 82         serv.setblocking(0)
 83
 84         # Bind the address.
 85         serv.bind(('', self.PORT))
 86         serv.listen(5)
 87         self.serv = serv
 88
 89     def accept(self):
 90         (newsock, newaddr) = self.serv.accept()

Calling code:

 57     con.listen()
 58     pydbgp_proc = subprocess.Popen(
 59         ('pydbgp.py', '-d', 'localhost:%i' % OPTIONS['pydbgp_port'],
 60         OPTIONS['debug_file']),
 61         stdout=subprocess.PIPE,
 62         stderr=subprocess.PIPE,)
 63     con.accept()

Error:

       File "/home/lee/projects/vim-debug/repo/vimbug/dbgp.py", line 90, in accept
    (newsock, newaddr) = self.serv.accept()
  File "/usr/lib/python2.6/socket.py", line 197, in accept
    sock, addr = self._sock.accept()
error: [Errno 11] Resource temporarily unavailable

Thoughts?

Edit 5: I changed the accept function to the following select() implementation, which resulted in 'Not ready..?' being printed.

 89     def accept(self):
 90         rfds, wfds, xfds = select.select([self.serv], [], [], 1)
 91
 92         if self.serv in rfds:
 93             print 'Read ready..?'
 94             (newsock, newaddr) = self.serv.accept()
 95         else:
 96             print 'Not ready..?'

---

Are you setting socket.setblocking(0) in your listening code?

After you spawn the listening server, you should be able to read the status with select() calls... example that runs fine under debian lenny and python 2.5 for me...

import socket
import select

SERVER_SOCKADDR = ("", 424242)

server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.setblocking(0)      # <------------------
server.bind(SERVER_SOCKADDR)
server.listen(5)

client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client.setblocking(0)

result = client.connect_ex(SERVER_SOCKADDR)

rfds, wfds, xfds = select.select([server], [client], [], 1)
if server in rfds:
    print "Server socket: accept does not block"
    sockfd, addr = server.accept()    # sockfd.send() and sockfd.recv() to 
                                      # write and read the stream...
    sockfd.setblocking(0)
    print sockfd, addr
else:
    print "Server socket: accept blocks"
if client in wfds:
    print "Client socket: write does not block"
else:
    print "Client socket: write blocks"


server.close()
client.close()

And when I run that...

[mpenning@Bucksnort ~]$ python socket_test.py
Server socket: accept does not block
<socket._socketobject object at 0xb75764c4> ('127.0.0.1', 35810)
Client socket: write does not block
[mpenning@Bucksnort ~]$

---

Doug Hellman's Python Module of the Week is a good place to find an introduction to a module's basic usage like subprocess.

Failing that, some code that is causing problems would help us answer your question.

---

There are a number of ways you can approach this. The simplest is for the subprocess to wait a few seconds before connecting.

It's unclear why you are doing this, can you fill us in on your goal?