Python Thread not returning the value

Using: Django with Python

Overall objective: Call a function which processes video conversion (internally makes a curl command to the media server) and should immediately return back to the user.

Using message queue would be an overkill for the app. So I had decided to use threads, I have written a class which overwrites the init and run method and calls the curl command

class process_video(Thread):
    def __init__ (self,video_id,video_title,fileURI开发者_如何学Python):
        Thread.__init__(self)
        self.video_id = video_id 
        self.video_title = video_title
        self.fileURI = fileURI
        self.status =-1

    def run(self):
        logging.debug("FileURi" + self.fileURI)
        curlCmd = "curl --data-urlencode \"fileURI=%s\" %s/finalize"% (self.fileURI, settings.MEDIA_ROOT)
        logging.debug("Command to be executed" + str(curlCmd))
        #p = subprocess.call(str(curlCmd), shell=True)
        output_media_server,error = subprocess.Popen(curlCmd,stdout = subprocess.PIPE).communicate()
        logging.debug("value returned from media server:")
        logging.debug(output_media_server)

And I instantiate this class from another function called createVideo which calls like this success = process_video(video_id, video_title, fileURI)

Problem: The user gets redirected back to the other view from the createVideo and the processVideo gets called, however for some reason the created thread (process_video) doesn't wait for the output from the media server.

---

I wouldn't rely on threads being executed correctly within web applications. Depending on the web server's MPM, the process that executes the request might get killed after a request is done (I guess).

I'd recommend to make the media server request synchronously but let the media server return immediately after it started the encoding without problems (if you have control over its source code). Then a background process (or cron) could poll for the result regularly. This is only one solution - you should provide more information about your infrastructure (e.g. do you control the media server?).

Also check the duplicates in another question's comments for some answers about using task queues in such a scenario.

BTW I assume that no exception occurs in the background thread?!

---

Here is the thing what I did for getting around the issue which I was facing.

I used django piston to create an API for calling the processvideo with the parameters passed as GET, I was getting a 403 CSRF error when I was trying to send the parameters as POST. and from the createVideo function I was calling the API like this cmd = "curl \"%s/api/process_video/?video_id=%s&fileURI=%s&video_title=%s\" > /dev/null 2>&1 &" %(settings.SITE_URL, str(video_id),urllib.quote(fileURI),urllib.quote(video_title))

and this worked. I feel it would have been helpful if I could have got the session_id and post parameters to work. Not sure how I could get off that csrf thing to work.