开发者

How to do parallel HTTP requests in Heroku?

开发者 https://www.devze.com 2023-01-23 13:28 出处:网络
I\'m building a Ruby on Rails app that access about 6-7 APIs, grabs information from them based on user\'s input, compares and display results to the users (the information is not sav开发者_JAVA百科ed

I'm building a Ruby on Rails app that access about 6-7 APIs, grabs information from them based on user's input, compares and display results to the users (the information is not sav开发者_JAVA百科ed in the database). I will be using Heroku to deploy the app. I would like those HTTP requests to access the APIs to be done in parallel so the answer time is better instead of doing it sequential. What do you think is the best way to achieve this in Heroku?

Thank you very much for any suggestions!


If you want to actually do the requests on the server side (tfe's javascript solution is a good idea), your best bet would be using EventMachine. Using EventMachine gives a simple way to do non-blocking IO.

Also check out EM-Synchrony for a set of Ruby 1.9 fiber aware clients (including HTTP).

All you need to do for a non-blocking HTTP request is something like:

require "em-synchrony"
require "em-synchrony/em-http"
EM.synchrony do
    concurrency = 2
    urls = ['http://url.1.com', 'http://url2.com']

    # iterator will execute async blocks until completion, .each, .inject also work!
    results = EM::Synchrony::Iterator.new(urls, concurrency).map do |url, iter|

        # fire async requests, on completion advance the iterator
        http = EventMachine::HttpRequest.new(url).aget
        http.callback { iter.return(http) }
        http.errback { iter.return(http) }
    end

    p results # all completed requests
    EventMachine.stop
end

Goodluck!


You could always make the requests client-side using Javascript. Then not only can you run them in parallel, but you won't even need the round-trip to your own server.


I haven't tried parallelizing requests like that. But I've tried parallel on heroku, works like a charm! This is my simple blog post about it.

http://olemortenamundsen.wordpress.com/2010/10/17/spawning-multiple-threads-at-heroku-using-parallel/


Have a look at creating each request as a background job: http://blog.heroku.com/archives/2009/7/15/background_jobs_with_dj_on_heroku/

The more 'Workers' you buy from Heroku, the more background jobs can be processed concurrently, leaving your 'Dynos' to serve your users.

0

精彩评论

暂无评论...
验证码 换一张
取 消

关注公众号