Limit method calls per second(s) (refuse when limit reached)

I have a method that does some IO, and I'd like to limit the calls (per second) to this method to avoid the backend to get bursts of concurrent requests it can't handle.

If 开发者_如何学Cthe requirement came without the "per second" I could just use a stack (basically just a counter) and offer() when starting a request, and poll() when done. With the "per second" requirement I'd somehow need to clean slots on the stack that are older than a given lapse of time.

How do I do that correctly? The structure should be thread-safe, obviously.

Thank you for your time!

---

Have a look at this question. The answer is equally applicable to this problem, even if the question is phrased rather differently.

I confess, I find it slightly mysterious, but it works beautifully.

---

You could use a queue but with some modifications. Whenever you want to add to the queue (the method gets called) you check how many elements are in it and when they were added. All elements that were inserted more than one second ago can be removed. Compare the number of remaining elements with the rate per second and you have a decision whether to reject the method execution or not. It might get a little bit more complicated if you intend to have the method block instead of reject.

There are likely to be many possible implementations and you have to check whether they are feasible for your problem. You could also take a look at implementations of Token buckets which is the general concept of your problem.

---

It sounds like you need to decouple the IO work from the requesting thread and offload it instead to a pool of threads with a given size. That way you can explicitly control the number of threads doing IO at the same time.

ExecutorService pool = Executors.newFixedThreadPool(10);

public void myMethod() {

  pool.submit(new Runnable() {
    public void run() {
      //Do IO work here
    }
  });

}

In this example there will never be more than 10 threads doing IO work. Its not limiting the number of requests per second but controlling the concurrency is probably a better way of throttling.