开发者

How should I manage expensive reporting crons for users who visit infrequently?

开发者 https://www.devze.com 2023-04-03 13:28 出处:网络
Let\'s say I have 5M users (for easy math) who vary widely in their visits per month. User loyalty, in visits per month

Let's say I have 5M users (for easy math) who vary widely in their visits per month.

User loyalty, in visits per month
1. 1M  <1      visits/month
2. 1M  1-10    visits/month
3. 1M  10-50   visits/month
4. 1M  50-100  visits/month
5. 1M  >100    visits/month

The goal for each user is to access data that takes (let's say) 1 CPU cycle to fetch (for example... the reality in our situation is much much less, but it's easier math with 1).

Each data fetch takes too long to load inline, so it's preferred to have data ready for them when they come. (via crons)

Let's say that in order to satisfy our most active users, we would need to run the cron 10 times a day to have it ready for them when they want it. (I say "when they want it" because typically that's 4 times within a 8 hour work day, not 4 times spread evenly over 24 hours). That's 1M(users) * 10(data fetches) per day. Or (at 1 CPU cycle per fetch), 10M CPU Cycles for these 1M most active users. The good news is that they're at least using the fetched data.

However, what about our less active users? What strategy do you recommend to still provide relevant fetched data results while protecting from wasted CPU cycles fetching data that will never or rarely be seen?

Here's a chart of the minimum cycles required based on the chart above. The ideal answer would get as close to this as possible.

Group   # Users   Visits/Month  CPU Cycles/Month
  1.      1M      0.1           0.1M
  2.      1M      1             1M
  3.      1M      10            10M
  4.      1M      50            50M
  5.      1M      100           100M
-------------------------------------------------
          5M      161.1         161.1M

If I did the same cron necessary to keep Group 5 happy for everyone, that'd be 500M CPU cycles (roughly 70% wasted).

What do you recommend to minimize wasted CPU cycles? But to still keep infrequent users happy (because we still开发者_运维问答 want them to turn into active users).

0

精彩评论

暂无评论...
验证码 换一张
取 消