开发者

Google App engine(python) Updating a db.StringListProperty contention/concurrency issues

开发者 https://www.devze.com 2023-01-06 00:48 出处:网络
Ive been looking at the principles of fan out of messages as described in the google IO \"building scalable complex apps\"

Ive been looking at the principles of fan out of messages as described in the google IO "building scalable complex apps"

In it it suggests that using a list property for say a list of receivers is a scalable solution.

In this scenario how does one update the list property so that contention issues don't step in, If the app is handling many users

using the IO example:

class message(db.model)
    sender=db.stringproperty()
    body=db.textproperty()

class messageindex(db.model)
    receivers=db.stringlistproperty()

To adapt their example i would also need

class followers(db.model)
    user=db.userproperty()
    followers=db.stringlistproperty()

(code is just in for an example and is not typed correctly - sorry)

The concept is if someone follows you, you add their key to your followers list in the followers model, If you make a message, you store your followers list in the message list - and using a simple query all users get the message - pretty simple stuff.

The issue is updating someones list of followers. Assuming that some accounts could have millions of followers - if i simply update the entity their is going to be contention issues - One would also need more than one entry as i think their is a limit of like 5000 entries per list. And of course requests may be sent to "add" or "remove" a person. What would be the best way to do this. I was thinking about using the task_q开发者_StackOverflow中文版ueue service. I was thinking about a work model that stores each follow request and triggers a task to run in say 60 seconds. The task gets all the work to be done for a persons followers list - and builds the new list. Not sure how this would work - but it would stop contention issues as only one thread could execute in one min.

Does anyone have any code examples good advice,help on how i can do this in a scalable manner - i don't think m cache can be used in the method as any loss would mean a follow request could be lost.


I have now found the solution for this using a fork-join-queue. Their is a post on google IO 2010 - regarding how this is done:

link text

0

精彩评论

暂无评论...
验证码 换一张
取 消

关注公众号