开发者

How to manage tasks in Celery?

开发者 https://www.devze.com 2023-03-20 15:27 出处:网络
I have a task with a certain parameter, and I want to know whether there is a task with the same parameter.

I have a task with a certain parameter, and I want to know whether there is a task with the same parameter. I have the following:

@task
def some_task(id):
    some_task.开发者_如何学运维update_state(state="PROGRESS", meta={"id": id})
    some_action_by_id(id)

But I would like this:

@task
def some_task(id):
    if !check_task(id):
    some_task.update_state(state="PROGRESS", meta={"id": id})
    some_action_by_id(id)   

How can I accomplish this?


You probably mean that you want only one "some_task" running for every unique id at every moment. So you have to implement a locking mechanism. Take a look here. Celery plays great with Redis!


here is my solution:

from celery.task.control import inspect
from celery.result import AsyncResult

def get_same_task(aTaskName, aArgs, aHosts):
    for jobs in aHosts.values():
        for job in jobs:
            if job['name'] == aTaskName and job['args']  == str(aArgs):
                return job['id']
    return None

class IgnoreSameArgumentsTask(Task):
    abstract = True
    inspect = inspect()

    def delay(self,  *args, **kwargs):
        vHosts_Jobs   = self.inspect.active()
        vTaskId = get_same_task(self.name, args, vHosts_Jobs)
        if vTaskId != None:
            return AsyncResult(vTaskId)
        else:
            return super(IgnoreSameArgumentsTask, self).delay(*args,  **kwargs)

@celery.task(base=IgnoreSameArgumentsTask)
def add(x, y):
    sleep(x+y)
    return x + y
0

精彩评论

暂无评论...
验证码 换一张
取 消