开发者

How to exit a loop of Python Multiprocessing?

开发者 https://www.devze.com 2023-02-18 00:35 出处:网络
I have a multiprocessing code that search database records for a certain condition, when it reaches the end of the database the loop must stop, how can I do that? Here is the code:

I have a multiprocessing code that search database records for a certain condition, when it reaches the end of the database the loop must stop, how can I do that? Here is the code:

import psycopg2
from multiprocessing import Pool

conn = psycopg2.connect(a database)
query=conn.cursor()
query.execute(some query)

def scanfile():
    try: row=query.fetchone()
    except: return False #here the loop must stop 
    #do something with row...

if __name__=='__main__':
    pool=Pool()
    while True:
        pool.apply_async(scanfile)
    pool.close()
    pool.joi开发者_运维技巧n()


Move your query / cursor over as the iterable and let multiprocessing distribute the work for you.

import psycopg2
from multiprocessing import Pool


def scanfile(row):
    #do something with row...


if __name__ == '__main__':
    conn = psycopg2.connect(a database)
    query = conn.cursor()
    query.execute(some query)
    pool = Pool()
    pool.map_async(scanfile, query).wait()
    pool.close()
    pool.join()
0

精彩评论

暂无评论...
验证码 换一张
取 消

关注公众号