栏目分类:
子分类:
返回
名师互学网用户登录
快速导航关闭
当前搜索
当前分类
子分类
实用工具
热门搜索
名师互学网 > IT > 面试经验 > 面试问答

multiprocessing.Pool在Linux / Python2.7上的terminate()之后生成新的子对象?

面试问答 更新时间: 发布时间: IT归档 最新发布 模块sitemap 名妆网 法律咨询 聚返吧 英语巴士网 伯小乐 网商动力

multiprocessing.Pool在Linux / Python2.7上的terminate()之后生成新的子对象?

在此页面上,多处理模块的作者Jesse Noller显示正确的处理方式

KeyboardInterrupt
是让子流程返回-
不会引发异常。这允许主进程终止池。

但是,如下面的代码所示,

except KeyboardInterrupt
直到运行了由生成的所有任务 之后
,主流程才到达该块
pool.map
。这就是为什么(我相信)在按下
run_nlin
后会看到对辅助函数的额外调用
Ctrl-C

一种可能的解决方法是,如果

multiprocessing.Event
设置了a,则对所有工作程序功能进行测试。如果事件已发生,则请工人提早纾困,否则,请继续进行长时间的计算。


import loggingimport multiprocessing as mpimport timelogger = mp.log_to_stderr(logging.WARNING)def worker(x):    try:        if not terminating.is_set(): logger.warn("Running worker({x!r})".format(x = x)) time.sleep(3)        else: logger.warn("got the message... we're terminating!")    except KeyboardInterrupt:        logger.warn("terminating is set")     terminating.set()    return xdef initializer(terminating_):    # This places terminating in the global namespace of the worker subprocesses.    # This allows the worker function to access `terminating` even though it is    # not passed as an argument to the function.    global terminating    terminating = terminating_def main():    terminating = mp.Event()        result = []    pool = mp.Pool(initializer=initializer, initargs=(terminating, ))    params = range(12)    try:         logger.warn("starting pool runs")         result = pool.map(worker, params)         pool.close()    except KeyboardInterrupt:        logger.warn("^C pressed")        pool.terminate()    finally:        pool.join()        logger.warn('done: {r}'.format(r = result))if __name__ == '__main__':    main()

运行脚本将产生:

% test.py[WARNING/MainProcess] starting pool runs[WARNING/PoolWorker-1] Running worker(0)[WARNING/PoolWorker-2] Running worker(1)[WARNING/PoolWorker-3] Running worker(2)[WARNING/PoolWorker-4] Running worker(3)

在这里按Ctrl-C;每个工人都设置

terminating
事件。我们确实只需要设置它,但是尽管效率很低,但它仍然有效。

  C-c C-c[WARNING/PoolWorker-4] terminating is set[WARNING/PoolWorker-2] terminating is set[WARNING/PoolWorker-3] terminating is set[WARNING/PoolWorker-1] terminating is set

现在,所有其他排队的任务

pool.map
都将运行:

[WARNING/PoolWorker-4] got the message... we're terminating![WARNING/PoolWorker-2] got the message... we're terminating![WARNING/PoolWorker-1] got the message... we're terminating![WARNING/PoolWorker-2] got the message... we're terminating![WARNING/PoolWorker-4] got the message... we're terminating![WARNING/PoolWorker-2] got the message... we're terminating![WARNING/PoolWorker-1] got the message... we're terminating![WARNING/PoolWorker-3] got the message... we're terminating!

最终,主要过程到达

except KeyboardInterrupt
块。

[WARNING/MainProcess] ^C pressed[WARNING/MainProcess] done: []


转载请注明:文章转载自 www.mshxw.com
本文地址:https://www.mshxw.com/it/405808.html
我们一直用心在做
关于我们 文章归档 网站地图 联系我们

版权所有 (c)2021-2022 MSHXW.COM

ICP备案号:晋ICP备2021003244-6号