栏目分类:
子分类:
返回
名师互学网用户登录
快速导航关闭
当前搜索
当前分类
子分类
实用工具
热门搜索
名师互学网 > IT > 软件开发 > 后端开发 > Python

python multiprocessing:多进程与进程池

Python 更新时间: 发布时间: IT归档 最新发布 模块sitemap 名妆网 法律咨询 聚返吧 英语巴士网 伯小乐 网商动力

python multiprocessing:多进程与进程池

创建新进程

使用 multiprocessing 模块中的 Process 类创建进程:

from multiprocessing import Process

创建进程与创建线程的语法十分相似。
多进程的问题: 进程之间不能共享资源,例如下面的例子,如果process2 是 ask_user 的进程,则会出现错误 EOFError: EOF when reading a line

process2 = Process(target=ask_user) # Will cause EOFError!!

因为 process2 无法从终端读取数据。

import time
from multiprocessing import Process

def ask_user():
    start = time.time()
    user_input = input("Enter your name: ")
    greet = f"Hello, {user_input}"
    print(greet)
    print(f'ask_user, {time.time() - start}')

def complex_calculation():
    start = time.time()
    print('Started calculating...')
    [x**2 for x in range(20000000)]
    print(f'complex_calculation, {time.time() - start}')

# for windows: put my_process.start(), my_process.join() 
# in this if statement is important
# or new processes will continuously start
if __name__ == '__main__':
    start = time.time()
    ask_user()
    complex_calculation()
    print(f'Single thread total time: {time.time() - start}')

    process = Process(target=complex_calculation)

	# if here target = ask_user, input cannot be read
	#  user_input = input("Enter your name: ")
	# EOFError: EOF when reading a line
	# because resources cannot be shared among processes
    process2 = Process(target=complex_calculation)

    process.start()
    process2.start()

    start = time.time()

    process.join()
    process2.join()

    print(f'Two processes total time: {time.time() - start}')
进程池

process pool 与 thread pool 的语法十分相似:

import time
from concurrent.futures import ProcessPoolExecutor

def ask_user():
    start = time.time()
    user_input = input("Enter your name: ")
    greet = f"Hello, {user_input}"
    print(greet)
    print(f'ask_user, {time.time() - start}')


def complex_calculation():
    start = time.time()
    print('Started calculating...')
    [x**2 for x in range(20000000)]
    print(f'complex_calculation, {time.time() - start}')


if __name__ == '__main__':
    start = time.time()
    complex_calculation()
    complex_calculation()
    print(f'Single thread total time: {time.time() - start}')

    start = time.time()
    with ProcessPoolExecutor(max_workers=2) as pool:
        pool.submit(complex_calculation)
        pool.submit(complex_calculation)
    print(f'Two processes total time: {time.time() - start}')

输出:

Started calculating...
complex_calculation, 3.7751059532165527
Started calculating...
complex_calculation, 3.809253215789795
Single thread total time: 7.584359169006348
Started calculating...
Started calculating...
complex_calculation, 3.9134361743927
complex_calculation, 3.94466495513916
Two processes total time: 4.0857954025268555

进程池的写法比创建进程简洁。

使用进程,程序运行时间明显缩短。两次调用 complex_calculation(), 如果不使用两个进程,调用时间累加,需要 7.5 秒,使用两个进程单独完成计算,则所需时间缩短为 4 秒。

一般情况下,如果是有需要等待的多任务程序,使用多线程。

如果 CPU 多核,并且需要执行多项使用CPU的复杂计算,使用多进程。

转载请注明:文章转载自 www.mshxw.com
本文地址:https://www.mshxw.com/it/1001201.html
我们一直用心在做
关于我们 文章归档 网站地图 联系我们

版权所有 (c)2021-2022 MSHXW.COM

ICP备案号:晋ICP备2021003244-6号