一:前言
協程又稱為微線程,纖程。英文名Coroutine:協程是一種用戶態的輕量級線程
協程擁有自己的寄存器上下文和棧。協程調度切換時,將寄存器上下文和棧保存到其他地方,在切回來的時候,恢復之前保存的寄存器上下文和棧。因此:
協程能夠保留上次一調用時的狀態,能夠進入上一次離開時所處的邏輯流的位置
協程的好處
無需線程上下文切換的開銷
無需原子操作(不會被線程調度機制打斷的操作)鎖定以及同步的開銷
方便切換控制流,簡化編程模型
高并發+高擴展性+低成文:一個CPU支持上完的協程都不是問題,所以很適合高并發處理
協程的缺點
無法利用多核資源:協程的本質是單線程,需要和進程配合才能運行在多CPU上
進行阻塞(Blocking)操作(如IO時)會阻塞掉整個程序
協程的條件
必須在只有一個單線程里實現并發
修改共享數據不需加鎖
用戶程序里自己保存多個控制流的上下文棧
一個協程遇到IO操作自動切換到其它協程
使用yield實現協程
def consumer(name):print("--->starting eating baozi...")while True:new_baozi = yieldprint("[%s] is eating baozi %s" % (name, new_baozi))def producer():next(con)next(con2)n = 0while n < 5:n += 1con.send(n)con2.send(n)print("\033[32;1m[producer]\033[0m is making baozi %s" % n)if __name__ == '__main__':con = consumer("c1")con2 = consumer("c2")p = producer()
二:Greenlet
greenlet是一個用C實現的協程模塊,相比與python自帶的yield,它可以使你在任意函數之間隨意切換,而不需把這個函數先聲明為generator
使用greenlet實現協程
from greenlet import greenletdef f1():print(12)gr2.switch()print(34)gr2.switch()def f2():print(56)gr1.switch()print(78)if __name__=='__main__':gr1 = greenlet(f1)gr2 = greenlet(f2)gr1.switch() #手動切換,gevent是對greenlet的封裝,實現自動切換
運行結果:
12
56
34
78
三:Gevent
Gevent是一個第三方庫(需要額外自己安裝),可以輕松通過gevent實現并發同步或異步編程,在gevent中主要用到的模式是Greenlet,它是以C擴展模塊形式接入Python的輕量級協程。Greenlet全部運行在主程序操作系統的內部,被協作式調度
使用gevent庫實現協程
import geventdef func1():print("func1 running")gevent.sleep(2) # 內部函數實現io操作print("switch func1")def func2():print("func2 running")gevent.sleep(1)print("switch func2")def func3():print("func3 running")gevent.sleep(0.5)print("func3 done..")if __name__=='__main__':gevent.joinall([gevent.spawn(func1),gevent.spawn(func2),gevent.spawn(func3),])
運行結果:
func1 running
func2 running
func3 running
func3 done..
switch func2
switch func1
同步與異步的性能區別
import geventdef task(pid):"""Some non-deterministic task"""gevent.sleep(0.5)print('Task %s done' % pid)def synchronous():for i in range(1, 10):task(i)def asynchronous():threads = [gevent.spawn(task, i) for i in range(10)]gevent.joinall(threads)if __name__ =='__main__':print('Synchronous:')synchronous()print('Asynchronous:')asynchronous()
運行結果:
Synchronous:
Task 1 done
Task 2 done
Task 3 done
Task 4 done
Task 5 done
Task 6 done
Task 7 done
Task 8 done
Task 9 done
Asynchronous:
Task 0 done
Task 1 done
Task 2 done
Task 3 done
Task 4 done
Task 5 done
Task 6 done
Task 7 done
Task 8 done
Task 9 done
將task函數封裝到Greenlet內部線程的gevent.spawn。 初始化的greenlet列表存放在數組threads中,此數組被傳給gevent.joinall 函數,后者阻塞當前流程,并執行所有給定的greenlet。執行流程只會在 所有greenlet執行完后才會繼續向下走。
遇到IO阻塞時會自動切換任務
from gevent import monkeymonkey.patch_all()
import gevent
from urllib.request import urlopendef f(url):print('GET: %s' % url)resp = urlopen(url)data = resp.read()print('%d bytes received from %s.' % (len(data), url))if __name__=='__main__':gevent.joinall([gevent.spawn(f, 'https://www.python.org/'),gevent.spawn(f, 'https://www.yahoo.com/'),gevent.spawn(f, 'https://github.com/'),])
通過gevent實現單線程下的多socket并發
server端
import sys
import socket
import time
import geventfrom gevent import socket,monkey
monkey.patch_all()def server(port):s = socket.socket()s.bind(('0.0.0.0', port))s.listen(500)while True:cli, addr = s.accept()gevent.spawn(handle_request, cli)def handle_request(conn):try:while True:data = conn.recv(1024)print("recv:", data)conn.send(data)if not data:conn.shutdown(socket.SHUT_WR)except Exception as ex:print(ex)finally:conn.close()
if __name__ == '__main__':server(8001)
client 端
import socketHOST = 'localhost' # The remote host
PORT = 8001 # The same port as used by the server
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, PORT))
while True:msg = bytes(input(">>:"),encoding="utf8")s.sendall(msg)data = s.recv(1024)#print(data)print('Received', repr(data))
s.close()
并發100socket連接
import socket
import threadingdef sock_conn():client = socket.socket()client.connect(("localhost",8001))count = 0while True:#msg = input(">>:").strip()#if len(msg) == 0:continueclient.send( ("hello %s" %count).encode("utf-8"))data = client.recv(1024)print("[%s]recv from server:" % threading.get_ident(),data.decode()) #結果count +=1client.close()for i in range(100):t = threading.Thread(target=sock_conn)t.start()
原文鏈接:https://blog.csdn.net/qq_39112646/article/details/86776107
感謝作者分享!