浅析Python多线程

　　今天看了几篇博客，主要讲解线程的实例以及如何避免线程间的竞争，觉得感觉对自己很有用，所以在此先写先来以备以后自己查阅.

　　实例一：我们将要请求三个不同的url

1.单线程：

 1 import time
 2 from urllib.request import urlopen
 3
 4
 5 def get_responses():
 6      urls = [
 7         ‘http://www.baidu.com‘,
 8                 ‘http://www.taobao.com‘,
 9                 ‘http://www.alibaba.com‘,
10     ]
11     start = time.time()
12     for url in urls:
13         print(url)
14         resp = urlopen(url)
15         print(resp.getcode())   #得到状态码
16     print("spent time:%s" % (time.time()-start))
17
18 get_responses()

解释：
url顺序的被请求
除非cpu从一个url获得了回应，否则不会去请求下一个url
网络请求会花费较长的时间，所以cpu在等待网络请求的返回时间内一直处于闲置状态。

输出为：
http://www.baidu.com
200
http://www.taobao.com
200
http://www.alibaba.com
200
spent time:1.1927924156188965

2.多线程：

from urllib.request import urlopen
import time
from threading import Thread

class GetUrlThread(Thread):
    def __init__(self, url):
        self.url = url
        super(GetUrlThread, self).__init__()
    def run(self):
        resp = urlopen(self.url)
        print(self.url, resp.getcode())

def get_responses():
    urls = [
        ‘http://www.baidu.com‘,
        ‘http://www.taobao.com‘,
        ‘http://www.alibaba.com‘,
    ]
    start = time.time()
    threads = []
    for url in urls:
        t = GetUrlThread(url)
        threads.append(t)
        t.start()
    for t in threads:
        t.join()
    print("spent time:%s" % (time.time()-start))

get_responses()

解释：
意识到了程序在执行时间上的提升
我们写了一个多线程程序来减少cpu的等待时间，当我们在等待一个线程内的网络请求返回时，这时cpu可以切换到其他线程去进行其他线程内的网络请求。
我们期望一个线程处理一个url，所以实例化线程类的时候我们传了一个url。
线程运行意味着执行类里的run()方法。
无论如何我们想每个线程必须执行run()。
为每个url创建一个线程并且调用start()方法，这告诉了cpu可以执行线程中的run()方法了。
我们希望所有的线程执行完毕的时候再计算花费的时间，所以调用了join()方法。
join()可以通知主线程等待这个线程结束后，才可以执行下一条指令。
每个线程我们都调用了join()方法，所以我们是在所有线程执行完毕后计算的运行时间。
关于线程：
cpu可能不会在调用start()后马上执行run()方法。
你不能确定run()在不同线程建间的执行顺序。
对于单独的一个线程，可以保证run()方法里的语句是按照顺序执行的。
这就是因为线程内的url会首先被请求，然后打印出返回的结果。

输出为：
http://www.baidu.com 200
http://www.alibaba.com 200
http://www.taobao.com 200
spent time:0.6294200420379639

实例二：全局变量的线程安全问题（race condition）

1.BUG版：

from threading import Thread
import time

#define a global variable
some_var = 0

class IncrementThread(Thread):
    def run(self):
        # we want to read a global variable
        # and then increment it
        global some_var
        read_var = some_var
        print("some_var in %s is %d" % (self.name, read_var))
        time.sleep(0.1)
        some_var = read_var + 1
        print("some_var in %s is %d" % (self.name, some_var))

def use_increment_thread():
    threads = []
    for i in range(50):
        t = IncrementThread()
        threads.append(t)
        t.start()
    for t in threads:
        t.join()
    print("After 50 modifications, some_var should have become 50")
    print("After 50 modifications, some_var is %d" % some_var)

use_increment_thread()

解释：

有一个全局变量，所有的线程都想修改它。
所有的线程应该在这个全局变量上加 1 。
有50个线程，最后这个数值应该变成50，但是它却没有。
为什么没有达到50？
在some_var是15的时候，线程t1读取了some_var，这个时刻cpu将控制权给了另一个线程t2。
t2线程读到的some_var也是15
t1和t2都把some_var加到16
当时我们期望的是t1 t2两个线程使some_var + 2变成17
在这里就有了资源竞争。
相同的情况也可能发生在其它的线程间，所以出现了最后的结果小于50的情况。
输出为：
some_var in Thread-1 is 0
some_var in Thread-2 is 0
some_var in Thread-3 is 0
some_var in Thread-4 is 0
some_var in Thread-5 is 0
some_var in Thread-6 is 0
some_var in Thread-7 is 0
some_var in Thread-8 is 0
some_var in Thread-9 is 0
some_var in Thread-10 is 0
some_var in Thread-11 is 0
some_var in Thread-12 is 0
some_var in Thread-13 is 0
some_var in Thread-14 is 0
some_var in Thread-15 is 0
some_var in Thread-16 is 0
some_var in Thread-17 is 0
some_var in Thread-18 is 0
some_var in Thread-19 is 0
some_var in Thread-20 is 0
some_var in Thread-21 is 0
some_var in Thread-22 is 0
some_var in Thread-23 is 0
some_var in Thread-24 is 0
some_var in Thread-25 is 0
some_var in Thread-26 is 0
some_var in Thread-27 is 0
some_var in Thread-28 is 0
some_var in Thread-29 is 0
some_var in Thread-30 is 0
some_var in Thread-31 is 0
some_var in Thread-32 is 0
some_var in Thread-33 is 0
some_var in Thread-34 is 0
some_var in Thread-35 is 0
some_var in Thread-36 is 0
some_var in Thread-37 is 0
some_var in Thread-38 is 0
some_var in Thread-39 is 0
some_var in Thread-40 is 0
some_var in Thread-41 is 0
some_var in Thread-42 is 0
some_var in Thread-43 is 0
some_var in Thread-44 is 0
some_var in Thread-45 is 0
some_var in Thread-46 is 0
some_var in Thread-47 is 0
some_var in Thread-48 is 0
some_var in Thread-49 is 0
some_var in Thread-50 is 0
some_var in Thread-6 is 1
some_var in Thread-5 is 1
some_var in Thread-2 is 1
some_var in Thread-4 is 1
some_var in Thread-1 is 1
some_var in Thread-3 is 1
some_var in Thread-12 is 1
some_var in Thread-13 is 1
some_var in Thread-11 is 1
some_var in Thread-10 is 1
some_var in Thread-9 is 1
some_var in Thread-7 is 1
some_var in Thread-8 is 1
some_var in Thread-21 is 1
some_var in Thread-20 is 1
some_var in Thread-19 is 1
some_var in Thread-18 is 1
some_var in Thread-17 is 1
some_var in Thread-15 is 1
some_var in Thread-14 is 1
some_var in Thread-16 is 1
some_var in Thread-26 is 1
some_var in Thread-25 is 1
some_var in Thread-24 is 1
some_var in Thread-22 is 1
some_var in Thread-23 is 1
some_var in Thread-31 is 1
some_var in Thread-29 is 1
some_var in Thread-28 is 1
some_var in Thread-27 is 1
some_var in Thread-30 is 1
some_var in Thread-38 is 1
some_var in Thread-37 is 1
some_var in Thread-36 is 1
some_var in Thread-35 is 1
some_var in Thread-32 is 1
some_var in Thread-33 is 1
some_var in Thread-34 is 1
some_var in Thread-44 is 1
some_var in Thread-43 is 1
some_var in Thread-42 is 1
some_var in Thread-41 is 1
some_var in Thread-40 is 1
some_var in Thread-39 is 1
some_var in Thread-50 is 1
some_var in Thread-49 is 1
some_var in Thread-48 is 1
some_var in Thread-47 is 1
some_var in Thread-45 is 1
some_var in Thread-46 is 1
After 50 modifications, some_var should have become 50
After 50 modifications, some_var is 1

解决竞争带锁版：

 1 from threading import Lock, Thread
 2 import time
 3 lock = Lock()
 4 some_var = 0
 5
 6 class IncrementThread(Thread):
 7     def run(self):
 8         #we want to read a global variable
 9         #and then increment it
10         global some_var
11         lock.acquire()
12         read_value = some_var
13         print("some_var in %s is %d" % (self.name, read_value))
14         time.sleep(0.1)
15         some_var = read_value + 1
16         print("some_var in %s after increment is %d" % (self.name, some_var))
17         lock.release()
18
19 def use_increment_thread():
20     threads = []
21     for i in range(50):
22         t = IncrementThread()
23         threads.append(t)
24         t.start()
25     for t in threads:
26         t.join()
27     print("After 50 modifications, some_var should have become 50")
28     print("After 50 modifications, some_var is %d" % (some_var,))
29
30 use_increment_thread()

解释： 

Lock 用来防止竞争条件
如果在执行一些操作之前，线程t1获得了锁。其他的线程在t1释放Lock之前，不会执行相同的操作
我们想要确定的是一旦线程t1已经读取了some_var，直到t1完成了修改some_var，其他的线程才可以读取some_var
这样读取和修改some_var成了逻辑上的原子操作。
输出为：
some_var in Thread-1 is 0
some_var in Thread-1 after increment is 1
some_var in Thread-2 is 1
some_var in Thread-2 after increment is 2
some_var in Thread-3 is 2
some_var in Thread-3 after increment is 3
some_var in Thread-4 is 3
some_var in Thread-4 after increment is 4
some_var in Thread-5 is 4
some_var in Thread-5 after increment is 5
some_var in Thread-6 is 5
some_var in Thread-6 after increment is 6
some_var in Thread-7 is 6
some_var in Thread-7 after increment is 7
some_var in Thread-8 is 7
some_var in Thread-8 after increment is 8
some_var in Thread-9 is 8
some_var in Thread-9 after increment is 9
some_var in Thread-10 is 9
some_var in Thread-10 after increment is 10
some_var in Thread-11 is 10
some_var in Thread-11 after increment is 11
some_var in Thread-12 is 11
some_var in Thread-12 after increment is 12
some_var in Thread-13 is 12
some_var in Thread-13 after increment is 13
some_var in Thread-14 is 13
some_var in Thread-14 after increment is 14
some_var in Thread-15 is 14
some_var in Thread-15 after increment is 15
some_var in Thread-16 is 15
some_var in Thread-16 after increment is 16
some_var in Thread-17 is 16
some_var in Thread-17 after increment is 17
some_var in Thread-18 is 17
some_var in Thread-18 after increment is 18
some_var in Thread-19 is 18
some_var in Thread-19 after increment is 19
some_var in Thread-20 is 19
some_var in Thread-20 after increment is 20
some_var in Thread-21 is 20
some_var in Thread-21 after increment is 21
some_var in Thread-22 is 21
some_var in Thread-22 after increment is 22
some_var in Thread-23 is 22
some_var in Thread-23 after increment is 23
some_var in Thread-24 is 23
some_var in Thread-24 after increment is 24
some_var in Thread-25 is 24
some_var in Thread-25 after increment is 25
some_var in Thread-26 is 25
some_var in Thread-26 after increment is 26
some_var in Thread-27 is 26
some_var in Thread-27 after increment is 27
some_var in Thread-28 is 27
some_var in Thread-28 after increment is 28
some_var in Thread-29 is 28
some_var in Thread-29 after increment is 29
some_var in Thread-30 is 29
some_var in Thread-30 after increment is 30
some_var in Thread-31 is 30
some_var in Thread-31 after increment is 31
some_var in Thread-32 is 31
some_var in Thread-32 after increment is 32
some_var in Thread-33 is 32
some_var in Thread-33 after increment is 33
some_var in Thread-34 is 33
some_var in Thread-34 after increment is 34
some_var in Thread-35 is 34
some_var in Thread-35 after increment is 35
some_var in Thread-36 is 35
some_var in Thread-36 after increment is 36
some_var in Thread-37 is 36
some_var in Thread-37 after increment is 37
some_var in Thread-38 is 37
some_var in Thread-38 after increment is 38
some_var in Thread-39 is 38
some_var in Thread-39 after increment is 39
some_var in Thread-40 is 39
some_var in Thread-40 after increment is 40
some_var in Thread-41 is 40
some_var in Thread-41 after increment is 41
some_var in Thread-42 is 41
some_var in Thread-42 after increment is 42
some_var in Thread-43 is 42
some_var in Thread-43 after increment is 43
some_var in Thread-44 is 43
some_var in Thread-44 after increment is 44
some_var in Thread-45 is 44
some_var in Thread-45 after increment is 45
some_var in Thread-46 is 45
some_var in Thread-46 after increment is 46
some_var in Thread-47 is 46
some_var in Thread-47 after increment is 47
some_var in Thread-48 is 47
some_var in Thread-48 after increment is 48
some_var in Thread-49 is 48
some_var in Thread-49 after increment is 49
some_var in Thread-50 is 49
some_var in Thread-50 after increment is 50
After 50 modifications, some_var should have become 50
After 50 modifications, some_var is 50

实例三：多线程环境下的原子操作

BUG版本：

 1 from threading import Thread
 2 import time
 3
 4 class CreateListThread(Thread):
 5     def run(self):
 6         self.entries = []
 7         for i in range(10):
 8             # time.sleep(0.1)
 9             self.entries.append(i)
10         for each in self.entries:
11             print(each, end = " ")
12             time.sleep(0.1)
13
14 def use_create_list_thread():
15     for i in range(3):
16         t = CreateListThread()
17         t.start()
18
19 use_create_list_thread()

解释：
当一个线程正在打印的时候，cpu切换到了另一个线程，所以产生了不正确的结果。我们需要确保print self.entries是个逻辑上的原子操作，以防打印时被其他线程打断。
因为打印的速度太快，我在此有意放大了这个时间，加了一个time.sleep(0.1)
输出为：
0 0 0 1 1 1 2 2 2 3 3 3 4 4 4 5 5 5 6 6 6 7 7 7 8 8 8 9 9 9

2.加锁保证操作的原子性

 1 from threading import Thread, Lock
 2 import time
 3
 4 lock = Lock()
 5
 6
 7 class CreateListThread(Thread):
 8     def run(self):
 9         self.entries = []
10         for i in range(10):
11             time.sleep(0.1)
12             self.entries.append(i)
13         lock.acquire()
14         for each in self.entries:
15             print(each, end = " ")
16             time.sleep(0.1)
17         lock.release()
18
19
20 def use_create_list_thread():
21     for i in range(3):
22         t = CreateListThread()
23         t.start()
24
25 use_create_list_thread()

输出为：
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9

时间： 2024-10-14 23:58:46

浅析Python多线程的相关文章

python多线程

http://blog.csdn.net/pipisorry/article/details/45306973 CPU-bound(计算密集型) 和I/O bound(I/O密集型) I/O bound 指的是系统的CPU效能相对硬盘/内存的效能要好很多,此时,系统运作,大部分的状况是 CPU 在等 I/O (硬盘/内存) 的读/写,此时 CPU Loading 不高.CPU bound 指的是系统的 硬盘/内存 效能 相对 CPU 的效能 要好很多,此时,系统运作,大部分的状况是 CPU Lo

Python多线程实现方法有几种

目前python多线程实现方式有很多种比如:thread,threading,multithreading ,其中thread模块比较底层,而threading模块是对thread做了一些包装,可以更加方便的被使用. 2.7 版本之前python对线程的支持还不够完善,不能利用多核CPU,但是2.7版本的python中已经考虑改进这点,出现了 multithreading模块.threading模块里面主要是对一些线程的操作对象化,创建Thread的class.一般来说,使用线程有两种 模式:

浅析python中_name_='_main_'

刚接触到python时,对代码中的_name_='_main_'比较疑惑,本文对其的讲解借鉴了其他博客讲述(见参考资料),希望和大家共同学习. Make a script both importable and executable 首先先看一个例子 1 #module.py 2 def main(): 3 print "we are in %s"%__name__ 4 if __name__ == '__main__': 5 main() 在这段函数中,定义main函数,当py文件被

python多线程、多进程以及GIL

多线程 使用threading模块创建线程 传入一个函数 这种方式是最基本的,即调用threading中的Thread类的构造函数,然后指定参数target=func,再使用返回的Thread的实例调用start()方法,即开始运行该线程,该线程将执行函数func,当然,如果func需要参数,可以在Thread的构造函数中传入参数args=(-).示例代码如下 import threading #用于线程执行的函数 def counter(n): cnt = 0; for i in xrange

Python多线程锁

[Python之旅]第六篇(四):Python多线程锁 python lock 多线程 多线程使用方法 多线程锁 摘要:   在多线程程序执行过程中,为什么需要给一些线程加锁以及如何加锁,下面就来说一说. 1.给线程加锁的原因     我们知道,不同进程之间的内存空间数据是不能够共享的,试想一下,如果可以随意共享,谈何安全?但是一个进程中的多个线程是可以共享这个进程的内存空间中的数据的,比如多个线程可以同时调用某一... 在多线程程序执行过程中,为什么需要给一些线程加锁以及如何加锁,下面就来说一

python多线程学习记录

1.多线程的创建 import threading t = t.theading.Thread(target, args--) t.SetDeamon(True)//设置为守护进程 t.start(),启动线程 t.join(),阻塞当前线程,即使得在当前线程结束时,不会退出.会等到子线程结束之后才退出. 如果不加join语句,主线程不会等到子线程结束才结束,但却不会立即杀死该线程. 但是如果添加了SetDaemon(True),如果不加join,则会在主线程结束后马上杀死子线程. 如果join

python多线程实现抓取网页

Python实现抓取网页 下面的Python抓取网页的程序比较初级,只能抓取第一页的url所属的页面,只要预定URL足够多,保证你抓取的网页是无限级别的哈,下面是代码: ##coding:utf-8 ''' 无限抓取网页 @author wangbingyu @date 2014-06-26 ''' import sys,urllib,re,thread,time,threading ''' 创建下载线程类 ''' class download(threading.Thread): def __

Python多线程（threading）学习总结

注:此文除了例子和使用心得是自己写的,很多都是Python核心编程中的原文.原文文风应该能看出来,就不每个地方单独表明出处了. 线程(有时被称为轻量级进程)跟进程有些相似,不同的是,所有的线程运行在同一个进程中,共享相同的运行环境.它们可以想像成是在主进程或"主线程"中并行运行的"迷你进程". 线程有开始,顺序执行和结束三部分.它有一个自己的指令指针,记录自己运行到什么地方.线程的运行可能被抢占(中断),或暂时的被挂起(也叫睡眠),让其它的线程运行,这叫做让步.一个

python多线程ssh爆破如何实现与防范？

本文和大家分享的主要是python多线程的ssh**与防范相关内容,一起来看看吧,希望对大家学习python多线程有所帮助. 0x01.About 这几天发现朋友的服务器22被人爆了,于是想想,也想去爆别人服务器. 爆弱口令时候写的一个python小脚本,主要功能是实现使用字典多线程**ssh,支持ip表导入,字典数据导入. 主要使用到的是python的paramiko模块和多线程threading模块. 那么,首先要准备的是字典dict.服务器ip表. 东西很简单,主要默认目录如下: |--s