python环境测试MySQLdb、DBUtil、sqlobject性能
首先介绍下MySQLdb、DBUtil、sqlobject:
(1)MySQLdb 是用于Python连接Mysql数据库的接口,它实现了 Python 数据库
API 规范 V2.0,基于 MySQL C API 上建立的。除了MySQLdb外,python还可以通过oursql, PyMySQL, myconnpy等模块实现MySQL数据库操作;
(2)DBUtil中提供了几种连接池,用以提高数据库的访问性能,例如PooledDB,PesistentDB等
(3)sqlobject可以实现数据库ORM映射的第三方模块,可以以对象、实例的方式轻松操作数据库中记录。
为测试这三者的性能,简单做一个例子:50个并发访问4000条记录的单表,数据库记录如下:
测试代码如下:
1、MySQLdb的代码如下,其中在connDB()中把连接池相关代码暂时做了一个注释,去掉这个注释既可以使用连接池来创建数据库连接:
(1)DBOperator.py
02 |
from stockmining.stocks.setting import LoggerFactory |
05 |
class DBOperator( object ): |
08 |
self .logger = LoggerFactory.getLogger( ‘DBOperator‘ ) |
12 |
self .conn = MySQLdb.connect(host = "127.0.0.1" ,user = "root" ,passwd = "root" ,db = "pystock" ,port = 3307 ,charset = "utf8" ) |
14 |
#self.conn=connectionpool.pool.connection() |
18 |
if ( self .conn ! = None ): |
21 |
def execute( self , sql): |
23 |
if ( self .conn ! = None ): |
24 |
cursor = self .conn.cursor() |
26 |
raise MySQLdb.Error( ‘No connection‘ ) |
28 |
n = cursor.execute(sql) |
30 |
except MySQLdb.Error,e: |
31 |
self .logger.error( "Mysql Error %d: %s" % (e.args[ 0 ], e.args[ 1 ])) |
33 |
def findBySQL( self , sql): |
35 |
if ( self .conn ! = None ): |
36 |
cursor = self .conn.cursor() |
38 |
raise MySQLdb.Error( ‘No connection‘ ) |
41 |
rows = cursor.fetchall() |
43 |
except MySQLdb.Error,e: |
44 |
self .logger.error( "Mysql Error %d: %s" % (e.args[ 0 ], e.args[ 1 ])) |
(2)测试代码testMysql.py,做了50个并发,对获取到的数据库记录做了个简单遍历。
06 |
operator = DBOperator() |
08 |
starttime = time.time() |
09 |
sql = "select * from stock_cash_tencent" |
10 |
peeps = operator.findBySQL(sql) |
14 |
diff = (endtime - starttime) * 1000 |
19 |
threading.Thread(target = run).start() |
22 |
if __name__ = = ‘__main__‘ : |
2、连接池相关代码:
(1)connectionpool.py
01 |
from DBUtils import PooledDB |
06 |
mincached = 10 #最小空闲连接 |
07 |
maxcached = 20 #最大空闲连接 |
08 |
maxshared = 30 #最大共享连接 |
09 |
connstring = "root#root#127.0.0.1#3307#pystock#utf8" #数据库地址 |
12 |
def createConnectionPool(connstring, dbtype): |
13 |
db_conn = connstring.split( "#" ); |
16 |
pool = PooledDB.PooledDB(MySQLdb, user = db_conn[ 0 ],passwd = db_conn[ 1 ],host = db_conn[ 2 ],port = string.atoi(db_conn[ 3 ]),db = db_conn[ 4 ],charset = db_conn[ 5 ], mincached = mincached,maxcached = maxcached,maxshared = maxshared,maxconnections = maxconn) |
19 |
raise Exception, ‘conn datasource Excepts,%s!!!(%s).‘ % (db_conn[ 2 ], str (e)) |
23 |
pool = createConnectionPool(connstring, dbtype) |
3、sqlobject相关代码
(1)connection.py
1 |
from sqlobject.mysql import builder |
3 |
conn = builder()(user = ‘root‘ , password = ‘root‘ , |
4 |
host = ‘127.0.0.1‘ , db = ‘pystock‘ , port = 3307 , charset = ‘utf8‘ ) |
(2)StockCashTencent.py对应到数据库中的表,50个并发并作了一个简单的遍历。(实际上,如果不做遍历,只做count()计算,sqlobject性能是相当高的。)
03 |
from connection import conn |
06 |
class StockCashTencent(sqlobject.SQLObject): |
09 |
code = sqlobject.StringCol() |
10 |
name = sqlobject.StringCol() |
11 |
date = sqlobject.StringCol() |
12 |
main_in_cash = sqlobject.FloatCol() |
13 |
main_out_cash = sqlobject.FloatCol() |
14 |
main_net_cash = sqlobject.FloatCol() |
15 |
main_net_rate = sqlobject.FloatCol() |
16 |
private_in_cash = sqlobject.FloatCol() |
17 |
private_out_cash = sqlobject.FloatCol() |
18 |
private_net_cash = sqlobject.FloatCol() |
19 |
private_net_rate = sqlobject.FloatCol() |
20 |
total_cash = sqlobject.FloatCol() |
23 |
starttime = time.time() |
24 |
query = StockCashTencent.select( True ) |
25 |
for result in query: pass |
27 |
diff = (endtime - starttime) * 1000 |
30 |
if __name__ = = ‘__main__‘ : |
32 |
threading.Thread(target = test).start() |
测试结果如下:
MySQLdb平均(毫秒) |
99.63999271 |
DBUtil平均(毫秒) |
97.07998276 |
sqlobject平均(毫秒) |
343.2999897 |
结论:其实就测试数据而言,MySQLdb单连接和DBUtil连接池的性能并没有很大的区别(100个并发下也相差无几),相反sqlobject虽然具有的编程上的便利性,但是却带来性能上的巨大不足,在实际中使用哪个模块就要斟酌而定了。
时间: 2024-10-24 22:50:08