实验需要,在一个含有几亿个字符的txt文件中查找特定的字符串,首先用re模块进行查找
1 from time import clock 2 import re 3 start=clock() 4 label_file = open("/home/ying/data/google_streetview_train_test1/label.txt") 5 label_str = label_file.read() 6 label_file.close() 7 filename = "2_0_pitch_95_yaw_95_lat_41.8975137_lng_-87.6268723.jpg" 8 start=clock() 9 for match in re.finditer(filename, label_str): 10 s = match.start() 11 e = match.end() 12 print(s) 13 print(e) 14 end=clock() 15 print(end-start)
re.finditer(filename, label_str)可以在label_str中查找filename的位置,s=match.start()返回字符串开始的索引,e=match.end(),返回字符串结束的索引。程序运行的结果是
304091635
304091689
304096479
304096533
1.003844
耗时1s左右
同样的,由于txt文件中为一行一行的数据,可以用readlines进行遍历读取比较,程序如下
1 from time import clock 2 start=clock() 3 data_label="/home/ying/data/google_streetview_train_test1/label.txt" 4 filename = "2_0_pitch_95_yaw_95_lat_41.8975137_lng_-87.6268723.jpg" 5 file = open(data_label) 6 lines = file.readlines() 7 print(len(lines)) 8 for line in lines: 9 cls = line.split() 10 fn = cls.pop(0) 11 if fn==filename: 12 break 13 end=clock() 14 print(end-start)
运行结果如下:
1
3.335657
可见耗时有3s多,用正则化模块要快的多
另外,由于label_str中存在1.2_0_pitch_95_yaw_95_lat_41.8975137_lng_-87.6268723.jpg,所以用re模块寻找时会返回两个结果,而用逐行读取的方式则返回一个值
时间: 2024-11-05 12:26:06