为什么要批量插入
要插入10000条数据,如果不批量插入的话,那么我们执行的sql语句将是10000条insert
insert into member (group_id, user_id, role, extend) values (101, 100, 3, NULL)
insert into member (group_id, user_id, role, extend) values (101, 101, 3, NULL)
insert into member (group_id, user_id, role, extend) values (101, 102, 3, NULL)
insert into member (group_id, user_id, role, extend) values (101, 103, 3, NULL)
......
如果是批量插入,一批次插入50条,那么10000条数据将生成200条insert语句
insert into member (group_id, user_id, role, extend) VALUES (101, 100, 3, NULL), (101, 101, 3, NULL), ..., (101, 149, 3, NULL)
insert into member (group_id, user_id, role, extend) VALUES (101, 100, 3, NULL), (101, 101, 3, NULL), ..., (101, 149, 3, NULL)
好了,以上是jdbc级别的批量插入,换言之,我们需要JPA提供批量插入功能
JPA批量插入
搜索了Stackoverflow,似乎JPA本身并没有对批量插入的支持
http://stackoverflow.com/questions/7440397/how-to-do-the-batch-insert-in-jpa
Hibernate批量插入
因为使用Hibernate作为JPA的provider实现,因此我们可以使用Hibernate提供的批量插入功能
查阅hibernate文档
http://docs.jboss.org/hibernate/core/3.3/reference/en/html/batch.html
(1) 首先要在hibernate配置文件中配置属性
<property name="hibernate.jdbc.batch_size" value="20" /> <!-- value文档建议在10-50间 -->
(2) 批量插入代码示例
1 for ( int i=0; i<100000; i++ ) { 2 Customer customer = new Customer(.....); 3 session.save(customer); 4 if ( i % 20 == 0 ) { //20, same as the JDBC batch size 5 //flush a batch of inserts and release memory: 6 session.flush(); 7 session.clear(); 8 } 9 }
注意
如果批量插入的实体主键为@GeneratedValue(strategy=IDENTITY)类型,则hibernate会透明关闭在jdbc级别的批量插入;因为实际上,每次插入该实体的一个实例,hibernate都需要搜索该实例的主键;想想在代码中,我们persist实例之后,就能够通过get方法获取实例的主键;
实验结果也说明了这点
实体定义
1 @Entity 2 @Table (name="member") 3 public class GroupUser { 4 @Id 5 @GeneratedValue(strategy=IDENTITY) 6 @Column(name="id", nullable=true, unique=true) 7 private Integer id; 8 9 ...... 10 11 }
实验代码
1 Session session = (Session)em.getDelegate(); 2 session.setFlushMode(FlushMode.MANUAL); 3 4 for (int i = 0; i < userIds.size(); ++i) { 5 session.save(new GroupUser(groupId, userIds.get(i), role)); 6 if (i % 20 == 0) { 7 session.flush(); 8 session.clear(); 9 } 10 }
日志输出
Hibernate: insert into member (group_id, user_id, role, extend) values (?, ?, ?, ?)
Hibernate: insert into member (group_id, user_id, role, extend) values (?, ?, ?, ?)
Hibernate: insert into member (group_id, user_id, role, extend) values (?, ?, ?, ?)
Hibernate: insert into member (group_id, user_id, role, extend) values (?, ?, ?, ?)
......