spring 官方文档:http://docs.spring.io/spring-batch/trunk/reference/html/configureStep.html#configuringSkip
当spring batch 处理过程中,可能要发生错误,就会导致整个批处理过程的失败,而有些错误不应该导致整个批处理的失败,那么通过配置 ???????????? <skippable-exception-classes> ????????????可以忽略掉某些错误,从而使 批处理继续运行。。
下面是我主要的处理skip逻辑的配置:
<!-- add people desc job begin --> <batch:job id="addPeopleDescJob"> <batch:step id="addDescStep" parent="abstractStep"> <batch:tasklet> <batch:chunk reader="peopleAddDescReader" processor="allowSkipProcessor" writer="addDescPeopleWriter" commit-interval="2" skip-limit="10"> <batch:skippable-exception-classes> <!--batch:include配置允许发生的异常 --> <batch:include class="com.lyx.batch.InvalidDataException" /> </batch:skippable-exception-classes> </batch:chunk> </batch:tasklet> </batch:step> </batch:job> <!-- add people desc job end -->
其中下面这段配置是配置的要忽略的异常:
<batch:skippable-exception-classes> <!--batch:include配置允许发生的异常 --> <batch:include class="com.lyx.batch.InvalidDataException" /> </batch:skippable-exception-classes>
下面是完整的配置:
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:batch="http://www.springframework.org/schema/batch" xmlns:context="http://www.springframework.org/schema/context" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-4.0.xsd http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd"> <!-- 包的扫描 --> <context:component-scan base-package="com.lyx.batch" /> <bean id="exceptionHandler" class="com.lyx.batch.ExceptionListener" /> <batch:step id="abstractStep" abstract="true"> <batch:listeners> <batch:listener ref="exceptionHandler" /> </batch:listeners> </batch:step> <bean id="abstractCursorReader" abstract="true" class="org.springframework.batch.item.database.JdbcCursorItemReader"> <property name="dataSource" ref="dataSource" /> </bean> <!-- add people desc job begin --> <batch:job id="addPeopleDescJob"> <batch:step id="addDescStep" parent="abstractStep"> <batch:tasklet> <batch:chunk reader="peopleAddDescReader" processor="allowSkipProcessor" writer="addDescPeopleWriter" commit-interval="2" skip-limit="10"> <batch:skippable-exception-classes> <!--batch:include配置允许发生的异常 --> <batch:include class="com.lyx.batch.InvalidDataException" /> </batch:skippable-exception-classes> </batch:chunk> </batch:tasklet> </batch:step> </batch:job> <!-- add people desc job end --> <bean id="peopleAddDescReader" parent="abstractCursorReader" scope="step"> <property name="sql"> <value><![CDATA[select first_name ,last_name from people where first_name like ? or last_name like ?]]></value> </property> <property name="rowMapper" ref="peopleRowMapper" /> <property name="preparedStatementSetter" ref="preparedStatementSetter" /> <property name="fetchSize" value="20" /> </bean> <bean id="peopleRowMapper" class="com.lyx.batch.PeopleRowMapper" /> <bean id="preparedStatementSetter" class="com.lyx.batch.PeoplePreparedStatementSetter" /> <bean id="allowSkipProcessor" class="com.lyx.batch.AllowSkipProcessor" /> <bean id="addDescPeopleWriter" class="com.lyx.batch.AddDescPeopleWriter" /> <!--tomcat jdbc pool数据源配置 --> <bean id="dataSource" class="org.apache.tomcat.jdbc.pool.DataSource" destroy-method="close"> <property name="poolProperties"> <bean class="org.apache.tomcat.jdbc.pool.PoolProperties"> <property name="driverClassName" value="com.mysql.jdbc.Driver" /> <property name="url" value="jdbc:mysql://localhost:3306/test" /> <property name="username" value="root" /> <property name="password" value="034039" /> </bean> </property> </bean> <!-- spring batch 配置jobRepository --> <batch:job-repository id="jobRepository" data-source="dataSource" transaction-manager="transactionManager" isolation-level-for-create="REPEATABLE_READ" table-prefix="BATCH_" max-varchar-length="1000" /> <!-- spring的事务管理器 --> <bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager"> <property name="dataSource" ref="dataSource" /> </bean> <!-- batch luncher --> <bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher"> <property name="jobRepository" ref="jobRepository" /> </bean> </beans>
定义的异常类:
InvalidDataException.java
package com.lyx.batch; public class InvalidDataException extends Exception { private static final long serialVersionUID = 1L; public InvalidDataException() { } public InvalidDataException(String message) { super(message); } }
抛出异常的processor
AllowSkipProcessor.java
package com.lyx.batch; import org.springframework.batch.item.ItemProcessor; public class AllowSkipProcessor implements ItemProcessor<People, PeopleDESC> { public PeopleDESC process(People item) throws Exception { System.out.println("process people desc"); if ("lyx".equals(item.getFirstName())) { System.out.println("skip invalid data!!!!!!!!!!!!!!!!!!!!!!!!"); throw new InvalidDataException("skip this data!!!!!!!!!!!!"); } return new PeopleDESC(item.getLastName(), item.getFirstName(), Thread .currentThread().getName()); } }
运行结果:
process people desc
skip invalid data!!!!!!!!!!!!!!!!!!!!!!!!
process people desc
skip invalid data!!!!!!!!!!!!!!!!!!!!!!!!
任务正常完成
程序运行时间: 7526ms
我在数据库中加了两条数据,这两条数据符合抛出异常的条件,当抛出异常后,没有导致批处理的失败,最终成功。
关于SkipPolicy的用法:http://www.cnblogs.com/gulvzhe/archive/2012/01/02/2309690.html
===================END===================
时间: 2024-10-20 02:59:31