Spring package dies when more than 1000 records are updated via itemwriter class

I am using Spring package to read and update the same DB table via JPAPagingItemReader and custom Itemwriter class using Open JPA.

But when trying to update more than 1000 records, some of the records are not updated and the package stops working with one of the package states STOPPED, FAILED, UNKNOWN (no exceptions during the batch process).

Below is what I am doing in my code

1) Reader -

<beans:bean id="dataReader"
class="org.springframework.batch.item.database.JpaPagingItemReader" scope="step">
    <beans:property name="entityManagerFactory" ref="entityManagerFactory" />
    <beans:property name="pageSize" value="1"/>
    <beans:property name="queryString"
    value="select b from Test b where b.testStatus = #{jobParameters[status]}"/>  <!β€”here status value is β€˜No’ -->
</beans:bean>

      

2) Processor - Execution of some business logic depends on some state

3) Writer -

<beans:bean id="dataWriter"
class="com.example.batch..DataWriter">
    <beans:property name="entityManager" ref="sharedEntityManager" />
</beans:bean>

      


public class DataWriter implements ItemWriter<TestData> {
    private EntityManager entityManager;
    public EntityManager getEntityManager() {
        return this.entityManager;
    }

    public void setEntityManager(EntityManager entityManager) {
        this.entityManager = entityManager;
    }

    @Transactional
    public void write  (List<? extends TestData> items) {
        try {
            for (TestData testData : items) {
                updateTestDataStatus(testData.getTestDataKey());
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    private void updateTestDataStatus(String testDataKey) {
        Test test = getEntityManager().find(Test.class, testDataKey); //getEntityManager is sharedEntityManager
        test.setTestStatus("Yes");  
        getEntityManager().persist(test);
    }
}

      

4) DB configuration

<tx:annotation-driven transaction-manager="transactionManager" />
<beans:bean id="sharedEntityManager"
    class="org.springframework.orm.jpa.support.SharedEntityManagerBean">
    <beans:property name="entityManagerFactory" ref="entityManagerFactory" />
</beans:bean>

<!-- Configuration for JPATransaction Manager -->
<beans:bean id="transactionManager"
    class="org.springframework.orm.jpa.JpaTransactionManager">
    <beans:property name="entityManagerFactory" ref="entityManagerFactory" />
</beans:bean>

<!-- Configuration for EntityManager -->
<beans:bean id="entityManagerFactory"
    class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
    <beans:property name="dataSource" ref="dataSource" />
    <beans:property name="persistenceUnitName" value="testdata-pu" />
    <beans:property name="jpaVendorAdapter">
        <beans:bean class="org.springframework.orm.jpa.vendor.OpenJpaVendorAdapter">
            <beans:property name="databasePlatform"
                value="org.apache.openjpa.jdbc.sql.OracleDictionary" />
            <beans:property name="database" value="ORACLE" />
            <beans:property name="showSql" value="true" />
        </beans:bean>
    </beans:property>
</beans:bean>

<!-- Configuration for Data Source -->
<beans:bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource"
    destroy-method="close">
    <beans:property name="driverClassName" value="${database.driverClassName}" />
    <beans:property name="url" value="${database.url}" />
    <beans:property name="username" value="${database.username}" />
    <beans:property name="password" value="${database.password}" />
</beans:bean>

<beans:bean id="dataSourcePropertiesplaceholder" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer" >
    <beans:property name="location" value="classpath:datasource.properties"/>
</beans:bean>

<!-- jobRepository - in-memory mapping configuration -->
<beans:bean id="jobRepository"
    class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean">
    <beans:property name="transactionManager" ref="jobRepository-transactionManager" />
</beans:bean>

<beans:bean id="jobRepository-transactionManager"
    class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />

      

+3


source to share


1 answer


Every thing looks great - you actually use it -

Share your assignments and step level settings -



You are also using - Skip Limit, and if so what limit have you set -

Track missing and failed writes - you can add listeners on top of the recording - and you can perform logarithmic faults using the onWriteError method.

Expect to see the configuration of the work level -

0


source







All Articles