com.atlassian.crowd.util.persistence.hibernate.batch

Class AbstractBatchProcessor<S>

java.lang.Object

com.atlassian.crowd.util.persistence.hibernate.batch.AbstractBatchProcessor<S>

All Implemented Interfaces:

BatchProcessor<S>

Direct Known Subclasses:

AbstractHibernateBatchProcessor

public abstract class AbstractBatchProcessor<S>
extends Object
implements BatchProcessor<S>

Thread-safe batch processor.

This processor is essentially a heavyweight generic DAO for processing batched operations over a collection of entities.

The batchSize defaults to batchSize and can be manually set (via Spring, for example) using the appropriate setter method. The batchSize should match the hibernate.jdbc.batch_size property defined in the Hibernate configuration.

Each batch operation is first divided into smaller sets of batchSize. If there is an error in processing the batch, the batched JDBC call is rolled-back and the batch is processed individually.

This mechanism ensures very fast (JDBC-batched) inserts and updates and follows it up with a fail-over retry for the failing batches. Callback methods are provided to allow you include the session and transaction management you desire. The processing flow is:

 call beforeProcessCollection()
 For each item in the collection
     Add the item to the batch collection
     If the batch collection size >= the batch size
         call beforeProcessBatch()
         For each item in the batch collection
            perform the operation on the item
         call afterProcessBatch() or rollbackProcessBatch() if an error occurred
         clear the batch collection
 If there are unprocessed items in the Batch Collection
     call beforeProcessBatch()
     For each item in the batch collection
        perform the operation on the item
     call afterProcessBatch() or rollbackProcessBatch() if an error occurred
 call beforeProcessCollection()
 

If an error occurred during the processing of a batch collection then:

 For each item in the batch collection
    call beforeProcessIndividual()
    perform the operation on the item
    call afterProcessIndividual() or rollbackProcessIndividual() if an error occurred
 

NOTE 1: Do not use this if your database is not transactional, *stab* MySQL ISAM.

Author:

Shihab Hamid, Matthew Jensen

Field Summary

Fields

Modifier and Type	Field and Description
protected int	batchSize
protected org.slf4j.Logger	log

Constructor Summary

Constructors

Constructor and Description
AbstractBatchProcessor()

Method Summary

Modifier and Type	Method and Description
protected abstract void	afterProcessBatch() Called after successfully processing each batch.
protected abstract void	afterProcessCollection() Called when processing the collection has completed successfully.
protected abstract void	afterProcessIndividual() Called after successully processing an item individually.
protected abstract void	auditOperations(List<AuditLogChangesetEntity> changesetEntities)
protected abstract void	beforeProcessBatch() Called before processing each batch.
protected abstract void	beforeProcessCollection() Called before the collection is processed into a batch.
protected abstract void	beforeProcessIndividual() Called before processing an individual item.
<E> BatchResult<E>	execute(HibernateOperation<S> op, Collection<E> objects)
<E> BatchResult<E>	execute(HibernateOperation<S> op, Collection<E> objects, BulkAuditMapper<E> bulkAuditMapper) The set is first divided into smaller sets of batchSize.
protected abstract S	getSession() Subclasses should use this method to provide the session to be used with any implementations of HibernateOperation.
protected abstract void	rollbackProcessBatch() Called after processing each batch where an exception was encountered.
protected abstract void	rollbackProcessIndividual() Called after processing an individual item where an exception was encountered.
void	setBatchSize(int batchSize) The batchSize value should be the same as the hibernate.jdbc.batch_size Hibernate property.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

log

protected final org.slf4j.Logger log

batchSize

protected int batchSize

Constructor Detail

AbstractBatchProcessor

public AbstractBatchProcessor()

Method Detail

execute

public final <E> BatchResult<E> execute(HibernateOperation<S> op,
                                        Collection<E> objects)

Specified by:

execute in interface BatchProcessor<S>

execute

public final <E> BatchResult<E> execute(HibernateOperation<S> op,
                                        Collection<E> objects,
                                        BulkAuditMapper<E> bulkAuditMapper)

The set is first divided into smaller sets of batchSize. Each batchSet is added via a batched JDBC call (using Hibernate's batching mechanism). If there is an error in processing the batch, the batched JDBC call is rolled-back and the batchSet is processed individually.

This mechanism ensures very fast (JDBC-batched) inserts and updates and follows it up with a fail-over retry for the failing batches.

NOTE: do not use this if your database is not transactional, *stab* MySQL ISAM.

Specified by:

execute in interface BatchProcessor<S>

Parameters:

op - Hibernate operation to perform (eg. replicate, saveOrUpdate).

objects - set of DirectoryEntity objects to batch add.

Returns:

batch result.

auditOperations

protected abstract void auditOperations(List<AuditLogChangesetEntity> changesetEntities)

getSession

protected abstract S getSession()

Subclasses should use this method to provide the session to be used with any implementations of HibernateOperation.

setBatchSize

public void setBatchSize(int batchSize)

The batchSize value should be the same as the hibernate.jdbc.batch_size Hibernate property.

Parameters:

batchSize - batch size used to group batches.

beforeProcessCollection

protected abstract void beforeProcessCollection()

Called before the collection is processed into a batch. Can be used to start session or transaction that will cover the execution of the entire collection which could involve multiple batches.

afterProcessCollection

protected abstract void afterProcessCollection()

Called when processing the collection has completed successfully.

beforeProcessBatch

protected abstract void beforeProcessBatch()

Called before processing each batch. Can be used to start session or transaction that will cover this particular batch.

afterProcessBatch

protected abstract void afterProcessBatch()

Called after successfully processing each batch.

rollbackProcessBatch

protected abstract void rollbackProcessBatch()

Called after processing each batch where an exception was encountered.

beforeProcessIndividual

protected abstract void beforeProcessIndividual()

Called before processing an individual item. This method will be called if the batch failed and the processor falls back to processing each item seperately.

afterProcessIndividual

protected abstract void afterProcessIndividual()

Called after successully processing an item individually.

rollbackProcessIndividual

protected abstract void rollbackProcessIndividual()

Called after processing an individual item where an exception was encountered.