The document describes how to use the Batch component in Mule applications. The Batch component allows processing messages in batches by splitting messages into individual records, performing actions on each record, and reporting results. The document includes an example Mule flow that uses a Batch job to iterate over a list of maps and perform database updates on each item. It logs output showing the batch job processing the records and handling any errors.
2. Abstract
• The main motto of this PPT is How to use
Batch component in our applications.
3. Introduction
• Mule possesses the ability to process messages
in batches. Within an application, you can initiate
a batch job which is a block of code that splits
messages into individual records, performs
actions upon each record, then reports on the
results and potentially pushes the processed
output to other systems or queues. This
functionality is particularly useful when working
with streaming input or when engineering "near
real-time" data integration between SaaS
applications.
8. • INFO 2016-05-28 18:51:48,882 [main] org.mule.module.launcher.MuleDeploymentService:
• ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
• + Started app 'Sample' +
• ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
• INFO 2016-05-28 18:51:48,886 [main] org.mule.module.launcher.DeploymentDirectoryWatcher:
• ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
• + Mule is up and kicking (every 5000ms) +
• ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
• INFO 2016-05-28 18:51:49,040 [main] org.mule.module.launcher.StartupSummaryDeploymentListener:
• **********************************************************************
• * - - + DOMAIN + - - * - - + STATUS + - - *
• **********************************************************************
• * default * DEPLOYED *
• **********************************************************************
• *******************************************************************************************************
• * - - + APPLICATION + - - * - - + DOMAIN + - - * - - + STATUS + - - *
• *******************************************************************************************************
• * Sample * default * DEPLOYED *
• *******************************************************************************************************
• INFO 2016-05-28 18:51:53,201 [[Sample].HTTP_Listener_Configuration.worker.01] org.mule.api.processor.LoggerMessageProcessor: --Hai
• INFO 2016-05-28 18:51:53,253 [[Sample].HTTP_Listener_Configuration.worker.01] org.mule.api.processor.LoggerMessageProcessor: --main flow--[{b=2, c=3}, {a=1}, {e=5}, {d=4}]
• INFO 2016-05-28 18:51:53,354 [[Sample].HTTP_Listener_Configuration.worker.01] com.mulesoft.module.batch.engine.DefaultBatchEngine: Created instance dc959d50-24fc-11e6-b3de-
30d420524153 for batch job SampleBatch
• INFO 2016-05-28 18:51:53,359 [[Sample].HTTP_Listener_Configuration.worker.01] com.mulesoft.module.batch.engine.DefaultBatchEngine: Batch job SampleBatch has no input phase.
Creating job instance
• INFO 2016-05-28 18:51:53,383 [[Sample].HTTP_Listener_Configuration.worker.01] com.mulesoft.module.batch.engine.queue.BatchQueueLoader: Starting loading phase for instance
'dc959d50-24fc-11e6-b3de-30d420524153' of job 'SampleBatch'
• INFO 2016-05-28 18:51:53,548 [[Sample].HTTP_Listener_Configuration.worker.01] com.mulesoft.module.batch.engine.queue.BatchQueueLoader: Finished loading phase for instance
dc959d50-24fc-11e6-b3de-30d420524153 of job SampleBatch. 4 records were loaded
• INFO 2016-05-28 18:51:53,562 [[Sample].HTTP_Listener_Configuration.worker.01] com.mulesoft.module.batch.engine.DefaultBatchEngine: Started execution of instance 'dc959d50-
24fc-11e6-b3de-30d420524153' of job 'SampleBatch'
• INFO 2016-05-28 18:51:53,711 [batch-job-SampleBatch-work-manager.01] org.mule.api.processor.LoggerMessageProcessor: --Batch step--{b=2, c=3}--
• INFO 2016-05-28 18:51:53,713 [batch-job-SampleBatch-work-manager.01] org.mule.api.processor.LoggerMessageProcessor: --for each--batch step--2
• INFO 2016-05-28 18:52:00,519 [batch-job-SampleBatch-work-manager.01] org.mule.api.processor.LoggerMessageProcessor: --for each--batch step--3
• INFO 2016-05-28 18:52:00,605 [batch-job-SampleBatch-work-manager.01] org.mule.api.processor.LoggerMessageProcessor: --Batch step--{a=1}--
• INFO 2016-05-28 18:52:00,605 [batch-job-SampleBatch-work-manager.01] org.mule.api.processor.LoggerMessageProcessor: --for each--batch step--1
• INFO 2016-05-28 18:52:00,617 [batch-job-SampleBatch-work-manager.01] org.mule.api.processor.LoggerMessageProcessor: --Batch step--{e=5}--
• INFO 2016-05-28 18:52:00,617 [batch-job-SampleBatch-work-manager.01] org.mule.api.processor.LoggerMessageProcessor: --for each--batch step--5
• INFO 2016-05-28 18:52:02,765 [batch-job-SampleBatch-work-manager.01] com.mulesoft.module.batch.DefaultBatchStep: Found exception processing record on step 'Batch_Step' for job
instance 'dc959d50-24fc-11e6-b3de-30d420524153' of job 'SampleBatch'.
• This is the first record to show this exception on this step for this job instance. Subsequent records with the same failureswill not be logged for performance and log readability reasons:
9. • ********************************************************************************
• Message : String or binary data would be truncated. (com.microsoft.sqlserver.jdbc.SQLServerException). Message payload is of type: String
• Type : org.mule.api.MessagingException
• Code : MULE_ERROR--2
• Payload : 5
• JavaDoc : http://www.mulesoft.org/docs/site/current3/apidocs/org/mule/api/MessagingException.html
• ********************************************************************************
• Exception stack is:
• 1. String or binary data would be truncated. (com.microsoft.sqlserver.jdbc.SQLServerException)
• com.microsoft.sqlserver.jdbc.SQLServerException:216 (null)
• 2. String or binary data would be truncated. (com.microsoft.sqlserver.jdbc.SQLServerException). Message payload is of type: String (org.mule.api.MessagingException)
• org.mule.module.db.internal.processor.AbstractDbMessageProcessor:93 (http://www.mulesoft.org/docs/site/current3/apidocs/org/mule/api/MessagingException.html)
• ********************************************************************************
• Root Exception stack trace:
• com.microsoft.sqlserver.jdbc.SQLServerException: String or binary data would be truncated.
• at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:216)
• at com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1515)
• at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.doExecutePreparedStatement(SQLServerPreparedStatement.java:404)
• at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement$PrepStmtExecCmd.doExecute(SQLServerPreparedStatement.java:350)
• at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:5696)
• at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:1715)
• at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:180)
• at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:155)
• at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.executeUpdate(SQLServerPreparedStatement.java:314)
• at org.mule.module.db.internal.domain.autogeneratedkey.NoAutoGeneratedKeyStrategy.executeUpdate(NoAutoGeneratedKeyStrategy.java:55)
• at org.mule.module.db.internal.domain.executor.UpdateExecutor.doExecuteQuery(UpdateExecutor.java:43)
• at org.mule.module.db.internal.domain.executor.AbstractSingleQueryExecutor.execute(AbstractSingleQueryExecutor.java:48)
• at org.mule.module.db.internal.processor.UpdateMessageProcessor.doExecuteQuery(UpdateMessageProcessor.java:59)
• at org.mule.module.db.internal.processor.AbstractSingleQueryDbMessageProcessor.executeQuery(AbstractSingleQueryDbMessageProcessor.java:42)
• at org.mule.module.db.internal.processor.AbstractDbMessageProcessor.process(AbstractDbMessageProcessor.java:66)
• at org.mule.execution.ExceptionToMessagingExceptionExecutionInterceptor.execute(ExceptionToMessagingExceptionExecutionInterceptor.java:24)
• at org.mule.execution.MessageProcessorNotificationExecutionInterceptor.execute(MessageProcessorNotificationExecutionInterceptor.java:107)
• at org.mule.execution.MessageProcessorExecutionTemplate.execute(MessageProcessorExecutionTemplate.java:44)
• at org.mule.processor.BlockingProcessorExecutor.executeNext(BlockingProcessorExecutor.java:88)
• at org.mule.processor.BlockingProcessorExecutor.execute(BlockingProcessorExecutor.java:59)
• at org.mule.execution.ExceptionToMessagingExceptionExecutionInterceptor.execute(ExceptionToMessagingExceptionExecutionInterceptor.java:24)
• at org.mule.execution.MessageProcessorNotificationExecutionInterceptor.execute(MessageProcessorNotificationExecutionInterceptor.java:107)
• at org.mule.execution.MessageProcessorExecutionTemplate.execute(MessageProcessorExecutionTemplate.java:44)
• at org.mule.routing.AbstractSelectiveRouter.processEventWithProcessor(AbstractSelectiveRouter.java:303)
• at org.mule.routing.AbstractSelectiveRouter.routeWithProcessors(AbstractSelectiveRouter.java:293)
10. • at org.mule.routing.AbstractSelectiveRouter.process(AbstractSelectiveRouter.java:193)
• at org.mule.execution.ExceptionToMessagingExceptionExecutionInterceptor.execute(ExceptionToMessagingExceptionExecutionInterceptor.java:24)
• at org.mule.execution.MessageProcessorNotificationExecutionInterceptor.execute(MessageProcessorNotificationExecutionInterceptor.java:107)
• at org.mule.execution.MessageProcessorExecutionTemplate.execute(MessageProcessorExecutionTemplate.java:44)
• at org.mule.processor.BlockingProcessorExecutor.executeNext(BlockingProcessorExecutor.java:88)
• at org.mule.processor.BlockingProcessorExecutor.execute(BlockingProcessorExecutor.java:59)
• at org.mule.execution.ExceptionToMessagingExceptionExecutionInterceptor.execute(ExceptionToMessagingExceptionExecutionInterceptor.java:24)
• at org.mule.execution.MessageProcessorExecutionTemplate.execute(MessageProcessorExecutionTemplate.java:44)
• at org.mule.processor.BlockingProcessorExecutor.executeNext(BlockingProcessorExecutor.java:98)
• at org.mule.processor.BlockingProcessorExecutor.execute(BlockingProcessorExecutor.java:59)
• at org.mule.routing.outbound.AbstractMessageSequenceSplitter.processParts(AbstractMessageSequenceSplitter.java:129)
• at org.mule.routing.outbound.AbstractMessageSequenceSplitter.process(AbstractMessageSequenceSplitter.java:59)
• at org.mule.execution.ExceptionToMessagingExceptionExecutionInterceptor.execute(ExceptionToMessagingExceptionExecutionInterceptor.java:24)
• at org.mule.execution.MessageProcessorNotificationExecutionInterceptor.execute(MessageProcessorNotificationExecutionInterceptor.java:107)
• at org.mule.execution.MessageProcessorExecutionTemplate.execute(MessageProcessorExecutionTemplate.java:44)
• at org.mule.processor.BlockingProcessorExecutor.executeNext(BlockingProcessorExecutor.java:88)
• at org.mule.processor.BlockingProcessorExecutor.execute(BlockingProcessorExecutor.java:59)
• at org.mule.execution.ExceptionToMessagingExceptionExecutionInterceptor.execute(ExceptionToMessagingExceptionExecutionInterceptor.java:24)
• at org.mule...
• ********************************************************************************
• INFO 2016-05-28 18:52:02,768 [batch-job-SampleBatch-work-manager.01] com.mulesoft.module.batch.DefaultBatchStep: Found exception processing record on step 'Batch_Step' for job
instance 'dc959d50-24fc-11e6-b3de-30d420524153' of job 'SampleBatch'.
• This is the first record to show this exception on this step for this job instance. Subsequent records with the same failureswill not be logged for performance and log readability reasons:
• ********************************************************************************
• Message : String or binary data would be truncated. (com.microsoft.sqlserver.jdbc.SQLServerException). Message payload is of type: String
• Type : org.mule.api.MessagingException
• Code : MULE_ERROR--2
• Payload : 5
• JavaDoc : http://www.mulesoft.org/docs/site/current3/apidocs/org/mule/api/MessagingException.html
• ********************************************************************************
• Exception stack is:
• 1. String or binary data would be truncated. (com.microsoft.sqlserver.jdbc.SQLServerException)
• com.microsoft.sqlserver.jdbc.SQLServerException:216 (null)
• 2. String or binary data would be truncated. (com.microsoft.sqlserver.jdbc.SQLServerException). Message payload is of type: String (org.mule.api.MessagingException)
• org.mule.module.db.internal.processor.AbstractDbMessageProcessor:93 (http://www.mulesoft.org/docs/site/current3/apidocs/org/mule/api/MessagingException.html)
• ********************************************************************************
• Root Exc
• com.microsoft.sqlserver.jdbc.SQLServerException: String or binary data would be truncated.
• at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:216)
• at com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1515)
• at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.doExecutePreparedStatement(SQLServerPreparedStatement.java:404)
• at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement$PrepStmtExecCmd.doExecute(SQLServerPreparedStatement.java:350)
• at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:5696) eption stack trace:
11. • at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:1715)
• at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:180)
• at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:155)
• at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.executeUpdate(SQLServerPreparedStatement.java:314)
• at org.mule.module.db.internal.domain.autogeneratedkey.NoAutoGeneratedKeyStrategy.executeUpdate(NoAutoGeneratedKeyStrategy.java:55)
• at org.mule.module.db.internal.domain.executor.UpdateExecutor.doExecuteQuery(UpdateExecutor.java:43)
• at org.mule.module.db.internal.domain.executor.AbstractSingleQueryExecutor.execute(AbstractSingleQueryExecutor.java:48)
• at org.mule.module.db.internal.processor.UpdateMessageProcessor.doExecuteQuery(UpdateMessageProcessor.java:59)
• at org.mule.module.db.internal.processor.AbstractSingleQueryDbMessageProcessor.executeQuery(AbstractSingleQueryDbMessageProcessor.java:42)
• at org.mule.module.db.internal.processor.AbstractDbMessageProcessor.process(AbstractDbMessageProcessor.java:66)
• at org.mule.execution.ExceptionToMessagingExceptionExecutionInterceptor.execute(ExceptionToMessagingExceptionExecutionInterceptor.java:24)
• at org.mule.execution.MessageProcessorNotificationExecutionInterceptor.execute(MessageProcessorNotificationExecutionInterceptor.java:107)
• at org.mule.execution.MessageProcessorExecutionTemplate.execute(MessageProcessorExecutionTemplate.java:44)
• at org.mule.processor.BlockingProcessorExecutor.executeNext(BlockingProcessorExecutor.java:88)
• at org.mule.processor.BlockingProcessorExecutor.execute(BlockingProcessorExecutor.java:59)
• at org.mule.execution.ExceptionToMessagingExceptionExecutionInterceptor.execute(ExceptionToMessagingExceptionExecutionInterceptor.java:24)
• at org.mule.execution.MessageProcessorNotificationExecutionInterceptor.execute(MessageProcessorNotificationExecutionInterceptor.java:107)
• at org.mule.execution.MessageProcessorExecutionTemplate.execute(MessageProcessorExecutionTemplate.java:44)
• at org.mule.routing.AbstractSelectiveRouter.processEventWithProcessor(AbstractSelectiveRouter.java:303)
• at org.mule.routing.AbstractSelectiveRouter.routeWithProcessors(AbstractSelectiveRouter.java:293)
• at org.mule.routing.AbstractSelectiveRouter.process(AbstractSelectiveRouter.java:193)
• at org.mule.execution.ExceptionToMessagingExceptionExecutionInterceptor.execute(ExceptionToMessagingExceptionExecutionInterceptor.java:24)
• at org.mule.execution.MessageProcessorNotificationExecutionInterceptor.execute(MessageProcessorNotificationExecutionInterceptor.java:107)
• at org.mule.execution.MessageProcessorExecutionTemplate.execute(MessageProcessorExecutionTemplate.java:44)
• at org.mule.processor.BlockingProcessorExecutor.executeNext(BlockingProcessorExecutor.java:88)
• at org.mule.processor.BlockingProcessorExecutor.execute(BlockingProcessorExecutor.java:59)
• at org.mule.execution.ExceptionToMessagingExceptionExecutionInterceptor.execute(ExceptionToMessagingExceptionExecutionInterceptor.java:24)
• at org.mule.execution.MessageProcessorExecutionTemplate.execute(MessageProcessorExecutionTemplate.java:44)
• at org.mule.processor.BlockingProcessorExecutor.executeNext(BlockingProcessorExecutor.java:98)
• at org.mule.processor.BlockingProcessorExecutor.execute(BlockingProcessorExecutor.java:59)
• at org.mule.routing.outbound.AbstractMessageSequenceSplitter.processParts(AbstractMessageSequenceSplitter.java:129)
• at org.mule.routing.outbound.AbstractMessageSequenceSplitter.process(AbstractMessageSequenceSplitter.java:59)
• at org.mule.execution.ExceptionToMessagingExceptionExecutionInterceptor.execute(ExceptionToMessagingExceptionExecutionInterceptor.java:24)
• at org.mule.execution.MessageProcessorNotificationExecutionInterceptor.execute(MessageProcessorNotificationExecutionInterceptor.java:107)
• at org.mule.execution.MessageProcessorExecutionTemplate.execute(MessageProcessorExecutionTemplate.java:44)
• at org.mule.processor.BlockingProcessorExecutor.executeNext(BlockingProcessorExecutor.java:88)
• at org.mule.processor.BlockingProcessorExecutor.execute(BlockingProcessorExecutor.java:59)
• at org.mule.execution.ExceptionToMessagingExceptionExecutionInterceptor.execute(ExceptionToMessagingExceptionExecutionInterceptor.java:24)
• at org.mule...
• ********************************************************************************
12. • INFO 2016-05-28 18:52:02,769 [batch-job-SampleBatch-work-manager.01] org.mule.api.processor.LoggerMessageProcessor: --Batch step--{d=4}--
• INFO 2016-05-28 18:52:02,769 [batch-job-SampleBatch-work-manager.01] org.mule.api.processor.LoggerMessageProcessor: --for each--batch step--
4
• INFO 2016-05-28 18:52:03,049 [batch-job-SampleBatch-work-manager.01] com.mulesoft.module.batch.DefaultBatchStep: Step Batch_Step finished
processing all records for instance dc959d50-24fc-11e6-b3de-30d420524153 of job SampleBatch
• INFO 2016-05-28 18:52:03,095 [batch-job-SampleBatch-work-manager.02] org.mule.api.processor.LoggerMessageProcessor: --step1--{e=5}
• INFO 2016-05-28 18:52:03,096 [batch-job-SampleBatch-work-manager.02] org.mule.api.processor.LoggerMessageProcessor: --for each--batch
step1--5--
• INFO 2016-05-28 18:52:03,488 [batch-job-SampleBatch-work-manager.02] com.mulesoft.module.batch.engine.DefaultBatchEngine: Starting
execution of onComplete phase for instance dc959d50-24fc-11e6-b3de-30d420524153 of job SampleBatch
• INFO 2016-05-28 18:52:03,508 [batch-job-SampleBatch-work-manager.02] org.mule.api.processor.LoggerMessageProcessor: --complete--
com.mulesoft.module.batch.ImmutableBatchJobResult@1fa547d1
• INFO 2016-05-28 18:52:03,509 [batch-job-SampleBatch-work-manager.02] com.mulesoft.module.batch.engine.DefaultBatchEngine: Finished
execution of onComplete phase for instance dc959d50-24fc-11e6-b3de-30d420524153 of job SampleBatch
• INFO 2016-05-28 18:52:03,509 [batch-job-SampleBatch-work-manager.02] com.mulesoft.module.batch.engine.DefaultBatchEngine: Finished
execution for instance 'dc959d50-24fc-11e6-b3de-30d420524153' of job 'SampleBatch'. Total Records processed: 4. Successful records: 3. Failed
Records: 1
• INFO 2016-05-28 18:52:03,511 [batch-job-SampleBatch-work-manager.02] com.mulesoft.module.batch.engine.DefaultBatchEngine:
• *************************************************************************************************************************
***************************************
• * - - + Exception Type + - - * - - + Step + - - * - - + Count + - - *
• *************************************************************************************************************************
***************************************
• * com.mulesoft.module.batch.exception.BatchException * Batch_Step * 1 *
• *************************************************************************************************************************
***************************************
• INFO 2016-05-28 18:52:03,524 [batch-job-SampleBatch-work-manager.02] com.mulesoft.module.batch.DefaultBatchStep: Step Batch_Step1
finished processing all records for instance dc959d50-24fc-11e6-b3de-30d420524153 of job SampleBatch
13. • Flow of execution:
1. URL to trigger the service from browser
http://localhost:8089
2. a. Service will update the status to ‘Success’ for
the records whose num is other than 5
b. It will fail while trying to update the status as
‘successsuccesssuccesssuccesssuccesssuccess’ for
the num 5 because the length of the status excides
the size of the column. Hence, the failed map will
process in Batch_Step1 and will update the status
as ‘batch fail’ for the record whose num is 5