Java batch is becoming a hot topic in enterprise environments these days, but how do you do it the right way? The project spring-boot-starter-batch-web offers a best practice approach to modern batch architectures, answering the following questions:
How do I actually deploy jobs?
How do I start, stop and monitor them?
How do I integrate them into my companies’ infrastructure?
How do I build job artifacts?
It builds on Spring Batch, Spring Boot and JSR-352, and it’s one of codecentric’s open source projects.
The just released version 1.3.0.RELEASE is available on Maven Central:
1<dependency> 2 <groupid>de.codecentric</groupid> 3 <artifactid>spring-boot-starter-batch-web</artifactid> 4 <version>1.3.0.RELEASE</version> 5</dependency>
Metrics
This release is all about metrics. You often need to count or sum up data while a job is running, either for business cases (How many items of which type have been processed? How much money did we transfer during job processing total etc.) or for monitoring (How is the processing time developing? Where are our performance bottlenecks?).
And metrics aren’t a trivial thing. Spring Batch has a very thoughtful transactional behaviour allowing skipping, retrying and restarting, but that also means that sometimes items are processed several times without us taking notice about that. Take a look at this blog post for more information about the transactional behaviour. So, if you naively do the counting and summing in memory, you might end up with wrong numbers. Which can be crucial.
We extended the Spring Boot metrics framework to allow for metrics aligned with the transactions, rolling back metrics when the transaction is rolled back.
So, how do you use it?
First of all, it’s not activated by default, you need to set the property batch.metrics.enabled to true. Then you may autowire our BatchMetrics into any configuration class and inject it into some component. In the following example BatchMetrics are injected into an ItemProcessor:
1@Configuration
2public class BatchMetricsConfiguration {
3
4 @Autowired
5 private BatchMetrics batchMetrics;
6
7 @Bean
8 public MetricsItemProcessor processor() {
9 return new MetricsItemProcessor(batchMetrics);
10 }
11
12}
The BatchMetrics interface contains methods for writing to counters and gauges, and for each type there is a transactional and a non-transactional method. There are different cases where you need one or the other, and the JavaDoc of the BatchMetrics interface tells you more about those cases.
The generated metrics get a prefix containing jobname, jobexecution-id and step name so that they are unique for the running step. At the end of the step they are copied to the step’s execution context omitting the prefix and deleted afterwards. If you want to keep the metrics you may set batch.metrics.deletemetricsonstepfinish to false. In addition to copying them to the step’s execution context they are also logged. You may change the log format by providing an implementation of MetricsOutputFormatter in the ApplicationContext, it will be picked up automatically.
Profiling method calls
Sometimes it’s good to know which methods have been called how many times and what the average duration was. You may use full blown APM solutions like AppDynamics or Dynatrace for that, but as a simple and free alternative you may also use Spring’s built in AOP capabilities. We provide the class AbstractBatchMetricsAspect which can be extended to build aspects, see ReaderProcessorWriterMetricsAspect as an example. The same metrics prefix like in BatchMetrics is used so that the resulting gauges are also transferred to the step’s execution context and logged. The ReaderProcessorWriterMetricsAspect can be activated by setting batch.metrics.profiling.readprocesswrite.enabled to true. It profiles the ItemReader, ItemProcessor and the ItemWriter.
Default listeners
If you have some listeners that need to be applied to every job, doing infrastructural things, it’s a little bit awkward to have to configure them in every job. spring-boot-starter-batch-web provides the interface ListenerProvider which can be extended by an @Configuration class to let the starter know about it. The starter then collects all listeners from implementations of ListenerProvider and applies them to every job being registered. That makes job configurations cleaner, separating business and technical concerns. As an example take a look at our MetricsConfiguration doing the configuration stuff for the metrics features above. It also adds a new default listener by implementing ListenerProvider.
Documentation update
We added a page describing all the available properties and code extensions here .
Next steps
Next on our todo list is doing XA transactions with more than one DataSource, JMS queue etc. The upcoming Spring Boot 1.2.0 will support XA transactions with different providers, so we’re gonna use that feature to implement XA support in our starter.
Was this post helpful?
More articles
fromTobias Flohre
16.2.2022 Your job at codecentric?
Jobs
Agile Developer und Consultant (w/d/m)
Alle Standorte
Gemeinsam bessere Projekte umsetzen.
Wir helfen deinem Unternehmen.
Du stehst vor einer großen IT-Herausforderung? Wir sorgen für eine maßgeschneiderte Unterstützung. Informiere dich jetzt.
Hilf uns, noch besser zu werden.
Wir sind immer auf der Suche nach neuen Talenten. Auch für dich ist die passende Stelle dabei.
Blog author
Tobias Flohre
Senior Software Developer
Do you still have questions? Just send me a message.
Do you still have questions? Just send me a message.