Spring Batch write Header and Footer in FlatFile

In previous post, we talked about how to read multiple queries and used MultiResourceItemReader to combine two set of data. Although it works, but required one more step to do a merging additionally the solution generated two extra files. Is there an alternative solution to not generate extra file but still achieve the same goal? In this post I will shared another trick to do so which is using appendAllowed. Beside this, I will also share on how to insert a header and footer with a total count.

1. Define Batch Job with two steps.

We used back the same example in previous post, in the car_job.xml, we define only two steps, first two steps will read in one set of data each from the database and write into a flat file. But in the second step, the writer instead of writing into a new file, it append into the first file. In order to write the header and footer, we need to create a custom writer and delegate the writing job.

2. JdbcCursorItemReader vs JdbcPagingItemReader

Below snippet code shows how to define a reader using JdbcCursorItemReader. If you have a complex SQL you may want to use org.springframework.batch.item.database.JdbcPagingItemReader. In secondReader, we used JdbcPagingItemReader as example, be noted that the sortkey is a compulsory property.

3. Writer to write Header and Footer in FlatFile

Now, we need to define two writer. The firstWriter will write the first set of data and header, in secondWriter instead of create new flatfile, it append to existing file and closing with a footer.

* For firstWriter we don’t include appendAllowed=true, it is because the data will keep append into same file new run.

In CarWriter.java we implements FlatFileFooterCallback, FlatFileHeaderCallback, ItemStream, StepExecutionListener. Before a step are executed, the spring-batch framework calls the open()-method of the ItemStream interface for Readers and Writers, and HeaderCallback will be called in open() method and FooterCallback in close() method. In this case, you will be expected that even your reader hit an exception, the file will be generated with a header and footer in flatfile.

To pass the step1 total count to next step, we need to use StepExecutionListener to listen step1 afterStep, and create an attribute. In the next step, also using StepExecutionListener to listen step2 beforeStep, to extract out the value and proceed.

Now, you should be able to write two set data into only one single file.