Sending data with Spring Batch can be done in several ways. The most used case with Spring Batch is to send the data as XML files, but this framework offers you a lot of other ways too.
One of these ways is to send the data to a secure FTP server. For this use case the Spring Integration framework has the correct interface.
Firstly you need, like in all cases to create files with Spring Batch, a marshaller with a “Reader” and a “Writer”.
The Reader:
1<bean id="sftpFileReader"> 2 <property name="dataSource" ref="dataSource" /> 3 <property name="sql" value="SELECT * FROM table" /> 4 <property name="rowMapper"> 5 <bean class="de.package.rowmapper.SftpFileRowMapper" /> 6 </property> 7</bean>
The RowMapper which is used by the Reader:
1package de.package.rowmapper;
2
3import java.sql.ResultSet;
4import java.sql.SQLException;
5
6import org.apache.commons.logging.Log;
7import org.apache.commons.logging.LogFactory;
8import org.springframework.jdbc.core.RowMapper;
9
10import de.package.domainObjects.SftpFileObject;
11
12public class SftpFileRowMapper implements RowMapper {
13
14 public Object mapRow(ResultSet rs, int rowNum) throws SQLException {
15 SftpFileObject fileLine = new SftpFileObject();
16 try {
17 fileLine.setDbField1(rs.getString("dbField1"));
18 fileLine.setDbField2(rs.getString("dbField2"));
19 fileLine.setDbField3(rs.getString("dbField3"));
20 fileLine.setDbField4(rs.getString("dbField4"));
21 fileLine.setDbField5(rs.getString("dbField5"));
22 } catch (SQLException e) {
23 System.out.println("Can't create data row for export File.");
24 }
25 return fileLine;
26 }
27}
The DomainObject which is used by the Reader:
1package de.package.domainObjects;
2
3public class SftpFileObject implements java.io.Serializable {
4
5 private static final long serialVersionUID = 1L;
6
7 public String dbField1;
8 public String dbField2;
9 public String dbField3;
10 public String dbField4;
11 public String dbField5;
12
13 public String getDbField1() {
14 return dbField1;
15 }
16
17 public void setDbField1(String dbField1) {
18 this.dbField1= dbField1;
19 }
20
21 public String getDbField2() {
22 return dbField2;
23 }
24
25 public void setDbField2(String dbField2) {
26 this.dbField2= dbField2;
27 }
28
29 public String getDbField3() {
30 return dbField3;
31 }
32
33 public void setDbField3(String dbField3) {
34 this.dbField3= dbField3;
35 }
36
37 public String getDbField4() {
38 return dbField4;
39 }
40
41 public void setDbField4(String dbField4) {
42 this.dbField4= dbField4;
43 }
44
45 public String getDbField5() {
46 return dbField5;
47 }
48
49 public void setDbField5(String dbField5) {
50 this.dbField5= dbField5;
51 }
52
53}
The Writer:
1<bean id="sftpFileWriter" scope="step"> 2 <property name="resource" value="file:path/to/file/filename.csv" /> 3 <property name="encoding" value="ISO-8859-1" /> 4 <property name="headerCallback"> 5 <bean class="de.package.helper.HeaderCallback" /> 6 </property> 7 <property name="lineAggregator"> 8 <bean class="org.springframework.batch.item.file.transform.DelimitedLineAggregator"> 9 <property name="delimiter" value=";" /> 10 <property name="fieldExtractor"> 11 <bean class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor"> 12 <property name="names" value="dbField1, dbField2, dbField3, dbField4, dbField5" /> 13 </bean> 14 </property> 15 </bean> 16 </property> 17</bean>
The HeaderCallback is only needed, if you want to give the file, in our case a csv file, a headline, which is not created by the Reader. Then you need a small java class which creates the line for the header.
1package de.package.helper;
2
3import java.io.IOException;
4import java.io.Writer;
5import org.springframework.batch.item.file.FlatFileHeaderCallback;
6
7public class HeaderCallback implements FlatFileHeaderCallback {
8
9 @Override
10 public void writeHeader(Writer writer) throws IOException {
11 writer.write("FieldHeadline1;FieldHeadline2;FieldHeadline3;FieldHeadline4;FieldHeadline5");
12 }
13}
For sending the files to a SFTP server you need a SftpSessionFactory, that includes the access data of the receiving server.
1<bean id="sftpSessionFactory" class="org.springframework.integration.sftp.session.DefaultSftpSessionFactory"> 2 <property name="host" value="host.of.receiver"/> 3 <property name="user" value="username"/> 4 <property name="password" value="secureSftpPassword"/> 5 <property name="port" value="22"/> 6</bean>
Then you need a channel. This channel is used to send the data.
1<int:channel id="outputChannel" />
For knowing the kind of sending, you have to tell Spring Batch the way to do. This is done by an “outbound-channel-adapter”, which includes the kind of sending way, links the channel to the SftpSessionFactory and tells the framework how the file should be named on the target server (remote-filename-generator).
1<int-sftp:outbound-channel-adapter id="sftpOutboundAdapter" 2 session-factory="sftpSessionFactory" 3 channel="outputChannel" 4 charset="UTF-8" 5 remote-directory="/target" 6 remote-filename-generator="fileNameGenerator" />
To make it easy, the target server should store the file with the same name like it is stored on the sending server. To do this, you need only the DefaultFilenameGenerator of Spring Integration.
1<bean id="fileNameGenerator" class="org.springframework.integration.file.DefaultFileNameGenerator" />
To really send the file now, you need at least a Tasklet and a BatchJob.
For the Tasklet you need a small java class where you give the filename and the channel as parameters.
1<bean id="sftpJobTasklet" class="package de.package.tasklets.SftpTasklet"> 2 <property name="fileName" value=" path/to/file/filename.csv" /> 3 <property name="sftpChannel" ref="outputChannel" /> 4</bean>
The javacode for sending is also really simple:
1package de.package.tasklets;
2
3import java.io.File;
4import org.springframework.batch.core.StepContribution;
5import org.springframework.batch.core.scope.context.ChunkContext;
6import org.springframework.batch.core.step.tasklet.Tasklet;
7import org.springframework.batch.repeat.RepeatStatus;
8import org.springframework.integration.Message;
9import org.springframework.integration.MessageChannel;
10import org.springframework.integration.support.MessageBuilder;
11
12public class SftpTasklet implements Tasklet {
13
14 private String fileName;
15 private MessageChannel sftpChannel;
16
17 @Override
18 public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception {
19
20 File file = new File(fileName);
21
22 if (file.exists()) {
23 Message<File> message = MessageBuilder.withPayload(file).build();
24 try {
25 sftpChannel.send(message);
26 } catch (Exception e) {
27 System.out.println("Could not send file per SFTP: " + e);
28 }
29 } else {
30 System.out.println("File does not exist.");
31 }
32
33 return RepeatStatus.FINISHED;
34 }
35
36 public String getFileName() {
37 return fileName;
38 }
39
40 public void setFileName(String fileName) {
41 this.fileName = fileName;
42 }
43
44 public MessageChannel getSftpChannel() {
45 return sftpChannel;
46 }
47
48 public void setSftpChannel(MessageChannel sftpChannel) {
49 this.sftpChannel = sftpChannel;
50 }
51
52}
At least the xml code for the BatchJob:
1<batch:job id="sftpJob" restartable="false"> 2 <batch:step id="sftpFileGenerateStep" next="sftpFileSendStep"> 3 <batch:tasklet> 4 <batch:chunk reader="sftpFileCreator" writer="sftpFileWriter" commit-interval="100" /> 5 <batch:listeners> 6 <batch:listener ref="fileNameListener" /> 7 </batch:listeners> 8 </batch:tasklet> 9 </batch:step> 10 <batch:step id="sftpFileSendStep"> 11 <batch:tasklet ref="sftpJobTasklet" /> 12 </batch:step> 13</batch:job>
This BatchJob can be started by a CronJob on the server.
At last part I will shortly post the header of the xml file called “applicationContext.xml”, which includes all the described beans:
1<xml version="1.0" encoding="UTF-8"?> 2<beans xmlns="http://www.springframework.org/schema/beans" 3xmlns:batch="http://www.springframework.org/schema/batch" 4xmlns:int="http://www.springframework.org/schema/integration" 5xmlns:int-file="http://www.springframework.org/schema/integration/file" 6xmlns:int-sftp="http://www.springframework.org/schema/integration/sftp" 7xmlns:int-stream="http://www.springframework.org/schema/integration/stream" 8xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd 9http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch-2.1.xsd 10http://www.springframework.org/schema/integration http://www.springframework.org/schema/integration/spring-integration.xsd 11http://www.springframework.org/schema/integration/file http://www.springframework.org/schema/integration/file/spring-integration-file-2.0.xsd 12http://www.springframework.org/schema/integration/sftp http://www.springframework.org/schema/integration/sftp/spring-integration-sftp-2.0.xsd 13http://www.springframework.org/schema/integration/stream http://www.springframework.org/schema/integration/stream/spring-integration-stream-2.0.xsd">
Was this post helpful?
More articles
fromCarsten Fröhlich
6.9.2011 Your job at codecentric?
Jobs
Agile Developer und Consultant (w/d/m)
Alle Standorte
Gemeinsam bessere Projekte umsetzen.
Wir helfen deinem Unternehmen.
Du stehst vor einer großen IT-Herausforderung? Wir sorgen für eine maßgeschneiderte Unterstützung. Informiere dich jetzt.
Hilf uns, noch besser zu werden.
Wir sind immer auf der Suche nach neuen Talenten. Auch für dich ist die passende Stelle dabei.
Blog author
Carsten Fröhlich
Do you still have questions? Just send me a message.
Do you still have questions? Just send me a message.