Stardust/Knowledge Base/Integration/Camel/Consuming A CSV File - at FTP or Local FileSystem

Use Case

Consuming a CSV file (delimter could be anything) arriving at either an FTP location or in a folder on local file system. E.g. a csv file arriving at:

• C:\temp\temp.csv named commissionRates.csv, or
• /FtpFeed/comException folder.

commissionRates.csv - containts commission rates. All the content except the first line would be transfered into an Stardust process data, named CommissionRates.

name;code;currency;vatIndicator;amount
A;B;USD;Y;123
A1;B1;HKD;N;11123

Process named LoadCommissionRates will be trigerred on arrival of this file (only 1 process instance to load entire file into Stardust process).

Process Definition of LoadCommissionRates -

commissionRatesException.csv - per exception a seperate process instance of Stardust process will be intiated.

A;B;USD;Y;123;AAAAAAAAAAAAAAAAAAAAAAAAA
A1;B1;HKD;N;11123;BBBBBBBBBBBBBBBBBBBBBBBBB

Process named CommissionExceptionMgmt will be triggered on arrival of this file (as many instances of the process as the number of rows in this file).

Process Definition of CommissionExceptionMgmt -

How do we achieve this?

We will use Stardust's Generic Camel Route Event for this purpose.

Note: Please change your profile from Business Analyst to Integrator in Business Process Modelling perspective.

For Case 1:

• Create Start Event, change its type from None to Message and then add the below route in the Configuration Tab:

<from uri="file://c:\temp\temp.csv?fileName=commissionRates.csv&amp;initialDelay=5000&amp;delay=5000&amp;readLock=markerFile"/>
<unmarshal>
<csv skipFirstLine="true"  delimiter=";"/>
</unmarshal>
<setHeader headerName="CamelLanguageScript">
<constant>
<![CDATA[
importClass(java.util.HashMap);
importClass(java.util.ArrayList);
var results=new HashMap();
var commissionRates=new ArrayList();
 
var lines=request.body;
for (var i = 0; i < lines.size(); i++) {
 
    var line = lines.get(i);
    print("----"+line );
    print("****length="+line.size());
    var commissionRate= new HashMap();
 
    commissionRate.put("name",line.get(0));
    commissionRate.put("code", line.get(1));
    commissionRate.put("currency",line.get(2));
    commissionRate.put("vatIndicator",line.get(3));
    commissionRate.put("amount",line.get(4));
 
    commissionRates.add(commissionRate);   
 
} 
results.put("commissionRates",commissionRates);
exchange.out.body=results;
]]>
</constant>
</setHeader>
<to uri="language:javascript"/>
<to uri="ipp:direct"/>

• Click on Paramters Tab and map data as shown below:

• Data Definition of CommissionRates:

For Case 2:

• Create Start Event, change its type from None to Message and then add the below route in the Configuration Tab:

<from uri="file://c:\temp\temp9.csv?fileName=commissionRatesException.csv&amp;initialDelay=5000&amp;delay=5000&amp;readLock=markerFile"/>
 <split>
 <tokenize token="\n" />
 <unmarshal>
<csv  delimiter=";"/>
</unmarshal>
<setHeader headerName="CamelLanguageScript">
<constant>
<![CDATA[
importClass(java.util.HashMap);
importClass(java.util.ArrayList);
var results=new HashMap();
var line=request.body.get(0);
print("----"+line );
print("****length="+line.size());
var commissionEx= new HashMap();
 
    commissionEx.put("name",line.get(0));
    commissionEx.put("code", line.get(1));
    commissionEx.put("currency",line.get(2));
    commissionEx.put("vatIndicator",line.get(3));
    commissionEx.put("amount",line.get(4));
    commissionEx.put("exception",line.get(5));
 
results.put("commissionException",commissionEx);
exchange.out.body=commissionEx;
]]>
</constant>
</setHeader>
<to uri="language:javascript"/>
<to uri="ipp:direct"/>
</split>

Note: Please mark the use of split and route ipp:direct inside it. This is the reason to start as many process instance as the number of rows in the file.

• Click on Paramters Tab and map data as shown below:

• Data Definition of ComssionException:

The current use case explains to trigger processes on the arrival of file(s) to a folder on local file system. Just change the first route as shown below to respond to file(s) arrival at an FTP location:

For Case 1 -

<from  uri="ftp://motu@localhost/FtpFeed/comRate?password=motu&amp;stepwise=false&amp;delay=5000&amp;delete=true"/>

For Case 2 -

<from  uri="ftp://motu@localhost/FtpFeed/comException?password=motu&amp;stepwise=false&amp;delay=5000&amp;delete=true"/>

Runtime behavior:

File commissionRates.csv arrives at destination folder:

File commissionRatesException.csv arrives at destination folder:

2 instance becasue we have 2 rows in the file

Please find the model here File:SBBCommissionCapture.zip.

In the example the files have ';' as delimiter. Incases you have other delimiter characters, please change the route section below accordingly:

<unmarshal>
<csv skipFirstLine="true"  delimiter=";"/>
</unmarshal>
..
..
..