FTP Tracking Sample Profile Workflow

The profile receives a trigger that contains information on what folders need to be downloaded from the FTP server. After the list of files to be downloaded is obtained by using the component FTP Command, the list is stored in a Data Storage. Then the next route downloads the required files from the FTP server by using another component FTP Command. If any errors occur during processing, the profile initiates the retry procedure. The process stops when the retry attempts limit count is exceeded, or all the data is successfully downloaded. You can retrieve the error statistics from the Data Storage by using Edifecs XESManager as described here.

This profile contains the following routes:

This route picks up a trigger file that specifies what data needs to be downloaded from the FTP server, emulates a lookup in the Trading Partner Management system, and stores the triggers in a Data Storage.

Route Structure

Route Components Description

Component Name Component Type Description
FS Inbound FS Inbound Picks up a trigger file that specifies what data needs to be downloaded from the FTP server from the file system.
Session Set Router Checks to see if the session for this trigger has already been set up. Adds the correlation ID and the trigger ID as the message properties.
TPM Lookup Emulation Groovy Runs a groovy script that emulation a lookup in the Trading Partner Management system.
Store Triggers Data Storage Outbound Stores triggers in a Data Storage.

This route checks to see if the incoming trigger has been processed before. The route sends all the new triggers for processing on the route 3)File Processing. By using the FTP Command component, the route obtains a list of files to be downloaded, the list is stored in a Data Storage. By using the component Groovy, the route keeps track of the number of the processing retry attempts for all the sessions that have not been previously completed successfully.

Route Structure

Replacing Delimiters Route

Route Components Description

Component Name Component Type Description
Triggers Data Storage Inbound Picks up triggers ready for processing from the Data Storage.
Save Active Trigger Data Storage Outbound Stores active triggers in the Data Storage.
FTP Command FTP Command Receives a list of folders that need to be downloaded from the FTP server and runs a script that forms a list of items to be downloaded from the FTP server. This component connects to the server through FTP Connection Service.
Process Files JMS Outbound Sends the data ready for processing to the JMS queue of the route 3)Files Processing.
Check File Not In List Data Storage Inbound Checks to see if the data to be downloaded from the FTP server is not already on the list.
Unique File Router Checks to see if the incoming data is unique.
Put to Files Data Storage Outbound Stores files metadata in a Data Storage.
Session Repeat Counter Groovy Runs a groovy script to set the session retry count.
Router Count Error Router Checks to see if the retry count has exceeded the limit.
Repeat Trigger Data Storage Outbound Stores triggers that have not reached the retry limit and are ready fro another processing attempt in a Data Storage.
Save to Incomplete Sessions Data Storage Outbound Saves sessions that have exceeded the retry limit in a Data Storage.
Delete Active Session Data Storage Inbound Removes the triggers that have exceeded the session retry limit from the list of active triggers in the Data Storage.

This route downloads the required files from the FTP server by using the component FTP Command. If any errors occur during processing, the route initiates the retry procedure.

Route Structure

Route Components Description

Component Name Component Type Description
Process JMS Inbound Picks up the data received from the route 2)Sessions Processing from the JMS queue.
Get File Data Storage Inbound Picks up the trigger file associated with the incoming data and sends it to the next route component for duplication check. Sends the original incoming data straight to the component Aggregation.
Tracking and Duplicates Tracking and Duplicates Checks the incoming data for duplicates by comparing file hashes.
Session Checker Subroute Sends the data for processing on the subroute. The subroute checks to see if all the files from this session have been processed. If the session was completed with errors, the subroute stores the session in a Data Storage and removes the corresponding session from the list of active ones.
Aggregation Aggregation Forms a multi-message exchange from the incoming data.
FTP Command FTP Command Runs a groovy script that downloads the required data from the FTP server.
Download Status Router Checks to see if the download was completed successfully.
3_2)Download Ok Processing Subroute For all the successful downloads, the subroute validates the hash (checks to see if {Prp[ftpFileHash]}={Prp[fileHash]} and {Prp[errorEmulator]}!='error'). In case of a successful validation, the subroute saves data to a file system folder. Otherwise, the subroute sends data for another processing attempt.
3_1)Download Errors Processing Subroute For all the unsuccessful downloads, the subroute tracks and checks the session retry count. For all the sessions that have not exceeded the retry limit, the subroute sends data for another round of processing on the route 3)Files Processing. The subroute saves the files that have exceeded the retry limit in a Data Storage and checks to see if the session has been completed successfully or with errors.