Wednesday, June 10, 2015

Automate e-Payment Directory updates

Recently I've faced a tricky case, related to integration. One e-business system used to update e-payment directory with shell concurrent program and data from Federal Reserve Bank Services web site. System has worked quite well with wget  utility and direct link to text file with routing information.
But once it delivered HTML document instead of text document. The reason is quite simple - from now on  you should agree with Service terms and then you will get access to the data.
This changes even has led to Github project FedACHDir. Well, with all my respect to GitHub and project authors, I'm not comfortable at all to get such sensitive information from the third party. So I have had to overcome it and automate download.
Let's rethink approach to the task. Well known wget utility allows you to download  data form internet but now we have to interact with site, POST information and maintain web session. Answer is simple - curl another well known web utility, included in the most of Linux repositories and compiled for  the rest of the operating systems.
Long story short below is a small but elegant Shell script to get data from Federal Reserve Bank Services.

Reminder: use this script means that you are agree with terms and conditions posted on the FRBS site



#!/bin/sh #check output file name if [ -z $1 ]; then achout=/tmp/FedACHdir.txt >&2 echo "Save data to /tmp/FedACHdir.txt" else achout=$1 fi # Set session cookies rm /tmp/fedc.txt 2>/dev/null curl -s -c /tmp/fedc.txt \
 https://www.frbservices.org/EPaymentsDirectory/FedACHdir.txt -o /dev/null # Submit agreement curl -s -b /tmp/fedc.txt -c /tmp/fedc.txt -d "agreementValue=Agree" \
 -H "Referer:https://www.frbservices.org/EPaymentsDirectory/agreement.html" \
 https://www.frbservices.org/EPaymentsDirectory/submitAgreement -o /dev/null #read FedACH catalog curl -b /tmp/fedc.txt \
 https://www.frbservices.org/EPaymentsDirectory/FedACHdir.txt -o $achout # Cleanup cookies rm /tmp/fedc.txt 2>/dev/null

If you find this useful: enjoy and have fun.
Post a Comment