As part of my banking with ABN Amro I get a credit card managed by ICS Cards.
The single most annoying thing about ICS Cards is that they don’t provide any means of exporting my statements. The next most annoying thing is that they haven’t responded to any of my emails request that feature.
Time to help myself then, in true developer fashion, I’ll write my own.
I’ve used loads of languages over the years and different languages suit different tasks. Python turns out to be my go-to language for scraping web-pages and pulling out information from them. Mostly because it’s really easy to knock up a script and because there are loads of great libraries already available to do most of the grunt work.
So here it is. A quick and dirty script to log into my account, parse the monthly statement page, fix the formatting and write it out as CSV.
This is by no means a finished script (have you seen how messy it is!!) and it’s kinda hard-coded what it will download.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 |
__author__ = 'Martin Gill' from bs4 import BeautifulSoup import requests import csv import re url = "https://www.icscards.nl/abnamro/mijn/accountstatements?period=201310" loginUrl = "https://www.icscards.nl/pkmslogin.form" payload = {'username': 'Username', 'password': 'password', 'login-form-type': 'pwd'} s = requests.Session() r = s.post(loginUrl, payload, verify=False) data = r.content soup = BeautifulSoup(data) print(soup.prettify()) r = s.get(url) data = r.content soup = BeautifulSoup(data) table = soup.select("table.expander-table")[0] headers = table.select("th") outputRows = [] headerItems = [] for th in headers: headerItems.append(th.getText()) outputRows.append(headerItems) rows = table.select("tr") for row in rows: cols = row.select("td") rowItems = [] for col in cols: # Replace newlines with whitespace text = col.getText().replace("\n", " ").strip() rowItems.append(text) # skips the "header" row if len(rowItems) > 0: # Replace commas with dots (Dutch decimal to English) rowItems[5] = rowItems[5].replace(',', '.') # Fix comma and eliminate currency symbol and other crap rowItems[6] = re.sub(r"^.+\s(\d+),(\d+).*", r"\1.\2", rowItems[6]) # Credit/Debit handling if re.match(r"Debet", rowItems[4]): rowItems[6] = '-' + rowItems[6] outputRows.append(rowItems) with open('test.csv', 'w', newline='') as csvfile: writer = csv.writer(csvfile, delimiter=',', quotechar='"', quoting=csv.QUOTE_MINIMAL) writer.writerows(outputRows) |
Here’s an example output once the script is done with it:
|
1 2 3 |
Datum,*,Omschrijving,Card-nummer,Debet / Credit,Valuta,Bedrag 03-09-2013,,GH *GITHUB.COM xxxx xxxxxx USA Land: Verenigde Staten Merchant code: Computers,xxxx,Debet,USD 7.00,-5.41 04-09-2013,,WORLD OF WARCRAFT SUB SCUNTHORPE NO GBR Land: Verenigd Koninkrijk Merchant code: Dir Market-Continuity,xxxx,Debet,GBP 8.99,-10.85 |