Python may not be the right tool here, but from what little I've read, it should do the trick.

I need to log into the financial POS and GL systems for about 80 recently acquired small businesses and pull historical data. This will need to be a weekly or monthly practice. Right now we offshore the practice where individuals log in, download, and send the information.

This seems tailor made for a script of some kind, but I'm out of my depth here. I'm pretty good in excel with power query and all that once I get my hands on the data.

Would it be sensible to create a separate script for each system that needs to be accessed, referencing a table with all the needed credentials etc? There are probably a half dozen or so separate systems that would need to be accessed.

If this is a non starter, that's fine. But if it's doable, can anyone point me to a reasonable resource to get started? Not looking to become a pro or anything.

Thanks!

I would be worried about security using some random guy from fiverr for this. Particularly if the data set from the point of sale machines has PII or credit card info, etc.

If there isn't any sensitive data being accessed, just delete the development accounts after the product is delivered. Oh, and always insist on getting the source code and not an executable from 3rd party vendors.

Trust but verify.

Definitely, i was thinking that you tell them the parameters, server addresses and field names and what you want it to do but never give credentials to access it to them.

For sure get the source code of the program.

Seems like it would really depend on the complexity of the systems that you need to login to. Do they have api's in place? Do they have development modes to where someone can write a script for it via a sandbox account? If they don't have an api, then you will essentially be doing web-scrapping which can really vary in complexity and going the third-party route might be a bit risky.

There are tools like this available as well:
Data Scraper Chrome Extension

There are also tools such as Puppeteer (Javascript) & Selenium (Python) that make it easier to automate web-tasks.

Personally I would write a single script with different interfaces for each system, but it will probably be easiest starting off writing seperate ones and then combine them later once you get more experience programming.

I wrote a VBA script that scraped TexAgs account histories a couple years ago. Fun project and only took a few nights after work to get it working. I'm not a coder, just an Excel wiz so I had to google EVERYTHING.

If OP is familiar with python already it shouldn't be that difficult.

There can be a big difference from site to site though. Sometimes you have to set session data or go through several layers of network requests and pull data (keys, tokens, etc.) from one request to utilize in another. I imagine TexAgs is pretty simple as it isn't going to be as security focused as financial sites are bound to be.

Fair enough. I've never had to perform any types of user authentication beyond enter a password and usually just relied on a browser already being logged in.

I imagine that could be challenging.

Quote:

---

log into the financial POS

---

If you are going to programmatically do this, then you need to be able to programmatically login to these systems and such systems likely need some amount of config on their side to allow it. Authentication mechanism between a named user (a human) and say an API user is (hopefully) different, if they have any inclination of security.

Some RESTful API interfaces will have Basic authentication (id/password) and some will have access key/secret key or some other tokenization.

Quote:

---

referencing a table with all the needed credentials etc

---

this would also be a security concern, depending on what you are using for a database/table. It needs to be done securely.

Give us an idea of what POS/GL system you are talking about - the product name and that might point us in a direction where we can investigate its integration capabilities for you.

But, let's say you have pulled all 80 "files" of data down (securely) and have them in a secure location...running some python to chug through it an output something usable is certainly doable.

If you are wanting to do this, in the cloud with proper tooling and security, my company can help (we are an AWS partner). Doing this very same thing for a client - calling out to 3rd party APIs to pull down some data, land it in secure storage and then send it through an ETL pipeline to something usable.

Bumping this back up. I got distracted in a new role, and I'm back to thinking about this

CapCity, how is best to contact you to pick your brain?