Feels like most of the work is not related to data science at all. Is this accurate?
Yes
I know this is not a data-driven company with a high-level data engineering department, but it is my opinion that data science requires minimum levels of data accessibility. Am I wrong?
You're not wrong, but such are the realities of real life.
Is this type of setup common for a company with serious data science needs?
Yes
From a technical standpoint, you need to look into ETL solutions that can make your life easier. Sometimes one tool can be much faster than another to read certain data. E.g. R's readxl is orders of mangnitudes faster than python's pandas at reading xlsx files; you could use R to import the files, then save them to a Python-friendly format (parquet, SQL, etc). I know you're not working on xlsx files and I have no idea if you use Python - it was just an example.
From a practical standpoint, two things:
- first of all, understand what is technically possible. In many cases,
the people telling you no are IT-illiterate people who worry about
business or compliance considerations, but have no concept of what is
and isn't feasible from an IT standpoint. Try to speak to the DBAs or
to whoever manages the data infrastructure. Understand what is
technically possible. THEN, only then, try to find a compromise. E.g.
they won't give you access to their system, but I presume there is a
database behind it? Maybe they can extract the data to some other
formats? Maybe they can extract the SQL statements that define the
First of all, understand what is technically possible. In many cases, the people telling you know are IT-illiterate people who worry about business or compliance considerations, but have no concept of what is and isn't feasible from an IT standpoint. Try to speak to the DBAs or to whoever manages the data infrastructure. Understand what is technically possible. THEN, only then, try to find a compromise. E.g. they won't give you access to their system, but I presume there is a database behind it? Maybe they can extract the data to some other formats? Maybe they can extract the SQL statements that define the data types etc?
data types etc? - business people are more likely to help you if you can make the case that doing so is in THEIR interest. If they don't even believe in what you're doing, tough luck...
Business people are more likely to help you if you can make the case that doing so is in THEIR interest. If they don't even believe in what you're doing, tough luck...