Data Infrastructure as a Data Analyst
Focused doing my first job, what about I teach myself data engineering as well.
Hello, it’s Brien, your fellow data coworker. I hope your week is going well.
This week I want to talk about a topic that has become a huge part of my role and something every data analyst should understand at least at a foundational level.
Data infrastructure.
It sounds technical, but once you break it down, it’s simply the combination of where your data lives and how it moves through your organization.
What is Data Infrastructure?
Data infrastructure is the system that stores your data, moves your data, and prepares your data for use. It includes:
Where the data is housed
How the data moves between systems
How it ends up in dashboards, reports, and tools that stakeholders rely on
Understanding this matters because without it, you end up spending most of your time manually pulling data from APIs, exporting spreadsheets, and cleaning the same messy files over and over again.
A strong data infrastructure removes that repetitive work and lets you focus on actual analysis.
My First Year: Manual Everything
In my first year as a data analyst, I pulled almost everything manually. I exported reports, cleaned them in Excel, filtered out duplicates, and repeated the same steps every week.
I wasn’t even using SQL yet. I relied on the built‑in reporting tools that come with education software.
Eventually I asked myself a simple question:
What if this data could just land somewhere automatically, and everyone could access it?
That question led me to the concept of a data warehouse.
Two Core Ideas in Data Infrastructure: ETL and ELT
To understand how data moves, you need to understand two processes: ETL and ELT. Both are types of data pipelines.
ETL: Extract, Transform, Load
This is the traditional approach.
Extract the data from a source system
Transform it into a usable format
Load it into a warehouse or analytics tool
In a typical organization, data might come from a software platform, land in a warehouse like Snowflake or Databricks, and then be cleaned and reshaped using SQL before analysts or data scientists use it.
The transformation happens before the data is loaded into its final destination.
ELT: Extract, Load, Transform
As computing has become cheaper, many organizations switched to ELT.
You extract the data, load it into the warehouse immediately, and then transform it inside that system.
This approach is faster and more flexible, especially when you’re working with large datasets.
What If You Don’t Have Snowflake or Databricks?
This was exactly my situation.
We used Infinite Campus (IC) as our student information system, and we were transitioning to Airtable for internal operations. Airtable is like a mix between a database and a project management tool.
The problem was that the tools available to connect IC to Airtable were either clunky or way too expensive.
So I had to pick up some software engineering skills.
I built an in‑house API that moved data from IC to Airtable in under a minute.
I handled the extract and transform steps inside IC, and my API handled the loading step. Now our Airtable tables update automatically, and stakeholders can access clean, structured data whenever they need it.
This also made it easier for me to build dashboards and track KPIs without constantly cleaning the same data.
All of this came from learning the basics of data infrastructure.
From Teacher to Analyst to… Part‑Time Data Engineer?
That’s honestly what it feels like sometimes.
But understanding data infrastructure has made my work more efficient, more scalable, and more impactful. It’s one of the biggest reasons I’ve been able to grow in my role.
I appreciate you coming back every week to learn from my journey. If you know someone who would enjoy learning about data or hearing about this career path, feel free to share this newsletter with them.
Data infrastructure is becoming a competitive advantage in itself. The better the data pipeline, the better the decisions built on top of it.