dbt Explained
dbt: The Data Engineer's Superhero Sidekick Let’s face it: writing SQL is fun… until you’re juggling 47 versions of the same query, hunting down broken dashboards, or explaining to your…
Building Your First DAG with Apache Airflow
Apache Airflow is a powerful platform for orchestrating complex workflows. After learning the Fundamentals and installing Airflow with Docker, it’s time to dive into one of its most essential features…
Warehouses, Lakes, and Lakehouse
The modern data ecosystem is like a real estate market for your bytes except instead of bidding wars, we’ve got schema-on-write vs. schema-on-read drama. Let’s break down the contenders: Data Warehouses, Data Lakes, Lakehouses, and crack open…
Installing Apache Airflow with Docker
Apache Airflow is a powerful workflow orchestration tool used for scheduling, monitoring, and managing complex workflows. Read my previous blog on the Fundamentals of Apache Airflow. Installing Airflow can sometimes…
Fundamentals of Apache Airflow
Modern data workflows involve numerous interconnected tasks, dependencies, and schedules. Manually managing these workflows or using basic schedulers like cron jobs quickly becomes inefficient as complexity grows. This is where…
Building Your First Report and Dashboard in Power BI: A Step-by-Step Guide
Introduction Embark on your Power BI journey by creating a stunning report and dashboard that bring your data to life! In this article, I'll walk you through each step -…
How to perform a large delete operation
Deleting data from your production databases can be tricky. You can either choose TRUNCATE or DELETE statements to suit your need. Here in this blog, I will focus more on…
What are partitions in SQL Server?
SQL Server supports table/index partitions that store multiple chunks of row data in different partitions. Starting in 2016 SP1, this is no longer just an enterprise edition-only feature. In a…
What is Business Intelligence?
You will find people defining BI, Business Intelligence in so many ways, and believe it or not all those different definitions do make sense for a specific need. In a…
How do you choose an index type?
Indexes are on-disk structures tied with a table/view that helps reduce I/O. Implementing a good indexing solution can have dramatic performance gains in the database. However, too many indexes will…