Warehouses, Lakes & Pipelines
Warehouses, Lakes & Pipelines groups the core ideas a learner needs at the 2 · data ingestion, storage & integration stage. Work through the lessons in order when new to the area, or use them independently as a reference when implementing an analysis.
Learn the mechanism one decision at a time
Work through the lessons in order if the topic is new. If you already know the basics, open the specific leaf lesson that matches the operation, diagnostic or failure mode you need.
1Definition→
2Mechanism→
3Example→
4Diagnostic→
5Decision
Operational databases vs analytical warehousesOperational databases vs analytical warehouses is a practical concept within Warehouses, Lakes & Pipelines. It helps turn the broader workflow stage “2 · Data Ingestion, Storage & Integration” into an explicit analytical decision that can be explained, implemented and checked. The concept should be understood in terms of purpose, mechanism, assumptions, evidence and downstream consequences.
02Star schemas: facts and dimensionsStar schemas: facts and dimensions is a practical concept within Warehouses, Lakes & Pipelines. It helps turn the broader workflow stage “2 · Data Ingestion, Storage & Integration” into an explicit analytical decision that can be explained, implemented and checked. The concept should be understood in terms of purpose, mechanism, assumptions, evidence and downstream consequences.
03Data lakes and lakehousesData lakes and lakehouses is a practical concept within Warehouses, Lakes & Pipelines. It helps turn the broader workflow stage “2 · Data Ingestion, Storage & Integration” into an explicit analytical decision that can be explained, implemented and checked. The concept should be understood in terms of purpose, mechanism, assumptions, evidence and downstream consequences.
04ETL vs ELTETL vs ELT is a practical concept within Warehouses, Lakes & Pipelines. It helps turn the broader workflow stage “2 · Data Ingestion, Storage & Integration” into an explicit analytical decision that can be explained, implemented and checked. The concept should be understood in terms of purpose, mechanism, assumptions, evidence and downstream consequences.
05Batch vs streaming pipelinesBatch vs streaming pipelines changes how raw variables are represented for analysis or modelling. The transformation should preserve the information needed by the task while making assumptions explicit and reproducible.
06Data contracts and schema evolutionData contracts and schema evolution is a practical concept within Warehouses, Lakes & Pipelines. It helps turn the broader workflow stage “2 · Data Ingestion, Storage & Integration” into an explicit analytical decision that can be explained, implemented and checked. The concept should be understood in terms of purpose, mechanism, assumptions, evidence and downstream consequences.