Files, Formats & Data Ingestion
Files, Formats & Data Ingestion groups the core ideas a learner needs at the 2 · data ingestion, storage & integration stage. Work through the lessons in order when new to the area, or use them independently as a reference when implementing an analysis.
Learn the mechanism one decision at a time
Work through the lessons in order if the topic is new. If you already know the basics, open the specific leaf lesson that matches the operation, diagnostic or failure mode you need.
1Definition→
2Mechanism→
3Example→
4Diagnostic→
5Decision
CSV and delimited textCSV and delimited text is a practical concept within Files, Formats & Data Ingestion. It helps turn the broader workflow stage “2 · Data Ingestion, Storage & Integration” into an explicit analytical decision that can be explained, implemented and checked. The concept should be understood in terms of purpose, mechanism, assumptions, evidence and downstream consequences.
02Excel workbooksExcel workbooks is a practical concept within Files, Formats & Data Ingestion. It helps turn the broader workflow stage “2 · Data Ingestion, Storage & Integration” into an explicit analytical decision that can be explained, implemented and checked. The concept should be understood in terms of purpose, mechanism, assumptions, evidence and downstream consequences.
03JSON and nested dataJSON and nested data is a practical concept within Files, Formats & Data Ingestion. It helps turn the broader workflow stage “2 · Data Ingestion, Storage & Integration” into an explicit analytical decision that can be explained, implemented and checked. The concept should be understood in terms of purpose, mechanism, assumptions, evidence and downstream consequences.
04Parquet and columnar formatsParquet and columnar formats is a practical concept within Files, Formats & Data Ingestion. It helps turn the broader workflow stage “2 · Data Ingestion, Storage & Integration” into an explicit analytical decision that can be explained, implemented and checked. The concept should be understood in terms of purpose, mechanism, assumptions, evidence and downstream consequences.
05Reading data safely with explicit dtypesReading data safely with explicit dtypes is a practical concept within Files, Formats & Data Ingestion. It helps turn the broader workflow stage “2 · Data Ingestion, Storage & Integration” into an explicit analytical decision that can be explained, implemented and checked. The concept should be understood in terms of purpose, mechanism, assumptions, evidence and downstream consequences.
06Chunked and large-file ingestionChunked and large-file ingestion is a practical concept within Files, Formats & Data Ingestion. It helps turn the broader workflow stage “2 · Data Ingestion, Storage & Integration” into an explicit analytical decision that can be explained, implemented and checked. The concept should be understood in terms of purpose, mechanism, assumptions, evidence and downstream consequences.