Data Collection, Sampling & Sources
Acquire data that represent the target population and deployment process, with known provenance, sampling logic and measurement quality. The topic is split into focused lessons so definitions, implementation decisions, diagnostics and common failure modes can be learned separately.
Learn the mechanism one decision at a time
Work through the lessons in order if the topic is new. If you already know the basics, open the specific leaf lesson that matches the operation, diagnostic or failure mode you need.
1Definition→
2Mechanism→
3Example→
4Diagnostic→
5Decision
Primary and secondary dataPrimary data are collected specifically for the project; secondary data are reused from existing systems, studies or public sources. The important practical question is not only how the technique is defined, but what assumptions it introduces, which data are allowed to influence it, and how its effect should be validated on unseen evidence.
02Sampling strategiesSampling determines which units from a population enter the dataset. The important practical question is not only how the technique is defined, but what assumptions it introduces, which data are allowed to influence it, and how its effect should be validated on unseen evidence.
03Measurement and instrumentationReliable measurements require calibrated instruments, stable definitions and timestamps. The important practical question is not only how the technique is defined, but what assumptions it introduces, which data are allowed to influence it, and how its effect should be validated on unseen evidence.
04APIs, files, databases and streamsData can arrive through static files, database queries, APIs, event streams or object storage. The important practical question is not only how the technique is defined, but what assumptions it introduces, which data are allowed to influence it, and how its effect should be validated on unseen evidence.
05Dataset versioning and lineageLineage records how a dataset was produced from source inputs and transformations. The important practical question is not only how the technique is defined, but what assumptions it introduces, which data are allowed to influence it, and how its effect should be validated on unseen evidence.