Follow the transformation
Choose features and scaling that define meaningful similarity.
Fit the clustering algorithm without target labels.
Inspect cluster size, separation/stability and representative observations.
Unsupervised Learning · Lesson 49
K Means
K Means is an unsupervised clustering technique that groups observations using a specific notion of similarity or density.
What K Means actually means
K Means is an unsupervised clustering technique that groups observations using a specific notion of similarity or density. Clusters are model-dependent structures, not automatically “true” categories in the world.
K Means matters because unsupervised methods impose a particular notion of structure—variance, distance, density or probability—without target labels. The discovered representation or clusters must therefore be interpreted relative to that chosen notion.
Deeper walkthrough
Read K Means as a mechanism, not a recipe
Treat this as a sequence of observable decisions rather than one opaque command. Stage 1: Choose features and scaling that define meaningful similarity. Stage 2: Fit the clustering algorithm without target labels. Stage 3: Inspect cluster size, separation/stability and representative observations. Final checkpoint: Evaluate usefulness against the downstream analytical purpose.
Know what would convince you
- Standardise/represent features deliberately and document the distance or latent objective.
- Repeat with nearby hyperparameters/seeds or resamples and examine whether major structure is stable.
Useful distinctionK-means: Partitions into k clusters by minimising within-cluster squared distance to centroids; favours roughly spherical equal-scale clusters.
Mathematical / formal view
K-means minimises the sum of squared distances from each point to the centroid of its assigned cluster.
Trace the mechanism step by step
- Choose features and scaling that define meaningful similarity.
- Fit the clustering algorithm without target labels.
- Inspect cluster size, separation/stability and representative observations.
- Visualise in original or reduced feature space without overclaiming separation.
- Evaluate usefulness against the downstream analytical purpose.
Make the concept concrete
Demonstration
Python / scikit-learn example
# Step 1 — Import the module so its functions/classes are available to the rest of this example.
import numpy as np
# Step 2 — Import only the named objects needed by the following steps, keeping dependencies explicit.
from sklearn.cluster import KMeans
# Step 3 — Construct `X` as an array so vectorised numerical operations can be applied consistently.
X = np.array([[1,1],[1.2,0.9],[5,5],[5.2,4.8]])
# Step 4 — Fit the model or transformer, learning its parameters from the supplied training data.
m = KMeans(n_clusters=2, n_init=10, random_state=0).fit(X)
# Step 5 — Display the current value explicitly so the result/state can be inspected during execution.
print(m.labels_)
# Step 6 — Display the current value explicitly so the result/state can be inspected during execution.
print(np.round(m.cluster_centers_, 2))Expected / illustrative result
Two label groups corresponding to the two compact clouds; centroids near [1.1, 0.95] and [5.1, 4.9].
Interpret the result.
For K Means, connect the displayed result to the specific input and mechanism above; independently verify one value/state change rather than treating successful execution as proof.
Know what this is — and what it is not
K-meansPartitions into k clusters by minimising within-cluster squared distance to centroids; favours roughly spherical equal-scale clusters.DBSCANDensity-based; finds arbitrary shapes and labels low-density points as noise; requires eps/min_samples.HierarchicalBuilds a nested dendrogram of merges/splits using a distance/linkage rule.Gaussian mixtureProbabilistic soft clustering with Gaussian component distributions.When it is appropriate
Use K Means when the goal is to discover or represent structure without a supervised target and the chosen similarity/density/latent assumptions match the data.
When to stop or reconsider
Reconsider the method when feature scaling, distance choice, cluster shape, density variation or embedding purpose makes the discovered structure unstable or uninterpretable.
Failure modes to recognise
- Treating discovered clusters/components/embeddings as objectively true categories.
- Ignoring feature scale and distance/metric choices that dominate the geometry.
- Selecting a visually pleasing result without checking stability or external/domain plausibility.
How to check the result
- Standardise/represent features deliberately and document the distance or latent objective.
- Repeat with nearby hyperparameters/seeds or resamples and examine whether major structure is stable.
- Check cluster/component/embedding summaries against original variables and domain expectations rather than relying on the plot alone.
Demonstrate understanding
Try this:
Build a tiny, inspectable example of K Means. First choose features and scaling that define meaningful similarity. Then fit the clustering algorithm without target labels. Write the expected result before running it, and explain one condition that would make the result misleading or invalid.
Use a small synthetic dataset where you know the rough structure. Change one scale or hyperparameter and observe which parts of the result are stable.
Check reasoning, not memorisation
Before trusting a result from K Means, which check provides the strongest evidence that you understand and applied it correctly?
Keep the important distinctions visible
Step 1Choose features and scaling that define meaningful similarity.Step 2Fit the clustering algorithm without target labels.Step 3Inspect cluster size, separation/stability and representative observations.Step 4Visualise in original or reduced feature space without overclaiming separation.What to remember
- K Means is an unsupervised clustering technique that groups observations using a specific notion of similarity or density. Clusters are model-dependent structures, not automatically “true” categories in the world.
- Choose features and scaling that define meaningful similarity.
- Treating discovered clusters/components/embeddings as objectively true categories.
- Standardise/represent features deliberately and document the distance or latent objective.