Representation Changes the Clusters

A clustering algorithm sees numbers, not the physical meaning of the objects behind them. That makes representation one of the most important choices in unsupervised learning.

Suppose the same machines can be described with:

• temperature and vibration;
• energy use and load;
• age and maintenance hours.

Each representation defines a different geometry. Two machines that are neighbors under one pair of features may be far apart under another.

Scaling changes distance

If we multiply one feature by 10, we stretch the space in that direction. A distance-based algorithm will respond even though the underlying observations have not changed.

Imagine points A and B differ by 0.2 in vibration and 5 degrees in temperature. If vibration is rescaled from 0–1 to 0–100, the vibration difference becomes 20. The relative influence of the two features has changed dramatically.

That may be appropriate or inappropriate depending on the question. The important part is to make the choice visible.

Outliers can move the story

A single extreme point can drag a mean-based centroid. Sometimes that outlier is the most interesting observation in the dataset. Sometimes it is a measurement error. Clustering cannot decide which interpretation is correct without additional evidence.

Useful clusters answer a stated question

Instead of asking “are these clusters true?”, ask:

• Are they stable when reasonable preprocessing choices change?
• Do they help us inspect the data?
• Do they support a defined downstream task?
• Can we describe the features and distance measure that produced them?
• Are we assigning human meaning that the algorithm never tested?

Before the lab

Predict which change will affect the clustering more in the provided simulator: adding one outlier or doubling the scale of the Y feature. Then test it instead of trusting the prediction.