Week 13 · lesson
Features, Labels, and the Question a Model Learns
A row in a dataset is not knowledge. It is one recorded case. Learning begins only after we decide what information the model may use and what answer, if any, it is expected to predict.
Suppose a fictional robot records distance to an obstacle:
| Example | Distance | Human label |
|---|---|---|
| A | 18 cm | stop |
| B | 44 cm | continue |
| C | 27 cm | stop |
| D | 39 cm | continue |
The feature is information available when a prediction is made. Here, distance is a feature. The label is the target answer attached to a supervised example. Here, stop or continue is the label.
That distinction matters because the model is supposed to infer the label from the features. If the true answer leaks into the feature set, the experiment becomes meaningless. A spreadsheet can contain a column named correct_action, but a deployed model is not allowed to secretly read the answer it is supposed to predict.
Classification and regression answer different questions
A classification model predicts one category from a defined set: stop/continue, spam/not-spam, defective/not-defective.
A regression model predicts a number: stopping distance, energy use, temperature, travel time.
The same dataset can support different tasks depending on the question. A robot-distance table might be used to classify an action or to estimate a numeric stopping distance. The important part is to state the prediction target before evaluating the model.
Consider the row distance = 27 cm, label = stop.
- Feature available to the model:
27 cm. - Training label:
stop. - Prediction target: one of the action categories.
- Missing information: speed, surface, sensor error, robot mass, and many other conditions that could matter in a real system.
That last line is not a technicality. The model can only learn relationships represented in the data it receives.
A useful feature is not automatically a justified feature
Imagine the table also contains robot_id. The identifier may correlate with the labels because one robot happened to collect more stop examples. That does not mean the identifier describes the mechanism we want the model to learn.
Before using a feature, ask:
- Will this value exist when the model is deployed?
- Could it leak the answer?
- Does it encode a shortcut that will disappear later?
- Does using it create a privacy or fairness problem?
A model will happily exploit a shortcut. It does not know which patterns are meaningful unless the experiment is designed to test that question.
Before moving on
Take one ordinary prediction problem—weather, music recommendations, machine maintenance, game matchmaking, anything you understand. Name one possible feature, one possible target, and one field that would be suspicious because it leaks or shortcuts the answer.
Experiment pipeline
From labeled examples to a bounded model claim
Follow one fixed result through expected and failure traces to a bounded system decision.
Input preparation
Labeled Data Split
Assign the twelve supplied rows to eight training jobs and four held-out test jobs.
Model configuration
Threshold Rule
Use distance as the feature, stop or continue as the label, and the smaller threshold as the tie break.
Evaluation
Training Search
Score every candidate boundary on the eight rows allowed to influence the fitted parameter.
Held-Out Test
Apply the fitted threshold to four labeled rows that did not participate in parameter selection.
Evidence Gap
Compare the perfect training fit with two held-out errors instead of reporting training success alone.
Decision
Bounded Model Decision
Retain the threshold for this classroom experiment while rejecting safety and deployment claims.
Read this concept flow as plain text
- Labeled Data Split. Assign the twelve supplied rows to eight training jobs and four held-out test jobs. BASELINE SPLIT: 8 TRAIN / 4 TEST.
- Threshold Rule. Use distance as the feature, stop or continue as the label, and the smaller threshold as the tie break. PREDICTION RULE: ≤ THRESHOLD → STOP.
- Training Search. Score every candidate boundary on the eight rows allowed to influence the fitted parameter. SPLIT A TRAINING: 32 CM · 0 ERRORS.
- Held-Out Test. Apply the fitted threshold to four labeled rows that did not participate in parameter selection. SPLIT A TEST: 2 / 4 CORRECT.
- Evidence Gap. Compare the perfect training fit with two held-out errors instead of reporting training success alone. GENERALIZATION GAP: 0 VS 2 ERRORS.
- Bounded Model Decision. Retain the threshold for this classroom experiment while rejecting safety and deployment claims. DECISION: RETAIN FOR COMPARISON.