External Data Can Fail

Core path: 25 minutes

What should a program assume about a system it does not control?

Very little.

An API call crosses a trust boundary. Your code depends on the network, routing, the remote service, the HTTP response, and the shape of the returned data.

Diagrams open at a readable shape-aware scale. Zoom or expand when you need more detail.

Three failure layers

1. Network failure: the request cannot complete, or it times out.
2. HTTP failure: the server responds, but the response status represents failure such as 404 or 500.
3. Data failure: the request succeeds, but the payload is missing or changes a value your program expected.

Diagrams open at a readable shape-aware scale. Zoom or expand when you need more detail.

Put a time limit on the request

import requests

try:
    response = requests.get(url, timeout=5)
    response.raise_for_status()
except requests.exceptions.Timeout:
    print("Request timed out.")
except requests.exceptions.RequestException as error:
    print(f"Request failed: {error}")

A timeout is not a prediction that the server will fail. It is a boundary that prevents your program from waiting forever.

Successful HTTP does not guarantee useful data

After a successful request:

data = response.json()
website = data.get("website", "No website provided")

.get() can provide a fallback when an optional dictionary key is missing. For required data, silently replacing a missing key may hide a real failure, so decide the contract before choosing the fallback.

Guided example: trace the failure layer

Suppose the dashboard reports:

Request failed: 404 Client Error

Use the layer model:

Network connection succeeded
        ↓
Server returned an HTTP response
        ↓
HTTP status says requested resource was not found

Do not debug JSON parsing first. The program never reached valid payload analysis. If instead the response were 200 but a required key were missing, the investigation would move to the data layer.

Offline evidence is still evidence

The Week 13 Lab includes a local JSON fixture. That does not simulate the network. It isolates the data-processing part of the program so a blocked or unavailable service does not stop you from proving your parsing and analysis logic.