Exploring distributions in Singapore HDB resale data
Question: What does the distribution of HDB resale prices reveal, and how does the apparent shape depend on the histogram bins?
Concepts: Statistical distributions, skewness, histograms, numerical summaries. Data: 2023 HDB teaching extract. Estimated activity: 20–30 minutes.
The data are a prepared teaching extract, not a census of all transactions. This learning notebook includes guided tasks; use the companion solution after making an attempt.
After you attempt the exercises: Open the complete solution notebook on GitHub.
from pathlib import Path
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
# Works in a repository checkout and remotely (including Colab).
name = "hdb_data_2023.csv.xz"
candidates = [
Path("../datasets/singapore_hdb_ura") / name,
Path("src/content/datasets/singapore_hdb_ura") / name,
Path("datasets/singapore_hdb_ura") / name,
]
source = next((p for p in candidates if p.is_file()), "https://raw.githubusercontent.com/wcchin/geo-stat-vis/8cf3ddb0a7eff7d94018940e91f32d7482f32aaa/src/content/datasets/singapore_hdb_ura/" + name)
df = pd.read_csv(source, index_col=0)
assert {"resale_price", "resale_age"}.issubset(df.columns)
print(f"Loaded {len(df):,} records from {source}")
df[["resale_price", "resale_age"]].describe()1. Establish a reference distribution¶
First inspect the supplied summary and histogram. Are the mean and median similar?
price = df["resale_price"].dropna()
print("Mean:", price.mean(), "Median:", price.median())
fig, ax = plt.subplots()
ax.hist(price, bins=30)
ax.set(xlabel="Resale price (SGD)", ylabel="Number of observations", title="HDB resale prices")
plt.show()2. Exercise — choice of bin width¶
TODO: Draw another histogram with 10 bins and compare it with the 30-bin reference. What changes in the visual appearance, and what does not change about the observations?
# TODO: make a 10-bin histogram of `price` and label the axes.
3. Exercise — skewness and transformation¶
TODO: Compute the skewness of price, then plot the distribution of the natural logarithm of strictly positive prices. Does the transformation change how you describe the shape?
# TODO: use price.skew() and np.log(price[price > 0]); make a labelled histogram.
Interpretation¶
Write 2–3 sentences about the distribution, bin sensitivity and whether transforming a variable alters the empirical observations or just their numerical representation.
Check your understanding¶
Try the exercises and write down your interpretation before comparing your work with the complete answers.