Lab: Handwritten Digit Recognition (Multiclass)

machine-learning
neural-networks
A softmax neural network trained on 5000 real handwritten digit images to recognize all ten digits, 0 through 9, with a prediction grid and a look at its mistakes.
Published

July 14, 2026

The earlier Handwritten Digit Recognition lab built a network to tell just two digits apart, zero and one. This lab extends that to the full problem: recognizing all ten digits, 0 through 9, using the softmax output layer and cross-entropy loss from the multiclass pages. The Sequential / Dense / compile / fit mechanics are the same as before, so this lab focuses on the new part: a real ten-class problem, with a look at the predictions and the mistakes.

Dataset

The dataset has 5000 handwritten digit images. Each image is 20 by 20 pixels, unrolled into a row of 400 numbers, so X has shape (5000, 400). The labels y hold the true digit (0 through 9) for each image.

%config InlineBackend.figure_formats = ['png']
import numpy as np
import matplotlib.pyplot as plt
import tensorflow as tf
from tensorflow.keras import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras.losses import SparseCategoricalCrossentropy
import logging
logging.getLogger("tensorflow").setLevel(logging.ERROR)
tf.autograph.set_verbosity(0)
plt.style.use("../../deeplearning.mplstyle")

X = np.load("../../media/digits-all-X.npy")
y = np.load("../../media/digits-all-y.npy")
print("X shape:", X.shape, " y shape:", y.shape)
print("digit labels present:", np.unique(y))
X shape: (5000, 400)  y shape: (5000, 1)
digit labels present: [0 1 2 3 4 5 6 7 8 9]

Each row of X is one image. To view it, reshape those 400 numbers back into a 20 by 20 grid (and transpose it, since the pixels were unrolled column by column). Here are 64 random images with their true labels.

rng = np.random.default_rng(1)
fig, axes = plt.subplots(8, 8, figsize=(5, 5))
fig.tight_layout(pad=0.1, rect=[0, 0.03, 1, 0.93])
for ax in axes.flat:
    i = rng.integers(X.shape[0])
    ax.imshow(X[i].reshape(20, 20).T, cmap="gray")
    ax.set_title(int(y[i, 0]), fontsize=8)
    ax.set_axis_off()
fig.suptitle("Label above each image", fontsize=13)
plt.show()

Model

The network has three layers: two hidden layers of 25 and 15 ReLU units, and an output layer of 10 units, one per digit. Following the numerically stable pattern, the output layer uses a linear activation and the softmax is folded into the loss with from_logits=True. The optimizer is Adam.

tf.random.set_seed(1234)
model = Sequential([
    tf.keras.Input(shape=(400,)),
    Dense(25, activation="relu",   name="L1"),
    Dense(15, activation="relu",   name="L2"),
    Dense(10, activation="linear", name="L3"),   # 10 outputs, logits
], name="digit_model")

model.compile(
    loss=SparseCategoricalCrossentropy(from_logits=True),
    optimizer=tf.keras.optimizers.Adam(learning_rate=0.001),
)
model.summary()
Model: "digit_model"

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ L1 (Dense)                      │ (None, 25)             │        10,025 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ L2 (Dense)                      │ (None, 15)             │           390 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ L3 (Dense)                      │ (None, 10)             │           160 │
└─────────────────────────────────┴────────────────────────┴───────────────┘

 Total params: 10,575 (41.31 KB)

 Trainable params: 10,575 (41.31 KB)

 Non-trainable params: 0 (0.00 B)

The summary() shows the layer sizes and the parameter counts. The first layer has \(400 \times 25 + 25 = 10025\) parameters (a weight per input-unit pair, plus one bias per unit), and the pattern continues through the network. Now train it. Each epoch is one pass over all 5000 images.

history = model.fit(X, y, epochs=40, verbose=0)

fig, ax = plt.subplots(figsize=(5, 3))
ax.plot(history.history["loss"], color="#0096ff")
ax.set_xlabel("epoch"); ax.set_ylabel("loss")
ax.set_title("Training loss", color="gray")
fig.tight_layout()
plt.show()

The loss falls steadily as the network learns to map images to digits.

Making a Prediction

To classify one image, call model.predict on it. Remember the output layer is linear, so the model returns 10 logits, not probabilities. Take image number 1015, which is a handwritten two.

logits = model.predict(X[1015].reshape(1, 400), verbose=0)
print("raw logits:", np.round(logits, 1))
print("largest at index:", np.argmax(logits))
raw logits: [[-10.2   3.5   5.3   0.9  -9.9  -3.7  -4.3   1.4  -3.3  -4.2]]
largest at index: 2

The largest logit is at index 2, so the network predicts a 2. To turn the logits into probabilities, pass them through a softmax; to get just the predicted digit, np.argmax is enough.

probs = tf.nn.softmax(logits).numpy()
print("probabilities:", np.round(probs, 3))
print("predicted digit:", np.argmax(probs))
probabilities: [[0.    0.135 0.836 0.01  0.    0.    0.    0.018 0.    0.   ]]
predicted digit: 2

Predictions on a Sample

Below are 64 random images again, now labeled with both the true digit and the model’s prediction, as true, predicted. Almost all match.

def predict_digit(img):
    logits = model.predict(img.reshape(1, 400), verbose=0)
    return int(np.argmax(logits))

def show_grid(idxs, title):
    fig, axes = plt.subplots(8, 8, figsize=(5, 5))
    fig.tight_layout(pad=0.1, rect=[0, 0.03, 1, 0.93])
    for ax, i in zip(axes.flat, idxs):
        yhat = predict_digit(X[i])
        ax.imshow(X[i].reshape(20, 20).T, cmap="gray")
        ax.set_title(f"{int(y[i,0])},{yhat}", fontsize=8)
        ax.set_axis_off()
    fig.suptitle(title, fontsize=13)
    plt.show()
sample = rng.integers(X.shape[0], size=64)
show_grid(sample, "true, predicted")

Looking at the Errors

Running the model on all 5000 images and comparing predictions to the true labels shows how many it gets wrong. On this dataset a well-trained network makes only a handful of mistakes.

all_logits = model.predict(X, verbose=0)
yhat = np.argmax(all_logits, axis=1)
errors = np.where(yhat != y[:, 0])[0]
print(f"{len(errors)} errors out of {len(X)} images "
      f"(accuracy {np.mean(yhat == y[:, 0]):.3f})")
38 errors out of 5000 images (accuracy 0.992)

The misclassified digits are usually the genuinely ambiguous ones, sloppy or unusual handwriting that even a person might hesitate over. Here are a few, labeled true, predicted.

n = min(8, len(errors))
fig, axes = plt.subplots(1, n, figsize=(6, 1.4))
for ax, j in zip(np.atleast_1d(axes), errors[:n]):
    ax.imshow(X[j].reshape(20, 20).T, cmap="gray")
    ax.set_title(f"{int(y[j,0])},{yhat[j]}", fontsize=9)
    ax.set_axis_off()
fig.suptitle("Mistakes (true, predicted)", fontsize=12)
fig.tight_layout(rect=[0, 0, 1, 0.85])
plt.show()

Training for more epochs can drive these last errors down further. With that, you have trained a real ten-class image classifier using a softmax output layer.

Review Questions

1. The output layer has 10 units with a linear activation, not softmax. Why, and how do you turn its outputs into probabilities?

A linear output plus SparseCategoricalCrossentropy(from_logits=True) is the numerically stable form: the network emits raw logits and the softmax is folded into the loss. To get probabilities from a prediction, apply a softmax afterward with tf.nn.softmax. For just the predicted digit, np.argmax on the logits is enough.

1. The first layer reports 10025 parameters. Where does that number come from?

The input has 400 values and the layer has 25 units, giving \(400 \times 25 = 10000\) weights, plus one bias per unit, \(25\) more, for \(10025\) total.

1. Why does the model still make a few errors, and what kind of images are they usually?

No model is perfect, and the misclassified images are typically the genuinely ambiguous ones, messy or unusual handwriting. Training for more epochs can reduce these remaining errors further.