Handwritten Digit Recognizer using Python and Streamlit

Introduction

This project is the classic machine-learning rite of passage, upgraded: instead of scoring 98% on MNIST in a notebook and stopping, you'll deploy the model behind a drawing canvas. Draw a digit with your mouse, and a trained neural network guesses what you wrote — live. It is the drawing-board sibling of the image classifier, but with a model you train yourself in five minutes.

Two lessons in one: training a small CNN end-to-end, and the surprisingly tricky preprocessing that stands between a canvas drawing and a correct prediction.

Features

Draw pad — mouse/touch drawing via streamlit-drawable-canvas.

Trained CNN — 99%+ accuracy on MNIST, trained in ~2 minutes on CPU.

Live prediction — top-3 guesses with confidence bars.

Probability chart — all ten digits' scores at once.

Model persistence — train once, save to disk, load forever.

Prerequisites

Python 3.9+ — from python.org.

Dependencies:

code

pip install streamlit streamlit-drawable-canvas tensorflow-cpu numpy

Step 1: Train the Model (once)

Save as train_model.py and run it:

code

import tensorflow as tf

(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()
x_train = x_train.reshape(-1, 28, 28, 1) / 255.0
x_test = x_test.reshape(-1, 28, 28, 1) / 255.0

model = tf.keras.Sequential([
    tf.keras.layers.Conv2D(32, 3, activation="relu", input_shape=(28, 28, 1)),
    tf.keras.layers.MaxPooling2D(),
    tf.keras.layers.Conv2D(64, 3, activation="relu"),
    tf.keras.layers.MaxPooling2D(),
    tf.keras.layers.Flatten(),
    tf.keras.layers.Dropout(0.3),
    tf.keras.layers.Dense(10, activation="softmax"),
])

model.compile(optimizer="adam", loss="sparse_categorical_crossentropy", metrics=["accuracy"])
model.fit(x_train, y_train, epochs=5, validation_split=0.1)
print("Test accuracy:", model.evaluate(x_test, y_test)[1])
model.save("digit_model.keras")

Step 2: Create the App

Save as digit_app.py:

code

import streamlit as st
import numpy as np
from PIL import Image, ImageOps, ImageDraw
from streamlit_drawable_canvas import st_canvas
from tensorflow.keras.models import load_model

st.set_page_config(page_title="Digit Recognizer", page_icon="✍️")
st.title("✍️ Draw a digit (0–9)")

model = load_model("digit_model.keras")

SIZE = 280
canvas = st_canvas(
    fill_color="black",
    stroke_width=20,
    stroke_color="white",
    background_color="black",
    height=SIZE, width=SIZE,
    drawing_mode="freedraw",
    key="pad",
)

if st.button("🔮 Predict") and canvas.image_data is not None:
    img = Image.fromarray(canvas.image_data.astype(np.uint8))
    gray = ImageOps.grayscale(img).resize((28, 28))
    arr = np.array(gray, dtype=np.float32) / 255.0

    if arr.sum() < 1:
        st.warning("Draw something first!")
        st.stop()

    probs = model.predict(arr.reshape(1, 28, 28, 1), verbose=0)[0]
    top3 = probs.argsort()[-3:][::-1]

    st.subheader(f"Prediction: **{top3[0]}** ({probs[top3[0]]:.0%} confident)")
    for digit in top3:
        st.progress(float(probs[digit]), text=f"Digit {digit}: {probs[digit]:.1%}")

    st.bar_chart({str(d): float(probs[d]) for d in range(10)})

Step 3: Run the App

code

python train_model.py
streamlit run digit_app.py

Draw a big, centered 7 and predict. Then draw a tiny corner slash and watch the model get humbled — that contrast *is* the preprocessing lesson.

How It Works

Training is the textbook CNN recipe: two convolution + pooling blocks learn to detect strokes and shapes at increasing abstraction, a dropout layer fights overfitting, and a 10-unit softmax outputs one probability per digit. Five epochs on CPU gets ~99% test accuracy because MNIST is famously friendly.

The app's real lesson is preprocessing symmetry. MNIST digits are: white-on-black, 28×28, centered, thick strokes. Your drawing must match that contract exactly — grayscale, downscale, normalize to 0–1. Get one detail wrong (black-on-white is the classic) and a 99% model reads your confident 5 as an 8. The canvas is configured black-background, white-stroke deliberately, to match the training data without inversion gymnastics.

probs.argsort()[-3:][::-1] is the numpy top-3 idiom: argsort gives ascending indices, slice the last three, reverse for descending — the same ranking idea as the image classifier's decode_predictions, done by hand.

Common Errors & Fixes

Predictions are terrible despite 99% test accuracy — preprocessing mismatch. Verify: white strokes on black, values 0–1, shape (1, 28, 28, 1). Compare your array's mean against a real MNIST sample.

`OSError: digit_model.keras does not exist` — you skipped Step 1; the app loads a saved model, it doesn't train one.

Canvas is blank in the prediction — canvas.image_data holds the RGBA array; make sure you read it after drawing, and that stroke_color contrasts background_color.

Everything predicts as 1 or 0 — your strokes are too thin; stroke_width=20 at 280px scales down to MNIST-like thickness.

Key Concepts

CNN building blocks — conv filters, pooling, dropout, softmax.

Train/deploy split — train once offline, serve forever.

Preprocessing symmetry — inference inputs must mirror training inputs exactly.

Softmax probabilities — ranked confidence, not just an answer.

What to Try Next

Add auto-predict on release using the canvas's update_streamlit real-time mode.

Add a center-of-mass recenter step so off-center drawings work better.

Train on EMNIST letters and recognize A–Z with the same app (26 outputs).

Compare against the pretrained MobileNet classifier — custom-trained vs off-the-shelf.

FAQ

Why does my hand-drawn digit fail when test accuracy is 99%?

The model is 99% accurate on *MNIST-like* inputs. Real drawings differ in stroke width, position, and style — the distribution shift eats the margin. Better preprocessing recovers most of it.

Can I skip training and download a model?

Plenty of pretrained MNIST models exist, but training takes two minutes and teaches the whole pipeline. Do it once.

Is 5 epochs enough?

For MNIST with a CNN, yes — accuracy plateaus around 99%. More epochs mostly overfit; watch the validation split to see it happen.