Project 13: Data Cleaning Toolkit

Introduction

Data scientists joke that 80% of the job is cleaning data. The joke is real, and this app is the response: an interactive toolkit where you upload a messy CSV and clean it with one-click actions — drop duplicates, fill or drop nulls, fix types, strip whitespace, standardize case — watching a before/after diff of your dataset the whole time.

It is the hands-on counterpart to the read-only CSV explorer: that app answers questions, this one fixes problems. Every operation is a classic Pandas one-liner; the app's value is wiring them to buttons with instant feedback.

Features

Issue scan — instant report of duplicates, nulls, mixed types, whitespace.

One-click cleaning — each fix is a button with an undo.

Null strategies — drop rows, fill with mean/median/mode, or a custom value.

Type coercion — convert columns to numeric/datetime with error handling.

Text standardization — trim, lowercase, collapse spaces across columns.

Clean CSV download — export the fixed dataset.

Prerequisites

Python 3.8+ — from python.org.

Dependencies:

code

pip install streamlit pandas

Step 1: Create the Script

Save as cleaning_toolkit.py:

code

import streamlit as st
import pandas as pd
import numpy as np

st.set_page_config(page_title="Data Cleaning Toolkit", page_icon="🧹", layout="wide")
st.title("🧹 Data Cleaning Toolkit")

uploaded = st.file_uploader("Upload a messy CSV", type=["csv"])

if uploaded:
    if "df" not in st.session_state:
        st.session_state.df = pd.read_csv(uploaded)
    df = st.session_state.df

    # --- Issue scan ---
    dupes = df.duplicated().sum()
    nulls = df.isna().sum()
    ws_cols = [c for c in df.select_dtypes("object") if (df[c].astype(str) != df[c].astype(str).str.strip()).any()]

    c1, c2, c3 = st.columns(3)
    c1.metric("Rows", len(df))
    c2.metric("Duplicate rows", int(dupes))
    c3.metric("Columns with nulls", int((nulls > 0).sum()))
    if ws_cols:
        st.caption(f"Whitespace issues in: {', '.join(ws_cols)}")

    st.divider()
    st.subheader("Cleaning actions")
    a1, a2, a3 = st.columns(3)

    if a1.button("🗑 Drop duplicate rows"):
        st.session_state.df = df.drop_duplicates().reset_index(drop=True)
        st.rerun()

    if a2.button("✂️ Trim whitespace"):
        for c in df.select_dtypes("object"):
            st.session_state.df[c] = df[c].astype(str).str.strip().str.replace(r"\s+", " ", regex=True)
        st.rerun()

    with a3.popover("🧯 Handle nulls"):
        target = st.selectbox("Column", nulls[nulls > 0].index)
        strategy = st.radio("Strategy", ["Drop rows", "Fill mean", "Fill median", "Fill mode", "Custom"])
        custom = st.text_input("Custom value") if strategy == "Custom" else None
        if st.button("Apply"):
            if strategy == "Drop rows":
                st.session_state.df = df.dropna(subset=[target])
            elif strategy == "Fill mean":
                st.session_state.df[target] = df[target].fillna(df[target].mean())
            elif strategy == "Fill median":
                st.session_state.df[target] = df[target].fillna(df[target].median())
            elif strategy == "Fill mode":
                st.session_state.df[target] = df[target].fillna(df[target].mode()[0])
            elif custom is not None:
                st.session_state.df[target] = df[target].fillna(custom)
            st.rerun()

    st.divider()
    st.subheader(f"Result — {len(st.session_state.df):,} rows × {len(st.session_state.df.columns)} cols")
    st.dataframe(st.session_state.df.head(50), use_container_width=True)

    st.download_button(
        "⬇️ Download cleaned CSV",
        st.session_state.df.to_csv(index=False).encode(),
        "cleaned.csv", "text/csv",
    )

Step 2: Run the App

code

streamlit run cleaning_toolkit.py

Make a messy file to practice on: duplicate some rows in Excel, blank a few cells, add trailing spaces — then clean it.

How It Works

Every cleaning action follows the same contract: mutate session state, then rerun. st.session_state.df is the working copy — the uploaded file stays pristine in memory, and each button applies one Pandas transformation to the working copy before st.rerun() repaints the metrics and table. The issue scan recomputes on every rerun, so the duplicate and null counts fall as you clean — visible progress is the whole UX.

The nulls popover demonstrates strategy selection: the same fillna call with different values (mean for symmetric numerics, median for skewed ones, mode for categories). Dropping rows is the honest option when nulls are few; filling is better when they're many.

Whitespace cleanup chains three string operations — strip plus a regex \s+→space replace — because real-world mess is usually *both* leading/trailing spaces and doubled internal ones.

Common Errors & Fixes

`SettingWithCopyWarning` — you assigned into a filtered view; always assign back to the full frame (st.session_state.df[col] = ...), as the code does.

Mean-fill crashes on text columns — mean() needs numeric dtype; coerce with pd.to_numeric(col, errors="coerce") first, which turns junk into NaN you can then fill.

Mode-fill fails on all-null columns — mode() of an empty series has no [0]; guard with if not df[target].dropna().empty.

Undo doesn't exist — correct, it doesn't; add a snapshot stack (st.session_state.history.append(df.copy())) before each mutation if you need it.

Key Concepts

Working-copy pattern — session state holds the mutable dataset.

Idempotent actions — every button is safe to press twice.

Null strategy menu — drop vs fill is a data-story decision, not a technical one.

Regex cleanup — \s+ → single space fixes most text mess.

What to Try Next

Add type coercion buttons — pd.to_numeric(errors="coerce") and pd.to_datetime per column.

Add outlier flagging — rows beyond 3 standard deviations, styled red like the Excel report.

Add an undo stack with the snapshot list described above.

Chain it to the sales analyzer — clean, then analyze the same file.

FAQ

Should I drop or fill nulls?

If nulls are under ~5% of rows and the rows aren't special, dropping is simplest. Fill when nulls are widespread or rows are precious — and prefer median over mean whenever the column might be skewed.

Why does my numeric column have object dtype?

One stray string ("N/A", "1,200") poisons the whole column. pd.to_numeric(errors="coerce") converts the valid values and quarantines the junk as NaN — which this app then handles.

Is cleaning in the UI reproducible?

Not by default — that's the trade for interactivity. Export the cleaned CSV as your record, or log each action to a list and offer a 'replay script' download.