Optimizer Comparison Lab for SGD, Adam, and RMSProp

Added Jun 2025 3 design docs

SGD, Adam, and RMSProp appear in every deep learning tutorial as a dropdown choice, but the differences between them are dynamic — visible only in how training actually unfolds over time. This project builds a playground for racing the optimizers against each other on equal terms. Working in Jupyter with Keras and TensorFlow, the intern trains identical copies of a simple neural network on a toy dataset, one per optimizer, and renders live loss and accuracy plots that update as training proceeds. The overlaid curves reveal each algorithm's temperament: plain SGD's steady but slow descent, RMSProp's adaptive pace, and Adam's typically rapid early progress. The intern extends the comparison by varying learning rates per optimizer, discovering that rankings between optimizers can flip when their settings change — a nuance most tutorials never mention. The project replaces cargo-cult optimizer selection with observed understanding. The intern learns what adaptive gradient methods actually adapt, how to design fair model comparisons, and how to present training dynamics visually — foundations for principled experimentation in any deep learning role.

Related projects

Interactive Loss Landscape Visualizer for Neural Network Training

Students meeting gradient descent for the first time are usually shown a single loss curve and asked to trust that optimization is happening somewhere underneath. Actually seeing the surface a network descends — its valleys, plateaus, and traps — makes concepts like local minima and learning rate behavior click in a way equations rarely do. The intern builds an interactive visualization tool in Python that constructs the loss landscape of a small neural network and renders it with Matplotlib as contour and surface plots. Working in Jupyter, users adjust weights by hand and watch their position on the surface move, then launch gradient descent with different learning rates to see convergence, overshooting, and getting stuck in local minima play out visually. The tool contrasts the regions around local and global minima so learners develop intuition for why initialization and step size matter so much in deep learning. The project deepens the intern's own understanding of optimization while exercising scientific Python skills — numerical computation, Matplotlib visualization, and notebook-driven experiment design — and produces a teaching asset that is genuinely useful to other learners studying how neural networks train.

educationresearchdata-sciencedeep-learning+3

View project

Easy

Beginner-Friendly Voting and Stacking Classifier Builder

Ensemble learning is usually taught as a formula, but the underlying idea is simple: several weak opinions, combined well, beat one confident guess. This project turns that idea into an application a beginner can experiment with directly, making one of machine learning's most powerful techniques concrete. The intern builds a Streamlit application where users train several simple scikit-learn classifiers on a chosen dataset and combine their predictions using hard and soft voting, stacking, and blending strategies. The interface shows each base model's individual accuracy next to the combined result, so the effect of ensembling is visible immediately rather than taken on faith. Users can swap base models in and out and watch how diversity among the learners changes the ensemble's performance, with Python handling training and evaluation behind a clean interactive front end. The project cements the intern's understanding of why and when ensembles work, gives them working knowledge of scikit-learn's ensemble APIs, and provides practice in building approachable educational tooling — demonstrating they can both apply a core machine learning technique and explain it to someone newer than themselves.

educationdata-sciencemachine-learning+3

View project

Easy

Searchable Case Study Library for Banking Education

Case studies are one of the best ways to learn banking, since real events such as failed risk controls, successful product launches, and landmark regulatory actions teach lessons that textbooks flatten. The difficulty is discovery: relevant cases are scattered across sources, and a student rarely knows which one matches the topic at hand. The intern builds a streamlit application that makes a case study collection searchable by meaning rather than keyword. Case documents are embedded and stored in a vector database, and when a user asks about a topic such as credit risk in small business lending, a retrieval-augmented generation pipeline pulls the most relevant cases and generative AI produces a concise summary of each one, highlighting the key lessons and outcomes. Everything is written in python, and the interface lets users browse results, read summaries, and drill into the underlying case text. The project is an approachable introduction to RAG: the intern learns how embeddings capture topical similarity, how retrieval grounds generated summaries in real sources, and how to package the whole system as a clean educational tool for banking learners.

bankingresearchgenerative-aipython+4

View project