Multilingual Stopword Removal Utility for Text Pipelines

Added Jun 2025 3 design docs

High-frequency function words like the, and, and of dominate raw text while carrying little standalone meaning, and removing them is a standard early step in search indexing, topic modeling, and text classification. The task sounds trivial until real requirements arrive: multiple languages, different stopword inventories between libraries, and cases where removing a word like not quietly destroys meaning. The intern builds a Python command-line utility that removes stopwords from input text using both NLTK and spaCy, with support for multiple languages drawn from each library's stopword resources. The tool accepts text or files, lets users choose the library and language, and outputs the filtered text alongside statistics such as how many tokens were removed. The intern implements a comparison mode showing how NLTK's and spaCy's stopword lists differ for the same language, and supports custom additions and exclusions so users can protect words that matter for their task. The project grounds the intern in practical tokenization and text filtering across languages, teaches the judgment call hiding inside every default word list, and demonstrates the ability to build a small, well-behaved preprocessing tool that other pipelines can depend on, complete with clear command-line ergonomics.

Related projects

Batch Grading Analytics Pipeline for Student Assignments

A university processing tens of thousands of assignment submissions each term needs more than spreadsheets to compute grades, detect anomalies, and report cohort statistics. This project recreates that challenge as a distributed data workflow that an intern can build and run end to end. Submission events are simulated with Kafka so assignments arrive as a realistic stream throughout the exercise, and raw submission data is stored durably in HDFS. Scheduled PySpark jobs then process the accumulated submissions in batches, computing grading summaries, score distributions, completion rates, and per-course statistics. Python scripts manage the surrounding workflow, handling ingestion, triggering batch runs, and exporting results as structured data files and reports for instructors and administrators. There is no web or mobile front end; the deliverables are the pipeline itself and the reports it produces, which keeps attention on data correctness and throughput. The intern comes away understanding the streaming-to-batch pattern used across industry big-data platforms: durable ingestion, distributed computation, and reproducible reporting, exercised on a domain where the meaning of every metric is easy to reason about.

educationtechnologybig-datahdfs+3

View project

Medium

Retrieval-Backed Course Material Generator for Banking Education

Trainers and educators in financial services spend enormous effort producing course notes, quizzes, and explanations for every topic they teach, and the material dates quickly as products and regulations change. A generator that assembles fresh, accurate learning content from a maintained resource library changes the economics of banking education. The intern builds this generator in python with a streamlit interface. Educational resources, including articles, definitions, and worked examples, are embedded and stored in a vector database. When a user enters a banking topic or uploads their own learning materials, a retrieval-augmented generation pipeline gathers the most relevant source content and generative AI composes tailored outputs: explanatory lessons, topic summaries, and quiz questions with answers. Because generation is grounded through RAG in the stored resources, the produced material reflects the curated corpus rather than the model's unverified memory, and users can regenerate or refine content interactively. The project teaches content pipeline design, grounded generation for educational quality, and quiz construction logic, giving the intern a strong demonstration of AI applied to learning-content operations in a regulated domain.

bankingeducationgenerative-aipython+4

View project

Medium

Personalized Learning Pathway Planner for Banking Students

Two people studying banking rarely need the same course: a new teller, a compliance trainee, and a fintech founder each need different topics in a different order. Static curricula cannot adapt to that, but a generator that assembles a personalized pathway from stated goals can. The intern builds the planner in python. Users select banking topics or describe their learning goals, and the system generates a structured pathway: an ordered sequence of topics, an explanation of each step, and quizzes to check understanding along the way. Langgraph manages the multi-stage flow from goal interpretation to pathway assembly to quiz creation, while an agentic-ai layer coordinates the stages and revisits weak sections, and generative AI through openai models writes the explanations and question sets. The output is a complete, structured learning plan with accompanying assessments, delivered as organized documents a learner or trainer can follow directly. The project teaches curriculum decomposition, AI-driven content generation with consistent structure, and workflow automation. The intern comes away with a substantial artifact demonstrating that they can direct language models to produce organized educational programs rather than isolated answers.

bankingeducationagentic-aigenerative-ai+3

View project