Multimodal Retrieval System for Game Statistics and Match Highlights

Sports analysis lives in two kinds of data at once: numbers, such as box scores, player statistics, and play-by-play logs, and images, such as frames from match footage, formation diagrams, and highlight stills. Analysts, journalists, and passionate fans constantly ask questions that span both, like what the statistics say about a team's second-half collapse and what the visual record shows about their defensive shape. Conventional tools handle one modality and ignore the other. The intern builds a sports analytics platform that answers such questions across both. Written in Python, the system uses LangChain to construct a multimodal retrieval-augmented generation pipeline: textual game data is indexed for retrieval, images are interpreted through OpenAI's GPT-4o vision capability, and a combined generation step composes answers that cite statistics while describing what the associated visuals show. The intern implements the ingestion of both data types, the retrieval logic that decides which modality a query needs, and the response formatting that keeps statistical claims traceable to source data. The project demonstrates one of the most current skills in applied AI, multimodal RAG: designing retrieval across mixed data types, integrating vision-language models into a pipeline, and delivering grounded analytical answers in a domain users genuinely care about.