CruxBit
Back to catalog

Sandbox for Building and Benchmarking Agentic AI Workflows

Added Jun 2025 3 design docs

Agentic AI systems are notoriously hard to evaluate: an agent that shines on one task fails quietly on another, and teams need somewhere to build, test, and compare agents rigorously before trusting them with real work. This project builds that environment.

The intern creates an experimentation platform where agents built with CrewAI and Langchain, powered by OpenAI models, can be constructed, configured, and benchmarked side by side. The system supports defining test scenarios, running agents against them, and capturing results for comparison, turning subjective impressions of agent quality into measurable outcomes. Agent customization tools let users adjust prompts, tools, and collaboration patterns, while user profile management keeps each experimenter's configurations and history organized. A FastAPI backend orchestrates runs and a Streamlit frontend, all written in Python, makes experimentation interactive and repeatable.

The project builds a skill the AI industry urgently needs, evaluation engineering: designing harnesses that reveal how agent systems actually behave. The intern finishes with both a working testing platform and demonstrated judgment about what makes agents reliable.

Related projects

You might also like

TechnologyMedium

Prompt Template Design and Testing Workbench for Educators

Teachers and instructional designers increasingly build AI-driven learning modules, such as quiz generators, explanation assistants, and feedback tools, and every one of them lives or dies by its prompts. Yet educators typically iterate on prompts by trial and error in a chat window, with no way to save what worked, compare variations, or enforce consistent output formats. Prompt engineering needs a workbench, the way code has an IDE. The intern builds that workbench as a Streamlit application in Python, using LangChain as the engine. The tool lets users create parameterized prompt templates with variables such as subject, difficulty, and student level; run them against OpenAI models; and compare outputs across template variations side by side. Output parsers enforce structured results, so a quiz template reliably yields well-formed questions and answers rather than loose prose, and chain sequencing lets users compose multi-step flows in which one prompt's output feeds the next, mirroring how real learning modules are assembled. The intern implements template storage, the comparison runner, and the parser configuration interface. The project demonstrates systematic prompt engineering as a discipline: template design, structured output enforcement, chain composition, and building a tool that turns an ad hoc craft into a repeatable process for a non-technical audience.

educationgenerative-ailangchainopenai+2
PRD · HLD · LLD includedView
TechnologyEasy

Lightweight Text Generation API Built With LangServe

The gap between a working LangChain script and a service other software can call is deployment, and LangServe exists to close it: it turns chains into REST APIs with typed endpoints, streaming, and an interactive playground almost for free. Knowing this path from notebook to network service is essential for any developer who wants their AI work used by anyone other than themselves, and this project walks it end to end at a friendly scale. The intern builds a FastAPI project in Python that exposes a text generation chain through LangServe. The intern first constructs the chain itself with LangChain, a prompt template feeding a language model with configurable parameters such as topic and style, then mounts it as a LangServe route, gaining automatically generated endpoints for invocation, batching, and streaming. Around this core the intern adds a health check, request logging, and clear API documentation, then exercises the service from the built-in playground and from an external client script to prove the full round trip works. The project teaches deployment fundamentals for LLM applications: how chains become HTTP services, how FastAPI and LangServe divide responsibilities, what streaming responses look like on the wire, and how to structure a small AI service so that adding the next endpoint is easy, foundations the intern will reuse in every production AI role.

educationfastapigenerative-ailangchain+1
PRD · HLD · LLD includedView