AWS IAM and S3 Misconfiguration Auditor for Financial Workloads

Added Jun 2025 3 design docs

Most cloud breaches in finance trace back not to exotic exploits but to misconfiguration: an IAM policy granted too broadly, an S3 bucket left quietly public. This project builds an analyzer that finds those mistakes systematically, before auditors or attackers do. The intern writes Python tooling against real AWS APIs to scan IAM policies for over-permissive grants, privilege escalation paths, and unused access, and to inspect S3 storage permissions for public exposure and weak encryption settings. Each finding comes with actionable remediation steps, turning the scan into a to-do list rather than an alarm. Docker packages the analyzer and its security lab environments for reproducible runs, while Prometheus and Grafana track scan results over time so teams can watch their security posture trend. The design follows recognized cloud infrastructure security and identity and access management best practices throughout. The intern comes away understanding cloud security the way practitioners do, as configuration reviewed continuously rather than a one-time setup, and can demonstrate hands-on IAM analysis, storage auditing, and security tooling development, capabilities in high demand across fintech and cloud operations teams.

Related projects

Farm Sensor Data Ingestion Pipeline with Kafka and HDFS

Modern farms generate continuous streams of readings for soil moisture, temperature, humidity, and equipment status, but the raw numbers are useless until they are reliably collected, stored, and summarized. Agritech companies solve this with streaming data platforms, and this project introduces that architecture at a scale a newcomer can build end to end. The intern writes Python producers that simulate realistic agricultural sensor streams and publish them to Kafka topics, then builds consumers that read the streams, validate and enrich the readings, and land them in HDFS for durable, distributed storage. On top of the stored data, Python analysis scripts compute aggregates such as average soil moisture per field, daily temperature ranges, and gaps that indicate sensor failures, producing summary files a farm manager could act on. The focus stays on the pipeline itself: partitioning topics sensibly, handling malformed messages, and organizing HDFS directories for easy downstream analysis. Through this big-data project the intern gains a concrete understanding of publish-subscribe streaming, distributed file storage, and pipeline design, which are foundational skills for any data engineering role.

agriculturetechnologybig-datahdfs+2

View project

Medium

Batch Grading Analytics Pipeline for Student Assignments

A university processing tens of thousands of assignment submissions each term needs more than spreadsheets to compute grades, detect anomalies, and report cohort statistics. This project recreates that challenge as a distributed data workflow that an intern can build and run end to end. Submission events are simulated with Kafka so assignments arrive as a realistic stream throughout the exercise, and raw submission data is stored durably in HDFS. Scheduled PySpark jobs then process the accumulated submissions in batches, computing grading summaries, score distributions, completion rates, and per-course statistics. Python scripts manage the surrounding workflow, handling ingestion, triggering batch runs, and exporting results as structured data files and reports for instructors and administrators. There is no web or mobile front end; the deliverables are the pipeline itself and the reports it produces, which keeps attention on data correctness and throughput. The intern comes away understanding the streaming-to-batch pattern used across industry big-data platforms: durable ingestion, distributed computation, and reproducible reporting, exercised on a domain where the meaning of every metric is easy to reason about.

educationtechnologybig-datahdfs+3

View project

Medium

Distributed Crop Yield Analysis Pipeline with PySpark

Predicting and improving crop yields starts with unglamorous plumbing: field sensor readings and harvest records must be collected reliably, stored at scale, and distilled into trends a agronomist can act on. This project has the intern build that plumbing with the same tools used in commercial agritech data platforms. Simulated sensor and yield data is streamed through Kafka, capturing the continuous, event-driven nature of readings arriving from many fields. The stream lands in HDFS, which provides distributed storage for the growing dataset. PySpark batch jobs then process the records to generate yield summaries by crop and region, seasonal trend reports, and comparisons across growing conditions. Python scripts coordinate the workflow end to end, from ingestion through processing to exporting results as clean data files. The project is deliberately data-focused, with all outputs delivered as files and reports rather than interfaces. By completing it, the intern learns the streaming-to-batch architecture at the heart of big-data engineering, gains real PySpark experience with joins, aggregations, and window-style analysis, and can speak concretely about applying data infrastructure to agricultural problems.

agriculturetechnologybig-datahdfs+3

View project