Streaming Anomaly Detection for Industrial Sensor Networks
A single failing pump or overheating transformer can cost a manufacturer or utility far more than the sensor that would have flagged it. The hard part is scale: thousands of devices emit readings continuously, and spotting the one abnormal pattern in that flood requires streaming infrastructure, not spreadsheets and morning reports.
In this project the intern builds a real-time analytics platform for industrial telemetry. Kafka ingests high-frequency sensor readings from simulated devices, and Spark Streaming jobs written in Python compute windowed aggregations over the live stream, including rolling averages, min-max bands, and rate-of-change metrics. Spark ML models trained on historical data score incoming readings for anomalies, raising alerts when equipment drifts outside learned norms, while HDFS stores the complete history for replay, auditing, and periodic model retraining.
The intern comes away understanding both halves of applied streaming machine learning: the engineering side, with partitioning, checkpointing, and fault-tolerant state, and the data science side, with feature windows, anomaly thresholds, and honest evaluation of detection quality. That combination is in high demand across energy and manufacturing.
