Projects
Streaming Observability Platform with Local LLMs
End-to-end real-time observability: Kafka ingestion, Spark Structured Streaming with tumbling windows and watermarks, rule-based anomaly detection with tiered severity, and local LLM inference for auto-generated incident summaries.
Kafka · Spark Structured Streaming · DuckDB · Ollama
Lakehouse + Streaming Pipeline (in progress)
Building in public: a lean, free-tier lakehouse — real-time API ingestion through Redpanda, Apache Iceberg tables on Cloudflare R2, incremental DuckDB/PyIceberg transforms, Dagster orchestration, and a live dashboard embedded in this site.
Redpanda · Apache Iceberg · Cloudflare R2 · DuckDB · PyIceberg · Dagster
GenAI Dynamic Task Builder for Airflow
GenAI-powered pipeline that dynamically generates and executes Airflow tasks from natural language input — an early exploration of agentic orchestration patterns for data engineering.
Python · Apache Airflow · GenAI · LLM
RAG Pipeline with LangChain and OpenAI
End-to-end retrieval-augmented generation: document chunking, embedding, vector retrieval, and grounded response generation.
Python · LangChain · OpenAI · Vector Search · Jupyter
English–Marathi Multimodal Translation App
Bidirectional English–Marathi multimodal translation deployed on Hugging Face Spaces — combining NLP and multimodal inputs for real-world language accessibility.
Python · Hugging Face · NLP · Multimodal