Available for AI/ML collaboration

Building intelligent systems
that reason, see & scale.

Applied AI engineer & team lead who takes frontier research to production — RL post-training (GRPO) of SOTA LLMs & VLMs, multimodal pipelines fusing 5+ models, and architecture-level tuning for fast inference. Full stack: from model weights to Kubernetes.

▋
View Work → ↓ Download Résumé Get in touch
✉thegigasurgeon@gmail.com 📞+91-9915503368
Delivered Value

Outcomes, not just models.

0
PDFs processed daily by AI platform I lead
0
Document extraction accuracy — up from 35%
0
Developer-hours saved weekly via LLM orchestration
0
Inference pipeline speedup from optimization
0
FPS gain — real-time video analytics on the same GPUs
0
Post-processing time eliminated in geospatial CV
0
Engineers led across validator jurisdictions
0
SOTA models fused in one production pipeline
About

Research-grade AI, engineered for production.

I'm an AI engineer working across NLP, vision, and everything multimodal — at the intersection of deep-learning research and large-scale systems. I pretrain and fine-tune state-of-the-art LLMs and VLMs (LLaMA 3.3, Qwen 3, Gemma, Gemini 3.5 Pro, GPT-4o) using LoRA/QLoRA and RL post-training with GRPO to build reasoning and thinking capabilities into domain models.

Today I lead the development of an enterprise-scale, agentic AI document intelligence platform processing 80K+ PDFs a day — multi-agent text + visual reasoning that fuses 5+ specialized models (detection, OCR, VLMs, reasoning LLMs, embeddings) in a single microservices-based application, with tool-calling, subagents, and skills. Along the way I've built RAG systems, multi-LLM orchestration infrastructure, and extraction pipelines that took accuracy from 35% to 99.85%.

I also go below the framework line — tinkering with model architectures, quantization, and graph-level optimization to hit aggressive inference targets. Those instincts come from years of computer vision: real-time sports analytics, satellite segmentation, pose estimation, and edge deployment. I care about the whole stack, from a novel architecture to a quantized engine shipping 3× the FPS.

At a glance

RoleAI/ML Engineer · Agentic AI, LLMs, CV
LocationDelhi, India
FocusAgents · Post-training · Vision
TeamLed 8 engineers
EducationB.Tech CSE · 8.28 GPA
Featured Work

Systems I've shipped in production.

// Work is proprietary — the diagrams and films are illustrations of each system, not source or screenshots. Ten case studies, each with an architecture diagram, a live illustration and a short film.

Seven more case studies

Multi-LLM orchestration film frame
Real Brokerage · 2025–26

Multi-LLM orchestration infrastructure

One interface over five model families, with intent routing, graceful fallback and shared evaluation.

1000s of developer-hours saved every week
Evaluation harness film frame
Real Brokerage · 2024–26

Evaluation harness and monitoring

Golden sets, regression gates, adversarial cases and production drift monitoring. How we know it works.

Certified the climb from 35% to 99.85%
Transaction auto-review film frame
Real Brokerage · 2024–26

Transaction auto-review with human checkpoints

Agents check the bundle, flag discrepancies with the page and field as evidence, draft the note. A person approves.

1 human gate before any write
Claude Agent SDK automation film frame
Real Brokerage · 2025–26

Claude Agent SDK automation

Triggered by pull requests, incidents and webhooks: reads the diff, runs the tests, inspects logs, delegates, reports.

First-pass reviews and investigations automated
Retrieval pipeline film frame
Real Brokerage · 2025

End-to-end RAG pipeline

Intent routing, semantic chunking, hybrid dense and keyword retrieval with reranking, context-optimised prompts, cited answers.

Retrieval backbone for 80K+ PDFs a day
Inference optimisation film frame
Real Brokerage · Streamingo AI · 2022–26

Inference optimisation

Precision, fusion, compilation and batching, applied one lever at a time and measured after each.

200% document pipeline speedup · 3× video FPS
Satellite segmentation film frame
Attentive AI · 2021–22

Satellite image segmentation

HRNet and Vision Transformer segmentation of property features, owned end to end across two products, Falcon and Respod.

+12% accuracy · 90% of post-processing eliminated
Experience

Where I've built.

AI/ML Engineer · Real Brokerage
Oct 2024 — Present
  • Made the Automated Broker Review platform fully agentic, reviewing 80K+ real-estate PDFs a day: tool-using Gemini agents on pydantic-ai (page overview, text search, render and zoom tools) replaced the OCR + detection + hand-written validator stack; form templates and per-form review hints are managed from S3 and an admin UI, so new forms ship without a code deploy.
  • Pretrained & fine-tuned reasoning LLMs and VLMs (Llama 3.3, Qwen 2.5 72B, Qwen 3, Gemma) with LoRA/QLoRA and GRPO-based RL post-training, plus Gemini Pro/Flash and GPT-4o fine-tunes, taking document review accuracy from 35% to 99.85%.
  • Designed the shared agent runtime behind every review agent: typed tool and output schemas, custom skills and MCP servers exposing document, template and transaction tools, per-page turn / tool-call / token budgets, retry policies with required vs. fail-open semantics, one trace row per run, and a shadow-then-switch rollout with legacy gates running side by side before cutover.
  • Shipped specialised agents on that runtime: entity-signature detection (zero false positives on a 128-document audit set), template version matching, form-metadata extraction, and escalation agents that comment on checklist items and tag brokers on transaction mismatches.
  • Built the agent eval and replay harness: CLI runners, per-document subprocess drivers, replay against human-audited ground truth, cost and latency tracking per model and per form.
  • Built a multi-LLM orchestration layer and an end-to-end RAG pipeline (intent routing, semantic chunking, hybrid retrieval, context optimization, structured outputs) used across the organization, saving thousands of developer-hours a week.
  • Built the production voice agent on LiveKit Agents (Gemini, Deepgram, Cartesia, ElevenLabs, Silero VAD) with realtime tool calling, hardened against upstream model timeouts.
  • Built custom skills and MCP servers (reZEN, Metabase, Datadog, Slack) for Claude Agent SDK and Claude Code agents, wired into engineering workflows: automated PR reviews, incident investigation, webhook-driven agents.
  • Moved review processing onto Temporal durable workflows (Kafka consumer → workflow → activities, 100 concurrent activities, scheduled template sync), removing duplicate processing and lost jobs, with Datadog tracing and structured logging across every agent run.
  • Built high-precision extraction pipelines with YOLOv9, Doctr OCR & custom VLMs, fusing 5+ models as Kubernetes microservices on AWS and GCP, and cut pipeline latency 200% through batching, quantization (INT8/FP16, AWQ/GPTQ) and graph optimization.
  • Led an 8-person team coordinating validator and agent development across US and Canadian jurisdictions.
Deep Learning Engineer · Streamingo AI
Nov 2022 — Oct 2024
  • Developed temporal event-understanding systems with Swin, VideoMAE, MVD, ViT, BEiT.
  • Built a real-time sports analytics engine — object detection, multi-object tracking, pose estimation & court localization.
  • Designed zero-shot & few-shot classification with CLIP, Detic & Prototypical Networks for rapid scaling to unseen classes.
  • Deployed optimized inference with TensorRT & ONNX, delivering 3× FPS.
Computer Vision Research Engineer · Attentive AI
Apr 2021 — Nov 2022
  • Built & trained SOTA networks for satellite image segmentation (HRNet, Vision Transformer).
  • End-to-end ownership of CV products (Falcon & Respod) across the company.
  • Improved model accuracy by 12% and cut post-processing time by 90%.
Software Engineer · Nokia
Jul 2020 — Mar 2021
  • Created a meta-learning platform for dynamic model selection & hyperparameter optimization.
  • Integrated Explainable AI (SHAP, LIME) for model interpretability & compliance.
Under the Hood

How I take models to production.

🚀 Deployment

Multi-cloud and self-hosted — each model lands where its cost, latency, and data-privacy profile fits best.

AWS · EC2 / ECS / SageMakerGCP · Vertex AI / GKE Kubernetes + DockerMicroservice architecture AutoscalingSelf-hosted GPU clusters Temporal + KafkaHelmDatadog · APM + LLM traces

⚡ Inference Optimization

The 200% pipeline speedup and 3× FPS came from stacking these — measured at every step, not guessed.

INT8 / FP16 quantizationAWQ / GPTQ TensorRT enginesONNX + graph fusion Continuous batchingKV-cache reusevLLM

🎯 Model Adaptation

Right-sizing intelligence: RL post-training for reasoning, low-rank adaptation for the domain, and architecture surgery when off-the-shelf isn't fast enough.

GRPO (RL post-training)LoRA / QLoRA DistillationPruningArchitecture modification LLaMA / Qwen / GemmaGemini 3.5 Pro / GPT-4o tuning
Toolkit

Technologies I work in depth.

🤖 Agentic AI

Tool-calling agent loopsMulti-agent & subagent orchestration Custom agent skillsMCP servers & tools Structured outputsContext engineering Token / turn budgetsRetry & fail-open policies Shadow-mode rolloutsAgent evals & replay harnesses LLM tracing & cost controlGuardrails

🧩 Agent Frameworks

pydantic-aiClaude Agent SDK Claude Code · skills, hooks, subagentsMCP · servers & clients LangGraphLangChainOpenAI Agents SDK Google ADKAgent2Agent (A2A)

🧠 Models & APIs

Gemini 3.x Pro / Flash · google-genaiClaude 5 family GPTGroq Llama 3.3Qwen 2.5 / 3Gemma SigLIPCLIPYOLODoctr OCR

🎯 Post-Training & Inference

LoRA / QLoRAGRPO (RL)Distillation Multimodal fine-tuningFew-shot prompting Quantization · INT8 / FP16, AWQ / GPTQvLLM TensorRTONNXBatched inference

🔁 Workflows & Data

Temporal · workflows, activities, schedules, task queues KafkaFastAPISQLAlchemy 2.0 AlembicPostgreSQLRedisS3

🎙️ Voice Agents

LiveKit AgentsGeminiDeepgram CartesiaElevenLabsSilero VAD Realtime tool calling

☁️ Infra & Observability

Python 3.12DockerKubernetesHelm AWSGCP Datadog · APM, LLM traces, monitorsstructlog GitHub Actionsuv

👁️ Vision & Video

YOLOv9Doctr OCRSwinVideoMAE ViT / BEiTCLIP / DeticHRNet Pose estimationMulti-object tracking OpenCVSLAM / ORB-SLAM3

💻 Languages

PythonC++JavaScript / TypeScript
Selected Projects

Things I've built for fun & research.

01

Basketball Video → 3D Mapping

Reconstructed player trajectories in 3D from monocular sports footage using deep video transformers and classical SLAM. Added temporal event summarization via transformer-based description generation.

PyTorchVisual SLAMORBv3BlenderVideoMAE
02

Tekken with Pose

A camera-driven interface that maps real-world body actions into virtual gameplay using pose estimation, latency-aware gesture recognition, and real-time control mapping.

YOLO-NASPose EstimationRaspberry Pi
03

Charlie — AI Virtual Assistant

An edge-deployable assistant with multi-modal recognition: speaker ID, facial login, offline speech-to-text, and contextual response generation — all running on-device.

Raspberry PiSTTFaceNetPython
Contact

Let's build something intelligent.

Open to conversations about AI/ML engineering, applied research, and hard problems in LLMs, VLMs, and computer vision.

Email thegigasurgeon@gmail.com
Phone +91-9915503368