Dhruv Makwana

Officers were burning an hour and a half per incident watching bodycam footage, listening to audio, and writing it all up by hand — with data-privacy rules that ruled out sending any of it to the cloud. I redesigned the pipeline around a simple principle: use deterministic techniques (OCR timestamp anchoring, detection + tracking + embeddings for identity, native-text parsing) everywhere possible, and reserve reasoning-model calls only for genuinely judgment-dependent steps like weapon attribution. A tiered, self-escalating Q&A layer lets follow-up questions get answered without maintaining a standing search index. The result: officer active time drops roughly 85%, from ~75-90 minutes to ~12-15 minutes — while the entire system runs on-premise for compliance.

No operator can attentively watch dozens of feeds for a weapon drawn for a split second, or a bag left unattended just long enough. I built a multi-camera pipeline — GPU-accelerated decode, detection, tracking, pose estimation — with deterministic rules for most alert categories and a vision-language model reserved specifically for the one judgment call geometry can't make: was this object abandoned, or is someone still with it. On real and staged footage, that swapped a legacy detector's 16.7% recall for 66.7% recall at 80% precision. Aggression and weapon alerts fired correctly on 89.5% of real incidents, with false alerts averaging under 5 per camera per hour across a live 23-camera evaluation. A decision-layer agent on top correlates related alerts into one coherent incident report instead of a flood of pings.

Most video QA models compress a clip the same way regardless of the question, throwing away exactly the evidence a specific question needs — and surgical answers come in five incompatible shapes (yes/no, classification, multi-label, counting, free-text timing) that no single reward can score fairly. I designed a dual-stream compressor — one stream for spatial/anatomical detail, one for instrument timing — each conditioned on the question via cross-attention, squeezing 32,805 raw tokens per clip into a fixed 671-token budget, a 49x reduction. Paired with an answer-type-aware GRPO reward and a new 80,636-question benchmark, the model hit 0.779 keyword accuracy on a sealed test set: a 10x BLEU-4 gain over zero-shot and nearly double the closest published comparable medical-video RL model. Disabling each stream at inference confirmed the architecture split evidence exactly as designed — the fast stream alone swings timing questions by up to 22 points.

Whisper handled English bodycam audio fine but fell apart on Spanglish — officers and suspects switching languages mid-sentence over street noise and radio chatter, where missing a name or a key verb has legal consequences. No usable Spanglish dataset existed for this domain, so I built one: concatenating existing English and Spanish speech datasets with noise injected at the seams to pretrain the model on language-switching, then fine-tuning on synthetic officer-suspect dialogue generated from expert-identified critical word lists via TTS. That two-stage pretrain-then-finetune approach beat fine-tuning on synthetic data alone, landing at 0.1 WER on real annotated Spanglish law-enforcement audio.

The client had fewer than 10 real images per part — nowhere near enough for standard detection training, and template matching failed outright on reflective, variably-lit industrial parts. I proposed capturing a 3D mesh of each object with a depth camera, then rendering hundreds of photorealistic synthetic images per class from arbitrary angles and lighting — since the render defines the exact pose, every image comes fully annotated for free. Training YOLOv8 on that synthetic data crushed template matching, and feeding class-specific detections into MegaPose produced full 6DoF poses accurate enough for real robotic pickup. When a shortcut was proposed that I knew YOLO's grid-based design couldn't support, I ran a quick controlled experiment to prove it rather than argue it — saving the team from building the wrong thing.

A food-delivery client's product data was a mess of freeform text — "half kg", "500g", and "0.5 kilograms" all meaning the same thing, diet tags written a dozen different informal ways. Regex and lookup tables couldn't keep up with the variability, so I built a LangChain pipeline that sends each row to GPT-4 alongside the client's schema of valid values, and iterated the prompt through zero-shot, few-shot, and a compressed JSON-schema format that cut hallucinated fields and token usage together. A second GPT-4 call then judges the output against the original input and scores it 0-1: anything scoring 0.8+ writes straight to the database, everything else routes to a human. That gate auto-approved roughly 90% of rows and cut manual review effort by about 85%, serving the whole thing as a FastAPI endpoint.