Home/Work/Compact models for a hard domain
Case study · Real Brokerage · Real estate · 2024–26
Compact models for a hard domain
Open LLMs and VLMs fine-tuned with LoRA/QLoRA and GRPO for real-estate auto-review, evaluated per task against hosted frontier models, and served on self-hosted GPUs inside a platform that reads more than 80,000 PDFs a day.
- EmployerReal Brokerage
- RoleAI/ML engineer
- Period2024 – 2026
- SectorReal estate brokerage
§ 01 — Problem
Hosted frontier models are general-purpose, priced per token, and send every document out of the building. The tasks in real-estate auto-review are the opposite: narrow, high-volume and repetitive. That is exactly where a smaller model trained on the task can win, if the evaluation exists to prove it.
Each task raised the same question: can a compact open model, adapted to the domain, match or beat the hosted model on that task's own evaluation set, and then be served at the platform's volume on hardware the team controls?
§ 02 — Architecture
Train, gate, serve.
Open models are adapted on self-hosted GPUs, compared with the hosted baseline per task, and only the tasks they win go to self-hosted serving.
§ 03 — How it works
One task through the post-training loop.
The animation follows one task family from domain data through SFT, GRPO, the evaluation gate, quantisation and serving.
Diagrams and films on this page are illustrations of the system's architecture and behaviour, drawn from my work at Real Brokerage. They are not screenshots or recordings of production systems, which remain confidential.
§ 04 — Approach
Adaptation
- Supervised fine-tuning with LoRA/QLoRA on domain data, one adapter per task family, across four open families: LLaMA 3.3, Qwen 2.5 72B, Qwen 3 and Gemma.
- GRPO-based reinforcement learning post-training to build task reasoning into the model, with the task's verifier as the reward.
- Every fine-tuned model evaluated against the hosted baseline on the task's own evaluation set before it goes near production.
- When off-the-shelf is too slow, the same toolkit extends to distillation, pruning and architecture modification.
Serving
- Served self-hosted where the compact model matches or beats the hosted model; routed to Gemini 3.5 Pro/Flash or GPT-4o where it does not, mixed per cost and latency profile.
- Quantised with AWQ/GPTQ, INT8 or FP16, depending on what the task's accuracy allows.
- Continuous batching and KV-cache reuse on self-hosted GPUs.
- The hosted/self-hosted mix is a per-task decision made by evaluation, cost and latency.
§ 05 — Impact
Fine-tuned open models serve production extraction inside the document platform, which reads more than 80,000 PDFs a day. For each task the hosted/self-hosted mix is chosen by evaluation, cost and latency, and the documents for self-hosted tasks stay on GPUs the team controls.
What generalises: a narrow, high-volume task is the right place to try a compact model; the evaluation set has to exist before the comparison means anything; and a fine-tuned checkpoint is only useful once it is quantised and batched for serving.
