Ask Your Data in Plain Language, a 30B LLM Running Entirely On-Premise
Business staff couldn't write SQL, so every question became a ticket. We deployed a 30-billion-parameter model on the client's own server, grounded on its schema, answering voice or text questions in seconds, with nothing leaving the network.
- Service
- AI Business Applications
- Industry
- Revenue Cycle & Finance
- Region
- United States
- Engagement
- Lab-type ODC (dedicated team)
Outcomes
- Answers in seconds, no SQL
- 100% on-premise, zero data egress
- Permission-scoped, read-only
Stack
On-prem LLM (30B) · SQL · Schema grounding · Voice input · Private server
Written byOriental Nippon Team
A US revenue-cycle operator holds account-level data on healthcare patients. Operations and commercial staff needed answers from that data all the time, but they could not write SQL, and the data team's queue was the only way in.
Hosted AI tools were not an option. The data includes personally identifiable and health information, and it was not allowed to leave the building.
The problem
- Every question was a ticket. Ad-hoc questions waited in the data team's queue.
- Capacity drained. The data team spent its time on one-off queries.
- Dashboards miss the unexpected. Self-service BI covers the questions you anticipate. The important ones are often the ones you didn't.
- Strict data residency. Nothing could cross the network boundary.
| Before | After | |
|---|---|---|
| How staff asked | Ticket to the data team | Plain language, voice or text |
| Time to answer | Hours to days | Seconds |
| Model hosting | n/a | 30B-parameter model, on-premise |
| Data leaving the network | n/a | None |
What we built
A locally hosted LLM
A thirty-billion-parameter model on the client's own server, with nothing crossing the network boundary.
Questions are translated into queries against the real database structure, not a guessed one.
A voice and text interface
Results in seconds.
Read-only, permission-scoped access
Users only see what they are already allowed to see.
A single question, end to end:
Technology and the role it played
| Technology | Role in this project |
|---|---|
| 30B-parameter open model | Turns plain-language questions into queries, on the client's server |
| Schema grounding | Gives the model the real tables and columns |
| SQL (read-only) | Answers questions without the ability to change data |
| Voice input | Lets staff ask without typing |
| Private server | Keeps every prompt and answer inside the network |
How we delivered it
Our ODC team settled the boundary first: what may run where, confirmed with the client's compliance position before any model was chosen.
- Boundary and model choice. Residency constraints, model selection and hardware sizing.
- Infrastructure. On-premise deployment, serving and monitoring.
- Schema grounding. Iterated against real questions staff had previously sent to the data team.
- Constrain the interface. Read paths only, scoped to the asking user's permissions.
- Pilot. Non-technical users, with answers compared to what the data team would have given.
01Boundary and model selection
1–2 wks
Residency constraints, model choice, hardware sizing
02Infrastructure
2 wks
On-premise deployment, serving, monitoring
03Schema grounding
3–4 wks
Structure mapping, query generation, evaluation set
04Interface and permissions
2–3 wks
Voice and text input, read-only scoping, user permissions
05Pilot
2–3 wks
Non-technical users, answers compared against the data team's
Risks and how we handled them
| Risk | How we handled it |
|---|---|
| Sensitive data leaving the network | Residency boundary agreed with compliance before choosing a model |
| The model invents columns | Grounding iterated against real past questions |
| Users seeing data they shouldn't | Read-only queries run with the asking user's permissions |
| Answers nobody can check | Pilot answers compared against what the data team would have produced |
Outcome
Business users get account-level answers in plain language, in seconds, without SQL and without a queue. No sensitive data leaves the client's network to make that possible.
Data residency was the reason a hosted approach was never viable. Building for it from day one is far cheaper than retrofitting it after a pilot.
What the client keeps
- The deployment and its configuration
- Schema grounding artefacts
- The evaluation question set
- Permission scoping rules
- Infrastructure documentation
Sources and further reading
- Top 10 for Large Language Model Applications, OWASP
- AI Risk Management Framework, NIST
- BIRD: a big benchmark for text-to-SQL, BIRD-bench
- vLLM documentation, vLLM project
Facing something similar? This is the kind of work our AI, Data Science & Automation team delivers every day. Talk to us about your stack.
