Licence fit.
Read the actual terms for your use: commercial use, redistribution, derivative models and any acceptable-use restrictions. Open weights and open source are not the same thing.
Smaller,
and better
at one thing.
A private LLM does not have to be a frontier model. It has to be one you can host, licence cleanly, and measure on the task you actually have.
READ THE DEFINITION ↓WEIGHTS · LICENCE · FIT
01 / DEFINITION
The short answer first.What is a private LLM?
A private LLM is a large language model whose weights an organisation holds and runs on its own infrastructure. In practice this means an open-weight model, since running a model privately requires access to the weights and a licence that permits the intended use.
Two properties follow from holding the weights, and both matter beyond privacy. The version is frozen until you change it, which makes results reproducible for as long as you keep the artefact. And the model can be fine-tuned, which is what allows a smaller model to outperform a larger general one on a narrow task.
02 / CHOOSING ONE
Five criteria, in the order they eliminate options.Size against your hardware.
A model that does not fit your memory budget at your context length and concurrency is not a candidate, whatever its benchmark scores.
Task fit, measured.
Run your own held-out set. General leaderboards do not predict performance on specialist vocabulary or your document types.
Fine-tuning viability.
Tooling maturity, community adoption and documented recipes. A model nobody has successfully fine-tuned is a research project.
Longevity.
You will be running this version for years. An abandoned family means no security fixes and no upgrade path.
Licence terms deserve genuine attention rather than a glance. Several widely used model families carry restrictions on scale of use, on training other models from their outputs, or on specific application domains. These are conditions that can affect whether a planned deployment is permitted at all, and reading them before selection is considerably cheaper than discovering them afterwards.
03 / THE SPECIALISATION ARGUMENT
First-party evidence for smaller models.An 8B model,
doing one job well.
The strongest argument for a private LLM is not that it is nearly as good as a frontier model. It is that on a narrow, well-defined task a fine-tuned smaller model can be better, and small enough to run on modest hardware while doing so.
MKC2 is Privilege AI's own instance of that argument. Fine-tuned from Qwen3-8B for legal supervision, it ranks defective work above sound work at 0.9623 against 0.7281 for the stock base model, with 93.17% accuracy against 83.09%, measured on 4,403 held-out items scored in the same run. The claim is specific to that supervision task and should not be read as a general statement about model size — but it is the shape of result that makes private deployment attractive rather than merely acceptable. Full record on the MKC2 page.
The corollary is that model selection and fine-tuning are one decision. If the plan is to specialise, the criteria that matter are tuning tooling, base capability in the relevant domain and licence permission for derivative models — not the base model's general leaderboard position. See AI model fine-tuning.
04 / OPERATING A PRIVATE MODEL
What you take on by holding the weights.Version discipline.
Weights, quantisation, serving version and prompt, all pinned and recorded. Without it, no evaluation result can be attributed to anything.
Artefact provenance.
Checksums and a recorded source for every model file, and a safe serialisation format. Model weights are an unreviewed dependency with deep access.
An upgrade path.
A newer base model means re-running fine-tuning and the full evaluation. Plan the second version while building the first.
The upgrade obligation is the part most often left out of a business case. A fine-tuned model is a maintained artefact: when the base family releases a materially better version, capturing that improvement means repeating the training and the held-out evaluation. Budgeting for that from the start is what keeps a private model from quietly becoming a legacy system.
05 / QUESTIONS
Asked when selecting a model to host.Are open-weight models good enough for production?
+
For well-defined tasks, frequently yes, particularly after fine-tuning. For broad open-ended reasoning the largest hosted models still lead. The question is answered by measuring candidates on your own held-out set rather than by comparing leaderboard positions.
What size model should we start with?
+
The smallest that clears your quality bar on your own evaluation, because everything downstream — memory, latency, concurrency, cost — improves with a smaller model. Models in the 7B to 13B range are a reasonable starting point for narrow tasks.
Is an open-weight model the same as open source?
+
No. Open weights means the parameters are downloadable; the licence governs what you may do with them and is frequently narrower than an open-source licence. Some families restrict scale of use, derivative models or specific application domains.
Can a private LLM be fine-tuned on our own data?
+
Yes, subject to the licence permitting derivative models, and it is often the reason to choose a private model in the first place. Sensitive training material calls for deliberate handling of the resulting weights, since they encode traces of what they were trained on. See AI data privacy.
How often should a private model be updated?
+
Only when a measured improvement justifies the re-evaluation, which is the honest cost of an upgrade. A frozen version that has been measured is more valuable than a newer one that has not.
Pick the model
for the task.
Privilege AI develops and fine-tunes specialised models, and measures them against the stock base on held-out data.