The Best Open Source LLMs for Australian Business
Open-weight models have closed most of the quality gap with proprietary APIs, and they are the reason a private, sovereign LLM is now a practical option for Australian organisations rather than a research project. But choosing between Llama, Mistral, Qwen, Gemma and DeepSeek is a licensing decision as much as a technical one, and getting parameter count wrong can triple your hardware bill for accuracy you never needed. This guide covers what matters before you commit.
Why Open-Weight Models Changed the Economics of Private AI
Two years ago, running a genuinely capable language model on your own infrastructure meant accepting a large quality penalty. That trade-off has largely closed. The best open-weight models are now good enough for most business tasks, which changes what a private deployment costs, who controls it, and where your data lives.
The Weights Are the Asset
An open-weight model is a file you download and keep. Once it is on your hardware, no vendor can deprecate it, reprice it, change its safety behaviour under you, or read your prompts. That is a materially different relationship to the one you have with an API. Proprietary providers retire and replace model versions on their own schedule, and a workflow tuned against a retired model has to be revalidated from scratch. A model you hold on disk behaves identically in year three as it did on day one.
Open Weights Is Not the Same as Open Source
Most of the models marketed as "open source" are not open source in the sense the Open Source Initiative means. Llama ships under Meta's community licence, Gemma under Google's terms of use, and several Mistral models under a research-only licence. Each carries conditions: acceptable-use policies, attribution requirements, user thresholds, or an outright ban on commercial deployment. The licence, not the benchmark score, is the first thing to check, and it is the thing most buyers discover last.
Sovereignty Becomes an Architecture Choice
When the model runs on infrastructure you control, cross-border disclosure under Australian Privacy Principle 8 stops being a compliance question you have to argue and becomes a fact about your network diagram. There is no offshore inference endpoint, no vendor sub-processor list to audit, and no data-residency clause to negotiate. For organisations answering to the OAIC, APRA or a Commonwealth agency, that difference is usually worth more than any benchmark delta between an open model and a frontier API.
Choosing an Open Model: Licensing, Families, Sizing and Hosting
Model selection is four separate decisions that get collapsed into one. Treat them separately and the choice becomes straightforward.
Licensing and Commercial Use
Every open-weight model ships with a licence that determines whether you can legally deploy it commercially, what you must attribute, and what you may not use it for. These terms differ sharply between families and even between sizes within one family.
- Apache-2.0 and MIT: genuine permissive licences with no commercial restriction
- Meta Llama Community Licence: permitted commercially below a large monthly-user threshold, with attribution and acceptable-use conditions
- Google Gemma Terms of Use: commercial use allowed, with use restrictions that flow through to anyone you redistribute to
- Research-only licences (some Mistral releases): no production use without a separate commercial agreement
The Main Model Families Compared
Five families dominate serious business deployment. They differ less in raw capability than in licence terms, size ladder, context length and how well they hold up after fine-tuning on domain data.
- Llama: broadest tooling and community support, strongest fine-tuning ecosystem, community licence
- Mistral: efficient for their size and strong in European languages; licence varies by release, so check each one
- Qwen: an unusually complete size ladder and strong coding and structured-output behaviour; most sizes Apache-2.0
- Gemma and DeepSeek: Gemma is compact and well-suited to constrained hardware, DeepSeek is strong at reasoning and maths-heavy work
Matching Model Size to the Task
Parameter count is the single biggest driver of hardware cost, and most organisations over-buy. The right question is not which model is best, but which is the smallest model that clears your accuracy bar on your own evaluation set.
- 7B to 13B: classification, extraction, routing, summarising a known document, drafting from a template
- 30B to 34B: multi-step reasoning, code, and most retrieval-grounded question answering over your own corpus
- 70B+: open-ended analysis, nuanced drafting, and tasks where a subtle error is expensive
- Mixed deployment: route cheap high-volume tasks to a small model and reserve the large model for the few that need it
Hardware and Sovereign Hosting
Open models run wherever you have GPU memory. In Australia that means an Australian cloud region, an IRAP-assessed sovereign provider, or hardware in your own rack or a local data centre.
- Australian cloud regions: AWS Sydney and Melbourne, Azure Australia East and Central, Google Cloud Sydney and Melbourne
- IRAP-assessed sovereign providers for government and Protected-level workloads
- On-premises in your own rack or a NEXTDC-class colocation facility for full physical control
- Quantisation to 8-bit or 4-bit typically cuts VRAM requirements by half or more with modest quality loss
Making an Open Model Yours
A base open model knows nothing about your business. The two mechanisms that fix this are complementary, not alternatives, and most production deployments use both.
- RAG grounds answers in your own documents and updates the moment a document changes
- Fine-tuning teaches domain vocabulary, house style, and output format the base model gets wrong
- LoRA and QLoRA adapters fine-tune large models on modest hardware and stay separable from the base weights
- Start with RAG, measure the accuracy ceiling, and only fine-tune once you know what RAG cannot fix
Security and Supply Chain
A downloaded model is a third-party binary artefact entering your environment. It deserves the same supply-chain scrutiny as any other dependency, and rather more than most organisations currently give it.
- Prefer safetensors over pickle-based checkpoints, which can execute arbitrary code on load
- Verify checksums and pull weights from the official publisher, not a community re-upload
- Pin an exact model revision and mirror the weights internally so a deployment is reproducible
- Scan and review any inference server, quantisation tool or adapter you add alongside the model
How We Benchmark and Select Models for Client Deployments
Public leaderboards tell you how a model performs on someone else's test set. The only benchmark that predicts your outcome is one built from your own tasks and your own documents.
Build an Evaluation Set From Your Work
We collect representative tasks and correct answers from your actual operations, typically 100 to 300 examples. This is the artefact that makes model selection an evidence-based decision instead of an opinion.
Screen on Licence and Sovereignty First
Before any model is benchmarked, we confirm its licence permits your intended commercial use and that it can be hosted within your data-residency and regulatory constraints. Models that fail this screen are never shortlisted.
Benchmark the Shortlist at Multiple Sizes
We run the surviving candidates across the size ladder against your evaluation set, measuring accuracy, latency and tokens per second per dollar, so you can see exactly what a larger model buys you and whether it is worth it.
Size the Deployment and Lock the Revision
We select the smallest model that clears your accuracy bar, size the hardware for your real concurrency, pin the exact model revision, and document the evaluation so the next upgrade is a comparison rather than a guess.
The Total Cost Picture, and Where Open Models Still Fall Short
Open weights are free. Running them is not. An honest comparison against a proprietary API has to count all of it, and has to admit what open models do not yet do well.
What Self-Hosting Actually Costs
The licence fee is zero and the inference is not. The economics favour self-hosting once volume is high enough to amortise fixed cost, and favour an API when volume is low and sporadic.
- GPU capacity is the dominant line item, whether rented monthly or bought once and depreciated
- Engineering time to deploy, evaluate, monitor and upgrade is real and recurring
- Cost per query falls as utilisation rises, which inverts the API model entirely
- Fixed cost means a budget you can forecast, not one that moves with adoption
- Eligible model development and evaluation work may qualify under the R&D Tax Incentive
Where an Open Model Is the Wrong Answer
We will tell you when self-hosting does not fit. Open models trail frontier APIs at the hardest end of reasoning, and low-volume use cases rarely justify the fixed cost.
- Very low or sporadic query volume, where an API subscription is simply cheaper
- Tasks at the frontier of reasoning ability, where the best proprietary models still lead
- No appetite to own evaluation and upgrades, which do not disappear once the model is deployed
- Non-sensitive data plus a small user base, where sovereignty buys you little
- Deep dependence on a proprietary vendor's multimodal or tool-calling ecosystem
Related AI Solutions
On-Premises LLM Deployment
Once you have chosen a model, this covers the hardware, rack and network architecture for running it entirely inside your own perimeter.
Explore on-premises deployment →LLM Fine-Tuning Services Australia
How we adapt an open-weight base model to your domain vocabulary, house style and output formats using LoRA and QLoRA adapters.
Explore fine-tuning options →RAG Architecture Australia
The retrieval layer that grounds an open model in your own documents, and the design decisions that determine whether it is accurate.
See the RAG architecture →Private LLM Cost Australia
Detailed implementation, infrastructure and operating cost ranges, plus the break-even analysis against proprietary API spend.
See cost breakdown →How to Choose an LLM Provider in Australia
A broader selection framework covering vendors, hosting models and contracts, for buyers not yet committed to self-hosting.
Compare provider options →Microsoft Copilot Alternative Australia
How a self-hosted open model compares with Copilot for organisations already inside the Microsoft 365 estate.
Compare with Copilot →Frequently Asked Questions
Most can, but the terms differ and "open source" is used very loosely in this market. Genuinely permissive licences such as Apache-2.0 and MIT impose no commercial restriction. Meta's Llama community licence permits commercial use below a large monthly-active-user threshold, requires attribution, and binds you to an acceptable-use policy. Google's Gemma terms allow commercial use but carry use restrictions that flow through to anyone you redistribute the model to. Some Mistral releases are research-only and need a separate commercial agreement. Licences also vary between sizes within one family, so the 72B and the 7B of the same model may not carry the same terms. Check the licence on the specific model and revision you intend to deploy, and have your legal team read it before it reaches production, not after. We run this screen before benchmarking anything.
For most business tasks, the gap no longer decides the outcome. Retrieval-grounded question answering over your own documents, classification, extraction, summarising, drafting from a template and structured output are all handled well by good open models in the 30B to 70B range, because accuracy in those tasks is driven far more by retrieval quality than by the raw model. The frontier proprietary models still lead on the hardest open-ended reasoning, long multi-step analysis and the broadest multimodal work. The practical answer is that the comparison is task-specific and public leaderboards will not settle it for you. We build an evaluation set from your actual work and measure candidates against it, because a model that ranks third on a public benchmark can easily rank first on your corpus.
Roughly, a model at 16-bit precision needs about two gigabytes of GPU memory per billion parameters, plus headroom for the KV cache that grows with context length and concurrent users. A 7B model therefore fits comfortably on a single 24GB card and will run acceptably on much less once quantised to 8-bit or 4-bit. A 13B model wants around 26GB at full precision, so a single 40GB or 80GB data-centre card. A 70B model needs roughly 140GB at 16-bit, which means two 80GB cards at minimum, or a single 80GB card once quantised to 4-bit with some quality cost. Concurrency changes this materially: sizing for one user and sizing for fifty simultaneous users are different exercises. We size against your real concurrency rather than a single-user benchmark.
Yes, and this is the main structural advantage over any API-based approach. Once the weights are downloaded, an open model has no need to contact the internet to run inference. A complete air-gapped stack is achievable with open components throughout: the model weights on local storage, an inference server such as vLLM or Ollama, a self-hosted vector database for retrieval, and a local embedding model. Nothing in that path makes an outbound request. In practice we build these with egress explicitly blocked at the network layer rather than merely unused, so the guarantee is enforced by your firewall and demonstrable to an auditor rather than resting on trust. Model updates and security patches are then handled as a controlled, scheduled import rather than a live connection.
Less often than the release cycle suggests. New open models appear constantly, but a deployed model that is clearing your accuracy bar does not become worse because a newer one exists. Most organisations we work with review annually, or when a specific capability gap emerges in production. That is the real value of holding the weights: you upgrade on your schedule, not the vendor's. An upgrade is a controlled process, not a swap. The new candidate is benchmarked against the same evaluation set as the incumbent, any fine-tuning adapters are retrained and revalidated, prompts are checked for behaviour changes, and the rollout is staged with the previous revision retained for rollback. Because you pinned the old revision, you can always go back, which is precisely what a deprecated API will not let you do.
Your training data is yours and the adapter weights produced by fine-tuning are generated from it, so in commercial substance the tuned artefact is your asset. The caveat is that it is a derivative of the base model and remains subject to that base model's licence. Fine-tuning Llama does not release you from the Llama community licence, and its attribution, acceptable-use and naming conditions continue to apply to the tuned result and to anything you distribute. Base models under Apache-2.0 or MIT carry far fewer strings, which is one reason we favour them when a client intends to build a durable internal asset or embed the model in a product. When we deliver a fine-tune, the adapter weights, the training data and the evaluation set are all handed over to you. Ownership is settled in the engagement contract before work starts.
Choose Your Open Model on Evidence, Not on a Leaderboard
Talk to us about building an evaluation set from your own work, screening candidates on licence and sovereignty, and benchmarking the shortlist so you deploy the smallest model that actually does the job. Call +61 3 9999 7398 or send us the details of your use case.