Skip to content
Skip to content

Practical LLM operations guide · By Yes AI

Who does what in a private LLM implementation?

Published 6 min read

Buyers often budget for the provider and forget the internal time. A private LLM touches documents, permissions, privacy and daily work, and only people inside the organisation can make those decisions. This guide sets out the roles, what each one decides, and the handovers between them.

Quick answer

A private LLM implementation needs an accountable business owner, a task owner from the team that will use it, document owners for each source collection, IT for identity, networking and hosting, a privacy or risk lead, and the provider who builds and tests the system. The provider can design, build and run it, but your people decide the scope, which documents are authoritative, who may see what, and whether answers are good enough to release.

Project team of five around a table in a Brisbane office planning with sticky notes and a laptop

Who is accountable for the system?

Someone senior has to own the assistant as a business system: its purpose, its risks and the decision to keep running it. The Australian Government's Guidance for AI Adoption puts this first among its six essential practices. It says organisations should make a specific person accountable for every AI system they use and make sure that person is familiar with the technology and understands its business implications. In a private LLM project, the accountable owner approves the scope, accepts residual risks, signs off each release and decides what happens when a provider retires a model. Without that person, decisions drift to whoever is most technical, who may have no authority over documents or policy.

What does the task owner do?

The task owner runs the work the assistant supports, for example the HR team lead for a policy assistant. They supply real questions, define what a good answer looks like, review test results and decide whether the assistant is ready for their team. They also own adoption: telling staff what the assistant covers, collecting feedback and deciding which gaps to fix first. Expect this role to need regular time during the build and testing stages, mostly spent reviewing answers.

  • Collect a set of real questions, with the expected answer and source for each.
  • Mark which questions must be refused or handed to a person.
  • Review test results and sign off each release for their team.

Why do document owners matter?

The assistant can only be as current as its sources. Each collection, such as HR policies, IT procedures or client templates, needs an owner who decides which version is authoritative, removes drafts and duplicates, and tells the support team when something is withdrawn. Plan for document clean-up as internal work, because it falls on these people. A provider can find duplicates and conflicts and report them, but it should not decide which of two contradictory policies is correct. Ask document owners to agree to the clean-up before the build starts.

What does IT need to provide?

IT connects the assistant to the organisation's identity, network and security controls. The provider usually does the design and much of the configuration, but IT approves and owns the parts inside your environment and keeps them working after the project team has moved on. Bring IT in at scoping, because hosting and identity choices shape the rest of the design, including which source systems can be connected and how. The typical tasks are below.

  • Single sign-on and group membership, so the assistant knows who each user is.
  • Service accounts or app registrations for each source system, with minimum permissions.
  • Network access, firewall rules and, for self-hosted options, servers or cloud subscriptions.
  • Security review, logging integration and backup arrangements.
  • Joiner, mover and leaver processes that update assistant access automatically.

Where do privacy and risk fit?

A privacy or risk lead checks the project against your obligations before the design is fixed. The OAIC describes a privacy impact assessment as a systematic assessment of a project that identifies its impact on individuals' privacy and sets out recommendations for managing it, and advises doing one early enough to influence the design. This person decides whether personal information may be indexed, which notices need updating and what the assistant must refuse. In regulated sectors they also check sector rules, such as APRA standards. Give them the data flow early: the sources, where processing happens and where logs are kept.

What should the provider own?

The provider designs the architecture, builds the retrieval and application layers, connects sources, runs evaluation and documents how the system works, so that your team or another provider could take it over. It should report test results in a form the task owner can judge, flag document conflicts instead of resolving them, and explain trade-offs in plain terms. Agree up front how decisions are recorded, who can approve scope changes and what is handed over at the end, including configuration, prompts, test sets and runbooks. Timing for each stage is confirmed in your written quote after scoping.

Where do handovers between roles break down?

The roles above are only useful if they survive staff changes and the end of the project. Watch the points where one person's decision depends on another's work. A document owner who leaves without a replacement leaves a collection that nobody updates. An IT change to group names can silently cut users off from sources. A task owner who moves role takes the knowledge of what a good answer looks like. Keep a deputy for each role, a shared decision log, and a short review of the role list at every release.

  • Name a deputy for the task owner and each document owner.
  • Tell support before renaming or restructuring identity groups.
  • Review the role list whenever someone in it changes job.

Roles, decisions and typical time demand

RoleDecidesMain tasksTime demand
Accountable business ownerScope, risk acceptance, release and retirementApprovals and escalationsLow, at decision points
Task ownerWhat a good answer is and when the team is readyQuestions, test review and adoptionModerate during testing
Document ownersWhich versions are authoritativeClean-up and ownership of updatesHigh early, then steady
ITIdentity, network and hosting arrangementsSingle sign-on, service accounts and security reviewModerate during the build
Privacy or risk leadUse of personal information, notices and refusalsPrivacy impact assessment and sign-offModerate early
ProviderTechnical design within the agreed scopeBuild, evaluation, documentation and supportMost of the build effort

Practical checklist

  • Each role in the table has a named person, with a deputy for the task owner.
  • The accountable owner has approved the scope in writing.
  • Every source collection has a document owner who has agreed to the clean-up work.
  • IT has reviewed the identity, hosting and network design before the build starts.
  • The privacy or risk lead has seen the data flow and set what the assistant must refuse.
  • The provider's handover list covers configuration, prompts, test sets and runbooks.

Common questions

Can the provider do everything?

A provider can do most of the technical work and can run the system afterwards. It cannot decide which policy is current, who should see which records or whether an answer meets your standard, because those are business decisions. A provider that offers to make those calls for you is taking on responsibility it cannot properly hold.

How much internal time should we expect?

It depends on the number of source collections, how tidy they are and how many teams are involved. Document owners and the task owner carry most of it. Ask the provider for a stage-by-stage estimate of internal hours as part of the written quote, and check it against your team's other commitments for the same period.

Do we need a dedicated project manager?

For a single team with one or two sources, the task owner can often coordinate. Once several departments, multiple systems or a formal privacy assessment are involved, a coordinator who tracks decisions and dependencies saves time for everyone else. Either way, keep one decision log that both your team and the provider can see.

What should happen at the end of the project?

A formal handover: the accountable owner signs off the release, IT confirms its operational responsibilities, and the provider delivers documentation, test sets and runbooks. Agree who runs support from then on and how model retirements and document changes will be handled. Our guide on support after launch lists the recurring tasks.

Sources and further reading

Continue planning

Plan the checks before your next LLM release

Describe the task, source documents and people who will review the answers. We can discuss evaluation, access boundaries and a staged implementation. Keep confidential documents and access credentials out of this enquiry.