Question 1

What is prompt injection and how serious is it in an enterprise context?

Accepted Answer

Prompt injection is an attack where an adversary inserts instructions into data the LLM processes in order to override the model's intended behaviour. In an enterprise context, the most serious variants involve an attacker injecting instructions into documents that end up in the model's context, or into user inputs, that cause the model to perform actions it should not or reveal information it should not. The severity depends on what capabilities the model has: a model that can only answer questions has limited blast radius from prompt injection; a model that can take actions like sending emails or querying databases has much higher exposure. Architectural controls, specifically limiting the model's capabilities and implementing human review for consequential actions, are the most effective mitigations.

Question 2

Can an attacker extract our proprietary documents from a fine-tuned model?

Accepted Answer

Training data extraction is a real but probabilistic risk. Research has shown that LLMs can reproduce verbatim training content through carefully crafted prompts, but typically only content that appeared many times in training. For fine-tuned models trained on your proprietary documents, the risk depends on: how many times each document appeared in training, how much the fine-tuning process has memorised vs generalised, and what access controls limit who can interact with the model. For most enterprise fine-tuning use cases involving moderate document volumes, verbatim extraction risk is low. However, the higher-risk approach is RAG: if your knowledge base is poorly access-controlled, an attacker can retrieve specific documents directly. Access controls on the RAG index are more important than fine-tuning memorisation risk.

Question 3

How does LLM security differ for a RAG system vs a fine-tuned model?

Accepted Answer

The security profiles are different. A RAG system's primary risk is retrieval manipulation: an attacker who can influence retrieval to return documents they should not access, or who can inject content into the knowledge base to influence future retrievals, can exfiltrate data or manipulate model behaviour. Access controls on both the document index and the retrieval pipeline are essential. A fine-tuned model's primary risk is training data extraction and the fact that model behaviour can be influenced by training data characteristics. Both systems share the prompt injection risk. In practice, most enterprise deployments combine both, so both attack surfaces must be addressed.

Question 4

What security testing should we do before deploying an enterprise LLM?

Accepted Answer

Pre-deployment security testing for enterprise LLMs should cover at minimum: prompt injection testing using the OWASP LLM Top 10 as a framework, jailbreak testing to verify system prompt hardening, access control testing confirming document access restrictions are enforced, RAG manipulation testing including indirect injection via document content, output monitoring testing confirming sensitive data detection works as intended, and authentication and session management testing. For higher-security deployments involving sensitive data categories, red team testing by a specialist AI security team is worthwhile. We include red team testing as a standard part of our enterprise deployment process.

Question 5

How do we handle security incidents involving our LLM deployment?

Accepted Answer

An LLM security incident response plan differs from a standard application incident response plan in that the primary evidence source is the audit log of prompts and responses. If an incident is suspected, the first step is to preserve and review audit logs to understand what queries were made, what documents were retrieved, and what outputs were generated. The second step is to determine whether the incident was an internal misuse, an external attack, or an accidental data leakage. Depending on the nature and data involved, you may have notification obligations under the Notifiable Data Breaches scheme. We include incident response procedures and audit log forensics guidance as part of our managed service.

Question 6

What is the difference in security posture between cloud-hosted and on-premises deployment?

Accepted Answer

On-premises deployment provides a fundamentally different security posture for the most sensitive data: the attack surface accessible from the internet is reduced to whatever management interface exists, model weights and training data cannot be accessed without physical access, and the entire inference pipeline is within your physical security perimeter. For government, defence, and highly regulated deployments, this is the appropriate architecture. Cloud-hosted sovereign deployment on Australian infrastructure provides strong data sovereignty and eliminates offshore access risk, but retains the network attack surface inherent in any cloud-connected system. For most commercial deployments, cloud-hosted with strong network controls is adequate; for the highest-sensitivity applications, on-premises is the appropriate choice.

LLM Security and Data Privacy for Australian Enterprise

Why LLM Security Is Different From Traditional Application Security

The Prompt Injection Threat

Training Data Extraction

RAG Retrieval Manipulation

The LLM Security Architecture for Enterprise Deployment

Input Validation and Prompt Hardening

Access Control and Authentication

Network Isolation and Egress Controls

Output Monitoring and Filtering

Comprehensive Audit Logging

Model Isolation and Sandboxing

How We Build a Secure Private LLM Deployment

Threat Modelling

Security Architecture Design

Red Team Testing

Production Monitoring and Incident Response

The OWASP Top 10 for LLMs: How Private Deployment Mitigates Each

Architecture-Level Mitigations (OWASP LLM01-05)

Data-Level Mitigations (OWASP LLM06-10)

Related AI Solutions

ISO 27001 AI Compliance

Sovereign AI Australia

On-Premises LLM Deployment

Frequently Asked Questions

Deploy AI That Is Secure by Architecture, Not Just by Policy