AI for SharePoint and Microsoft 365 Without Giving Up Control of Your Data
If your organisation runs on Microsoft 365, then a decade of decisions, procedures, contracts and corporate memory is already sitting in SharePoint. The problem is that none of it is findable. Staff know the answer exists somewhere, so they ask a colleague instead, or they rewrite the document. A private AI layer indexes that content, answers questions in plain language with citations, respects every permission you have already configured, and keeps the whole thing on Australian sovereign infrastructure.
Why Knowledge Buried in SharePoint Is So Hard to Find
SharePoint is an excellent place to store documents and a frustrating place to find them. This is not a criticism of the product so much as a description of what happens to any document store after ten years, three restructures and a migration. The failure is not that the content is missing. It is that finding it requires knowing what it was called and who filed it.
Search Matches Words, Not Meaning
SharePoint search is fundamentally a keyword engine. If the policy says "flexible working arrangements" and your new starter searches "work from home", the document is invisible to them. It exists, it is indexed, and it will never appear. Every organisation develops internal vocabulary that new staff do not have, and keyword search punishes exactly the people who most need to find things. Semantic retrieval matches on meaning rather than string overlap, so the question finds the document regardless of which words the author happened to choose years ago.
Sprawl That Nobody Owns
Every Team created spawns a SharePoint site. Over a few years that becomes hundreds of site collections, an archive nobody dares delete, and four documents named "Expense Policy FINAL" with different effective dates. Staff cannot tell which version governs, so they ask someone, and that person guesses. The content problem is really a governance problem, and no amount of retraining staff on folder structures fixes it. A retrieval layer that reads document metadata, modified dates and library structure can prioritise the current, authoritative version and tell the user which one it used.
The Answer Lives Across Five Documents
Real questions rarely map to one file. "What do we pay a contractor in WA for weekend work, and what do we need on file before they start?" touches an enterprise agreement, a state licensing requirement, an insurance certificate policy and a finance procedure. Search can return five documents; it cannot read them and give you the answer. That synthesis step is the actual work, and it is what staff currently do by hand, badly, or delegate to whoever has been there longest. This is precisely what a grounded language model does well.
What a Private AI Layer Over Microsoft 365 Looks Like in Practice
This is not a product you install into your tenant and switch on. It is a retrieval and generation pipeline that reads from Microsoft Graph, indexes into a sovereign store you control, and answers through a private model. Each component is a deliberate choice.
Microsoft Graph Ingestion
Content is read through the Microsoft Graph API using an application registration in your own Entra ID tenant, scoped to exactly the sites and libraries you approve. Nothing is copied out of Microsoft 365 that you have not explicitly put in scope.
- SharePoint Online site collections, document libraries and list content
- OneDrive for Business, scoped per site or excluded entirely
- Teams channel messages and wiki content where in scope
- Exchange Online mailboxes and shared mailboxes, opt-in only
Permission-Aware Retrieval
Every chunk in the index carries the access control list of its source item. At query time the retrieval layer resolves the asking user's Entra ID group membership and filters the candidate set before the model ever sees it. The AI cannot answer from a document the user could not open.
- Entra ID group and SharePoint role assignments mirrored onto each chunk
- Security trimming applied at retrieval, not as a post-filter on the answer
- Item-level unique permissions and broken inheritance honoured
- Microsoft Purview sensitivity labels readable as retrieval filters
Sovereign RAG Index
Document chunks and their embeddings live in a vector store on infrastructure you control, in an Australian region or on your own hardware. The embedding model runs locally, so document text is never sent to an offshore embedding API to be vectorised.
- Hosting in Azure Australia East or Australia Southeast, or on-premises
- Locally hosted embedding models with no outbound document traffic
- Hybrid retrieval combining semantic search with exact keyword matching
- Citations back to the source SharePoint item, with version and modified date
Private Model Generation
The generative model runs on your infrastructure alongside the index. Prompts, retrieved context and generated answers stay inside the boundary, which is what makes the Privacy Act and contractual confidentiality conversations straightforward rather than fraught.
- Open-weight models deployed on sovereign infrastructure you control
- No prompt or document content used to train any third-party model
- Grounded answering that declines when the documents do not contain the answer
- Optional fine-tuning on your organisation's vocabulary and document conventions
Delta Sync and Freshness
The index tracks change rather than rebuilding nightly. Graph delta queries and change notifications push new, modified and deleted items into the pipeline continuously, so answers reflect the document set as it stands rather than as it stood last week.
- Microsoft Graph delta queries for incremental change detection
- Change notification subscriptions for near-real-time updates
- Deletion and tombstone handling so removed content stops being retrievable
- Scheduled reconciliation passes to catch anything the delta stream missed
Governance and Audit Logging
Every query, every retrieved chunk and every generated answer is logged against the requesting user. This is what turns a promising pilot into something your risk committee will actually sign off on.
- Full audit trail of who asked what and which documents informed the answer
- Evidence artefacts supporting APRA CPS 234 and ISO 27001 control reviews
- Retention and logging aligned to your existing records management policy
- Administrative controls to exclude sites, libraries or classifications from scope
How We Deploy AI Over Your Microsoft 365 Tenant
The work that determines whether this succeeds happens before any model is deployed. Most of it is understanding what is actually in your tenant and who is genuinely allowed to see it.
Tenant and Permission Assessment
We map your site collections, document volumes, file types and permission structure, and identify where inheritance is broken or over-shared. This assessment frequently surfaces existing exposure that has nothing to do with AI, and you get that finding regardless of whether you proceed.
Scope, Classification and Coexistence Design
We agree which sites are in scope, how sensitivity labels map to retrieval rules, and where this layer sits alongside anything you already run, including Copilot. Data classification drives the boundary, not convenience.
Build, Index and Validate Trimming
We build the Graph ingestion pipeline, index your corpus, and then test permission trimming adversarially with real accounts across your access tiers, deliberately trying to retrieve documents those users should not reach. This gate is not negotiable.
Pilot, Measure and Roll Out
A pilot group works with the system against real questions while we measure retrieval accuracy and answer faithfulness. We tune chunking and retrieval against observed failures, then roll out to the organisation with training and a feedback channel.
Permission-Aware Answers, and Where Copilot Fits
Two questions decide almost every Microsoft 365 AI project: whether the AI can leak something, and whether you should just buy Copilot instead. Both deserve a straight answer.
How Permission Trimming Actually Works
The nightmare scenario is an AI that cheerfully summarises the redundancy list to the person on it. Preventing that is an architectural property, not a prompt instruction, because a model asked politely enough will ignore an instruction. The permission check has to happen before retrieval.
- Access control lists are captured at ingestion and stored on every chunk
- The asking user's group membership is resolved against Entra ID at query time
- Filtering happens pre-retrieval, so out-of-scope chunks never enter the prompt
- Permission changes in SharePoint propagate through the delta sync pipeline
- Adversarial trimming tests run before go-live and on every index schema change
How This Differs From Microsoft 365 Copilot
Copilot is genuinely good, and for in-application work it is often the better tool. It drafts in Word, summarises a Teams meeting you missed, and triages an Outlook inbox with a fluency a separate chat interface will not match. Where a private layer wins is grounding, boundary and unit economics.
- Reaches systems outside Microsoft 365, such as Xero, MYOB, LEAP or JobAdder
- Priced on query volume rather than per-seat licences, so cost does not track headcount
- Model, index and prompts on infrastructure you control and can audit directly
- Tunable on your document conventions rather than a generic model over a generic index
- Coexistence is normal: Copilot for in-app drafting, private layer for sensitive corpora
Related AI Solutions
AI Knowledge Base for Enterprise
The broader case for an organisational knowledge base, including document types and query patterns beyond the Microsoft stack.
Explore enterprise knowledge base →RAG Architecture Australia
The retrieval engineering underneath this page: chunking, hybrid search, re-ranking and the evaluation that proves it is accurate.
See the RAG architecture →Microsoft Copilot Alternative Australia
A full comparison of Copilot against a private custom LLM on residency, customisation and per-seat cost, including where Copilot wins.
Compare against Copilot →LLM Security and Data Privacy
How prompts, embeddings and audit logs are secured, and what your obligations are under the Privacy Act and APP 11.
Review security and privacy →Sovereign AI Australia
What data sovereignty means in practice for Australian organisations, and how hosting and jurisdiction decisions are made.
Understand sovereign AI →AI Document Processing Australia
When the goal is extracting structured data from documents rather than answering questions about them, this is the adjacent capability.
Explore document processing →Frequently Asked Questions
Yes, and this is an architectural guarantee rather than a promise the model makes. At ingestion, every chunk inherits the access control list of its source item, including item-level unique permissions where inheritance has been broken. At query time we resolve the asking user's Entra ID group membership and filter the candidate set before retrieval, so content the user cannot open in SharePoint never enters the prompt context. That ordering matters: filtering after generation would mean the model had already read the document, and a sufficiently clever prompt could extract it. We validate this adversarially before go-live, using real accounts from each of your access tiers and deliberately attempting to retrieve restricted content. Where the tenant assessment finds over-shared or broken-inheritance sites, we surface that as a finding for you to remediate first, because AI does not create that exposure but it does make it far easier to discover.
They solve overlapping but distinct problems, and running both is a normal outcome rather than a compromise. Copilot lives inside the applications and is excellent there: drafting in Word, summarising a Teams meeting, triaging Outlook. A private layer is better when answers must draw on systems outside Microsoft 365 such as Xero, MYOB, LEAP or JobAdder, when the model needs tuning on your organisation's vocabulary rather than a generic index, or when a corpus is sensitive enough that you want the model, index and prompts on infrastructure you control and can audit. Cost also diverges, though not in the way a per-seat comparison suggests: Copilot is licensed per user and rises with headcount, while our pricing tiers on query volume rather than seats, starting at $2,999 per month on the Starter tier for a single custom model and up to 100,000 queries. Adding staff does not by itself add cost, but heavier use does, so the comparison turns on how much your people actually ask rather than on headcount alone — see our pricing page for the current tiers. The usual design splits by data classification: Copilot for everyday in-app productivity, the private layer for the corpora your risk committee cares about. We will tell you if Copilot alone is the right answer.
Chunks, embeddings, the vector index and the generative model all sit on infrastructure you control, either in an Australian cloud region such as Azure Australia East in Sydney or Australia Southeast in Melbourne, or on your own hardware for a fully on-premises deployment. The embedding model runs locally, which is the detail most implementations get wrong: if embeddings are generated by a hosted API, your document text has left the country to be vectorised regardless of where the resulting vectors are stored. Nothing about your content is used to train a third-party model. Your Microsoft 365 tenant remains the system of record and this layer holds a derived index alongside it. Worth noting separately: Microsoft offers Advanced Data Residency as a paid add-on covering core Microsoft 365 workloads, and exactly what that covers for AI processing is worth confirming directly with Microsoft or your reseller, as the commitments are repositioned periodically.
Technically yes, through the same Microsoft Graph application registration, and both are opt-in rather than included by default. Teams channel messages are often valuable because decisions get made in channels and never make it into a document, so a question about why an approach was chosen frequently only has an answer in Teams history. Outlook is more delicate. Mailboxes contain personal correspondence, HR matters and legally privileged material, and staff have a reasonable expectation that mail is not searchable by colleagues. Where we do include Exchange, it is usually shared and functional mailboxes such as accounts or enquiries rather than individual mailboxes, and permission trimming applies exactly as it does to SharePoint. We would rather scope Outlook out at the start and add it deliberately later than have a staff member discover their mail was indexed without a conversation first.
The index tracks change continuously rather than rebuilding on a schedule. Microsoft Graph delta queries and change notification subscriptions push new, modified and deleted items into the pipeline, so a typical update is re-chunked, re-embedded and retrievable within minutes of being saved. Deletions matter more than most people expect: if a document is removed from SharePoint or a permission is revoked, that change has to propagate or the AI keeps answering from content that no longer exists or is no longer permitted. We handle tombstones explicitly and treat permission changes as index events in their own right. On top of the delta stream we run scheduled reconciliation passes, because change feeds occasionally miss events during throttling or service interruptions, and a corpus that silently drifts out of sync is a quiet correctness problem rather than a loud one.
Eight to twelve weeks from scoping engagement to production is typical for a mid-sized tenant, and the distribution of that time surprises people. Roughly the first two to three weeks are tenant and permission assessment, which is where most of the real complexity surfaces: broken inheritance, orphaned sites, archives nobody will make a decision about. Ingestion and indexing is fast, often a couple of weeks. Validating permission trimming and tuning retrieval against real questions takes longer than building the pipeline did, and it is the part we will not compress, because an AI that is fast and wrong is worse than no AI. Tenants that take longer usually do so for governance reasons rather than technical ones, most often waiting on decisions about which sites are in scope. If your permission structure is clean and scope is a single well-defined corpus, the shorter end is realistic.
Find Out What Your Microsoft 365 Tenant Is Actually Sitting On
Start with a tenant and permission assessment. You get a map of your site collections, document volumes and permission exposure, and a straight answer on whether a private AI layer, Copilot, or both is the right call for your organisation. Call us on +61 3 9999 7398 or send through your details to get started.