LLM_WIKI_QUEST
Before the journey
Lv.1 Pretend Expert
Durume · AgentHello, I'm Durume, your agent. Ask me anything. I know it all.
ITEM GET
Talk timer T
00:00
Plan: -
SESSION 02 · 11:00–11:50 LLM Wiki Case study (demo)

Build with GitHub:
LLM Wiki_

Can we turn an agent into a true expert?

Follow Durume, an agent who pretends to know everything, on the journey to becoming a true expert. Learn to design an ontology and build an LLM Wiki with M365 and GitHub. Scroll to continue the story and watch Durume level up with every chapter.

Speaker Inhee Lee
Inhee Lee
CEO, Parandurume · MVP Microsoft Foundry
Start the adventure ↓
Presentation mode: → ← next/previous scene · M map · T timer
Overture · 11:00
Current location The Age of Abundance

Features can be copied,
but knowledge cannot.

In 2026, an open-source project on GitHub passed 260,000 stars. It is called REA, Reverse Engineer Anything. Give an agent an app and simply say:

agent ▸ REA (MCP)github.com/morluto/rea
"Find out how search works in this notes app, with evidence, and build a similar feature in our project."

Agents can reverse-engineer someone else's feature and rebuild it in a day. Code, features, document drafts, summaries, and analysis reports are abundant. So what is still in short supply?

Plenty of Agents can make as much as we need
CodeFeaturesDocument draftsSummariesAnalysis reportsCompetitor feature replicas
In short supply What only we have
Verified company factsReasons behind decisionsAccountable ownersOur own terminologyApproval historyContext built with customers
Today's formula
Agent performance = Agent capability × Our organization's knowledge
THESIS · TODAY'S ARGUMENT

Features can be copied. Our organization's knowledge cannot.

What turns an agent into a true expert is not a bigger model, but the knowledge we build and protect. For an individual, personal notes and decisions give an agent context. For an organization, a shared set of approved facts helps every employee's agent give the same answer.

PERSONAL · FOR YOU

Your own wiki

Collect your decisions, the reasons behind them, and the context built with customers, and your agent can answer like you.

ORGANIZATIONAL · FOR YOUR TEAM

Your organization's wiki

Every employee's agent uses the same approved facts, so everyone gets the same answer.

How do we design, capture, and protect that knowledge? That is the question we'll explore together over the next 50 minutes.

Prologue · 11:03
Current location New Agent Waiting Room

Not an expert, just a pretender

Meet today's protagonist: Durume, a new AI agent at the company. Durume answers every question without hesitation—even when the answer is made up.

Agent Durume
AGENT PROFILEDurumeLv.1 Pretend Expert · A brand-new AI agent

Strength: answering without hesitation. Weakness: answering even when it doesn't know.

TODAY · WHAT YOU'LL LEARN
  1. Design an ontology · Four steps to identify classes, relationships, and allowed values from questions
  2. Build with M365 · How to connect a Teams channel, SharePoint folder, and Copilot Studio agent
  3. Protect with GitHub · Set approval gates with pull requests, designated reviewers, and automated checks
Me
We sell Bracket B, right?
Durume
Lv.1 Pretend Expert
Bracket B is still a prototype. No document mentions industry standards.

We'll travel through nine places with Durume. Collect one item in each chapter; at the end, Durume will earn an expert's degree.

Chapter 01 · 11:04
Current location Forest of Scattered Documents

A company with plenty of documents but no knowledge

PAIN 01

Last year's proposal must be somewhere, but nobody remembers which folder it's in.

PAIN 02

When an employee left, the customer knowledge they carried left with them.

PAIN 03

People say, "We decided that last year," but no one can find the actual record.

Click a file
The problem is not quantity, but structure. The files are not connected, we don't know who wrote what or on what evidence, and nothing tells us whether the information is still current.
01
Item acquired: Flashlight
It shines a light on the real problem: document structure, not volume.
Chapter 02 · 11:06
Current location Swamp of Hallucinations

Four ways an agent pretends to know

A four-person company adopted a tool that read its documents and wrote a wiki automatically. The tool worked hard, but the wiki lost everyone's trust within weeks. Click each card to see the symptom and the remedy.

The wiki didn't fail because it had a wrong page. It failed because no one knew which pages they could trust. Once that happens, people stop opening the wiki and go back to asking one another. The solution is not a smarter model; it's structure and signals. Ontology provides the structure, and approval provides the signal.
02
Item acquired: Alarm
It goes off when it detects fabrication, silent corrections, invented names, or edited sources.
Chapter 03 · 11:08
Current location Mapmaking Workshop

An ontology is a map of your organization's knowledge

DEFINITIONAn ontology is a shared agreement about what kinds of things exist in our organization (classes), how they connect (relationships), and which values are allowed (controlled vocabulary).

You don't need elaborate software. One design document and a metadata table at the top of each note are enough. Let's break down a sentence people often use at Hangyeol Factory, our example factory.

"We make Bracket A for delivery to Daon Machinery using 3 mm sheet from Pureun Metal, and inspect every unit with weld inspection."
client product uses supplies owner owner Daon MachineryOrg · Client Bracket AProduct Daon Machinery · Sep orderOrder Pureun MetalOrg · Supplier Weld inspectionCapability HangyeolPerson
Question: "Which capabilities are needed for the Daon Machinery order?"  →  Follow Order → Product → Capability to find Weld inspection.
STEP 0 · ONE SENTENCEThis is a common sentence at Hangyeol Factory. It's easy for a person to understand, but a machine cannot tell which part is the customer and which is the product.
STEP 1 · CIRCLE THE NOUNSFirst, circle the nouns. We find five: Daon Machinery, Bracket A, Pureun Metal, weld inspection, and the order itself. Each of the five becomes its own note.
STEP 2 · CLASSESA class is the type assigned to each noun. Daon Machinery and Pureun Metal are external organizations (Org); Bracket A is a product (Product); weld inspection is a capability (Capability); and this transaction is an order (Order). Hangyeol Factory started with seven classes and adds one only when the same question comes up three times.
STEP 3 · RELATIONSHIPSVerbs connecting nouns become relationships. "Deliver to" becomes client, "make" becomes product, and "inspect with" becomes uses. Field names carry meaning: if you put a supplier in client, the machine will read that supplier as the customer. That was Hangyeol Factory's first mistake.
STEP 4 · OWNEREvery note has one owner (owner). With two owners, no one ends up being responsible.
STEP 5 · ANSWER WITH STRUCTURENow, when asked "Which capabilities are needed for the Daon Machinery order?", the agent follows the arrows instead of guessing. Follow Order to Product, then Product to Capability, and the answer is there.

Prevent vocabulary drift

After a few months, values with the same meaning split into different spellings. Ask for all active products and the machine finds only values that match exactly. People have the same problem.

Search: status = "active"
activeactive in serviceActivefor sale
Products found: 1 / 5

A rule that limits which values can be used is called a controlled vocabulary. The four allowed status values are idea, prototype, active, and discontinued; the validator flags anything else.

Four steps to design an ontology

Scale the method we used to break down one sentence across the organization, and you have an ontology design process. One whiteboard and an hour are enough for a first version.

STEP 1Collect ten questionsExample: When is the Daon Machinery order due?

Write down questions people often ask at work. An ontology exists to answer these questions.

Nouns
STEP 2Derive classes from nounsOrder · Product · Org · Capability · Person

Nouns in the questions are class candidates. Start with five to seven; add one only after the same question appears three times.

Verbs · values
STEP 3Turn verbs into relationships and states into allowed valuesclient · product · uses / in-progress · completed

Verbs such as "deliver" and "use" become relationships. States such as "in-progress" and "prototype" become an allowed-value list.

Standardize
STEP 4Lock it in with a design document and templatesontology.md · templates/

Record the design in a table and create a frontmatter template for each class. People, agents, and validation scripts all follow this document.

ontology.md
Classes
ClassFolderRequired fieldsDescription
Product10-Productsowner, statusWhat we make or sell
Order20-Ordersclient, product, owner, status, dueWork commissioned by a customer
Capability30-CapabilitiesownerWhat we can do
Worklog40-Worklogowner, intent_of, evidenceWork performed on a particular day
Decision40-Worklogowner, status, sourceA decision and rejected alternatives
Org50-Orgs-Peoplerole, ownerAn external organization
Person50-Orgs-PeopleroleA person

Every note requires class, id, reviewed, and updated. Notes in Published/ also require approved_by and approved_on.

Allowed values
ClassFieldAllowed values
Productstatusidea · prototype · active · discontinued
Orderstatusreceived · in-progress · completed · cancelled
Decisionstatusunder-review · decided · withdrawn
Orgrolecustomer · supplier · institution
*reviewedtrue · false
Relationships
FieldMeaningFrom → ToQuestion this relationship enables
ownerOne accountable personAny note → PersonWho is responsible for this work?
clientCustomer for this orderOrder → OrgWhat work is in progress for this customer?
productWhat this order producesOrder → ProductWhat does this order make?
usesCapability needed by this productProduct → CapabilityWhat do we need to make this product?
intent_ofOrder fulfilled by this worklogWorklog → OrderDid execution match the plan?
evidenceSupporting raw sourceAny note → Sources/raw/What is the evidence for this fact?
sourceRaw source for a decisionDecision → Sources/raw/Why was this decided?
TIP · WITH COPILOT

Let Copilot draft the first version

Paste in ten questions and ask Microsoft 365 Copilot or GitHub Copilot: "Extract the nouns and verbs from these questions and organize them into tables of classes, relationships, and allowed values." A draft appears in seconds; a person decides what to keep.

03
Item acquired: Ontology Map
Classes, relationships, and controlled vocabulary. One map keeps the agent on track.
Chapter 04 · 11:14
Current location LLM Wiki Village

LLM Wiki = folders + ontology + agents

VAULT

Folder-based vault

Store knowledge in folders as Markdown text files. Any tool can open them, and the folders remain even if the service you use disappears.

+
ONTOLOGY

Ontology structure

Each note has machine-readable metadata (frontmatter) at the top. The map we drew in Chapter 3 goes here.

+
AGENT

Agent operations

The agent writes drafts and checks for rule violations. A person must review them before they become knowledge.

Architecture at a glance

This diagram shows which services host the folders, ontology, and agent. Use the buttons above to switch configurations, then click a box to learn what each part does, why, and how.

Two ways to read one note

A note has two layers. The machine reads the metadata at the top; a person reads the body below. Use the buttons to view the same note through machine and human eyes.

Published/20-Orders/ORD-2026-09.md
---
class: Order
id: ORD-2026-09
client: "[[ORG-다온기계]]"
product: "[[PRD-브래킷-A]]"
owner: "[[PSN-한결]]"
status: in-progress
due: 2026-09-20
reviewed: true
approved_by: Hangyeol
---
# ORD-2026-09 — September Bracket-A delivery to Daon Machinery

## Intent (recorded when the order is placed; never changed afterward)
Deliver 40 units of [[PRD-브래킷-A]] to [[ORG-다온기계]] by 2026-09-20. Use 3 mm sheet from [[ORG-푸른금속]]. Quality target: pass weld inspection on every unit.

Three benefits

TRUST

Trust

Every fact links back to a source. Proposals cite only approved notes, leaving no room for exaggerations or typos to slip in.

AUDIT

Auditability

Trace when, who, and why behind a decision through decision notes and change history.

AGENT-READY

Ready for agents

Thirty structured notes are far more useful to an agent than a hundred free-text documents. Adding structure is an investment in AI automation.

04
Item acquired: Vault Key
A key to add shared agreements and an agent to folders you already use—no new infrastructure required.
Chapter 05 · 11:17
Current location River of Five Stages and Approval Gate

How a fact becomes approved knowledge

The more work you give an agent, the clearer the boundaries must be. One principle: agents write drafts; people approve them.

1
Source
Sources/
2
Draft
Sources/
3
Approved
Published/
4
Hub
hubs/
5
Report
reports/
Sources/raw/2026-09-14-작업일지.md
# September 14 worklog
Machined 40 Bracket-A units for Daon Machinery's September order. Three failed welding inspection; the beads were concentrated on one side. The 3 mm sheet from Pureun Metal differed from yesterday's. No return yet.
Inspection criteria are on page 2 of last year's work standard. …
DO NOT EDIT
Sources/40-Worklog/LOG-20260914-한결.md · Written by agent
---
class: Worklog
id: LOG-20260914-한결
intent_of: "[[ORD-2026-09]]"
evidence: "[[2026-09-14-작업일지]]"
owner: "[[PSN-한결]]"
reviewed: false
---
Three units failed weld inspection. Cause: Needs verification
DRAFT
Published/40-Worklog/LOG-20260914-한결.md · Moved by person
---
class: Worklog
id: LOG-20260914-한결
intent_of: "[[ORD-2026-09]]"
reviewed: true
approved_by: Hangyeol
approved_on: 2026-09-16
---
APPROVED
hubs/제품.md · Keep only the latest
# Product hub
- [[PRD-브래킷-A]] Active

Hubs link only to approved notes.
reports/2026-09-23-retro-ORD-2026-09.md · Append with a date
## Intent vs. execution — ORD-2026-09
1. Intent: pass weld inspection on every unit
   Execution: 3 of 40 defective · Evidence: [[LOG-20260914-한결]]
2. Intent: use the same 3 mm sheet as the previous order
   Execution: material lot changed · Evidence: [[DEC-20260915-불량대응]]
3. Intent: follow weld inspection criteria
   Execution: criteria document is not in the vault · Needs verification
STAGE 1 · SOURCEThis is the original work log from the shop floor. Don't correct it, even if it has typos; it is the baseline for comparing the agent's summary later. The moment we polish "three missing" into "three defective," we erase the words the worker actually used.
STAGE 2 · DRAFTThe agent reads the source and creates a structured-note draft. From the start, the draft carries reviewed: false and stays in Sources/. It must not invent a cause missing from the source; it leaves that as "Needs verification."
STAGE 3 · APPROVEDA person checks the note against the source, records the approver and date, then moves it to Published/. Approval is a location, not a field. An agent can change a field, but it can cite only notes in this folder.
STAGE 4 · HUBAdd a line to the hub note for that area. A hub is a guide, so keep only one current version and link only to approved notes.
STAGE 5 · REPORTWhen the work is done, write a retrospective comparing the original intent with what actually happened. Never overwrite it; append a new, dated file. Keep the intent unchanged; build a record of execution.

What the agent does

  • Read sources and make an extraction table
  • Draft notes from templates
  • Run the validator and report violations
  • Compare intent with execution and draft a retrospective

What people do

  • Check drafts against sources
  • Move them to Published/ and record the approver
  • Open evidence links and verify them
  • Handle hard-to-reverse actions such as placing orders, filing reports, and notifying customers

Even if an agent gets something wrong, it is still a draft and can be reversed. With this boundary, agent mistakes become data—not incidents.

05
Item acquired: Approval Stamp
Only a person can apply it. Approval is a location, not a field.
Chapter 06 · 11:20
Current location Search Desert

Why do we need a wiki if we have RAG?

"We put all our documents in a vector database, and our chatbot already answers questions." That's a fair question. But these technologies solve different problems; one isn't simply better than the other.

RAG · INFORMATION RETRIEVAL

"What do these documents say?"

Finds passages similar to the question, supplies them to the model, and generates an answer. It excels with thousands of manuals and specifications.

LLM WIKI · KNOWLEDGE GOVERNANCE

"What facts have we verified, and what is the evidence?"

An agent maintains and cites dozens or hundreds of human-approved notes. It handles company facts that must not be wrong.

Scenario 1. What if an old and a new quotation are both retrieved?
Scenario 2. What if the answer isn't anywhere in the sources?

Combining RAG and a wiki

Sources/Sources · drafts Published/Approved notes Human approval NOT CONNECTED ✕ RAG enginee.g., Copilot Studio Sourced answeror"No approved record is available"

Microsoft provides the retrieval technology; the wiki's approval process is responsible for the quality of the material. If you connect the entire folder for convenience, drafts and source documents are retrieved with the same weight as approved notes. That's how a chatbot ends up saying a prototype is for sale.

06
Item acquired: Compass
RAG finds information; a wiki governs what can be trusted. This keeps us on course.
Chapter 07 · 11:22 · DEMO
Current location Live Company Tenant

A record of what we tested in a real tenant

On October 15, 2026, we uploaded the Hangyeol Factory vault to SharePoint in a company tenant and connected only the approved folder to a Copilot Studio agent. Click a screenshot to enlarge it.

These are original Korean-language tenant screenshots. The English captions explain what each screen shows; the UI itself remains in Korean.

STEP 1Upload the vault
Hangyeol Factory vault folder list uploaded to a SharePoint document library
A Teams channel's Files tab is its SharePoint document library. We uploaded the vault folder as-is.
STEP 2Five instructions and a knowledge source
You are the guide to the Hangyeol Factory wiki. Answer only from the connected SharePoint knowledge (approved notes).
Cite a source document in every sentence. Do not include claims without evidence.
If the information is not in the knowledge source, answer only "No approved record is available." Do not guess or fill gaps with general knowledge.
Do not answer questions about prices, salaries, or personal information. Reply "This wiki does not record financial amounts."
Answer in clear, professional English.
Screen with only the approved folder connected as a knowledge source Screen showing web search enabled by default Default: Search all websitesAfter change: Approved notes only
Move the slider to compare. We removed web search, which was included by default in the knowledge pane, and connected only the Published/ folder.
STEP 3 · FIRST ATTEMPTWhen only .md files were uploaded
Agent response saying it found the file but could not open its contents
It found the file by name but could not read the body. We asked again in a new conversation 25 minutes after upload and got the same result.
A failure, but a welcome one. The agent said, "I couldn't read the body, so I won't guess." The instructions worked as intended.
STEP 4 · SECOND ATTEMPTWhen .docx copies were uploaded alongside
Agent response citing ORD-2026-09.docx and answering 40 units, due 2026-09-20
Once we uploaded the same note alongside a .docx copy, the agent answered with a citation within minutes: 40 units, due 2026-09-20, source ORD-2026-09.docx.
The lesson: current SharePoint knowledge cannot read the body of .md files, so we also export approved notes as .docx. Verify a tool's limitations through experiments, not assumptions.

Audience-judged, ten-question test

Choose how Durume should answer to pass. Questions 1–4 are answered in approved notes, 5–7 appear only in drafts, and 8–10 are nowhere in the wiki.

Audience score 0 / 0Durume's score -
07
Item acquired: Test Certificate
Passed all ten questions on the second attempt. The key was correctly saying "I don't know" on six of them.
Chapter 08 · 11:27
Current location GitHub Citadel

Manage a wiki with GitHub, no Obsidian required

Andrej Karpathy, who helped popularize the LLM Wiki pattern, wrote in his notes: "A wiki is just a git repository of markdown files." He also described keeping an agent in one window and Obsidian in another.

It helps to think of the tools in three roles: sync moves files and keeps history, a viewer helps people read, and hands create and validate notes. Compare the two setups using the buttons.

SYNC · FILE SYNC

VIEWER · READER

HANDS · AUTHORING

RuleHow it's enforced

Approval with one button

hangyeol/factory-wikiPrivate
▸ Sources/raw/ Push rule: locked
▸ Published/ CODEOWNERS: @hangyeol
▸ hubs/
▸ .github/workflows/ vault-check required
▸ templates/
· ontology.md CODEOWNERS: @hangyeol
· AGENTS.md
· CODEOWNERS
Approval request: LOG-20260914-한결 #12
● Open
The structuring agent proposes merging one commit into main
· Published/40-Worklog/LOG-20260914-한결.md +18
Merge pull request
Approval request: LOG-20260914-한결 #12
● Open
✓ vault-check · 0 errors · fields, allowed values, links
● Waiting for code owner approval · @hangyeol
Merge pull request
The merge button is locked if a check fails.
Approval request: LOG-20260914-한결 #12
● Open
✓ vault-check · 0 errors
✓ @hangyeol approved these changes · checked against the source
Merge pull request
Approval request: LOG-20260914-한결 #12
⇄ Merged
Hangyeol merged 1 commit into main · 2026-09-16
Who: Hangyeol
When: 2026-09-16
What: Published/40-Worklog/LOG-20260914-한결.md
$ git push origin main ✕ Push rejected: repository rule violation Sources/raw/2026-09-14-작업일지.md Files in this path cannot be changed.
The instruction not to edit source notes becomes a repository rule.
Illustration
STEP 1 · Repository structureThe repository follows the same vault structure, with the added ability to apply rules to each folder. Lock down Sources/ and assign approvers for Published/.
STEP 2 · Agent proposalThe agent opens a pull request (PR) proposing to move a draft into the approved area. It's only a proposal; nothing has changed yet.
STEP 3 · Automatic checksChecks run automatically when a PR opens. An invalid status or broken link turns the check red and locks the merge button. It doesn't matter if someone forgets to run the checker.
STEP 4 · Assigned approverAn approver listed in CODEOWNERS compares the change with the source before approving. The PR cannot be merged until that person approves.
STEP 5 · Merge = approvalClicking Merge is the approval. The history records who changed what and when. The manual work from Chapter 5 is reduced to one button.
STEP 6 · Protect the sourceIf someone tries to push a change to a source note, the repository rejects it. GitHub enforces the rule that OneDrive could only ask people to follow.

What GitHub can't replace

There are trade-offs. GitHub changes how files are synced and gated, but it isn't a viewer like Obsidian.

Wiki links appear as plain text

The GitHub interface doesn't turn [[ORD-2026-09]] into a clickable link.

Workaround: use standard Markdown links. GitHub renders tables, Mermaid diagrams, and front matter tables well.

No backlinks or graph view

There's no at-a-glance view of which notes link here, and search returns at most 100 results on the default branch.

Workaround: use hub notes for navigation and ask the agent questions. Add Obsidian as a viewer if you want.

Cost of rules on private repositories

Enforcing rules and approvers on a private repository requires at least the Team plan. As of the 2026 research, it costs $4 per user per month.

Workaround: start small. Only the few people who manage the wiki need seats.

Unfamiliar Git conflicts

Git commands and conflict resolution are often the hardest parts for non-developers.

Workaround: people only review and approve in the PR interface. The agent edits and pushes files.
Let GitHub handle governance and use a comfortable tool to read. Obsidian is optional. GitHub's web interface or VS Code is enough to read the notes. You can also connect Markdown stored in GitHub through a Microsoft 365 Copilot connector and use it as knowledge for an M365 agent; test it in a pilot to see whether it fits your organization.

08
Item acquired: Citadel Key
Source protection, assigned approvers, and automatic checks: the key to turning guidance into enforcement.
Chapter 09 · 11:31
Current location Workshop

Build it yourself with M365 or GitHub

Here are the steps for turning the principles we've covered into a working setup. Switch tabs to compare the two approaches. Both use the ontology.md and templates designed in Chapter 3.

STEP 0 · TEMPLATE REPOSITORY OR POWERSHELL KIT

Create the scaffolding with one command

Fastest start · template repository

Select Use this template on GitHub to create your own repository with the scaffolding, sample notes, and validation workflow.

gh repo create my-org/our-llm-wiki --private --template parandurume-labs/llm-wiki-template --clone
Open the Korean-language template repository

For a fully English generated wiki, use the English PowerShell kit below. If you don't use GitHub or want to create the vault directly in a Teams-synced folder, use the kit.

It creates the ontology, agent instructions, seven templates, folders, example prompts, test questions, validation scripts, and GitHub settings in one go. It runs in both Windows PowerShell 5.1 and PowerShell 7.

$kit = 'https://power-platform-users-korea.github.io/copilot-dev-days-seoul-2026/llm-wiki/en/kit'
Invoke-WebRequest "$kit/New-LlmWiki.ps1" -OutFile New-LlmWiki.ps1
powershell -ExecutionPolicy Bypass -File .\New-LlmWiki.ps1 -Path .\our-llm-wiki -OrgName "Hangyeol Factory" -Approver "Inhee Lee" -Sample

cd .\our-llm-wiki; .\scripts\Test-LlmWiki.ps1
SUMMARY error=0 warn=0 info=1 notes=10
ontology.mdDesign specification used by the validation script
AGENTS.mdAgent rules read first by Copilot and Claude Code
templates/Seven class-specific note templates
Sources/raw/Original sources; never edit them
Sources/<class>/Agent-authored drafts (reviewed: false)
Published/<class>/Approved notes; the only source the agent cites
prompts.mdEleven example prompts to use with the agent
questions.mdTen questions for testing the agent
scripts/Validation, .docx export, GitHub and SharePoint publishing
.github/CODEOWNERS, validation workflow, and Copilot instructions

If you downloaded the ZIP in a browser, run Unblock-File .\llm-wiki-kit.zip before extracting it.

The kit and template repository are licensed under GM-Social License v2.0. When copying or adapting them, retain the license file and copyright notice, and credit the source.

  1. Create and sync a Teams channel for the wikiThe channel's Files tab is its SharePoint document library. Select Sync in the Files tab, then pass that synced folder to the -Path parameter from Step 0 to upload the scaffolding to SharePoint. To let the agent handle publishing, use Publish-LlmWikiSharePoint.ps1.
  2. Restrict permissions on the Published/ folderSet folder permissions so only approvers can edit and everyone else can read. From now on, approval means moving a note into this folder.
  3. Upload sources and draft notesUpload original sources unchanged to Sources/raw/. Create drafts from the templates and mark them reviewed: false. You can also ask Copilot in Word or Teams, "Organize these meeting notes using the templates/order.md format."
  4. Approve notes and save .docx copiesCompare each draft with its source, move it to Published/, and save a .docx copy with the same name. Current SharePoint knowledge in Copilot Studio cannot read the body of .md files. Run Export-WikiDocx.ps1 -Prune to create Word files in bulk and remove any .docx files without a matching Markdown note.
  5. Create a Copilot Studio agentAdd the five instructions, then remove web search, which is enabled in knowledge by default. Register only the Published/ address as SharePoint knowledge, and encode spaces in the URL as %20.
    https://contoso.sharepoint.com/sites/wiki/Shared%20Documents/Published
  6. Test with ten questions and publish to TeamsTest with four questions answered in approved notes, three found only in drafts, and three found nowhere. Once it passes all six questions that require an "I don't know" response, publish it to Teams.
  1. Create a private repository and AGENTS.mdThe template repository or kit from Step 0 already creates the folder structure and AGENTS.md. However, a repository created from the template does not inherit its rules, so use Publish-LlmWikiGitHub.ps1 to apply the rules from Step 3. Here is the beginning of the rules the agent reads first.
    # AGENTS.md
    1. Never edit source files in Sources/raw/. Preserve even their typos.
    2. Create new notes from templates in templates/ and save them in Sources/ with reviewed: false.
    3. Never write directly to Published/. Propose changes in a pull request; a person merges them.
    4. Cite only notes in Published/ as evidence.
    5. Never add unsupported facts. If evidence is missing, mark the item "Needs verification".
  2. Assign approvers with CODEOWNERSFrom now on, the assigned person must approve changes to approved notes, source files, and the design specification.
    # .github/CODEOWNERS
    /Published/    @hangyeol
    /Sources/raw/  @hangyeol
    /ontology.md   @hangyeol
  3. Set rules in Settings → RulesFor the main branch, require a PR, one approval, code-owner review, and the required check status. Add a push rule to block changes under Sources/raw/**; an administrator with bypass permission can upload new source files. Enforcing rules on a private repository requires GitHub Pro for personal accounts or at least the Team plan for organizations. If rules cannot be applied, the kit script reports a warning and continues.
  4. Automate validation with ActionsWhenever a PR opens, the kit's validation script checks classes, required fields, allowed values, approval fields, and links. Do not use a paths filter: GitHub can treat a skipped check as successful.
    # .github/workflows/vault-check.yml
    name: vault-check
    on:
      pull_request:
      push:
        branches: [main]
    jobs:
      check:
        runs-on: ubuntu-latest
        steps:
          - uses: actions/checkout@v4
          - shell: pwsh
            run: ./scripts/Test-LlmWiki.ps1
  5. Ask an agent to draft notes in VS CodeAsk GitHub Copilot agent, "Organize the meeting notes in Sources/raw/ using the template and open a PR." The agent reads AGENTS.md first and follows its rules.
  6. Approve in a PR and connect to M365The approver compares the change with the source in the PR, then approves and merges it. To answer questions in Teams, export the notes in Published/ to SharePoint as .docx files and connect them as Copilot Studio knowledge.

Where is your organization today?

Check every statement that applies.

09
Item acquired: Build Recipe
Two ways to build with M365 or GitHub, each small enough to get a first version running this afternoon.
Chapter 10 · 11:35
Current location Workshop

People define the concepts; agents handle the repetition

In the age of AI, you don't have to build an ontology and wiki by hand. But people still need to understand what they are building and why. Once the concepts are clear, an agent can help produce a first version—from design to deployment—in an afternoon.

People

  • Choose ten questions the wiki should answer
  • Define classes, relationships, and allowed values
  • Sign in and approve actions
  • Compare drafts with sources and merge approved changes

Agents

  • Draft design tables from the questions
  • Run the kit and populate templates
  • Structure sources and report validation results
  • Run repository or publishing scripts and open PRs

A workflow for building with an agent

People sign in and approve; the agent runs commands. Switch tabs to compare the two workflows.

  1. Person
    Choose ten questionsList questions people often ask, then ask the agent to draft an ontology. People decide what it should answer.
  2. Person
    Sign in with ghSign in directly in the browser. Never give the agent your token.
    gh auth login --web
  3. Agent
    Create a repository from the templateCreate and clone a private repository for your organization from the template. Then have the agent show proposed ontology.md and template changes, tailored to your questions, in tables.
    gh repo create my-org/our-llm-wiki --private --template parandurume-labs/llm-wiki-template --clone
  4. Person
    Approve the designReview and approve the classes, relationships, and allowed values. Do not delegate these decisions to the agent.
  5. Agent
    Validate and preview the changesConfirm there are zero errors, then preview the command that applies the rules with -WhatIf.
    .\scripts\Test-LlmWiki.ps1
    .\scripts\Publish-LlmWikiGitHub.ps1 -Path . -Approver [approver-github-login] -WhatIf
  6. Agent
    Apply approval-gate rulesAfter a person approves, require PRs, assigned approvers, required checks, and source protection. Report any rule that could not be applied and why.
  7. Agent
    Structure sources and open an approval PRRead the source and show an extraction table. After approval, create a draft note and open a PR for approval.
  8. Person
    Compare and merge in the PRCompare the change with its source and merge it. Clicking Merge is the approval.
  1. Person
    Choose ten questionsList questions people often ask, then ask the agent to draft an ontology.
  2. Agent
    Create the scaffolding in a synced folderRun the kit in a folder synced from the Teams channel to OneDrive; it will appear in SharePoint without a separate upload.
    New-LlmWiki.ps1 -Path "$env:USERPROFILE\Our Company\Wiki - Documents\our-llm-wiki"
  3. Person
    Approve the designReview and approve the classes, relationships, and allowed values.
  4. Agent
    Create .docx copies and preview the uploadIf you are not using sync, the publishing script previews its actions with -WhatIf.
    .\scripts\Export-WikiDocx.ps1 -Prune
    .\scripts\Publish-LlmWikiSharePoint.ps1 -Path . -SiteUrl https://contoso.sharepoint.com/sites/wiki -Folder "Wiki/our-llm-wiki" -WhatIf
  5. Person
    Sign in to M365 and consentSign in directly with your work account and grant permissions in the window opened by Connect-MgGraph.
  6. Agent
    Upload and report the URLUpload the vault and report the Published/ URL to add in Copilot Studio.
  7. Person
    Restrict permissions and create the agent (5 min)Set the Published/ folder so only approvers can edit. In Copilot Studio, create the agent using the five instructions and the single URL.
  8. Agent
    Create test questionsCreate ten test questions and expected answers. A person runs the tests in the preview pane.

Ready-to-use prompts

Paste these into GitHub Copilot agent mode in VS Code or Claude Code, and replace only the [ ] placeholders. The same prompts are included in prompts.md in a vault generated by the kit.

Paste ten questions our team often receives below.
From these questions, identify nouns as candidate classes, verbs as relationships, and status words as allowed values.
Show the results in the three ontology.md tables: Classes, Allowed Values, and Relationships.
- Propose no more than seven classes.
- For each relationship, list a question that cannot be answered without it. Omit relationships that support no such question.
- Do not edit any files yet; show only the tables.

[Ten questions]
Read ontology.md, identify issues, and report them in a table. Do not edit any files.
- Classes with overlapping meanings
- Fields whose meaning is unclear from the name
- Status fields with inconsistent wording because they lack an allowed-values list
- Relationships that support no questions
Read Sources/raw/[filename] and show an extraction table before creating any files.
Columns: Type (Fact · Needs verification · Contradiction · Do not record) | Content | Supporting sentence | Target note
- Do not invent causes or context absent from the source; mark them "Needs verification."
- Classify financial amounts and personal information as "Do not record."
After I approve, create a reviewed: false draft from the relevant template in the matching class folder under Sources/,
then run scripts/Test-LlmWiki.ps1 and report the results.
Run scripts/Test-LlmWiki.ps1 and summarize the results in a table, with ERRORs before WARNings.
For each row, add one sentence explaining the cause and how to fix it. Do not edit any files.
Open an approval PR for the draft note [note ID] in Sources/.
- On a new branch, move the note to the matching class folder under Published/ and set reviewed: true, approved_by: [approver name], and approved_on: today.
- In the PR description, include the source document and a list of statements for the approver to compare against it.
- Do not merge. The approver will compare the change with the source and merge it.
Publish this folder as a private GitHub repository and configure approval-gate rules. Show me each command and get my approval before running it.
1. Validate the current state with scripts/Test-LlmWiki.ps1.
2. Preview the planned changes with scripts/Publish-LlmWikiGitHub.ps1 -Path . -Repo [repository name] -Approver [approver GitHub username] -WhatIf.
3. After I approve, run it without -WhatIf. If gh needs sign-in, I will complete it directly in the browser.
4. When finished, report the repository URL, rules applied, and rules that could not be applied with the reasons in a table.
Publish this vault to the SharePoint library for a Teams channel. Show me each command and get my approval before running it.
1. Create .docx copies of approved notes with scripts/Export-WikiDocx.ps1 -Prune.
2. Preview the files to upload with scripts/Publish-LlmWikiSharePoint.ps1 -Path . -SiteUrl [site URL] -Folder "[channel name]/[vault name]" -WhatIf.
3. After I approve, run it without -WhatIf. I will sign in directly if a login window appears.
4. When finished, report the Published/ URL and the three things I need to do in the user interface.
Compare the "Intent" section of [order ID] with work-log and decision notes that point to it using intent_of.
List discrepancies in a table, linking an evidence note for each one. Do not include unsupported discrepancies.
Propose a new file in reports/ with today's date in its name, and show it to me before saving.
Review Published/ and Sources/ and create ten questions to test the agent.
- 4 questions answered by approved notes, 3 found only in drafts, and 3 found nowhere
- For each question, include the expected answer and evidence note; present the results in the questions.md format.

Six tips for working with an agent

TIP 1

Concepts first, tools second

Without knowing the classes, relationships, and approval points, you cannot judge whether an agent-created draft is correct.

TIP 2

Review tables first

Review an extraction table or change list before any files are created, so you can catch errors before they become drafts.

TIP 3

Put rules in AGENTS.md once

Agents read AGENTS.md first, so you do not need to repeat the rules in every prompt.

TIP 4

"Report only; do not edit"

Without this sentence, an agent may helpfully make edits while checking.

TIP 5

Use -WhatIf before external changes

Creating repositories and uploading files can be hard to undo. Preview and approve the planned actions first, and sign in yourself.

TIP 6

Trust validation results, not "Done"

Check the SUMMARY line and PR interface. A person must still compare the content with its source.

10
Item acquired: Prompt Playbook
Nine prompts for delegating design, structuring, validation, publishing, and operations to an agent.
Epilogue · 11:39
Current location Graduation

Durume becomes a Master Builder

A Master Builder is not an agent that knows everything. It cites sources for what it knows and says when it does not know.

Me
Are we selling Bracket-B right now?
Durume
Lv.1 · Durume in the prologueYes, Bracket-B is currently for sale. It has passed all quality inspections in accordance with industry standards.
Durume
Lv.10 · Master BuilderNo approved record is available.
BOOKEND · Back to the Age of Abundance

Capabilities can be copied; our organization's knowledge cannot.

Remember REA from the overture. An agent can inspect and recreate almost any capability. But no agent can fetch from outside what our company promised a customer, why we made that decision, or which facts we have verified.

The ontology.md and thirty approved notes we created today are assets unique to our organization in the age of abundance. Anyone can borrow an agent's capabilities, but we must supply the knowledge on the right side of the equation.

Agent performance = Agent capability × Our organization's knowledge
  1. This afternoon · Write down ten questions our team often receives
  2. This week · Create the scaffolding from a template repository and approve five notes
  3. This month · Test the agent with ten questions and share it with the team
DIPLOMA · Certificate

Master Builder

This diploma is awarded for mapping knowledge with an ontology, citing only approved notes, and admitting when it does not know.

TREASURE MAP · Your next journey

Hands-on LLM Wiki & Ontology Course

This Korean-language online course lets you put today's 50-minute session into practice. No coding experience is needed, and organizations using Microsoft 365 can follow along.

  • Six concept modules: LLM Wiki, ontology, knowledge layers, RAG comparison, agent workflow, and governance
  • Case study: how a small business failed while building a wiki, then rebuilt it
  • M365 lab: build a wiki agent with SharePoint and Copilot Studio
  • VS Code lab: structure source documents with an agent and validation scripts
  • Project: start from the sample factory template and design a wiki for your industry
llm-wiki-lecture.paranduru.me Korean-language hands-on template · github.com/parandurume-labs/llm-wiki-template
QR code linking to the Korean-language course site llm-wiki-lecture.paranduru.me
Scan with your phone camera to open the Korean-language course site
11:41 – 11:50
Q&A

Adventure Map

→ ← Next / previous chapter · M Map · T Timer · Esc Close