# AI Harness — repository-wide static audit

Pinned snapshot fetched: 2026-09-18T07:24:18.010228+00:00.

| Accounting | Count |
|---|---:|
| Baseline tracked files | 13,538 |
| Secondary mobile files | 2,708 |
| Total accounted paths | 16,246 |
| Text-discovery candidate files (all scopes) | 2,771 |
| Baseline product-text candidates | 1,473 |
| TS/JS/Python files parsed (all eligible scopes) | 8,615 |
| Parse-error files | 0 |
| Baseline product syntax invocation candidates | 151 |
| Curated prompt/source entries | 511 |
| Unique embedded source files | 508 |
| Finding-anchor files | 48 |

## Scope and what “exhaustive” means here

This is a **repository-wide static audit**, not just the original representative map.
Every tracked path in the six baseline Git trees is accounted for, plus a separately
reported mobile design-revamp tree. No new fetch was performed during this audit:
it uses the latest-fetched snapshot already approved by the user. Web is pinned to
`origin/dev-sunbeam`. Untracked working-tree files are not part of the audit.

**Exhaustive:** the Git-tree census, explicit inclusion/exclusion ledger, rule-based
text discovery over eligible UTF-8 files, and the recorded TS/JS/Python syntax passes.
**Not certified exhaustive:** semantic identification of every possible runtime prompt,
whole-program reachability, externally supplied instructions, deployed features, or
all live model requests. This distinction remains visible rather than replacing a
disclaimer with an unsupported “100% audited” claim.

## Method

1. Enumerate every pinned tree using `git ls-tree -r -z`; record mode, blob ID and size.
2. Read blob bytes via `git cat-file --batch`. Do not execute repository code.
3. Assign explicit dispositions. Never dereference tracked symlinks; record gitlinks.
   Sensitive environment/key files are metadata-only. Binary/vendor/generated files
   are not interpreted as application prompts. Test/eval/developer artifacts are separate.
4. Search eligible text using the published rules in `audit.py`, recording line numbers,
   not arbitrary configuration values. A match is a candidate, not proof of a model call.
5. Parse eligible TS/JS and Python with TypeScript and Python 3.12 ASTs. Record call
   expressions, imports and prompt-like declarations; distinguish syntax from comments
   and definitions. These passes are not interprocedural data-flow analysis.
6. Review important newly exposed systems and record evidence-backed findings. Expand
   the prompt explorer with missing code and the full selected runtime skill/instruction
   asset roots. The evidence level of each file is explicit; discovery is never labeled
   as manual line-by-line review.
7. Classify known human/UI prompts, bookkeeping, ACP input loops and model probes
   separately from model/agent invocation syntax. Wrappers and nested calls remain
   separate sites; the resulting counts are not unique prompts, requests or features.

## Open verification boundaries

- Live environment variables, tenant-installed skills, stored prompts, project settings,
  private session history and model/provider flags are not captured.
- SDK presets, native dependencies, remote AgentCore, workflow-service, Scribe/meeting
  services, Genie/provider internals and tenant-generated applications can add prompts
  outside these Git trees. A local append does not reveal the full external preset.
- `frontend/web/qa-aic` is a gitlink. Its pinned object is recorded; the submodule tree
  was not fetched or recursively audited. Its apparent QA purpose does not justify
  silently dropping it from the ledger.
- Generic transport/subprocess sites are inventoried, not all proven model-related or
  exhaustively traced through runtime aliasing/reflection. Other language files get text
  discovery, not TS/Python AST analysis. Negative matches are not semantic proof of absence.
- Runtime reachability, prompt quality, injection robustness, retention/deletion and
  production logging need authorized deployment inspection and test fixtures.

## Read the counts correctly

An instruction asset may be a reference, example, developer procedure or conditional
template. A prompt-like declaration may be a DTO/type/UI form. A model invocation site
may be a wrapper, embedding call, model probe or repeated path to the same prompt.
Nothing here is advertised as a count of all deployed features or live model requests.

## Findings and map corrections

### A01 — Gateway graph memory has four prompt modes.

**Omitted memory subsystem.** The earlier map omitted graph flush, dreaming, summary indexing and live-read instructions. These run through a bounded tool loop. Configuration gates the subsystem; SaaS local graph access is deliberately restricted. It is not Knowledge L1 or authoritative UMS hierarchy.

**Action:** Added a dedicated graph-memory walkthrough and memory-table row.

- [Graph memory · flush, dream, summary and read prompts](https://github.com/cdc-internal/sunshine-sunbeam/blob/8d960e8d97c0b5fd399d231647129198d031af68/src/graph/agent_kb/prompts.ts#L1-L231)
- [Graph memory · bounded tool loop](https://github.com/cdc-internal/sunshine-sunbeam/blob/8d960e8d97c0b5fd399d231647129198d031af68/src/graph/agent_kb/loop.ts#L1-L160)
- [Graph memory · enablement and SaaS backend gate](https://github.com/cdc-internal/sunshine-sunbeam/blob/8d960e8d97c0b5fd399d231647129198d031af68/src/graph/integration/config.ts#L1-L103)
- [Graph memory · runtime seams](https://github.com/cdc-internal/sunshine-sunbeam/blob/8d960e8d97c0b5fd399d231647129198d031af68/src/graph/integration/adapter.ts#L1-L581)

### A02 — Not all gateway recall is tool-initiated.

**Optional plugin behavior.** The optional LanceDB plugin can prepend retrieved memory at before_agent_start and capture short candidate text at agent_end. Its heuristics and embeddings are distinct from the topic/experience flush prompt. A blanket claim that all recall requires memory_search would be wrong.

**Action:** Documented as a conditional plugin path, not a proven live deployment.

- [LanceDB · optional auto-recall and capture hooks](https://github.com/cdc-internal/sunshine-sunbeam/blob/8d960e8d97c0b5fd399d231647129198d031af68/extensions/memory-lancedb/index.ts#L1-L590)
- [LanceDB · plugin defaults](https://github.com/cdc-internal/sunshine-sunbeam/blob/8d960e8d97c0b5fd399d231647129198d031af68/extensions/memory-lancedb/config.ts#L1-L114)

### A03 — Dashboard Discuss was only one of several AI paths.

**Previously missing prompts.** Direct chart generation/edit includes schema grounding, read-only SQL validation and bounded repair. Dashboard TL;DR has its own five-bullet system prompt. Genie remote chart follow-ups and the universal viewer Python NL-to-SQL prompt are separate again.

**Action:** Added each prompt builder and an end-to-end chart walkthrough.

- [Charts · generation, edit and repair prompts](https://github.com/cdc-internal/sunshine-sunbeam/blob/8d960e8d97c0b5fd399d231647129198d031af68/extensions/databricks-genie/src/generate-chart.ts#L1-L1255)
- [Charts · schema grounding and bounded candidates](https://github.com/cdc-internal/sunshine-sunbeam/blob/8d960e8d97c0b5fd399d231647129198d031af68/extensions/databricks-genie/src/schema-context.ts#L1-L929)
- [Dashboard TL;DR · separate five-bullet prompt](https://github.com/cdc-internal/sunshine-sunbeam/blob/8d960e8d97c0b5fd399d231647129198d031af68/extensions/aic_app_builder/src/dashboard-builder/dashboard-summarize.ts#L1-L291)
- [Genie · remote chart instructions and SQL follow-ups](https://github.com/cdc-internal/sunshine-sunbeam/blob/8d960e8d97c0b5fd399d231647129198d031af68/extensions/databricks-genie/src/genie-one-dashboard.ts#L1-L680)
- [Universal viewer · NL-to-SQL model prompt](https://github.com/cdc-internal/sunshine-sunbeam/blob/8d960e8d97c0b5fd399d231647129198d031af68/extensions/universal_dashboard_viewer/python/nl_to_sql.py#L1-L147)

### A04 — The app-coder append is not the full SDK prompt.

**Nested effective prompt.** The local SDK runner uses systemPrompt type=preset, preset=claude_code, plus an append, setting sources, a local plugin and persisted sessions. The preset implementation and live project settings are not contained in this snapshot. AgentCore is another remote boundary.

**Action:** Marked external preset/runtime dependencies explicitly; no invented flattened prompt.

- [App coder · SDK system append](https://github.com/cdc-internal/sunshine-sunbeam/blob/8d960e8d97c0b5fd399d231647129198d031af68/extensions/aic_app_builder/src/agent-sdk-slim-system-prompt.ts#L1-L312)
- [App coder · SDK preset, query, settings and persistence](https://github.com/cdc-internal/sunshine-sunbeam/blob/8d960e8d97c0b5fd399d231647129198d031af68/extensions/aic_app_builder/src/claude-agent-sdk-runner.ts#L1-L1158)
- [App coder · AgentCore request boundary](https://github.com/cdc-internal/sunshine-sunbeam/blob/8d960e8d97c0b5fd399d231647129198d031af68/extensions/aic_app_builder/src/agentcore-coder-runner.ts#L1-L323)
- [App coder · CLI harness prompt](https://github.com/cdc-internal/sunshine-sunbeam/blob/8d960e8d97c0b5fd399d231647129198d031af68/extensions/aic_app_builder/src/coder-backends/cli-harness-prompt.ts#L1-L117)

### A05 — Knowledge can rewrite queries and rerank results with a model.

**Context before answering.** The rewrite prompt preserves names/dates and forbids invented scope. Reranking requests a permutation of existing candidate indices; application parsing checks it. These are distinct calls from the final Ask answer and have separate preference resolution.

**Action:** Added retrieval-stage prompt inspection and a dedicated walkthrough.

- [Knowledge retrieval · query rewrite and reranking prompts](https://github.com/cdc-internal/sunshine-aic-knowledge/blob/6b5384c63036ea636325a4fd1d1dc59a70fe376c/src/lib/memory/ask-retrieve.ts#L1-L254)
- [Knowledge · feature-level model preference resolution](https://github.com/cdc-internal/sunshine-aic-knowledge/blob/6b5384c63036ea636325a4fd1d1dc59a70fe376c/src/lib/agent/llm-prefs.ts#L1-L60)
- [Knowledge · user model overrides](https://github.com/cdc-internal/sunshine-aic-knowledge/blob/6b5384c63036ea636325a4fd1d1dc59a70fe376c/src/lib/agent/llm-user-prefs.ts#L1-L125)

### A06 — Small helper and multimodal calls were missing.

**More model consumers.** Follow-up suggestions, session slugs, Matrix DM names, vision captions, image understanding, voice replies, TTS summary/translation and music generation are source-visible consumers. Speech and embedding requests are not all chat-style prompts.

**Action:** Added auxiliary and media walkthroughs; do not equate invocation sites with runtime request counts.

- [Follow-up suggestions · auxiliary model call](https://github.com/cdc-internal/sunshine-sunbeam/blob/8d960e8d97c0b5fd399d231647129198d031af68/src/auto-reply/reply/suggestion-generator.ts#L1-L211)
- [Session slug · naming sub-run](https://github.com/cdc-internal/sunshine-sunbeam/blob/8d960e8d97c0b5fd399d231647129198d031af68/src/hooks/llm-slug-generator.ts#L1-L96)
- [Matrix DM · display-name extraction](https://github.com/cdc-internal/sunshine-sunbeam/blob/8d960e8d97c0b5fd399d231647129198d031af68/extensions/matrix/src/matrix/monitor/dm-displayname-extractor.ts#L1-L152)
- [App coder · text-only model vision fallback](https://github.com/cdc-internal/sunshine-sunbeam/blob/8d960e8d97c0b5fd399d231647129198d031af68/extensions/aic_app_builder/src/coder-backends/vision-caption-fallback.ts#L1-L170)
- [Media understanding · default modality prompts](https://github.com/cdc-internal/sunshine-sunbeam/blob/8d960e8d97c0b5fd399d231647129198d031af68/src/media-understanding/defaults.ts#L1-L36)
- [Image tool · image prompt and model fallback](https://github.com/cdc-internal/sunshine-sunbeam/blob/8d960e8d97c0b5fd399d231647129198d031af68/src/agents/tools/image-tool.ts#L1-L438)
- [Voice call · conversation response prompt](https://github.com/cdc-internal/sunshine-sunbeam/blob/8d960e8d97c0b5fd399d231647129198d031af68/extensions/voice-call/src/response-generator.ts#L1-L171)
- [TTS · summary/translation prompts and speech egress](https://github.com/cdc-internal/sunshine-sunbeam/blob/8d960e8d97c0b5fd399d231647129198d031af68/src/tts/tts.ts#L1-L2215)
- [Music · generation prompt payload](https://github.com/cdc-internal/sunshine-sunbeam/blob/8d960e8d97c0b5fd399d231647129198d031af68/src/music/music.ts#L1-L255)

### A07 — Research carries multiple task prompts and a source ledger.

**Research orchestration.** The coordinator builds planner, worker, coverage, synthesis and verification prompts under workstream/source-policy budgets. Child-run initialization and tools supply additional context beyond each task string.

**Action:** Added the full coordinator source and research walkthrough.

- [Deep research · planner, worker, coverage, synthesis, verifier](https://github.com/cdc-internal/sunshine-sunbeam/blob/8d960e8d97c0b5fd399d231647129198d031af68/src/research/coordinator.ts#L1-L571)
- [Subagents · inherited context and session spawning](https://github.com/cdc-internal/sunshine-sunbeam/blob/8d960e8d97c0b5fd399d231647129198d031af68/src/agents/subagent-spawn.ts#L1-L397)

### A08 — Authored applications can themselves call models.

**Generated runtime boundary.** The generated-app template contains an LLM SDK accepting caller-supplied messages plus embedding-backed document search. A snapshot of the builder repository cannot enumerate prompts later authored by every tenant app.

**Action:** Separate generated application calls from the app-authoring agent.

- [Generated apps · caller-supplied messages through LLM SDK](https://github.com/cdc-internal/sunshine-sunbeam/blob/8d960e8d97c0b5fd399d231647129198d031af68/extensions/aic_app_builder/templates/webapp-template/packages/sunbeam-llm-sdk/src/index.ts#L1-L361)
- [Generated apps · document-search embedding client](https://github.com/cdc-internal/sunshine-sunbeam/blob/8d960e8d97c0b5fd399d231647129198d031af68/extensions/aic_app_builder/templates/webapp-template/server/src/search/embedding-client.ts#L1-L72)

### A09 — Runtime bootstrap templates live under docs/.

**Instruction-loading inventory.** Gateway workspace.ts reads docs/reference/templates. A path-only exclusion of every docs/ file would miss real runtime instructions. The expanded catalog includes that directory and all checked-in skill/instruction assets in the selected runtime roots, while development skills remain separately classified.

**Action:** Recorded a runtime-template exception in the census and added conditional asset badges.

- [Workspace files · identity, user, soul and memory](https://github.com/cdc-internal/sunshine-sunbeam/blob/8d960e8d97c0b5fd399d231647129198d031af68/src/agents/workspace.ts#L1-L854)
- [Workspace bootstrap · filtering and hooks](https://github.com/cdc-internal/sunshine-sunbeam/blob/8d960e8d97c0b5fd399d231647129198d031af68/src/agents/bootstrap-files.ts#L1-L104)
- [Skill discovery and prompt exports](https://github.com/cdc-internal/sunshine-sunbeam/blob/8d960e8d97c0b5fd399d231647129198d031af68/src/agents/skills.ts#L1-L53)

### A10 — Source-present is not the same as default or reachable.

**Legacy distinction.** Retained wiki digest/vault extraction/team composition and the reference experience-compaction prompt must not be silently counted as the default workspace pipeline. The native summary prompt remains the active summary source. Legacy route reachability is not certified by file presence.

**Action:** Kept reference/retained labels and exact source links rather than claiming deployed parity.

- [Retained wiki · document/activity digest prompt](https://github.com/cdc-internal/sunshine-aic-knowledge/blob/6b5384c63036ea636325a4fd1d1dc59a70fe376c/src/lib/wiki/digest.ts#L1-L58)
- [Retained vault · incremental extract instructions](https://github.com/cdc-internal/sunshine-aic-knowledge/blob/6b5384c63036ea636325a4fd1d1dc59a70fe376c/src/lib/wiki/vault-extract.ts#L1-L327)
- [Retained team compose · structured page prompt](https://github.com/cdc-internal/sunshine-aic-knowledge/blob/6b5384c63036ea636325a4fd1d1dc59a70fe376c/src/lib/team/compose-llm.ts#L1-L109)
- [Experience compaction · reference module; no src importer found](https://github.com/cdc-internal/sunshine-sunbeam/blob/8d960e8d97c0b5fd399d231647129198d031af68/src/memory/compaction-prompts.ts#L1-L197)
- [Native compaction · summary, update and split-turn prompts](https://github.com/cdc-internal/sunshine-sunbeam/blob/8d960e8d97c0b5fd399d231647129198d031af68/src/agents/aic-core-agent/session/generate-summary.ts#L1-L552)

### A11 — Injection wrappers prevent verbatim echo, not semantic influence.

**Trust boundary.** The aibot_internal wrapper is an outbound echo-stripping envelope. Its comments explicitly say the model still reads and acts on wrapped text. It must not be described as an authorization or prompt-injection isolation boundary.

**Action:** Documented the distinction between formatting/sanitization, prompt policy and application access controls.

- [Injected blocks · source wrappers](https://github.com/cdc-internal/sunshine-sunbeam/blob/8d960e8d97c0b5fd399d231647129198d031af68/src/agents/pi-embedded-helpers/injection-wrapper.ts#L1-L44)
- [External content · trust-boundary instructions](https://github.com/cdc-internal/sunshine-sunbeam/blob/8d960e8d97c0b5fd399d231647129198d031af68/src/security/external-content.ts#L1-L276)
- [Ask · private memory/persona capabilities](https://github.com/cdc-internal/sunshine-aic-knowledge/blob/6b5384c63036ea636325a4fd1d1dc59a70fe376c/src/lib/agent/features/workspace-context.ts#L1-L333)

### A12 — Prompt/request artifacts need a separate operational check.

**Operational privacy review.** The gateway has non-production system-prompt inspection files; Matrix DM name extraction logs its constructed request; CC has optional local-evaluation request artifacts. These are source-visible logging surfaces, not evidence that a particular deployment stores or exposes user data.

**Action:** Flagged for environment/retention review. No live logs or credentials were read.

- [Main agent · assembly, injection and session opening](https://github.com/cdc-internal/sunshine-sunbeam/blob/8d960e8d97c0b5fd399d231647129198d031af68/src/agents/pi-embedded-runner/run/attempt.ts#L1-L2965)
- [Matrix DM · display-name extraction](https://github.com/cdc-internal/sunshine-sunbeam/blob/8d960e8d97c0b5fd399d231647129198d031af68/extensions/matrix/src/matrix/monitor/dm-displayname-extractor.ts#L1-L152)
- [CC · provider payload, retries and optional input artifacts](https://github.com/cdc-internal/sunshine-command-center-service/blob/b4709582e9413bcf59401f388991b15c18a4d68d/apps/cc/pipeline/llm.py#L1-L687)

### A13 — Provider adapters and tenant/user preferences shape effective input.

**Configuration is part of context.** Native chat/Responses request builders, tenant model/runtime injection, Knowledge feature/user model preferences and CC stage context selection all affect the payload. Default templates alone cannot reproduce a live user-specific request.

**Action:** Added assembly/transport code to the explorer and retained explicit live-config gaps.

- [Provider input · native chat request assembly](https://github.com/cdc-internal/sunshine-sunbeam/blob/8d960e8d97c0b5fd399d231647129198d031af68/src/agents/aic-core-agent/llm/request-builder.ts#L1-L429)
- [Provider input · Responses request assembly](https://github.com/cdc-internal/sunshine-sunbeam/blob/8d960e8d97c0b5fd399d231647129198d031af68/src/agents/aic-core-agent/llm/responses-request-builder.ts#L1-L373)
- [Tenant settings · model and runtime injection](https://github.com/cdc-internal/sunshine-sunbeam/blob/8d960e8d97c0b5fd399d231647129198d031af68/src/tenancy/saas-gateway-agent-injection.ts#L1-L295)
- [Knowledge · feature-level model preference resolution](https://github.com/cdc-internal/sunshine-aic-knowledge/blob/6b5384c63036ea636325a4fd1d1dc59a70fe376c/src/lib/agent/llm-prefs.ts#L1-L60)
- [Knowledge · user model overrides](https://github.com/cdc-internal/sunshine-aic-knowledge/blob/6b5384c63036ea636325a4fd1d1dc59a70fe376c/src/lib/agent/llm-user-prefs.ts#L1-L125)
- [CC · per-prompt context allowlists and JSON repair](https://github.com/cdc-internal/sunshine-command-center-service/blob/b4709582e9413bcf59401f388991b15c18a4d68d/apps/cc/pipeline/stage_assembly.py#L1-L552)

## Reproduce

```sh
python3 docs-product/ai-harness/audit.py
node docs-product/ai-harness/audit-syntax.cjs
python3.12 docs-product/ai-harness/audit-python.py
python3 docs-product/ai-harness/build.py
python3 docs-product/ai-harness/build_audit.py
python3 docs-product/ai-harness/validate-audit.py
node docs-product/ai-harness/smoke.cjs
```