Understand Tenant Boundaries
Start by defining tenant boundaries (customers, departments, subsidiaries) and map each source system accordingly. Every Markdown file must carry tenant metadata so downstream services can enforce isolation. Recommended front matter:
--- tenant: acme-co classification: confidential retention: 365d source_system: sharepoint access_roles: - acme-admins - global-compliance ---
Conversion & Redaction Pipeline
- Ingest & Tag. Check documents into a staging bucket segregated by tenant. Attach metadata (tenant ID, owner, classification).
- Convert to Markdown. Use the MDConvert API with tenant-scoped credentials. Output Markdown plus extracted assets (images, attachments) into tenant-specific folders.
- Run PII/Secret Scans. Execute deterministic redaction rules and machine-learning scanners. Replace matches with tokens referencing secure vault entries.
- Approve via Pull Request. Commit Markdown to
tenants/{tenant}/docs. Require code-owner review from account managers and security.
Automate redaction logs: store masked value hash, original author, and reason for redaction so auditors understand the decision trail.
Immutable artifacts: once Markdown is approved, generate a signed hash stored in your compliance system.
Serving Content Safely
- Signed URLs: when serving Markdown to AI components, sign URLs with tenant scoping and short TTL.
- Row-Level Security: replicate tenant metadata into your vector database so embeddings are filtered server-side.
- Context Filters: injection layer should discard any chunk whose tenant doesn’t match the request token.
- Audit Journal: append read/write actions to an append-only log (CloudTrail, Azure Monitor) with tenant ID and user principal.