Skip to content

Latest commit

 

History

3 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 

Repository files navigation

AuditAI: AI Workflow Auditing Platform

A multi-tenant SaaS platform that helps IT auditors and GRC professionals run scoped governance audits of organizations' AI/ML systems against the frameworks examiners and regulators actually use: NIST AI RMF, ISO/IEC 42001, OWASP LLM Top 10, the EU AI Act, MITRE ATLAS, and the Federal Reserve's SR 11-7 model-risk guidance.

This is a public showcase. The source code is private. This repository documents the architecture, security posture, and capabilities of the platform for recruiters, hiring managers, and industry peers. Screenshots use seeded demonstration data, not real client engagements.


What it is

Most "AI governance" tools are runtime observability platforms that watch live model traffic. AuditAI is the other half of the market: audit engagement management, the system of record an auditor uses to plan, execute, and deliver a governance audit. Think AuditBoard or Workiva, purpose-built for AI/ML risk.

The platform takes an auditor from engagement scoping, to AI inventory discovery, to control testing against multiple frameworks, to findings, to remediation tracking, to client-ready deliverables, with assistive AI drafting the heavy-text artifacts (findings, executive summaries, threat models) under human review.

Engagement overview: maturity scoring, risk heatmap, control coverage

Engagement overview: per-domain maturity radar across NIST AI RMF / ISO 42001 / ISACA AAIA / NIST AI 600-1, a priority-by-severity risk heatmap, and control-coverage rollups. (Seeded demo engagement.)


Why it's interesting

It was built by a Tech auditor and cybersecurity expect who also engineers, so the governance model and the codebase are designed together rather than bolted on. The hard parts aren't the CRUD; they're the controls that make the platform's own evidence trustworthy to a tier-one financial-institution risk reviewer.

Framework coverage

160+ controls across eight catalogs, with per-control crosswalks so a single test maps to its equivalents in every framework:

Framework Scope
NIST AI RMF 1.0 Govern / Map / Measure / Manage
NIST AI 600-1 Generative AI profile
ISO/IEC 42001:2023 AI management system, Annex A
OWASP LLM Top 10 (2025) LLM application security
EU AI Act Regulation (EU) 2024/1689: tiering, Annex III, Art. 4/5
MITRE ATLAS Adversarial threat landscape for AI systems
Fed SR 11-7 / OCC 2011-12 Model risk management (FI examiner standard)
US state laws CO AI Act, NYC LL144 (AEDT), CA AB-2013, etc.

AI-generated framework citations are validated against the canonical control catalogs before they reach the auditor. Hallucinated control IDs are stripped, never surfaced.

Evidence integrity: the part regulators care about

The platform's audit trail is engineered to be trustworthy even against a fully-privileged platform operator:

  • Database-enforced immutability: Postgres BEFORE UPDATE/DELETE/TRUNCATE triggers reject mutations to the audit log; application bugs can't rewrite it.
  • Per-row SHA-256 hash chain: each row carries the previous row's hash; a daily verifier walks the chain and raises a security incident on any break.
  • External Merkle-root anchoring: each day's audit activity is hashed into a Merkle root and emitted to a customer-owned sink (their S3 bucket, HTTPS collector, or public log). Tampering then requires also forging the customer's own storage, which is outside the operator's reach.
  • Offline three-witness verification: a customer can independently prove "my Day-N evidence was not retroactively rewritten" by reconciling three sources: a verifiable CSV export, the platform's anchor record, and the copy in their own sink.

Bring Your Own Key (BYOK)

Envelope encryption with adapters for AWS KMS, GCP Cloud KMS, and Azure Key Vault. A per-blob data-encryption key is wrapped by the customer's key in their own KMS. When the customer disables that key, the platform loses the ability to read their data, and the system fails closed (it refuses to fall back to a platform key), which is the actual property a regulated buyer is purchasing.

Tenant isolation (defense in depth)

Two independent layers: application-level auditorOrgId filters and PostgreSQL Row-Level Security with FORCE ROW LEVEL SECURITY on every table, verified continuously in CI so policies can't drift from the schema.

AI safety

  • User input wrapped in delimited blocks to blunt prompt injection
  • Per-tenant monthly AI cost ceilings (fail-safe 429 before runaway spend)
  • A nightly eval suite: ground-truth scoring on workflow classification plus schema-validation evals on every other AI route, published to a scorecard
  • Adversarial prompt-injection test suite across all assistive-AI surfaces

Compliance + privacy

  • SOC 2 evidence pipeline: controls mapped to Trust Service Criteria (CC1-CC9, A1, C1), with a weekly evidence-pack snapshot written into the tamper-evident log.
  • OSCAL System Security Plan export: machine-readable control evidence for federal / FedRAMP-adjacent procurement diligence.
  • GDPR Art. 17 erasure: soft-delete plus 30-day grace plus PII-scrub cascade that preserves the audit chain.
  • Separation of duties: an auditor cannot verify their own completed remediation task (four-eyes control).

Architecture

Next.js 15 (App Router, RSC)  ::  React 19 / TypeScript
        |
        |- Clerk            authentication + org management
        |- Prisma           ORM over PostgreSQL (Supabase)
        |     \- Row-Level Security + immutability triggers + hash chain
        |- Anthropic Claude assistive drafting (cost-capped, eval-gated)
        |- Inngest          background jobs (anchoring, verification, sweeps)
        |- Cloudflare R2     evidence storage (versioned, signed-URL access)
        |- Stripe           subscription billing + trial gating
        \- KMS adapters     BYOK envelope encryption (AWS / GCP / Azure)

Observability: Sentry (errors), PostHog (product analytics)
Hosting: Vercel (push-to-deploy)

See docs/architecture.md for a deeper walk through the integrity and isolation design.


Engineering practices

  • ~550 automated tests (Vitest unit + Playwright E2E)
  • CI/CD: typecheck, lint, test, and production build gate every PR
  • Security scanning: CodeQL (SAST), Trivy (dependency CVEs), gitleaks (secret scanning), npm audit, on every PR plus a weekly cron
  • Dependabot for grouped dependency + GitHub Actions updates
  • DB-validation CI: migrations + RLS + audit-chain verifier run against a live Postgres service container on schema-touching PRs
  • Push-to-deploy via Vercel's native Git integration

Tech stack

Frontend Next.js 15, React 19, TypeScript, Tailwind CSS, Radix UI, Recharts Backend Next.js API routes / Server Actions, Prisma, PostgreSQL AI Anthropic Claude (Opus / Sonnet / Haiku, ZDR) Infra Vercel, Supabase, Cloudflare R2, Inngest Auth / Billing Clerk, Stripe Crypto AES-256-GCM, SHA-256 Merkle trees, AWS/GCP/Azure KMS envelope encryption Quality Vitest, Playwright, CodeQL, Trivy, gitleaks


About

Built and maintained by William Asare, IT auditor and GRC engineer (CISA, CISM). The platform reflects a deliberate thesis: the people who understand audit methodology and the people who build audit tooling are usually different people, and the gap shows in the product. This closes it.


All figures and engagements shown are seeded demonstration data. No real client or proprietary information appears in this repository.

About

Multi-tenant AI governance audit platform: NIST AI RMF, ISO 42001, EU AI Act, SR 11-7. Tamper-evident audit trail, BYOK, SOC 2 evidence pipeline. Built by an IT auditor (CISA/CISM). Public showcase; source private.

Topics

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors