← Back to Explore
    NoteActive

    MLNNA Solution Implementation (original)

    Pipeline architecture from ingestion through AI synthesis, export, and dashboard presentation.

    BuilderEntrepreneur
    content:notesarchitecture

    Metadata

    Type
    Note
    Entity Type
    Standard
    Status
    Active

    Links

    Notes

    Source Summary

    Document Metadata

    • title: Solution Implementation
    • description: How the project implements portfolio strategy from ingestion through dashboard delivery
    • status: stable
    • lastUpdated: 2026-02-12
    • owner: Engineering # Solution Implementation ## Executive Summary The project solves the strategy problem with a staged pipeline: 1. Ingest GitHub + manual context into Notion.

    Imported Context

    Document Metadata

    • title: Solution Implementation
    • description: How the project implements portfolio strategy from ingestion through dashboard delivery
    • status: stable
    • lastUpdated: 2026-02-12
    • owner: Engineering

    Solution Implementation

    Executive Summary

    The project solves the strategy problem with a staged pipeline:

    1. Ingest GitHub + manual context into Notion.
    2. Synthesize repo and portfolio intelligence with structured AI outputs.
    3. Export a normalized snapshot.
    4. Render a static-data dashboard with health, trends, and drill-down views.
    5. Run the full loop on schedule through GitHub Actions.

    This architecture prioritizes reliability and repeatability over real-time complexity.

    System Architecture

    Canonical store: Notion

    The system bootstraps and maintains a Notion workspace model through src/dashboard/bootstrap_notion.py.

    Primary databases:

    • Projects
    • GitHub Activity
    • Conversations
    • Tasks
    • Goals
    • AI Runs

    Key implementation detail: schema bootstrap is idempotent and can add missing properties to existing databases.

    Ingestion layer: GitHub + CLI logging

    GitHub data comes through src/dashboard/github_client.py and is written via src/dashboard/sync_github_to_notion.py.

    Manual context is captured with:

    • scripts/log_convo.py for single conversations
    • scripts/log_convos.py for batch imports

    Both attach conversations/tasks to project records, preserving decision context next to repo telemetry.

    Intelligence layer: two AI passes

    Repo pass (src/dashboard/ai_synthesis.py):

    • builds per-repo evidence bundles
    • calls Responses API with strict JSON schema
    • synthesizes goals with status, priority, confidence, rationale, and evidence links
    • writes/upserts goals to Notion and records audit rows in AI Runs

    Portfolio pass (src/dashboard/ai_portfolio.py):

    • builds cross-project context from projects/goals/tasks/activity/conversations
    • infers purpose, phase, scope notes, top goal, and risk flag per project
    • returns portfolio-level focus insights and recommendations
    • writes strategic fields back to Projects

    Export layer: snapshot generation

    src/dashboard/export_snapshot.py produces a stable payload with:

    • projects
    • goals
    • activity
    • conversations
    • tasks
    • portfolio summary block

    Outputs are written to:

    • output/observable/snapshot.json
    • output/observable/snapshot.csv

    Optional Codex Wrapped enrichment is included when available.

    Presentation layer: React dashboard

    The app in apps/dashboard/ reads /data/snapshot.json and renders:

    • KPI strip
    • Portfolio insights
    • Project health cards
    • Weekly activity visualizations
    • Goals board
    • Activity feed
    • Project detail pages

    Implementation design choices:

    • snapshot-driven UI (no backend runtime)
    • modular component decomposition
    • derived health scoring via apps/dashboard/src/utils/healthScore.ts
    • time-range filtering and repo filtering in the client

    End-to-End Runtime Flow

    Local or scheduled pipeline order:

    1. python scripts/sync.py
    2. python scripts/ai_synthesize.py --apply ...
    3. python scripts/ai_portfolio.py --apply ...
    4. python scripts/export_snapshot.py
    5. python scripts/publish_dashboard_data.py

    Published dashboard data is copied into:

    • apps/dashboard/public/data/snapshot.json
    • apps/dashboard/public/data/snapshot.csv
    • apps/dashboard/public/data/snapshot.generated.json

    Automation and Deployments

    Scheduled orchestration

    .github/workflows/sync_export.yml runs three times per day and performs sync, AI, export, publish, and commit of refreshed snapshot files.

    Dashboard hosting

    • Vercel serves the React SPA from apps/dashboard/dist.
    • Runtime data path is /data/snapshot.json.

    Docs hosting

    • VitePress builds from DOCS/.
    • Docs site deploy target is Cloudflare Pages.

    Manual trigger path

    cloudflare/notion-sync-trigger/ provides a secure webhook-style trigger that dispatches the GitHub workflow and can be used by a one-click Notion action.

    Reliability and Quality Controls

    Idempotency patterns

    • Projects upsert by stable Key (owner/repo)
    • Activity upsert by typed deterministic keys (issue:..., pr:..., commit:..., etc.)
    • Goal upsert by stable goal keys

    Bounded external calls

    • caps on issues/PRs/commits/releases/workflow runs
    • sync window filters to reduce API and Notion pressure
    • retry/backoff in src/dashboard/http.py

    Test coverage

    Unit tests validate core behavior in tests/:

    • sync idempotency
    • conversation/task linkage
    • snapshot shape/export correctness
    • synthesis key stability and writeback behavior
    • bootstrap schema helper shapes
    • Codex Wrapped integration behavior

    Practical Constraints and Tradeoffs

    • Snapshot architecture is near-real-time, not live real-time.
    • AI quality depends on available context and token limits.
    • Strategic fields are inferential and may need manual adjustment.
    • Portfolio intelligence is strongest when both sync and goal synthesis run consistently before export.

    How This Solves the Original Problem

    Compared to a raw repo list, the implemented system provides:

    • structured context consolidation (activity + tasks + decisions)
    • explicit strategic interpretation (purpose, phase, goals, risks)
    • recurring focus feedback (portfolio summary and trend views)
    • a stable operational loop that keeps insight fresh without manual spreadsheet-style upkeep

    That is the core solution: a repeatable strategy pipeline, not just a dashboard UI.

    Provenance

    Source Extracts

    • excerpt-1
      --- title: Solution Implementation description: How the project implements portfolio strategy from ingestion through dashboard delivery status: stable lastUpdated: 2026-02-12 owner: Engineering ---
      Path: notion-project-dashboard-automation/DOCS/solution-implementation.md
    • excerpt-2
      The project solves the strategy problem with a staged pipeline:
      Path: notion-project-dashboard-automation/DOCS/solution-implementation.md
    • excerpt-3
      1. Ingest GitHub + manual context into Notion. 2. Synthesize repo and portfolio intelligence with structured AI outputs. 3. Export a normalized snapshot. 4. Render a static-data dashboard with health, trends, and drill-down views. 5. Run the full loop on schedule through GitHub Actions.
      Path: notion-project-dashboard-automation/DOCS/solution-implementation.md
    • excerpt-4
      This architecture prioritizes reliability and repeatability over real-time complexity.
      Path: notion-project-dashboard-automation/DOCS/solution-implementation.md
    • excerpt-5
      The system bootstraps and maintains a Notion workspace model through `src/dashboard/bootstrap_notion.py`.
      Path: notion-project-dashboard-automation/DOCS/solution-implementation.md