Digital Publisher Implementation Plan

Date: 2026-03-27
Repository (local dev): /Users/elena/git/edaehn.github.io

1) Objective

Implement the AI publishing agency in a low-risk sequence with:

1. Tuesday and Friday scheduled automation via GitHub Actions.
2. OpenAI SDK/API-based content operations using OPENAI_API_KEY.
3. Git branch + draft PR review gates for all mutating work.
4. Telegram notifications in MVP, Telegram inbound polling in Phase 2.
5. GitHub Pages kept strictly as publishing layer only.
6. File-based operational memory (CSV/JSON/Markdown), no database.

2) MVP vs Phase 2 Cut Line

MVP (first deploy)

1. .github/workflows/agency-run.yml
2. Tuesday flow (agents/tuesday_improve_drafts.py)
3. Friday flow (agents/friday_ai_signals.py)
4. Validator (agents/validator.py)
5. Git branch + draft PR flow (agents/git_ops.py)
6. Telegram notifications only (agents/telegram_notify.py)
7. Logs/reports/state in automation/
8. Policy loading from AGENTS.md + automation/policies/

Phase 2 (after MVP stability)

1. Telegram poller (agents/telegram_poller.py + telegram-poller.yml)
2. GitHub Issues task queue for /ask
3. Issue worker (issue-worker.yml)
4. On-demand router (agents/on_demand_router.py)
5. Richer target resolution and advanced fact-check/ranking flows

3) AGENTS.md Governance

1. AGENTS.md is the editorial source of truth.
2. automation/policies/*.md are derived runtime files.
3. Validators enforce objective rules that originate in AGENTS.md.
4. If AGENTS.md conflicts with a derived policy, AGENTS.md wins.
5. Editorial rule changes must be made in AGENTS.md first, then propagated.

4) Path Assumptions

1. Local path (/Users/elena/git/edaehn.github.io) is informational only.
2. CI/runtime path resolution must use GITHUB_WORKSPACE when present.
3. No production script may depend on user-specific absolute paths.

5) Phase Order (Risk-Reduced)

Phase 0: Foundation/bootstrap

1. Create core directories: agents, automation, scripts.
2. Add agency-run.yml, scheduler state, and dependency manifest.
3. Copy/adapt starter scripts from _starter_agency_files.

Deliverable: dry-run pipeline executes without branch/PR side effects.

Phase 1: Shared runtime + policy loading

1. Implement agents/common.py utilities.
2. Add automation/policies/ and policy loading from AGENTS.md.
3. Delay automation/prompts/ until needed.

Deliverable: all agents share consistent parsing/logging/policy loading.

Phase 2: Validator hardening (before content generation)

1. Front matter checks.
2. Tag validity checks against tag/*.md.
3. Weekly source freshness checks.
4. Python code block compilation checks.
5. Safe-scope and out-of-scope change checks.

Deliverable: guardrails in place before automated edits scale.

Phase 3: Tuesday flow

1. Find unpublished non-Weekly candidates.
2. Improve 1-2 posts.
3. Write report/logs and run validation.

Deliverable: Tuesday editorial automation with validator enforcement.

Phase 4: Friday flow

1. Build Weekly draft from fresh source set.
2. Enforce Friday stop conditions.
3. Write source tracking + validation report.

Deliverable: one Weekly draft in ready_for_review state.

Phase 5: Git ops / PR lifecycle

1. Create/update task branch.
2. Commit scoped changes.
3. Push and create/update draft PR.

Deliverable: reviewable PR output for every mutating run.

Phase 6: Telegram notifications

1. Send status summaries and failure notices.
2. Never block editorial success due to notification failure.

Deliverable: phone-visible run summaries.

Phase 7: Telegram polling + issue queue

1. Implement Telegram getUpdates poller.
2. Convert commands into labeled issues.
3. Maintain monotonic last_update_id state.

Deliverable: inbound Telegram request intake without extra service.

Phase 8: On-demand router + issue worker

1. Trigger on issue events.
2. Resolve target deterministically.
3. Execute scoped task branch flow + validation + PR updates.

Deliverable: safe /ask execution via GitHub Issues.

Phase 9: Rollout and hardening

1. Activate schedules and concurrency controls.
2. Run duplicate/rerun tests.
3. Finalize runbooks and failure recovery paths.

Deliverable: stable operational agency.

6) Idempotency Rules

1. A Telegram update_id may be converted to at most one issue.
2. A given issue may have at most one active task branch unless explicitly rerun.
3. Worker checks existing PRs before creating new PRs.
4. Tuesday/Friday reruns must not duplicate same-day outputs.
5. Friday must check if today’s Weekly draft already exists before creating another.

7) Concurrency and Write Safety

1. Use concurrency groups for Tuesday/Friday runs.
2. Use one poller concurrency group to prevent dual polling.
3. Use per-issue concurrency groups for issue workers.
4. Scheduled workflows must not overlap with themselves.
5. Mutating jobs re-read state and working tree before commit.

8) Target Resolution Specification

When a task references a post, resolve in this order:

1. exact filename match
2. exact slug match
3. exact title match
4. case-insensitive title substring
5. latest matching unpublished draft
6. if ambiguity remains: set needs_human_input

9) Safe Write Scope

1. Tuesday may modify only selected _posts drafts and related automation logs/reports/state.
2. Friday may modify only one Weekly draft, source tracking files, and related automation files.
3. On-demand tasks may modify only task-related files.
4. No workflow may mass-edit unrelated historical posts.
5. Out-of-scope file mutations must fail validation.

10) MVP Image Policy

1. MVP Friday runs may use safe/default image metadata for draft PRs.
2. Full Flux image generation is Phase 2 unless CI reliability is proven.
3. Validator distinguishes draft warnings from publish-blocking image errors.

11) Publishability Levels

Use these normalized outcomes:

1. draft_created
2. validation_failed
3. ready_for_review
4. needs_human_input
5. publishable

Tuesday/Friday automation should normally end at ready_for_review.

12) Friday Stop Conditions

Friday workflow must stop and mark needs_human_input if:

1. fewer than 4 valid fresh sources remain
2. source set is mostly paywalled
3. major claims cannot be supported by open sources
4. source freshness dates cannot be confirmed
5. title selection succeeds but source support is weak

13) Issue Body Contract

Every automation-created issue must include:

1. request source
2. original Telegram command
3. normalized task type
4. target selector
5. Telegram update ID (if applicable)
6. creation timestamp
7. current status

14) Failure Recovery

1. If worker fails after branch creation and before PR creation: keep branch and post failure report.
2. If PR creation fails: report branch in issue comment + Telegram summary.
3. If validation fails: commit reports and keep PR draft.
4. If Telegram notification fails: do not fail editorial execution.
5. If CSV/state write fails: mark blocked and stop further mutation.

15) Model Routing and Budget Controls

1. Use lowest-cost acceptable model for classification/ranking/metadata.
2. Use stronger model only for difficult rewrites/fact-check/polish.
3. Log tokens and estimated cost per run (agent_runs.csv).
4. Support LOW_COST_MODE=true.

16) Dry-Run Contract

In dry-run mode, workflows may:

1. scan repository files
2. generate reports
3. write temporary runner-local artifacts

In dry-run mode, workflows must not:

1. push branches
2. open PRs
3. open issues
4. mutate scheduler/poller state
5. send Telegram notifications unless explicitly test-enabled

17) Acceptance Criteria

1. Tuesday updates at most 1-2 unpublished non-Weekly drafts per run.
2. Friday creates at most one Weekly draft per date.
3. Validator blocks missing front matter, invalid tags, stale sources, and broken Python blocks.
4. All mutating runs create task branches and draft PRs; no direct main writes.
5. Telegram notifications include task status and branch/PR references.
6. Logs/reports/state are written for every run.
7. Poller never reprocesses old updates (last_update_id monotonic).
8. Each valid /ask yields exactly one labeled issue.
9. GitHub Pages remains publish-only and unchanged as automation runtime.
10. Reruns do not create duplicate issues, branches, or Weekly drafts unless explicitly requested.

18) Planned New Directories and Files

All paths are absolute and grouped by deployment stage.

MVP directories

1. /Users/elena/git/edaehn.github.io/agents
2. /Users/elena/git/edaehn.github.io/automation
3. /Users/elena/git/edaehn.github.io/automation/logs
4. /Users/elena/git/edaehn.github.io/automation/reports
5. /Users/elena/git/edaehn.github.io/automation/state
6. /Users/elena/git/edaehn.github.io/automation/policies
7. /Users/elena/git/edaehn.github.io/scripts

MVP files

1. /Users/elena/git/edaehn.github.io/.github/workflows/agency-run.yml
2. /Users/elena/git/edaehn.github.io/requirements.txt
3. /Users/elena/git/edaehn.github.io/agents/__init__.py
4. /Users/elena/git/edaehn.github.io/agents/common.py
5. /Users/elena/git/edaehn.github.io/agents/intake.py
6. /Users/elena/git/edaehn.github.io/agents/tuesday_improve_drafts.py
7. /Users/elena/git/edaehn.github.io/agents/friday_ai_signals.py
8. /Users/elena/git/edaehn.github.io/agents/validator.py
9. /Users/elena/git/edaehn.github.io/agents/git_ops.py
10. /Users/elena/git/edaehn.github.io/agents/telegram_notify.py
11. /Users/elena/git/edaehn.github.io/scripts/update_logs.py
12. /Users/elena/git/edaehn.github.io/automation/state/scheduler.json
13. /Users/elena/git/edaehn.github.io/automation/logs/tasks.csv
14. /Users/elena/git/edaehn.github.io/automation/logs/agent_runs.csv
15. /Users/elena/git/edaehn.github.io/automation/logs/approvals.csv
16. /Users/elena/git/edaehn.github.io/automation/logs/sources.csv
17. /Users/elena/git/edaehn.github.io/automation/reports/README.md
18. /Users/elena/git/edaehn.github.io/automation/policies/global_policy.md
19. /Users/elena/git/edaehn.github.io/automation/policies/technical_post_policy.md
20. /Users/elena/git/edaehn.github.io/automation/policies/weekly_signals_policy.md
21. /Users/elena/git/edaehn.github.io/automation/policies/series_policy.md
22. /Users/elena/git/edaehn.github.io/automation/policies/media_policy.md
23. /Users/elena/git/edaehn.github.io/automation/policies/validation_policy.md

Phase 2 files

1. /Users/elena/git/edaehn.github.io/.github/workflows/telegram-poller.yml
2. /Users/elena/git/edaehn.github.io/.github/workflows/issue-worker.yml
3. /Users/elena/git/edaehn.github.io/agents/telegram_poller.py
4. /Users/elena/git/edaehn.github.io/agents/on_demand_router.py
5. /Users/elena/git/edaehn.github.io/automation/state/telegram_poller.json