Community Forums & Groups — Consolidation Plan

Completion note (2026-04-14)

Steps 1–4 complete. The old 02-areas/community-forums-groups/ folder has been deleted; all content now lives under 02-areas/practice-platform/community-forums-groups/. check-wikilinks.py reports zero broken links in the migration-affected files. VPS cron re-enabled at the end of the session.

Deviations from the original plan, worth noting:

1. Genevieve deferral was not thread 43. The plan assumed the 2026-02-12 deferral-request manual file corresponded to synced DM thread 43. On inspection, thread 43 (2026-01-24/25) is about a platform bug (lesson progression), not the deferral — two entirely different conversations. The manual file was moved into direct-messages/ as a standalone editorial record with no thread-id injected. Four hand-merges became three; the deferral file stands alone.

2. Editorial preservation helper added to the sync script. The plan called for appending ## Editorial Notes sections to synced files, but the sync script regenerates content whenever a topic gets new replies or a DM gets new messages — which would silently erase editorial work. A preserve_editorial_notes() helper was added to scripts/forum-content-sync.py that splices any existing ## Editorial Notes section into the regenerated content before write_if_changed. Editorial notes now survive re-generation.

3. Pre-existing sync bug fixed. generate_topic_md was emitting [](/areas/practice-platform/community-forums-groups/groups/) as a wikilink for topics with no group (primer-community topics, group_id=0). Fixed by only emitting the group wikilink when group_slug is non-empty. Ten existing topic files were cleaned up in place.

4. .gitignore updated. Added __pycache__/, *.pyc, and scripts/_migration-map.json — the pycache dir and migration map file were auto-committed by the vault sync cron before I could untrack them.

See the “Verification Checklist” section below for final state.

Original session state (2026-04-13, paused overnight)

Where we are: Step 1 of the plan is done and committed. Step 2 is next. VPS cron is still paused. Resume here tomorrow.

What was done today:

1. Discovered and fixed script drift (Step 0 + Option A fix): The VPS was running ~/scripts/forum-content-sync.py (1069 lines, included DM sync + SQL-based activity sync + translation from 2026-04-09) but the repo was frozen at the 714-line pre-DM-sync version. Root cause: ~/scripts/ and /srv/baseworks/knowledge-base/scripts/ were two separate copies with nothing linking them.

   • Pulled the VPS running copies into the repo (captured in vault-sync commit 9a2233c).
   • Repointed the crontab on baseworks-agents from /home/patrick/scripts/forum-content-sync.sh to /srv/baseworks/knowledge-base/scripts/forum-content-sync.sh — one canonical location, edit-through-git.
   • Renamed the ~/scripts/ copies on the VPS to .bak-20260414-015531 as safety backups (forum-content-sync.py, forum-content-sync.sh, translate-community-content.py).
   • Documented the new pattern in 00-inbox/claude-code-shared-context.md under “Forum Content Sync (System 3)” (commit a6b1ed1).

2. Step 1 implemented (commit df7f972): All preparatory code changes to scripts/forum-content-sync.py:

   • Added helpers: truncate_slug, extract_descriptive_slug, derive_cohort, classify_topic_type, build_id_index, _yaml_escape.
   • generate_topic_md now writes cohort + topic-type frontmatter fields and YAML-escapes title/group/author.
   • generate_dm_thread_md now picks better titles ({FirstName} — {first clause} fallback when no real subject), writes title-auto-generated: true flag, adds cohort + topic-type fields, returns the full filename instead of a base.
   • All four sync functions (sync_topics_and_replies, sync_forums, sync_groups, sync_direct_messages) now use build_id_index for lookup and rename old-format files on next touch.
   • Important side effect: even without the Step 2 migration script, the next cron run after Step 2 would eventually rename everything to the new format just by touching each file. The migration script just does it all at once up front.

Cron state: PAUSED. Crontab line is prefixed with # PAUSED 2026-04-13 consolidation migration: */15 * * * * /bin/bash /srv/baseworks/knowledge-base/scripts/forum-content-sync.sh. Re-enable at the end of the migration by removing the prefix. Overnight forum activity will accumulate in BuddyBoss and be picked up on the first sync after re-enable.

Pickup plan for tomorrow:

1. git pull origin main — catch any overnight vault-sync auto-commits.
2. Read this section + the rest of the plan (Steps 2 onward).
3. Step 2-auto — write scripts/migrate-community-folders.py in dry-run-first mode. Script handles:
   • 2a: rename synced files in topics/ and direct-messages/ to new convention, using build_id_index from forum-content-sync.py.
   • 2c: move manual-only files into practice-platform structure (see table in Step 2c).
   • 2d: migrate _community-index.md + update internal wikilinks.
   • Records every action in _migration-map.json for reversibility.
4. Step 2-manual — three hand-merges via Edit (not scripted, because each needs editorial judgment):
   • mimi-touch-preferences ↔ thread-48 DM
   • any-movement-beyond-forms ↔ topic-21816 (exploring-movement-beyond-the-forms)
   • terminology-ignition-assimilation ↔ topic-21907 (definitions)
5. Step 2e — update external wikilinks (any-guay-continuing-education-certificate-inquiry.md and anything else found by python3 scripts/check-wikilinks.py).
6. Step 4 — delete 02-areas/community-forums-groups/ after wikilink check passes.
7. Verification — run check-wikilinks.py, compare file counts, dry-run python3 scripts/forum-content-sync.py --full on VPS (still with cron paused) to confirm no duplicates.
8. Re-enable cron on the VPS — edit crontab, remove the # PAUSED 2026-04-13 consolidation migration: prefix.
9. Update changelog in ~/Documents/baseworks-changelog/CHANGELOG.md covering the full session (drift discovery + fix, Step 1, migration).

Safe to revert at any point:

• git revert df7f972 → undoes Step 1 script changes
• git revert a6b1ed1 → undoes shared-context doc + cron-path rationale (but doesn’t undo the VPS crontab itself)
• On VPS, restore ~/scripts/ copies from .bak-20260414-015531 and change crontab path back to /home/patrick/scripts/forum-content-sync.sh
• Scripts on VPS /srv/baseworks/knowledge-base/scripts/ auto-restore via git pull after a git revert

---

Context

The vault has two overlapping community-forums-groups folders:

• 02-areas/community-forums-groups/ — manually curated (~7 files), rich metadata (cohort, topic-type, sensitivity, descriptive names like 2026-02-12-genevieve-deferral-request.md)
• 02-areas/practice-platform/community-forums-groups/ — auto-synced from BuddyBoss (83+ files), opaque naming (260124-genevieve-trépanier-thread-43.md), generic titles (“DM Thread 43”)

The auto-synced folder is the living one (updated by cron every ~15 min on the VPS). The manual folder stopped being updated in March 2026. We’re consolidating into the practice-platform location, improving naming and metadata in the process.

This plan extends the work done in forum-content-ingestion-plan, which established the automated sync (System 3).

---

Step 1: Update the Sync Script (scripts/forum-content-sync.py)

Goal: Future syncs produce well-named files and match by ID, not filename.

1a. Add build_id_index() helper

Scans a directory, reads each .md file’s frontmatter, returns {id_value: filepath} dict. Used by all sync functions to find existing files by their ID field (topic-id, forum-id, group-id, thread-id) instead of computing a filename and checking existence.

1b. Change topic filename format (line 571)

• Before: YYMMDD-{full-slug}.md → 260217-exploring-movement-beyond-the-forms-when-the-body-wants-to-investigate.md
• After: YYYY-MM-DD-{short-slug}.md → 2026-02-17-exploring-movement-beyond-forms.md
• Slug truncated to ~60 chars max, stop at word boundary

1c. Change DM filename format

• Before: YYMMDD-{participant-slug}-thread-{id}.md
• After: YYYY-MM-DD-{participant-firstname}-{descriptive-slug}.md
• Thread ID stays in frontmatter only
• Descriptive slug derived from first message content (first ~5 meaningful words)
• Add title-auto-generated: true flag for later human review

1d. Improve DM title generation

• Before: title: "DM Thread 43" or entire first message crammed in
• After: Short descriptive title: "Geneviève — Done Button Issue (Lesson 2.12)"
• Pattern: "{Participant} — {topic summary}"

1e. Add metadata fields to generated frontmatter

• cohort: auto-derived from group name → cohort mapping dict
• topic-type: keyword-based classification (soreness→intensity-modification, bug→technical-issue, etc.)
• These default to empty if not determinable — better than wrong

1f. Modify each sync function to use ID-based matching

• sync_topics_and_replies(): build index by topic-id, look up before creating
• sync_forums(): build index by forum-id
• sync_groups(): build index by group-id
• If an existing file is found by ID but has the old filename, rename it to the new convention and write updated content

---

Step 2: One-Time Migration Script (scripts/migrate-community-folders.py)

2a. Rename all existing synced files to new convention

For each file in topics/ and direct-messages/:

1. Read frontmatter for ID and created date
2. Compute new filename per the rules in Step 1
3. os.rename() (git will detect as rename)
4. Record mapping in _migration-map.json

2b. Merge overlapping files (3 confirmed overlaps)

Manual file	Synced file	Action
mimi-touch-preferences	thread-48 DM	Merge manual’s rich frontmatter (cohort, topic-type, sensitivity, language) into synced file. Append editorial sections as ## Editorial Notes.
any-movement-beyond-forms	topic-21816	Merge manual’s frontmatter fields. Append editorial context (group-feed origin note, bilingual separation).
terminology-ignition-assimilation	topic-21907 (definitions)	Merge manual’s frontmatter (lesson, segment, follow-up actions). Append editorial content.

2c. Move manual-only files into practice-platform structure

File	Destination	Notes
2026-02-12-genevieve-deferral-request.md	direct-messages/	Add thread-id: 43 so future syncs don’t duplicate
2026-02-16-sara-neck-pain.md	direct-messages/	No synced counterpart — personal coaching DM
study-group-access-codes-interview-followup.md	direct-messages/2026-02-19-access-codes-interview-followup.md	Add date prefix, mark as type: broadcast-template
2026-02-05-mimi-soreness-intensity.md	community-posts/	Group feed response with editorial treatment

2d. Migrate the index

• Move _community-index.md → 02-areas/practice-platform/community-forums-groups/_community-index.md
• Update all wikilinks inside it to point to new file locations
• Update community-forums-groups.md MOC to reference it and add direct-messages/ to the Structure section

2e. Update wikilinks across the vault

• External reference in any-guay-continuing-education-certificate-inquiry.md → update to point to new topics/ location
• Internal links within moved files: update relative paths
• Run python3 scripts/check-wikilinks.py to catch anything missed

---

Step 3: Update Shell Wrapper (scripts/forum-content-sync.sh)

Add direct-messages/ and community-posts/ to SYNC_PATHS on line 41 so future DM sync commits are captured.

---

Step 4: Delete Old Folder

Remove 02-areas/community-forums-groups/ entirely (all content either merged or moved). Verify with check-wikilinks.py first.

---

Order of Operations

1. Pause VPS cron — SSH in, comment out the cron entry, verify no sync running
2. Pull latest — git pull origin main on local
3. Update forum-content-sync.py with new naming/ID-matching logic (Step 1)
4. Run migration script locally (Step 2) — renames, merges, moves
5. Update shell wrapper (Step 3)
6. Update wikilinks (Step 2e)
7. Delete old folder (Step 4)
8. Run check-wikilinks.py — verify no broken links
9. Commit and push — single commit: “Consolidate community-forums-groups: ISO dates, descriptive names, merged metadata”
10. Deploy to VPS — git pull, test with python3 forum-content-sync.py --full
11. Verify — same file count, no duplicates, wikilinks intact
12. Re-enable cron

---

Verification Checklist

• python3 scripts/check-wikilinks.py — zero broken links
• No old-format filenames remain (no 2[0-9][0-9][0-9][0-9][0-9]-* pattern in topics/)
• python3 scripts/forum-content-sync.py --full on VPS — no duplicates, all matched by ID
• Obsidian graph view — community-forums-groups node connected, no orphans
• 02-areas/community-forums-groups/ no longer exists
• Changelog updated in ~/Documents/baseworks-changelog/CHANGELOG.md

---

Files to Modify

File	Change
scripts/forum-content-sync.py	ID-based matching, new filename format, DM titles, metadata fields
scripts/forum-content-sync.sh	Add direct-messages/ to SYNC_PATHS
scripts/migrate-community-folders.py	NEW — one-time migration
02-areas/practice-platform/community-forums-groups/community-forums-groups.md	Update Structure section
02-areas/practice-platform/community-forums-groups/direct-messages/index.md	Minor update for new naming
83+ synced files	Renamed
3 overlapping files	Metadata merged
4 manual-only files	Moved
1 external file	Wikilink updated
02-areas/community-forums-groups/	DELETED

---

Related

• forum-content-ingestion-plan — established the automated sync (System 3) that this plan extends
• community-forums-groups — the MOC file being updated