Observation & Detection Guide

Overview

Key Concepts

How It Works

1. Session Observer

The Session Observer captures the structural fingerprint of each Claude Code session. It hooks into two lifecycle events: session start and session end.

Session Start

When a session starts, the observer caches metadata into .planning/patterns/.session-cache.json. The cached data includes:

• sessionId — The unique identifier for this session
• transcriptPath — Path to the JSONL transcript file Claude Code writes to
• cwd — The working directory at session start
• source — How the session was initiated: startup (fresh), resume (continuing), clear (after /clear), or compact (after context compaction)
• model — Which AI model is active (e.g., claude-opus-4-6)
• startTime — Unix timestamp for duration calculation

This cache is needed because the session end event does not carry start-time information. By caching at start, the observer can calculate accurate session durations at end.

Session End: The Full Pipeline

When the session ends, the observer executes a multi-step pipeline:

Step 1: Parse the transcript. The TranscriptParser stream-reads the JSONL transcript file line by line. Each line is a JSON object representing one event in the session — a user message, an assistant response, a tool call, or a tool result. The parser extracts tool call entries (type tool_use), filters out sidechain entries from subagent Task calls (which would pollute the observation with sub-task noise), and skips corrupted lines gracefully. From the parsed entries, it extracts:

• File paths from Read, Write, and Edit tool calls (via tool_input.file_path or tool_input.path)
• Command patterns from Bash tool calls (the first word of the command string, e.g., vitest from vitest run src/)
• Tool usage counts across all tool types using a Map<string, number>
• Tool execution pairs matching each tool_use entry with its corresponding tool_result, by tool_use_id or sequential ordering

Step 2: Summarize. The PatternSummarizer compresses the parsed data into a SessionObservation — a compact JSON object containing:

• Session timing (start, end, duration in minutes)
• Session metadata (source, end reason, active skills)
• Metrics: user messages, assistant messages, tool calls, unique files read, unique files written, unique commands run
• Top commands, files, and tools — the N most frequent items in each category, sorted by count

Step 3: Rate limit check. The ObservationRateLimiter checks whether this session ID has already produced a write recently. If so, the observation is dropped to prevent flood writes during rapid session cycling (common during development and testing).

Step 4: Promotion evaluation. The PromotionEvaluator examines the observation's metrics and decides whether it deserves persistent storage or should go to the ephemeral buffer. Significant observations — those with high tool-call counts, diverse file patterns, or long durations — are written directly to sessions.jsonl via the PatternStore with their tier set to persistent. Lightweight observations are written to .ephemeral.jsonl via the EphemeralStore with tier set to ephemeral.

Step 5: Anomaly detection. The observer runs detectAnomalies() on the observation, which checks for unusual patterns that might indicate data corruption or system misbehavior. Detected anomalies are logged as warnings to the console (INT-04).

Step 6: Ephemeral promotion. After writing the current observation, the observer evaluates all accumulated ephemeral entries for batch promotion. The ObservationSquasher aggregates all ephemeral entries into a single composite observation by merging their topCommands, topFiles, and topTools arrays (union, sorted by cross-entry frequency) and summing their metrics. This squashed observation is re-evaluated by the PromotionEvaluator. If the combined signal is strong enough, the squashed entry is promoted to persistent storage. The ephemeral buffer is cleared regardless of the promotion outcome.

Step 7: Retention pruning. The RetentionManager prunes the persistent JSONL file, removing entries older than maxAgeDays (default 30) and keeping only the most recent maxEntries (default 100). Pruning uses atomic writes — writing surviving entries to a temp file in the OS temp directory, then renaming the temp file over the original — to prevent corruption if the process crashes mid-write.

2. Pattern Analyzer

The Pattern Analyzer reads the accumulated session observations and identifies recurring patterns that are candidates for skill generation. It operates as a streaming frequency analysis engine with recency weighting and co-occurrence tracking.

Streaming Analysis

The analyzer processes the JSONL file using Node.js readline with createReadStream, processing one line at a time. This streaming approach maintains constant memory usage regardless of file size — even a JSONL file with thousands of entries will only have one entry in memory at a time during counting.

Each parsed line can be either a raw SessionObservation or a pattern envelope ({ timestamp, category, data }). The analyzer handles both formats transparently by checking for the data wrapper field.

Frequency Maps

For each session, the analyzer updates three frequency maps:

• Commands map (Map<string, number>): Counts sessions containing each command, excluding common developer commands like git, npm, node, npx, pnpm, yarn, bun, ls, cd, cat, echo, pwd, mkdir, rm, cp, mv, grep, find, which, head, tail, wc, sort, uniq, curl, wget, ssh, and scp.
• Files map (Map<string, number>): Counts sessions containing each file path.
• Tools map (Map<string, number>): Counts sessions containing each tool, excluding common tools (Read, Write, Edit, Bash, Glob, Grep, Task).

Alongside the frequency counts, the analyzer maintains three metadata structures:

• Co-occurrence map (Map<string, Set<string>>): For each command or tool key, tracks which files appeared in the same session. This data is used to generate file-based trigger patterns in the generated skills.
• Session timestamps (Map<string, number[]>): For each pattern key, records the timestamps of sessions where it appeared. Used for recency calculations.
• Session IDs (Map<string, string[]>): For each pattern key, records which distinct sessions contained it. Used for evidence tracking.

Candidate Extraction and Confidence Scoring

After building frequency maps, the analyzer extracts candidates that exceed the configured threshold (default: 3 occurrences). Each candidate receives a confidence score computed from two components:

• Base confidence: Math.min(occurrences / 10, 0.7) — a pattern needs 10 occurrences to reach the frequency maximum of 0.7. This prevents over-weighting patterns that appear many times in a short burst.
• Recency boost: Math.min(recentCount / occurrences, 0.3) — up to 30% additional confidence if the pattern was seen within the recency window (default 14 days). A pattern that appeared 5 times, all within the last 2 weeks, gets the full 0.3 boost. A pattern that appeared 10 times but only 2 were recent gets a smaller boost of 0.06.

The final confidence is capped at 1.0: Math.min(baseConfidence + recencyBoost, 1.0).

Each candidate is packaged as a SkillCandidate object with a generated ID (cmd-vitest, tool-notebook-edit), the pattern type, occurrence count, confidence score, auto-generated name and description, and a PatternEvidence block that records first-seen and last-seen timestamps, up to 10 session IDs, up to 5 co-occurring files, and co-occurring tools.

The description is generated using an activation-friendly "Use when" pattern following Claude Code skill documentation conventions. For example: "Workflow for running vitest commands. Use when running vitest commands, setting up vitest, editing vitest.config.ts or pattern-analyzer.test.ts."

Candidates are sorted by confidence descending and capped at maxSuggestions (default 10).

3. Skill Generator

The Skill Generator transforms a SkillCandidate into a complete SKILL.md scaffold — a markdown file with YAML frontmatter that Claude Code can load as an active skill. The generation process involves name sanitization, metadata construction, body generation, and six safety validation passes.

Name Sanitization

The suggested name from the candidate is sanitized for use as a directory name: converted to lowercase, non-alphanumeric characters replaced with hyphens, consecutive hyphens collapsed, leading and trailing hyphens stripped, and the result truncated to 64 characters. This prevents path traversal attacks and ensures cross-platform compatibility.

Metadata Construction

The generator builds YAML frontmatter with the following fields:

• name: The sanitized skill name
• description: The auto-generated description with "Use when" activation patterns
• allowed-tools: Inferred from the candidate type and pattern via inferAllowedTools() (SEC-07). Command-type candidates typically get Bash, Read, and Edit. Tool-type candidates get the specific tool they reference.
• argument-hint: Generated if the body contains $ARGUMENTS placeholders (SPEC-02)
• context: Set to fork if the candidate describes a research or analysis workflow (SPEC-05), causing it to run in a subagent context
• agent: Suggested agent name if context forking is enabled
• metadata.extensions.gsd-skill-creator: Extension block with enabled: true, trigger configuration, and default threshold of 0.5

Trigger configuration varies by candidate type:

Body Generation

The body is assembled from a template containing: the skill name as an H1 heading, a Purpose section with the description, a Pattern Evidence section with formatted evidence data (first seen, last seen, session count, common files, common tools), placeholder Guidelines and Examples sections for user customization, and a footer noting the auto-generated origin.

If GSD is installed, the injectGsdReferences() function adds cross-references to relevant GSD commands in the skill body (QOL-04).

Safety Validations (6 Passes)

Before the scaffold is returned, the body passes through six safety validation layers:

1. Dangerous command scanning (SEC-05): Scans the body for patterns matching dangerous shell commands (rm -rf /, sudo, DROP TABLE, chmod 777, and others). Detected commands are blocked and a warning comment is prepended to the body.
2. Content sanitization: The sanitizeGeneratedContent() function strips or escapes potentially harmful content that might have leaked through from corrupted pattern data.
3. Injection risk checking (SPEC-07): Detects if $ARGUMENTS appears inside !command preprocessing directives. This combination could enable shell injection attacks. High-risk findings trigger a warning comment.
4. Context fork detection (SPEC-05): Analyzes the description and body for keywords indicating research or analysis workflows. If detected, sets context: fork in the metadata to run the skill in an isolated subagent.
5. GSD reference injection (QOL-04): Adds references to relevant GSD slash commands when GSD is installed alongside skill-creator.
6. Progressive disclosure (DISC-01/DISC-02): If the generated body exceeds 2000 words, the ContentDecomposer splits it into a main SKILL.md (kept under the character budget) and separate reference files and executable scripts placed in references/ and scripts/ subdirectories.

Scaffold vs Save

The generator returns a GeneratedSkill object containing the name, metadata, body, and any decomposed reference and script files. It does not write to disk. The scaffold is presented to the user for review and confirmation. Only when the user explicitly accepts via createFromCandidate() does the skill get persisted through the SkillStore, with decomposed files written to the skill directory.

4. Pattern Discovery CLI

The Pattern Discovery CLI provides manual, on-demand pattern scanning through the discover command. While the observer-analyzer pipeline runs automatically during sessions, the CLI lets you explicitly search for patterns across all your projects at any time.

Project Enumeration

The command enumerates project directories under ~/.claude/projects/, finding all sessions.jsonl files across your Claude Code project history. Each project directory corresponds to a codebase you have used with Claude Code. The CLI collects all observation files and processes them through the same PatternAnalyzer used by the automatic pipeline.

Incremental Scanning

With the --incremental flag, the CLI creates watermark files alongside each scanned JSONL to track the byte offset of the last-processed position. On subsequent runs, only new entries (those appended after the watermark position) are processed. This makes repeated scans efficient even when observation files contain thousands of entries. Watermark files are small (<100 bytes) and safe to delete — the next scan will simply reprocess from the beginning.

Noise Filtering (Dual-Threshold)

The CLI applies a dual-threshold system that is more sophisticated than the PatternAnalyzer's single threshold. Candidates must meet both:

1. A minimum occurrence count (configured by --threshold)
2. A minimum cross-project occurrence count — the pattern must appear in at least 2 distinct projects to qualify

This dual filter separates genuinely useful patterns from project-specific noise. A command that appears 50 times in one project but nowhere else is less interesting than one that appears 5 times across 3 projects, because the latter suggests a generalizable workflow.

Deduplication

Discovered candidates are compared against existing skills in both .claude/commands/ (project-level) and ~/.claude/commands/ (user-level). Patterns that already have corresponding skills are filtered out, preventing duplicate suggestions. The deduplication matches on skill name slugs derived from the pattern.

CLI Flags Reference

npx skill-creator discover [options]

Options:
  --threshold N     Minimum occurrences to surface a pattern       [default: 3]
  --since Nd        Only scan sessions from the last N days        [default: 14d]
  --incremental     Use watermarks to skip already-scanned data    [default: false]
  --project PATH    Scan only a specific project directory         [default: all]
  --max N           Maximum number of candidates to return         [default: 10]
  --json            Output results as JSON instead of table        [default: false]
  --help            Show help text

Usage Examples

# View the last 5 observation entries
tail -5 .planning/patterns/sessions.jsonl | jq .

# Example output (one entry):
{
  "timestamp": 1708732800000,
  "category": "sessions",
  "data": {
    "sessionId": "abc-123-def",
    "startTime": 1708730000000,
    "endTime": 1708732800000,
    "durationMinutes": 47,
    "source": "startup",
    "reason": "clear",
    "metrics": {
      "userMessages": 12,
      "assistantMessages": 14,
      "toolCalls": 38,
      "uniqueFilesRead": 9,
      "uniqueFilesWritten": 4,
      "uniqueCommandsRun": 6
    },
    "topCommands": ["vitest", "npm", "tsc"],
    "topFiles": [
      "src/detection/pattern-analyzer.ts",
      "src/types/detection.ts",
      "vitest.config.ts"
    ],
    "topTools": ["Read", "Edit", "Bash", "Grep"],
    "activeSkills": ["beautiful-commits"],
    "tier": "persistent"
  },
  "_checksum": "a7f3c2e..."
}

# Count ephemeral entries waiting for promotion
wc -l .planning/patterns/.ephemeral.jsonl
# Output: 3 .planning/patterns/.ephemeral.jsonl

# View ephemeral entries
cat .planning/patterns/.ephemeral.jsonl | jq '.data.durationMinutes'
# Output:
# 4
# 2
# 7
# These are short sessions that individually were not significant enough
# for persistent storage. After a few more, the squasher will combine them.

# Discover patterns from the last week with a low threshold
npx skill-creator discover --threshold 2 --since 7d

# Output:
Pattern Discovery Results (scanned 23 sessions across 3 projects)
================================================================

  #  Pattern                Type      Count  Confidence  Description
  1  vitest                 command   8      0.82        Workflow for running vitest commands
  2  tsc --noEmit           command   6      0.71        Workflow for running tsc commands
  3  prisma migrate         command   5      0.65        Workflow for running prisma migrate commands
  4  NotebookEdit           tool      4      0.48        Guide for using NotebookEdit tool

4 candidates found. Run `npx skill-creator suggest` to review and accept.

# Same scan with JSON output for scripting
npx skill-creator discover --threshold 2 --since 7d --json

# Output:
{
  "sessionsScanned": 23,
  "projectsScanned": 3,
  "candidates": [
    {
      "id": "cmd-vitest",
      "type": "command",
      "pattern": "vitest",
      "occurrences": 8,
      "confidence": 0.82,
      "suggestedName": "vitest-workflow",
      "suggestedDescription": "Workflow for running vitest commands...",
      "evidence": {
        "firstSeen": 1707580800000,
        "lastSeen": 1708732800000,
        "sessionIds": ["abc-123", "def-456", "..."],
        "coOccurringFiles": ["vitest.config.ts", "*.test.ts"],
        "coOccurringTools": []
      }
    }
  ]
}

# Review pending suggestions
npx skill-creator suggest

# Accept a suggestion (generates SKILL.md scaffold)
npx skill-creator suggest --accept vitest-workflow

# The generated scaffold at .claude/commands/vitest-workflow/SKILL.md:
---
name: vitest-workflow
description: >-
  Workflow for running vitest commands. Use when running vitest commands,
  setting up vitest, editing vitest.config.ts or pattern-analyzer.test.ts.
allowed-tools:
  - Bash
  - Read
  - Edit
metadata:
  extensions:
    gsd-skill-creator:
      enabled: true
      triggers:
        intents:
          - vitest
          - vitest workflow
        threshold: 0.5
---

# vitest-workflow

## Purpose

Workflow for running vitest commands. Use when running vitest commands,
setting up vitest, editing vitest.config.ts or pattern-analyzer.test.ts.

## Pattern Evidence

This skill was suggested based on detected patterns:

- **First seen:** 2/10/2026
- **Last seen:** 2/24/2026
- **Sessions:** 8 occurrences
- **Common files:** vitest.config.ts, pattern-analyzer.test.ts
- **Common tools:** (none beyond standard)

## Guidelines

When working with vitest:

1. [Add step 1]
2. [Add step 2]
3. [Add step 3]

## Examples

```
# Example usage
```

---
*Generated from pattern detection. Edit this skill to customize for your workflow.*

# View all suggestions with their states
npx skill-creator suggest --all

# Output:
  #  State      Pattern            Confidence  Created
  1  pending    vitest             0.82        2026-02-20
  2  pending    tsc --noEmit       0.71        2026-02-20
  3  deferred   docker-compose     0.55        2026-02-15 (until 2026-02-22)
  4  dismissed  eslint             0.43        2026-02-10 (reason: "handled by editor")

# Defer a suggestion (will re-surface after 7 days)
npx skill-creator suggest --defer tsc-workflow

# Dismiss a suggestion permanently
npx skill-creator suggest --dismiss eslint-guide --reason "IDE handles this"

# First run: full scan (processes all entries)
npx skill-creator discover --incremental
# Output: Scanned 847 entries across 12 projects (first run, no watermarks)

# Second run: incremental (only new entries since last scan)
npx skill-creator discover --incremental
# Output: Scanned 23 new entries across 2 projects (watermarks used)

interface FrequencyMap {
  commands: Map<string, number>;           // command -> session count
  files: Map<string, number>;              // file path -> session count
  tools: Map<string, number>;              // tool name -> session count
  coOccurrences: Map<string, Set<string>>; // pattern key -> co-occurring files
  sessionTimestamps: Map<string, number[]>; // pattern key -> timestamps
  sessionIds: Map<string, string[]>;        // pattern key -> session IDs
}

const recencyMs = config.recencyDays * 24 * 60 * 60 * 1000;  // 14 days default
const recentCount = timestamps.filter(t => (now - t) < recencyMs).length;
const recencyBoost = recentCount > 0 ? Math.min(recentCount / occurrences, 0.3) : 0;
const baseConfidence = Math.min(occurrences / 10, 0.7);
const confidence = Math.min(baseConfidence + recencyBoost, 1.0);

# Check ephemeral buffer size
wc -l .planning/patterns/.ephemeral.jsonl

# Check ephemeral buffer disk usage
du -h .planning/patterns/.ephemeral.jsonl

# Manual cleanup (safe to truncate at any time)
: > .planning/patterns/.ephemeral.jsonl

# In .gitignore
.planning/patterns/

Troubleshooting

Understanding the Observation Lifecycle

To summarize the complete lifecycle in one view: a Claude Code session produces a JSONL transcript. When the session ends, the transcript is parsed into structured entries, summarized into a compact observation, evaluated for significance, routed to either ephemeral or persistent storage, and then the ephemeral buffer is checked for batch promotion. Over time, the persistent store is pruned to stay within retention bounds. Separately, the Pattern Analyzer reads the persistent store and produces ranked candidates. The Skill Generator transforms top candidates into SKILL.md scaffolds for user review.

This entire lifecycle is designed to be invisible during normal work. You do not need to manage observations, trigger analysis, or clean up files. The system handles all of this automatically. The CLI commands (discover, suggest, status) exist for when you want to inspect or manually intervene, but they are optional.

The lifecycle also respects project boundaries. Observations from one project are stored in that project's .planning/patterns/ directory and do not leak into other projects. The CLI's cross-project discovery feature reads from all projects but generates suggestions in the current project's skill directory. User-level skills at ~/.claude/commands/ are the designated path for patterns that span projects.

No observations are being recorded

Symptom: sessions.jsonl does not exist or is empty after multiple sessions.

Causes and fixes:

• The .planning/patterns/ directory may not exist. The observer creates it automatically, but check file permissions. Run mkdir -p .planning/patterns to create it manually.
• The session may have been too short or empty. Sessions with zero transcript entries produce no observations. Verify by checking whether the Claude Code transcript file exists and contains entries.
• Rate limiting may be rejecting writes. If you are rapidly starting and stopping sessions (e.g., during testing), the rate limiter may throttle writes. Wait a few minutes between sessions.
• Check that the Session Observer hooks are wired correctly in the Claude Code hooks configuration. The observer needs both SessionStart and SessionEnd events.
• Verify disk space. The JSONL append operation will fail silently if the filesystem is full.

Pattern discovery returns no candidates

Symptom: npx skill-creator discover reports 0 candidates.

Causes and fixes:

• Not enough session data. The default threshold requires 3 occurrences. Lower the threshold: --threshold 2 or even --threshold 1 for exploratory scanning.
• All patterns may be common commands (git, npm, node, etc.) that are filtered out by design. This is correct behavior — these commands do not make good skill candidates on their own.
• The --since window may be too narrow. Try --since 30d or --since 90d to scan a wider range.
• If using --incremental, all entries may have been scanned in a previous run. Delete the watermark files and re-run.
• The dual-threshold filter may be excluding single-project patterns. If you work on only one project, try scanning that project directly: --project /path/to/project.

Ephemeral buffer growing without bound

Symptom: .ephemeral.jsonl keeps growing past hundreds of entries.

Causes and fixes:

• The promotion evaluation may be failing silently. Check console output for errors during session end.
• All observations may be classified as ephemeral (too lightweight to promote individually). This is normal for very short sessions. The squasher will eventually promote aggregated entries once the combined signal is strong enough.
• If the file is large and you want to start fresh, truncate it: : > .planning/patterns/.ephemeral.jsonl. This is always safe.

Generated skill has WARNING comments about dangerous commands

Symptom: The generated SKILL.md starts with <!-- WARNING: N dangerous command(s) were detected and blocked -->.

Causes and fixes:

• The Skill Generator detected potentially dangerous commands in the pattern data (e.g., rm -rf, sudo, shell injection patterns). These are automatically sanitized from the output for safety.
• Review the generated skill carefully. The warnings indicate the original pattern contained risky operations that were blocked.
• If the commands are legitimate for your workflow, manually add them to the skill body after generation. The safety system errs on the side of caution.

Checksum verification failures on read

Symptom: Observations seem to disappear or the analyzer processes fewer entries than expected.

Causes and fixes:

• Entries that fail checksum verification are silently skipped. This can happen if the JSONL file was manually edited or if there was a partial write during a process crash.
• Inspect the file for corrupted entries: cat .planning/patterns/sessions.jsonl | while IFS= read -r line; do echo "$line" | jq . >/dev/null 2>&1 || echo "CORRUPT: $line"; done
• Remove corrupted lines manually, or delete the file entirely to start fresh. Observations are non-critical data that will be rebuilt from future sessions.

Related Capabilities