Pattern Discovery

What Is Pattern Discovery?

Pattern Discovery is the intelligence engine behind gsd-skill-creator. It watches how you work with Claude Code—the tools you use, the commands you run, the files you touch—and finds the recurring workflows buried in your session history. When it spots something you do often enough, it offers to turn that pattern into a reusable skill so Claude can learn from your habits.

Think of it as a code reviewer for your process, not your code. Instead of asking “is this function correct?” it asks “do you always run these three commands after editing a config file?” If the answer is yes, it creates a skill that encodes that workflow.

What It Does

  • Scans all your Claude Code session transcripts
  • Extracts tool sequences (Read → Edit → Bash)
  • Identifies recurring Bash command patterns
  • Clusters similar user prompts by meaning
  • Ranks candidates by frequency, recency, and breadth
  • Generates draft skills from top candidates

What It Does NOT Do

  • Read your code content or file bodies—only tool names and command prefixes
  • Send data anywhere—all processing is local
  • Auto-apply skills without your confirmation
  • Modify existing skills without the bounded learning guardrails (3 corrections minimum, 7-day cooldown, 20% max change)
  • Include subagent (sidechain) activity in analysis

Getting Started with Pattern Discovery

Prerequisites

  • gsd-skill-creator installed (npm install -g gsd-skill-creator or local install)
  • Claude Code session history in ~/.claude/projects/ (generated automatically when you use Claude Code)
  • At least a few sessions of working with Claude Code (the more history, the better the results)

Your First Pattern Discovery Session

Follow these steps to run your first discovery scan and understand what comes back.

Step 1: Preview What Will Be Scanned

Before committing to a full scan, use --dry-run to see what the scanner finds without processing any content:

$ skill-creator discover --dry-run

┌  Skill Discovery
│
◇  [DRY RUN] Scanned 247 sessions across 12 projects
│
◆  [DRY RUN] Would process: 83 new, 14 modified, 150 skipped, 0 excluded
│
└  [DRY RUN] Discovery complete (no content was read).

This tells you the scanner found 12 projects with 247 total sessions. Of those, 83 are new (never scanned before), 14 have been modified since the last scan, and 150 are unchanged. The dry run reads no session content—it only checks file metadata.

Step 2: Run the Full Discovery

$ skill-creator discover

┌  Skill Discovery
│
◇  Scanned 247 sessions across 12 projects
│
○  Scan summary: 12 projects, 97 sessions processed, 18 patterns found, 7 candidates ranked
│
○  Tool Pattern Candidates:
│
│   1  0.823  bigram   Read -> Edit                      4p  37 occ
│   2  0.756  trigram  Read -> Edit -> Bash               3p  24 occ
│   3  0.691  bash     git-workflow                       5p  19 occ
│   4  0.544  bigram   Grep -> Read                       2p  12 occ
│
◆  Select candidates to generate skills from (space to toggle):
│   ● Read -> Edit workflow (score: 0.823 | 4 projects | 37 occurrences)
│   ○ Read -> Edit -> Bash workflow (score: 0.756 | 3 projects | 24 occurrences)
│   ○ Git-workflow patterns (score: 0.691 | 5 projects | 19 occurrences)
│   ○ Grep -> Read workflow (score: 0.544 | 2 projects | 12 occurrences)
│
└  Generated 2 skill draft(s). Review and customize in ~/.claude/skills/

Step 3: Review Generated Skills

Selected candidates become draft SKILL.md files in ~/.claude/skills/. Each draft includes:

  • A Purpose section describing what the skill does
  • A Pattern Evidence section showing when and where the pattern was seen
  • A Guidelines section with TODO placeholders for you to customize
  • Trigger configuration so the skill activates at the right time
$ cat ~/.claude/skills/read-edit-workflow/SKILL.md

# read-edit-workflow

## Purpose

Guides Read -> Edit workflow. Use when reading and analyzing
and editing and modifying files.

## Pattern Evidence

This skill was suggested based on detected patterns:

- **First seen:** 1/15/2026
- **Last seen:** 2/22/2026
- **Sessions:** 10 occurrences
- **Common files:** config.json, package.json
- **Common tools:** Grep, Bash

## Guidelines

<!-- TODO: Add specific guidelines for this pattern -->

When working with Read -> Edit:

1. [Add step 1]
2. [Add step 2]
3. [Add step 3]

Step 4: Customize and Activate

Edit the SKILL.md to add your actual workflow steps. The TODO placeholders are where your domain knowledge goes. Once customized, the skill becomes part of your Claude Code toolkit—it will activate when the trigger conditions match your current session context.

Common Patterns the System Finds

Pattern discovery operates on two parallel pipelines. Here are the kinds of patterns each one surfaces.

Tool Sequence Patterns (N-grams)

The system extracts bigrams (2-tool sequences) and trigrams (3-tool sequences) from your session transcripts. These represent the order in which Claude uses tools during your sessions.

Pattern Type What It Means
Read → Edit Bigram You frequently read a file before editing it (inspect-then-modify pattern)
Read → Edit → Bash Trigram Read, edit, then run a command (edit-and-test cycle)
Grep → Read Bigram Search for something, then read the matching file (search-and-inspect pattern)
Glob → Read → Edit Trigram Find files by pattern, read them, then edit (bulk modification workflow)

Bash Command Patterns (Categories)

Bash commands are classified into categories based on the command prefix. The system groups related commands rather than tracking individual invocations.

Category Commands Included Example Skill
git-workflow git add, git commit, git push, git log, git diff Commit message conventions, branch naming
npm/build npm test, npm run build, npm install Build-and-test cycle automation
docker docker build, docker compose, docker run Container workflow patterns
test-runner vitest, jest, pytest, cargo test Test execution and failure handling

Prompt Clusters (Semantic Grouping)

The second pipeline collects your natural language prompts, generates embeddings, and clusters similar prompts using DBSCAN. This finds intent-based patterns—things you ask Claude to do repeatedly across sessions and projects.

For example, if you frequently ask variations of “add error handling to this function” or “write tests for this module,” the clusterer groups these into a single candidate that could become a skill for your error-handling or testing workflow.

How to Interpret Results

The Confidence Score (0.000 – 1.000)

Each candidate receives a composite score computed from four weighted factors:

Factor Weight What It Measures
Frequency 25% How often the pattern occurs (log2-scaled so 1000 occurrences does not overwhelm 10)
Cross-Project 30% Fraction of all projects where this pattern appears (higher = more universal skill)
Recency 25% Exponential decay with a 14-day half-life from the most recent session
Consistency 20% Fraction of total sessions containing this pattern (higher = more habitual)

Reading the score: A score of 0.800+ means the pattern is frequent, recent, cross-project, and consistent—a strong skill candidate. Scores between 0.500–0.800 are worth reviewing. Below 0.500, the pattern may be too niche or stale to justify a skill.

The Evidence Trail

Each candidate includes evidence showing exactly where and when the pattern was observed:

  • Projects — which projects contained the pattern (sorted alphabetically)
  • Sessions — up to 10 most recent session IDs (sorted by timestamp, newest first)
  • Total occurrences — raw count across all sessions
  • Example invocations — tool sequences as Read → Edit → Bash or bash category names
  • First/last seen — ISO timestamps from the earliest and most recent session

This evidence is embedded directly into the generated SKILL.md file so you can always trace a skill back to the sessions that inspired it.

When to Accept vs. Reject Discovered Patterns

Accept When

  • You recognize the pattern as something you genuinely do often
  • The pattern spans multiple projects (cross-project score is high)
  • You have specific guidelines to encode (not just “do this sequence”)
  • The pattern involves nuance Claude gets wrong without guidance
  • Score is above 0.600 with recent activity

Reject When

  • The pattern is too generic (e.g., “Read then Edit” with no special handling needed)
  • It appeared only in one project and is project-specific, not reusable
  • The pattern is stale—high frequency but last seen months ago
  • An existing skill already covers the same workflow (dedup should catch this, but review anyway)
  • The pattern represents framework boilerplate, not your personal workflow

If you are unsure, use the defer action. Deferred suggestions resurface in future discovery runs, giving the system more data to confirm or disconfirm the pattern. Dismissed suggestions are permanently removed from future results.

Audience Deep Dives

For Students: What Is Pattern Detection?

Pattern detection is the process of finding recurring structures in data. In everyday life, you already do this intuitively—you notice that you always check your phone after sitting down at a café, or that you always run git status before committing.

gsd-skill-creator automates this for your coding sessions. It reads through your session history as a stream of events (tool calls, commands, prompts) and looks for sequences that appear more often than chance would predict. The key concepts:

  • N-grams: A sequence of N consecutive items. A bigram is 2 items (Read → Edit), a trigram is 3 items (Read → Edit → Bash). These are borrowed from natural language processing where they analyze word sequences.
  • Frequency counting: Simply tallying how many times each n-gram appears. More frequent = more likely to represent a real habit.
  • Noise filtering: Some patterns appear everywhere because they are fundamental (like Read being the most common tool). The system filters these out using a dual threshold: the pattern must appear in 15+ projects AND in 80%+ of all projects to be considered noise.
  • Clustering: Grouping similar things together. The prompt clustering pipeline groups your natural-language requests by meaning, not exact wording, so “write tests for this” and “add unit tests here” end up in the same cluster.

For ML Practitioners: Under the Hood

The prompt clustering pipeline uses production ML techniques adapted for small corpora (20–500 prompts per project):

  • Embeddings: User prompts are embedded via bge-small-en-v1.5 (384-dimensional vectors). Embeddings are cached in a PromptEmbeddingCache with model-version keying to avoid recomputation.
  • DBSCAN: Density-Based Spatial Clustering of Applications with Noise. Unlike K-means, it does not require specifying the number of clusters upfront, and it naturally handles noise points (prompts that do not belong to any cluster). The implementation uses O(n²) region queries, which is acceptable for the expected corpus sizes.
  • Distance metric: Cosine distance (1 - cosine_similarity) between embedding vectors. This measures semantic dissimilarity: 0 = identical meaning, 1 = orthogonal, 2 = opposite.
  • Epsilon auto-tuning: Instead of a hardcoded epsilon, the system uses the k-NN knee method. It computes each point’s k-th nearest neighbor distance, sorts these distances, and finds the “knee” (maximum perpendicular distance from the line connecting the first and last points). The k-distance at the knee becomes epsilon, clamped to [0.05, 0.50]. A flat curve (max distance < 0.01) falls back to 0.30.
  • Scoring formula: score = 0.25×frequency + 0.30×crossProject + 0.25×recency + 0.20×consistency. Frequency is log2-scaled. Recency uses exponential decay with a 14-day half-life: recency = e(-ln2 × days / 14). Cross-project is the raw fraction of total projects. Consistency is the session fraction.
  • Deduplication: Candidates are compared against existing skills using both exact name matching and Jaccard similarity on extracted keywords (threshold: 0.50). A minimum-results guarantee ensures all candidates are returned if dedup would otherwise eliminate everything.

For System Administrators: Data Flow and Storage

Understanding where data lives and how it flows is critical for managing deployments:

  • Input: JSONL session files under ~/.claude/projects/{project-hash}/. These are Claude Code’s native transcript format. The scanner stream-parses them line by line to handle 23MB+ files without loading into memory.
  • State: Scan watermarks live in ~/.gsd-skill-creator/discovery/scan-state.json. This tracks which sessions have been processed and their file modification times for incremental scanning.
  • Observation layer: Session summaries (not full transcripts) are stored in .planning/patterns/sessions.jsonl. These go through a two-tier storage system: ephemeral observations are held in memory and periodically evaluated for promotion to persistent storage based on a multi-factor score (tool calls, duration, file activity, user engagement, metadata richness).
  • Output: Generated skills are written to ~/.claude/skills/{skill-name}/SKILL.md. Large skills (>2000 words) are decomposed via progressive disclosure into a main SKILL.md plus references/ and scripts/ subdirectories.
  • Embedding cache: Prompt embeddings are cached locally, keyed by model version. Cache invalidation is automatic when the model version changes.

Practical Examples

Example 1: Scan Only Specific Projects

If you only want to analyze patterns from particular projects, use the allowlist:

# Only scan two specific projects
$ skill-creator discover --allow=my-web-app,my-api-server

┌  Skill Discovery
│
◇  Scanned 89 sessions across 2 projects
│
○  Found 5 patterns, 3 candidates ranked

Example 2: Exclude Private Projects

# Skip projects with sensitive data
$ skill-creator discover --exclude=client-secrets,internal-tools

Example 3: Force Full Rescan

After updating gsd-skill-creator to a new version with improved pattern extraction, you may want to rescan everything:

# Ignore watermarks and reprocess all sessions
$ skill-creator discover --rescan

Example 4: Using the Suggestion Manager (Programmatic)

For integration with other tools, the SuggestionManager API provides programmatic access:

import { SuggestionManager } from 'gsd-skill-creator';

const manager = new SuggestionManager('.planning/patterns');

// Run detection against session history
const result = await manager.runDetection();
console.log(`${result.newCandidates} new, ${result.totalPending} pending`);

// Review pending suggestions
const pending = await manager.getPending();
for (const suggestion of pending) {
  console.log(suggestion.candidate.suggestedName, suggestion.candidate.confidence);
}

// Accept a suggestion (creates the skill file)
await manager.accept('cmd-vitest-workflow');

// Defer for later
await manager.defer('tool-grep-guide');

// Dismiss permanently
await manager.dismiss('cmd-docker-workflow', 'not relevant to current work');

Troubleshooting

Issue Cause Solution
“No patterns found after noise filtering” All detected patterns were classified as framework noise (appeared in 80%+ of 15+ projects) Use --allow to scan fewer projects, or you may need more diverse sessions before meaningful patterns emerge
“Prompt clustering skipped (embedding unavailable)” The embedding service (bge-small-en-v1.5) is not available locally This is non-fatal. Tool pattern discovery still works. Install the embedding model for prompt clustering support.
Scan shows 0 new sessions All sessions already scanned and watermarks are current Use --rescan to force reprocessing, or wait until you have new Claude Code sessions
All candidates say “duplicates existing skill” Deduplication (Jaccard similarity ≥ 0.50) matched all candidates to skills you already have This means your skills are well-covered. New patterns will surface as your workflows evolve.
Scan takes a very long time Large session files (23MB+) with many projects The scanner uses stream parsing to avoid memory issues. Use --exclude to skip large, less relevant projects. Subsequent scans are incremental and much faster.
Generated skill has TODO placeholders This is by design—the system generates a scaffold, not a finished skill Edit ~/.claude/skills/{name}/SKILL.md to replace TODO sections with your actual workflow guidelines

Technical Reference

The discover command scans your Claude Code session history to find recurring interaction patterns and generate draft skills from them.

How It Works

  1. Scan — Enumerates all projects under ~/.claude/projects/ and stream-parses JSONL session files
  2. Extract — Identifies tool sequence n-grams (Read->Edit->Bash) and Bash command patterns (git workflows, build commands)
  3. Cluster — Groups similar user prompts using DBSCAN with automatic epsilon tuning
  4. Rank — Scores candidates using frequency, cross-project occurrence, recency, and consistency
  5. Present — Shows ranked candidates with evidence (which sessions, which projects, examples)
  6. Draft — Generates SKILL.md files with pre-filled workflow steps for selected candidates

Key Features

Feature Description
Incremental scanning Only processes new/modified sessions on subsequent runs via watermarks
Noise filtering Filters framework patterns appearing in 15+ projects (dual-threshold)
Deduplication Skips patterns that match existing skills
Semantic clustering Groups similar prompts using embeddings + DBSCAN
Stream parsing Handles 23MB+ session files without loading into memory
Subagent support Includes subagent session directories in analysis

Usage

# Discover patterns from all projects
skill-creator discover

# Exclude specific projects from scanning
skill-creator discover --exclude my-private-project

# Force full rescan (ignore previous watermarks)
skill-creator discover --rescan

# Preview what would be scanned (v1.10)
skill-creator discover --dry-run

# Explicitly allow a project (v1.10)
skill-creator discover --allow my-project

The command displays progress during scanning (project count, session count, patterns found) and presents an interactive selection UI for choosing which candidates to turn into skills.


Creating Agent Teams: Source Code Reference

Agent teams are built using the TeamStore for persistence and template generators for common topology patterns. The source code lives in src/teams/team-store.ts and src/teams/templates.ts.

TeamStore: Persistence Layer

The TeamStore class provides CRUD operations for team configuration files with path traversal protection. Teams are stored as {teamsDir}/{teamName}/config.json.

export class TeamStore {
  constructor(private teamsDir: string) {}

  async save(config: TeamConfig): Promise<string> {
    this.assertSafeName(config.name);
    const validation = validateTeamConfig(config);
    if (!validation.valid) {
      throw new Error(`Invalid team config: ${validation.errors.join('; ')}`);
    }

    const teamDir = join(this.teamsDir, config.name);
    const configPath = join(teamDir, 'config.json');
    this.assertSafeTeamPath(teamDir);

    await mkdir(teamDir, { recursive: true });
    await writeFile(configPath, JSON.stringify(config, null, 2) + '\n', 'utf-8');
    return configPath;
  }

  async read(teamName: string): Promise<TeamConfig> {
    this.assertSafeName(teamName);
    const configPath = join(this.teamsDir, teamName, 'config.json');
    this.assertSafeTeamPath(configPath);
    const content = await readFile(configPath, 'utf-8');
    return JSON.parse(content) as TeamConfig;
  }

  async list(): Promise<string[]> {
    try {
      const entries = await readdir(this.teamsDir, { withFileTypes: true });
      const teamNames: string[] = [];
      for (const entry of entries) {
        if (entry.isDirectory()) {
          const configPath = join(this.teamsDir, entry.name, 'config.json');
          try { await stat(configPath); teamNames.push(entry.name); } catch {}
        }
      }
      return teamNames;
    } catch { return []; }
  }
}

Leader/Worker Template Generator

Template generators are pure functions that produce valid TeamConfig objects with pattern-specific members, tools, and sample tasks. Five patterns are supported: leader-worker, pipeline, swarm, router, and map-reduce.

export function generateLeaderWorkerTemplate(opts: TemplateOptions): TemplateResult {
  const workerCount = opts.workerCount ?? 3;
  const description = opts.description ?? `Leader/worker team with ${workerCount} workers`;

  const lead: TeamMember = {
    agentId: `${opts.name}-lead`,
    name: 'Lead',
    agentType: 'coordinator',
    tools: LEADER_TOOLS,
  };

  const workers: TeamMember[] = Array.from({ length: workerCount }, (_, i) => ({
    agentId: `${opts.name}-worker-${i + 1}`,
    name: `Worker ${i + 1}`,
    agentType: 'worker',
    tools: WORKER_TOOLS,
  }));

  const config: TeamConfig = {
    name: opts.name,
    description,
    leadAgentId: lead.agentId,
    createdAt: new Date().toISOString(),
    members: [lead, ...workers],
    version: 1,
    topology: 'leader-worker',
  };

  return { config, sampleTasks, patternInfo: {
    topology: 'leader-worker',
    description: 'One coordinator delegates work to parallel workers',
    memberSummary: `1 lead + ${workerCount} workers`,
  }};
}

Tool Constant Arrays by Pattern

Each topology pattern assigns different tool sets to its agents, controlling what each member can do.

export const LEADER_TOOLS = [
  'Read', 'Write', 'Bash', 'Glob', 'Grep',
  'TaskCreate', 'TaskList', 'TaskGet', 'TaskUpdate',
  'SendMessage', 'TeammateTool',
];

export const WORKER_TOOLS = [
  'Read', 'Write', 'Edit', 'Bash', 'Glob', 'Grep',
  'WebFetch', 'WebSearch',
  'TaskGet', 'TaskUpdate', 'SendMessage',
];

export const SWARM_WORKER_TOOLS = [
  'Read', 'Write', 'Edit', 'Bash', 'Glob', 'Grep',
  'TaskList', 'TaskGet', 'TaskUpdate', 'SendMessage',
];

export const ROUTER_TOOLS = [
  'Read', 'Bash', 'Glob', 'Grep',
  'TaskCreate', 'TaskList', 'TaskGet', 'TaskUpdate',
  'SendMessage', 'TeammateTool',
];

Team Topology Patterns

Pattern Generator Function Lead Role Worker Tools
Leader/Worker generateLeaderWorkerTemplate() coordinator WORKER_TOOLS
Pipeline generatePipelineTemplate() orchestrator PIPELINE_STAGE_TOOLS
Swarm generateSwarmTemplate() coordinator SWARM_WORKER_TOOLS
Router generateRouterTemplate() router WORKER_TOOLS
Map-Reduce generateMapReduceTemplate() orchestrator WORKER_TOOLS