Learning & Refinement Guide

Overview

Data Flow Diagram

User Corrections
      |
      v
[FeedbackStore] ----> feedback.jsonl (append-only)
      |
      v
[ContradictionDetector] ----> Check for conflicting signals
      |
      v (if no conflicts)
[RefinementEngine] ----> analyzePatterns() -> generateSuggestion()
      |
      v
[DriftTracker] ----> Compare against original (via Git history)
      |
      v (if within bounds)
[User Confirmation] ----> REQUIRED: human approves the change
      |
      v
[SkillStore.update()] ----> Updated SKILL.md + audit log entry

Key Concepts

How It Works

1. Feedback Capture (FeedbackStore)

The FeedbackStore in src/learning/feedback-store.ts records every feedback event as a single JSON line in an append-only JSONL file at .planning/patterns/feedback.jsonl. This format ensures that feedback history is never overwritten — each event is appended to the end of the file, creating an immutable chronological record.

Feedback Event Structure

Every feedback event includes these fields:

interface FeedbackEvent {
  id: string;           // UUID, auto-generated on record
  timestamp: string;    // ISO 8601, auto-generated on record
  type: FeedbackType;   // 'correction' | 'override' | 'rating'
  skillName: string;    // Which skill produced the output
  sessionId: string;    // Links feedback to the session context

  // For corrections — the primary learning signal
  original?: string;    // What the skill-guided output was
  corrected?: string;   // What the user changed it to
  diff?: Change[];      // Word-level diff between original and corrected

  // For overrides — skill suggestion was rejected entirely
  rejected?: boolean;   // User rejected the skill suggestion
  reason?: string;      // Why it was rejected (free-text)

  // For ratings — explicit quality assessment
  score?: number;       // 1-5 explicit quality rating
}

The three feedback types serve different roles in the learning loop. Corrections are the primary signal — they contain the exact before/after text that the Refinement Engine analyzes for patterns. Overrides indicate that the skill was activated but not useful, which can signal trigger mismatches. Ratings provide aggregate quality tracking over time.

Recording Feedback

The record() method accepts a partial event (without id and timestamp, which are generated automatically using crypto.randomUUID() and new Date().toISOString()) and appends it to the JSONL file. Write operations are serialized through a promise queue to prevent interleaving when multiple agents write concurrently.

const store = new FeedbackStore('.planning/patterns');

// Record a correction — the most valuable feedback type
await store.record({
  type: 'correction',
  skillName: 'lint-on-save',
  sessionId: 'sess-abc123',
  original: 'npx eslint --fix .',
  corrected: 'npx eslint --fix --ext .ts,.tsx .',
});

// Record an override — skill suggestion was rejected
await store.record({
  type: 'override',
  skillName: 'test-runner',
  sessionId: 'sess-abc123',
  rejected: true,
  reason: 'Tests need to run in watch mode for this task',
});

// Record a rating — explicit quality assessment
await store.record({
  type: 'rating',
  skillName: 'beautiful-commits',
  sessionId: 'sess-abc123',
  score: 5,
});

Querying Feedback

The store provides several query methods for downstream analysis. All queries read the entire JSONL file and filter in memory, which is efficient for the expected scale (hundreds to low thousands of events per project).

• getBySkill(skillName) — All events for a specific skill, regardless of type
• getCorrections(skillName) — Only correction events for a skill (the primary signal for the Refinement Engine)
• getSince(timestamp, skillName?) — Events since a given ISO timestamp, optionally filtered by skill
• getAll() — Every recorded event across all skills
• count(skillName?) — Total event count, optionally per skill

// Get all corrections for a specific skill
const corrections = await store.getCorrections('lint-on-save');
console.log(`${corrections.length} corrections recorded`);

// Get events from the last 7 days
const weekAgo = new Date(Date.now() - 7 * 24 * 60 * 60 * 1000).toISOString();
const recentFeedback = await store.getSince(weekAgo);

// Count all feedback events
const totalCount = await store.count();
const skillCount = await store.count('lint-on-save');

Corrupted lines in the JSONL file are skipped with a console warning rather than causing the entire file read to fail. This ensures graceful degradation even if the feedback file is partially damaged by a process crash during a write operation.

Write Serialization

The appendLine() method serializes writes through a promise chain (this.writeQueue). Each write waits for the previous write to complete before starting. This prevents line interleaving that could occur when multiple async operations (e.g., from parallel agent actions in the same process) try to append simultaneously. The directory is created automatically on first write using mkdir({ recursive: true }).

The serialization works by chaining promises: each new write is appended to the queue by calling this.writeQueue = this.writeQueue.then(() => doWrite()). This means writes are guaranteed to complete in the order they were submitted, even in highly concurrent scenarios where multiple GSD agents record feedback events within the same process. If a write fails (disk full, permissions error), subsequent writes in the queue continue — the failure is isolated to the individual event.

Session and Skill Correlation

Every feedback event includes a sessionId that links it to the session context where the correction occurred. This enables several important analysis patterns:

• Session replay: Query all events for a session ID to reconstruct the complete feedback history of a single work session.
• Cross-session patterns: When the same correction appears across different sessions, it is a stronger signal than multiple corrections within one session (which might reflect a single task context).
• Session quality: Sessions with many corrections may indicate a skill that needs refinement or a task that falls outside the skill's intended scope.

2. Refinement Engine

The RefinementEngine in src/learning/refinement-engine.ts is the central component of the bounded learning system. It analyzes accumulated correction feedback, detects repeating patterns, and generates refinement suggestions that are constrained by strict safety bounds. The engine coordinates with the Drift Tracker to enforce cumulative change limits.

Bounded Learning Configuration

Every refinement is governed by these non-negotiable constraints, defined in the BoundedLearningConfig interface:

const DEFAULT_BOUNDED_CONFIG: BoundedLearningConfig = {
  minCorrectionsForRefinement: 3,    // Need 3+ corrections before suggesting
  minConfidence: 0.7,                // Minimum confidence score (0-1)
  maxRefinementsPerSkill: 1,         // One refinement per cooldown period
  cooldownDays: 7,                   // Wait 7 days between refinements
  maxContentChangePercent: 20,       // Cannot change more than 20% of content
  maxMetadataChanges: 3,             // Maximum metadata field changes
  requireUserConfirmation: true,     // ALWAYS requires user approval
  preserveOriginalOnRefinement: true // Keep original in version history
};

These defaults can be partially overridden through the constructor, but requireUserConfirmation is hardcoded as a safety invariant — it cannot be disabled through configuration.

Eligibility Checking

Before generating a suggestion, the engine checks three eligibility conditions via checkEligibility():

1. Skill existence: Does the skill still exist on disk? If not, return ineligible.
2. Cooldown check: Has the mandatory 7-day waiting period elapsed since the last refinement? The engine reads the learning.lastRefined timestamp from the skill's extension metadata and computes days elapsed. If still in cooldown, it returns the number of days remaining.
3. Correction count: Are there enough correction events? At least 3 corrections (fetched from the FeedbackStore) are required. If insufficient, the response includes how many more corrections are needed.

const engine = new RefinementEngine(feedbackStore, skillStore);

const eligibility = await engine.checkEligibility('lint-on-save');
if (!eligibility.eligible) {
  switch (eligibility.reason) {
    case 'cooldown':
      console.log(`In cooldown: ${eligibility.daysRemaining} days remaining`);
      break;
    case 'insufficient_feedback':
      console.log(`Need ${eligibility.correctionsNeeded} more corrections`);
      console.log(`Current count: ${eligibility.correctionCount}`);
      break;
  }
}

Pattern Analysis

When a skill is eligible, the engine analyzes all its corrections to find repeating patterns using the analyzePatterns() method. It groups corrections by a pattern key derived from the first 50 characters of the original and corrected text, then counts frequency. Only patterns that appear at least twice contribute to the suggestion — single corrections are treated as potential noise.

The patterns are sorted by frequency (highest first), so the most consistent corrections have the strongest influence on the suggestion. For example, if five different corrections all change npx eslint --fix . to npx eslint --fix --ext .ts,.tsx ., this pattern gets a frequency of 5 and becomes a high-confidence change.

The pattern analysis process follows these steps internally:

1. Fetch corrections: Read all correction-type events from the FeedbackStore for the specified skill name.
2. Build pattern keys: For each correction with both original and corrected text, generate a composite key from the first 50 characters of each. This truncation allows minor variations in longer text to be grouped together while keeping short corrections matched exactly.
3. Count frequencies: Increment the count for each pattern key. A single correction contributes a frequency of 1; repeated identical corrections build higher frequencies.
4. Filter noise: Discard patterns with frequency 1 (one-off corrections that may be situational rather than systematic).
5. Sort and rank: Order remaining patterns by frequency, highest first. The top patterns become the basis for the refinement suggestion.

This frequency-based approach means the system responds to consistent, repeating corrections rather than reacting to individual edits. A user who corrects the same output pattern three times across different sessions sends a clear signal; a user who makes three different corrections to three different parts of a skill sends no clear signal at all.

Suggestion Generation

The generateSuggestion() method produces a complete refinement proposal with full transparency about what would change and why:

const suggestion = await engine.generateSuggestion('lint-on-save');
// {
//   skillName: 'lint-on-save',
//   currentVersion: 2,
//   suggestedChanges: [
//     {
//       section: 'body',
//       original: 'npx eslint --fix .',
//       suggested: 'npx eslint --fix --ext .ts,.tsx .',
//       reason: 'Pattern observed 3 times in user corrections',
//     }
//   ],
//   confidence: 0.75,
//   basedOnCorrections: 5,
//   preview: '# Lint on Save\n\nWhen a TypeScript file...[CHANGED: npx eslint...]'
// }

The confidence score is calculated as the ratio of total pattern frequency to twice the number of corrections, capped at 1.0. This means a skill with 5 corrections where all corrections match the same pattern gets 5 / (5*2) = 0.5 confidence — higher frequency relative to total corrections means higher confidence. Suggestions below the minimum confidence threshold (default 0.7) are discarded and null is returned.

The preview shows the skill body with changes marked as [CHANGED: new text], truncated to 500 characters for display purposes.

Change Validation

Before any change is applied, the validateChange() method computes the change percentage using the diff npm package's word-level diffing. The change percentage measures how much text was added plus removed, relative to the original length:

const validation = engine.validateChange(
  'Run eslint on all files in the project directory',
  'Run eslint --ext .ts,.tsx on all TypeScript files in the src directory'
);
// { valid: true, changePercent: 15 }

const tooLarge = engine.validateChange(
  'Run eslint.',
  'Execute comprehensive linting with ESLint including TypeScript strict checks, ' +
  'import validation, accessibility rules, and performance optimization hints'
);
// { valid: false, changePercent: 85, reason: 'exceeds_bounds' }

Confidence Scoring

The confidence score determines whether a suggestion is worth presenting to the user. It is computed as:

confidence = totalPatternFrequency / (2 * totalCorrections)

This formula caps at 1.0 and rewards consistency. A skill with 6 corrections where 4 match the same pattern gets 4 / (6*2) = 0.33 — below the default 0.7 threshold. A skill with 6 corrections where all 6 match the same pattern gets 6 / (6*2) = 0.50 — still below threshold because the denominator is twice the count (representing a "perfect" scenario where every correction appears twice). This deliberate conservatism means the engine only suggests changes when correction patterns are overwhelmingly consistent.

The minimum confidence threshold is configurable through BoundedLearningConfig.minConfidence. Lowering it below 0.5 is not recommended because it would produce suggestions based on minority correction patterns.

Applying Refinements

The applyRefinement() method applies a confirmed suggestion to the skill. The method enforces a strict safety protocol:

1. User confirmation gate: The userConfirmed parameter must be true. Passing false throws an error immediately — this is a hard safety boundary.
2. Pre-apply drift check: If a DriftTracker is available, check that cumulative drift has not already exceeded the threshold.
3. Individual change validation: Every change in the suggestion is validated against the 20% content change bound.
4. Content modification: Apply string replacements to the skill body.
5. Post-apply drift projection: Compute what drift would be after applying the changes. If it would exceed the threshold, reject.
6. Metadata update: Increment version, update updatedAt and learning.lastRefined timestamps, increment applicationCount.
7. Disk write: Save the updated skill through SkillStore.update().

const result = await engine.applyRefinement(
  'lint-on-save',
  suggestion,
  true  // User explicitly confirmed
);

if (result.success) {
  console.log(`Refined to version ${result.newVersion}`);
} else {
  console.error(`Refinement failed: ${result.error}`);
  // Possible errors:
  // - "User confirmation required for skill refinement"
  // - "Cumulative drift (62%) exceeds threshold (60%)"
  // - "Change exceeds bounds (25% > 20%)"
  // - "Projected drift (65%) would exceed threshold (60%)"
}

3. Drift Tracker

The DriftTracker in src/learning/drift-tracker.ts monitors cumulative content changes from a skill's original version to its current state. It implements two critical safety requirements: LRN-01 (track cumulative change) and LRN-02 (halt at 60% drift).

Why Drift Tracking Matters

Each individual refinement is bounded to 20% content change, but without cumulative tracking, a skill could be refined repeatedly — 20% at a time — until it bears no resemblance to its original. After just four successive 20% refinements, the skill could theoretically be 80% different from its starting point. The Drift Tracker prevents this by measuring the total divergence from the very first version of the skill, using Git history as the source of truth for the original content.

At 60% drift, the system halts automatic refinements entirely. At this point, the skill has changed substantially enough that it should be reviewed holistically by a human rather than being incrementally modified further.

How Drift Is Measured

The tracker uses the VersionManager to retrieve the skill's complete Git commit history and extract the original content from the oldest commit. It then strips YAML frontmatter (using gray-matter) to compare body content only, and computes a word-level diff between the original body and the current body:

driftPercent = (addedChars + removedChars) / originalLength * 100

The result is rounded to one decimal place for readability. This produces a single number representing how much the skill's instructional content has changed over its entire lifecycle, regardless of how many individual refinements contributed to that change.

Drift Computation

const tracker = new DriftTracker(versionManager, skillStore);

const drift = await tracker.computeDrift('lint-on-save');
// {
//   originalContent: '# Lint on Save\n\nRun eslint on...',
//   currentContent: '# Lint on Save\n\nRun eslint --ext .ts...',
//   cumulativeDriftPercent: 35.2,
//   thresholdExceeded: false,
//   threshold: 60
// }

The result includes both the original and current content strings, enabling UI layers to display a visual diff alongside the numerical drift percentage.

Threshold Checking

The checkThreshold() method extends computeDrift() by comparing the result against a configurable threshold (default 60%). When the threshold is exceeded, the thresholdExceeded flag is set to true. A custom threshold can be passed for skills that should tolerate more or less drift than the default.

// Check against default 60% threshold
const check = await tracker.checkThreshold('lint-on-save');
if (check.thresholdExceeded) {
  console.warn(
    `Drift: ${check.cumulativeDriftPercent}% (threshold: ${check.threshold}%). ` +
    `Automatic refinement halted.`
  );
}

// Check with custom threshold for a critical skill
const strictCheck = await tracker.checkThreshold('deploy-production', 40);
// Uses 40% threshold instead of default 60%

Projected Drift

The computeDriftWithContent() method is the critical integration point with the Refinement Engine. Before applying a refinement, the engine can compute the projected drift of a hypothetical change without writing to disk. This catches the edge case where cumulative drift is at 58% and a 20% individual change would push it over 60%:

const projected = await tracker.computeDriftWithContent(
  'lint-on-save',
  proposedNewBody
);
if (projected.thresholdExceeded) {
  // Reject: this change would push drift to 62%
  return {
    success: false,
    error: `Projected drift (${projected.cumulativeDriftPercent}%) ` +
           `would exceed threshold (${projected.threshold}%).`
  };
}

Edge Cases

• New skills (no Git history): Skills with 0 or 1 commits report 0% drift. This is correct — a brand-new skill has no drift by definition.
• Frontmatter-only changes: Drift is computed on body content only (frontmatter is stripped using gray-matter). Metadata changes (triggers, description updates, version increments) do not count toward drift. This means you can refine triggers and descriptions freely without consuming your drift budget.
• Non-Git environments: The DriftTracker is optionally injected into the Refinement Engine. If omitted (no Git repository), drift checking is skipped entirely, and only the per-change 20% bound applies.
• Manual skill edits: If you edit a skill directly in your editor (outside of the refinement workflow), those changes still affect drift. The Drift Tracker compares current content against the oldest Git commit, regardless of how the changes were made.
• Empty original content: If the original body is empty (zero characters), the drift formula would produce a division-by-zero. The tracker handles this by returning 0% drift for zero-length originals.

4. Contradiction Detector

The ContradictionDetector in src/learning/contradiction-detector.ts implements LRN-03: flag contradictory feedback rather than silently averaging. When different corrections point in opposing directions, the system surfaces the contradiction for human resolution instead of producing confused refinements.

What Counts as a Contradiction

The detector identifies two types of contradictory patterns by performing pairwise comparison of all correction events for a skill:

1. Full reversals (severity: conflict) — One correction changes text A to text B, and a later correction changes text B back to text A. Both the original and corrected text must match (after normalization) for this to be detected. This is the clearest signal of disagreement and always merits investigation.
2. Partial reversals (severity: warning) — Corrections overlap in opposing directions. The detector checks if the corrected text of one correction contains the original text of another, and vice versa, indicating that changes are pushing content in opposite directions. A minimum length threshold (10 characters or 50% of the shorter text) prevents trivially short matches from being flagged.

Detection Process

The detector normalizes all original and corrected text (trim whitespace, convert to lowercase) before comparison. It skips corrections where either the original or corrected text is missing. The result includes all detected contradictions with their severity, human-readable descriptions, and the original feedback events for full context.

const detector = new ContradictionDetector(feedbackStore);

const result = await detector.detect('lint-on-save');
// {
//   contradictions: [
//     {
//       correction1: { ... original: 'eslint .', corrected: 'eslint src/' },
//       correction2: { ... original: 'eslint src/', corrected: 'eslint .' },
//       field: 'body',
//       description: "Correction reversal: changed 'eslint .' to 'eslint src/'
//                      then back to 'eslint .'",
//       severity: 'conflict',
//     }
//   ],
//   hasConflicts: true,
//   summary: 'Found 1 contradiction(s): 1 conflict(s), 0 warning(s)'
// }

How This Integrates with Refinement

The Refinement Engine should check for contradictions before proposing suggestions. When contradictions exist at conflict severity, the engine can lower its confidence score or skip the suggestion entirely. The user-facing CLI presents contradictions as action items that need human resolution.

The integration follows a recommended workflow:

1. Run contradiction detection for the skill before generating a refinement suggestion.
2. If hasConflicts is true, present the contradictions to the user and halt refinement until resolved.
3. If only warnings exist, proceed with refinement but include the warning information in the suggestion output so the user can assess whether the warnings are relevant.
4. If no contradictions exist, proceed normally with the full refinement pipeline.

Common resolution strategies include:

• Split the skill: Create two focused skills with different trigger conditions. For example, split a "deploy" skill into "deploy-staging" and "deploy-production" if corrections conflict based on environment.
• Add context triggers: Use the contexts trigger field to differentiate when each behavior is appropriate, allowing the skill to activate differently based on the current workflow.
• Discard corrections: After team review, remove one set of contradictory corrections from the feedback file. The FeedbackStore does not support deletion directly (append-only), but you can create a cleaned copy.
• Rewrite with conditionals: Update the skill body to handle both cases with explicit conditional logic (e.g., "If in staging, do X. If in production, do Y.").

Summary Format

The result includes a human-readable summary string for easy display in CLIs and dashboards:

// Contradictions found:
"Found 3 contradiction(s): 1 conflict(s), 2 warning(s)"

// No contradictions:
"No contradictions detected"

The hasConflicts boolean provides a quick check: it is true only when at least one conflict-severity contradiction exists (warnings alone do not set it to true). This distinction matters because warnings represent partial overlaps that may be coincidental, while conflicts represent direct reversals that require human attention.

Normalization and Matching

Before comparing corrections, the detector normalizes all text by trimming whitespace and converting to lowercase. This means "ESLint --fix" and "eslint --fix" are treated as identical for reversal detection. Corrections that are missing either the original or corrected field are skipped entirely — only complete correction events participate in contradiction analysis.

The minimum length threshold for partial reversals (10 characters or 50% of the shorter text, whichever is greater) prevents trivially short string matches from being flagged. Without this threshold, common short strings like "the" or "and" appearing in multiple corrections would generate false warnings.

Usage Examples

import { FeedbackStore } from 'gsd-skill-creator/learning/feedback-store';
import { RefinementEngine } from 'gsd-skill-creator/learning/refinement-engine';
import { DriftTracker } from 'gsd-skill-creator/learning/drift-tracker';
import { ContradictionDetector } from 'gsd-skill-creator/learning/contradiction-detector';
import { SkillStore } from 'gsd-skill-creator/storage/skill-store';
import { VersionManager } from 'gsd-skill-creator/learning/version-manager';

const feedbackStore = new FeedbackStore('.planning/patterns');
const skillStore = new SkillStore('.claude/skills');
const versionManager = new VersionManager('.claude/skills');
const driftTracker = new DriftTracker(versionManager, skillStore);
const engine = new RefinementEngine(feedbackStore, skillStore, {}, driftTracker);
const contradictionDetector = new ContradictionDetector(feedbackStore);

// Step 1: Corrections accumulate over sessions
await feedbackStore.record({
  type: 'correction', skillName: 'deploy-check',
  sessionId: 's1',
  original: 'npm run build',
  corrected: 'npm run build -- --production',
});
// ...two more similar corrections in later sessions...

// Step 2: Check for contradictions first
const contradictions = await contradictionDetector.detect('deploy-check');
if (contradictions.hasConflicts) {
  console.warn('Contradictory feedback detected — manual review needed');
  console.warn(contradictions.summary);
  // Stop here; don't refine with conflicting signals
} else {
  // Step 3: Check eligibility
  const eligible = await engine.checkEligibility('deploy-check');
  if (eligible.eligible) {
    // Step 4: Generate suggestion
    const suggestion = await engine.generateSuggestion('deploy-check');

    if (suggestion) {
      // Step 5: Present to user for review
      console.log('Proposed refinement:');
      console.log(`  Confidence: ${(suggestion.confidence * 100).toFixed(0)}%`);
      console.log(`  Based on ${suggestion.basedOnCorrections} corrections`);
      for (const change of suggestion.suggestedChanges) {
        console.log(`  "${change.original}" -> "${change.suggested}"`);
        console.log(`  Reason: ${change.reason}`);
      }
      console.log(`  Preview: ${suggestion.preview}`);

      // Step 6: Apply with user confirmation
      const result = await engine.applyRefinement(
        'deploy-check', suggestion, true
      );
      if (result.success) {
        console.log(`Applied: version ${result.newVersion}`);
      } else {
        console.error(`Failed: ${result.error}`);
      }
    }
  }
}

npx skill-creator refine --check lint-on-save

Refinement Eligibility: lint-on-save
======================================
Corrections recorded: 5
Minimum required: 3
Last refined: 2025-06-15 (39 days ago)
Cooldown period: 7 days
Status: ELIGIBLE

Correction patterns detected:
  Pattern 1 (freq: 3): "npx eslint --fix ." -> "npx eslint --fix --ext .ts,.tsx ."
  Pattern 2 (freq: 2): "eslint ." -> "eslint src/"

Run 'npx skill-creator refine lint-on-save' to generate a suggestion.

Refinement Eligibility: lint-on-save
======================================
Corrections recorded: 7
Minimum required: 3
Last refined: 2025-07-20 (2 days ago)
Cooldown period: 7 days
Status: IN COOLDOWN (5 days remaining)

Next eligible: 2025-07-27

npx skill-creator drift lint-on-save

Drift Report: lint-on-save
===========================
Current version: 4
Original commit: a3b7c9d (2025-06-15)
Cumulative drift: 42.3%
Threshold: 60%
Status: WITHIN BOUNDS

Remaining capacity: 17.7% before automatic refinements are halted.
Estimated refinements remaining: ~2 (at average 8% per refinement)

Drift Report: deploy-check
===========================
Current version: 7
Original commit: f1a2b3c (2025-05-01)
Cumulative drift: 63.8%
Threshold: 60%
Status: THRESHOLD EXCEEDED

Automatic refinement is halted for this skill.
Options:
  1. Review the skill manually and reset drift tracking
  2. Create a new skill based on current content
  3. Increase the drift threshold (not recommended)

npx skill-creator contradictions deploy-check

Contradiction Report: deploy-check
===================================
Corrections analyzed: 6
Contradictions found: 2

CONFLICT  Reversal detected
          Session s3 (2025-07-15): changed 'build' to 'build --production'
          Session s5 (2025-07-22): changed 'build --production' to 'build'
          Action: Review whether production flag is always needed

WARNING   Partial reversal
          Session s2 (2025-07-10): expanded command with --verbose flag
          Session s4 (2025-07-18): simplified command back toward original
          Action: Consider adding context condition for verbose mode

Recommendation: This skill has contradictory feedback. Consider:
  1. Adding trigger conditions to differentiate use cases
  2. Splitting into deploy-check-dev and deploy-check-prod
  3. Reviewing with the team to establish consensus

npx skill-creator feedback lint-on-save

Feedback History: lint-on-save
===============================
Total events: 8 (5 corrections, 1 override, 2 ratings)

Corrections:
  [2025-07-05] "npx eslint --fix ." -> "npx eslint --fix --ext .ts,.tsx ."
  [2025-07-08] "npx eslint --fix ." -> "npx eslint --fix --ext .ts,.tsx ."
  [2025-07-12] "eslint ." -> "eslint src/"
  [2025-07-15] "npx eslint --fix ." -> "npx eslint --fix --ext .ts,.tsx ."
  [2025-07-18] "eslint ." -> "eslint src/"

Overrides:
  [2025-07-10] Rejected — "Tests need to run in watch mode for this task"

Ratings:
  [2025-07-06] Score: 4/5
  [2025-07-14] Score: 5/5
  Average rating: 4.5/5

# Count corrections per skill
grep '"type":"correction"' .planning/patterns/feedback.jsonl | \
  grep -o '"skillName":"[^"]*"' | sort | uniq -c | sort -rn

# Find skills in cooldown (refined in the last 7 days)
grep '"type":"refine"' .claude/.audit-log.jsonl | \
  jq -r 'select(.timestamp > "'$(date -d '7 days ago' -Iseconds)'") | .filePath'

# Check feedback file growth rate (useful for capacity planning)
ls -la .planning/patterns/feedback.jsonl
# Typical: ~200 bytes per event, ~1000 events = ~200 KB

# Verify feedback file is backed up
ls -la .planning/patterns/feedback.jsonl

# Quick integrity check: validate every line is parseable JSON
while IFS= read -r line; do
  echo "$line" | jq -e . > /dev/null 2>&1 || echo "CORRUPT: $line"
done < .planning/patterns/feedback.jsonl

# Rotate feedback file quarterly
cp .planning/patterns/feedback.jsonl \
   ".planning/patterns/feedback-$(date +%Y-Q$(($(date +%m)/4+1))).jsonl"
# Truncate original — only recent corrections matter for pattern analysis
tail -200 .planning/patterns/feedback.jsonl > /tmp/feedback-recent.jsonl
mv /tmp/feedback-recent.jsonl .planning/patterns/feedback.jsonl

Troubleshooting

Refinement suggestion returns null despite multiple corrections

Cause: Corrections may not form repeating patterns (each correction is unique), the confidence score may be below 0.7, the skill may be in cooldown, or the skill may not exist.

Solution: Check eligibility first with checkEligibility() to see the specific reason. If insufficient_feedback, more corrections with consistent patterns are needed — the corrections must change the same text in the same way. If cooldown, wait for the remaining days. If corrections exist but no suggestion is generated, the patterns are too varied (each correction is different) to form a reliable recommendation.

Refinement fails with "exceeds bounds"

Cause: The proposed change would modify more than 20% of the skill's body content.

Solution: The bounded learning constraints are working as designed. If the skill genuinely needs a large change, consider creating a new skill rather than refining the existing one. Alternatively, wait for the change to be split across multiple smaller patterns that each stay within the 20% bound.

Refinement fails with "Cumulative drift exceeds threshold"

Cause: The skill has been refined so many times that its content has drifted more than 60% from the original.

Solution: This is a safety limit. Options: (a) Review the skill holistically and decide if the current version is correct. If so, consider creating a fresh skill with the current content as the new baseline. (b) If the skill has drifted in the wrong direction, use Git history to revert to an earlier version. (c) Increase the drift threshold only if you have thoroughly reviewed the cumulative changes.

Drift tracker reports 0% despite multiple refinements

Cause: The skill has no Git history (never been committed), or has only one commit.

Solution: Ensure that skill changes are committed to Git after each refinement. The Drift Tracker needs at least two commits (original and current) to compute meaningful drift. If using GSD, atomic commits happen automatically during execute-phase.

Contradiction detector misses implicit contradictions

Cause: The detector looks for exact or near-exact text reversals. Subtler contradictions where two corrections use completely different wording but imply opposing directions are not detected.

Solution: The current detector is designed for high-precision reversal detection. For implicit contradictions, manually review all corrections for a skill using the CLI feedback listing command or by querying feedbackStore.getCorrections(skillName).

Feedback file contains corrupted lines

Cause: A process was interrupted during a write, or the file was manually edited.

Solution: The FeedbackStore automatically skips corrupted lines with a console warning. No data loss occurs for valid entries. To clean the file, read all valid events with getAll(), write them to a new file, and replace the original.

Feedback events are not being recorded

Cause: The .planning/patterns/ directory does not exist and the FeedbackStore's auto-creation failed (permissions), or the feedback recording is not wired into the session observation pipeline.

Solution: Verify the directory exists: ls -la .planning/patterns/. If missing, create it manually: mkdir -p .planning/patterns. Verify write permissions. If the directory exists but the file is empty, check that session observation is active and corrections are being captured — the feedback pipeline only records events that pass through the record() method with valid type, skillName, and sessionId fields.

Refinement confidence is always below threshold

Cause: Corrections are varied — each correction changes different text, producing many low-frequency patterns instead of a few high-frequency ones. The confidence formula divides total pattern frequency by twice the correction count, so diverse corrections produce low confidence.

Solution: This is working as intended. The system requires consistent correction signals before suggesting changes. If the corrections genuinely represent improvements, consider manually applying the most common correction pattern by editing the skill directly, then committing the change. Alternatively, lower the minConfidence threshold in the BoundedLearningConfig (not recommended for production).

Related Capabilities