Documentation > Architecture > Data Flows

Data Flow Diagrams

This document details how data moves through gsd-skill-creator for the three core pipelines: Skill Creation, Session Activation, and Pattern Detection to Agent Composition. Each flow includes the actual function calls and module paths from the codebase.

---

Flow 1: Skill Creation Pipeline

User runs `npx skill-creator create`
  → CLI parses command, determines scope (user/project)
  → Workflow collects input (name, description, body)
  → SkillValidator.validate(metadata)
  → BudgetValidator.checkSingleSkill(content.length)
  → GenerationSafety.scanForDangerousCommands(body)
  → PathSafety.validateSafeName(name)
  → PathSafety.assertSafePath(resolvedPath, baseDir)
  → SkillStore.create(name, metadata, body)
  → SkillIndex.rebuild()
  → .claude/skills/{name}/SKILL.md written to disk

Validation Gates

Three critical checkpoints before any skill reaches disk:

// Gate 1: Name validation (src/validation/skill-validation.ts)
const nameValidation = validateSkillNameStrict(skillName);
if (!nameValidation.valid) {
  const suggestion = nameValidation.suggestion;
  throw new Error(`Invalid skill name "${skillName}": ${nameValidation.errors.join('; ')}`);
}

// Gate 2: Reserved name check (src/validation/reserved-names.ts)
const reservedCheck = await validateReservedName(skillName);
if (!reservedCheck.valid) {
  throw new Error(reservedCheck.error);
}

// Gate 3: Budget check (src/validation/budget-validation.ts)
const budgetValidator = BudgetValidator.load();
const budgetCheck = budgetValidator.checkSingleSkill(content.length);
if (budgetCheck.severity === 'error') {
  console.warn(`Skill exceeds character budget (${budgetCheck.charCount} / ${budgetCheck.budget} chars)`);
}

Security Pipeline

// src/validation/path-safety.ts - Defense-in-depth traversal prevention
export function validateSafeName(name: string): SafeNameResult {
  // Checks: empty, null byte, ".." traversal, path separators, reserved names
}
export function assertSafePath(resolvedPath: string, baseDir: string): void {
  // Verifies resolved path stays within base directory
}

// src/validation/generation-safety.ts - Dangerous command detection
export const DANGEROUS_COMMANDS: DangerousCommandPattern[] = [
  { name: 'recursive-delete', pattern: /rm\s+(?:-[a-zA-Z]*)?r.../, description: '...' },
  { name: 'recursive-delete-rf', pattern: /rm\s+-rf\s+[\/~*].../, description: '...' },
  // ... additional patterns for chmod 777, curl|bash, dd if=/dev/zero, etc.
];
export function scanForDangerousCommands(content: string): DangerousFinding[];
export function sanitizeGeneratedContent(body: string): { sanitized: string; findings: DangerousFinding[] };

Progressive Disclosure Path

For skills exceeding 2000 words, the creation flow takes an alternate path through SkillStore.createWithDisclosure():

Large skill body (>2000 words)
  → ContentDecomposer.decompose(name, metadata, body)
  → Splits into compact SKILL.md + references/*.md + scripts/*.sh
  → ReferenceLinker.validateSkillReferences(skillDir)
  → Checks for circular references between files
  → Writes all files to .claude/skills/{name}/

---

Flow 2: Session Activation Pipeline

Session starts
  → SessionObserver.onSessionStart(data)
     Caches: { sessionId, transcriptPath, cwd, source, model, startTime }
  → SkillPipeline.process(context)
     Stage 1: ScoreStage    → match intent via TF-IDF or embeddings, score relevance
     Stage 2: ResolveStage  → resolve inheritance chains via SkillResolver
     Stage 3: ModelFilterStage → filter by quality/balanced/budget profile
     Stage 4: CacheOrderStage  → reorder to maximize cache hits
     Stage 5: BudgetStage   → enforce 2-5% context budget with tiered priority
     Stage 6: LoadStage     → inject selected skills into context
  → Skills active for session

The PipelineContext Object

All data flows through a single context object passed between stages:

// src/application/skill-pipeline.ts
export interface PipelineContext {
  // Inputs (read-only to stages)
  readonly intent?: string;
  readonly file?: string;
  readonly context?: string;

  // Intermediate results (stages read/write)
  matches: SkillIndexEntry[];         // ScoreStage writes
  scoredSkills: ScoredSkill[];        // ScoreStage writes
  resolvedSkills: ScoredSkill[];      // ResolveStage writes
  conflicts: ConflictResult;          // ResolveStage writes

  // Budget results
  budgetSkipped: SkippedSkill[];      // BudgetStage writes
  budgetWarnings: BudgetWarning[];    // BudgetStage writes
  contentCache: Map<string, string>;  // BudgetStage writes, LoadStage reads

  // Model profile for model-aware filtering
  readonly modelProfile?: string;     // 'quality' | 'balanced' | 'budget'

  // Final outputs
  loaded: string[];                   // LoadStage writes
  skipped: string[];                  // LoadStage writes
  earlyExit: boolean;                 // Any stage can set
}

ScoreStage Detail

// src/application/stages/score-stage.ts
export class ScoreStage implements PipelineStage {
  readonly name = 'score';

  constructor(
    private skillIndex: SkillIndex,
    private scorer: RelevanceScorer,
    private router?: AdaptiveRouter,        // Optional: TF-IDF vs embedding routing
    private embeddingService?: EmbeddingService,
  ) {}

  async process(context: PipelineContext): Promise<PipelineContext> {
    const matches = await this.skillIndex.findByTrigger(
      context.intent, context.file, context.context
    );
    context.matches = matches;
    if (matches.length === 0) {
      context.earlyExit = true;
      return context;
    }
    // Score each match by relevance...
  }
}

BudgetStage Detail

// src/application/stages/budget-stage.ts
export class BudgetStage implements PipelineStage {
  readonly name = 'budget';

  constructor(
    private tokenCounter: TokenCounter,
    private profile: BudgetProfile,
    private skillStore: SkillStore,
    private contextWindowSize: number = 200_000
  ) {}

  async process(context: PipelineContext): Promise<PipelineContext> {
    const standardBudget = this.tokenCounter.calculateBudget(
      this.contextWindowSize, this.profile.budgetPercent   // 2-5%
    );
    const hardCeiling = this.tokenCounter.calculateBudget(
      this.contextWindowSize, this.profile.hardCeilingPercent
    );
    // Partition into critical/standard/optional tiers
    // Critical loads even past standard budget, up to hard ceiling
    // Optional gets skipped first when budget is tight
  }
}

Pipeline Extensibility

// The pipeline supports inserting stages without modifying existing code
const pipeline = new SkillPipeline();
pipeline.addStage(new ScoreStage(index, scorer));
pipeline.addStage(new ResolveStage(resolver));
pipeline.addStage(new ModelFilterStage(profiles));
pipeline.addStage(new CacheOrderStage());
pipeline.addStage(new BudgetStage(counter, profile, store));
pipeline.addStage(new LoadStage(store));

// Insert a custom stage before budget enforcement
pipeline.insertBefore('budget', new MyCustomStage());

// Insert after scoring
pipeline.insertAfter('score', new MyPostScoreStage());

---

Flow 3: Pattern Detection to Agent Composition

Session ends
  → SessionObserver.onSessionEnd(data)
     → TranscriptParser.parse(transcriptPath)    // Parse Claude Code transcript
     → PatternSummarizer.summarize(entries)       // Extract behavioral patterns
     → ObservationRateLimiter.checkLimit(id)      // Rate limit check
     → PromotionEvaluator.evaluate(summary)       // Ephemeral or persistent?
     → PatternStore.append('sessions', summary)   // Append to JSONL

Pattern Detection (periodic)
  → PatternAnalyzer.analyze(sessionsPath)
     → Stream JSONL, build frequency maps
     → Filter by threshold (3+ occurrences)
     → Calculate recency-weighted confidence
     → Return SkillCandidate[] sorted by confidence

User Approves Candidate
  → SkillGenerator.generateScaffold(candidate)
     → scanForDangerousCommands(body)            // SEC-05
     → sanitizeGeneratedContent(body)             // Block dangerous content
     → inferAllowedTools(candidate)               // SEC-07
     → detectArguments(body)                      // SPEC-02
     → shouldForkContext(description, body)        // SPEC-05
     → ContentDecomposer.decompose(name, meta, body) // DISC-01
  → SkillStore.create(name, metadata, body)

Agent Composition (after N sessions)
  → CoActivationTracker.analyze(sessions)
     → Find skill pairs with 3+ co-activations
     → Score by frequency and recency
  → ClusterDetector.detectClusters(coActivations)
     → Group related skills into clusters
  → AgentGenerator.generateContent(cluster)
     → Load skill descriptions
     → validateAgentFrontmatter()
     → Format agent markdown with skills list
  → TeamValidator.validateTeamFull(config)        // If team topology
  → .claude/agents/{name}.md written to disk

PatternAnalyzer Detail

// src/detection/pattern-analyzer.ts
export class PatternAnalyzer {
  private config: DetectionConfig;

  async analyze(sessionsPath: string): Promise<SkillCandidate[]> {
    const freq = await this.countPatterns(sessionsPath); // Stream JSONL
    return this.extractCandidates(freq);
  }

  // Frequency map tracks commands, files, tools, co-occurrences, timestamps
  private initFrequencyMap(): FrequencyMap {
    return {
      commands: new Map(),
      files: new Map(),
      tools: new Map(),
      coOccurrences: new Map(),
      sessionTimestamps: new Map(),
      sessionIds: new Map(),
    };
  }

  // Confidence = base (occurrences/10, max 0.7) + recency boost (max 0.3)
  // Total capped at 1.0
}

CoActivationTracker Detail

// src/agents/co-activation-tracker.ts
export class CoActivationTracker {
  analyze(sessions: SessionObservation[]): SkillCoActivation[] {
    // Filter to recent sessions (within recencyDays)
    // Generate all skill pairs per session (combinations, not permutations)
    // Alphabetically sort pairs for consistent keys
    // Count co-activations, track sessions and timestamps
    // Return pairs exceeding minCoActivations threshold
  }

  getCoActivationScore(skillA: string, skillB: string, sessions: SessionObservation[]): number {
    // Score = frequencyScore * 0.7 + recencyBoost * 0.3
    // frequencyScore = coActivationCount / totalRecentSessions
    // recencyBoost = 1 - (daysSinceLastSeen / recencyDays)
  }
}

Observation Tier System

Session observation arrives
  → PromotionEvaluator.evaluate(summary)
     → Sufficient signal? (tool diversity, file count, command count)
     YES → tier = 'persistent' → PatternStore.append()
     NO  → tier = 'ephemeral'  → EphemeralStore.append()

After each session end:
  → ObservationSquasher.squash(ephemeralEntries)
     → Aggregate all ephemeral entries into one
     → Re-evaluate aggregate for promotion
     → Strong enough? → Promote to persistent
  → EphemeralStore.clear()
  → RetentionManager.prune(sessionsFile)

---

Shared Components Across Flows

Component	Flow 1: Create	Flow 2: Activate	Flow 3: Detect/Compose
SkillStore	Write skills	Read skills (LoadStage)	Read metadata, write generated skills
SkillIndex	Rebuild after create	Trigger matching (ScoreStage)	Not used directly
EmbeddingService	Not used	Semantic scoring (ScoreStage)	Not used
BudgetValidator	Pre-write budget check	Runtime budget enforcement (BudgetStage)	Generated skill budget check
PathSafety	Name/path validation	Not used	Agent file path validation
SkillResolver	Not used	Inheritance resolution (ResolveStage)	Not used
PatternStore	Not used	Not used	Append observations, read for analysis