Module: PWN::Migrate

Defined in:
lib/pwn/migrate.rb

Overview

PWN::Migrate — the ~/.pwn state doctor and auto-migrator.

PWN persists a growing set of files under ~/.pwn (encrypted config, memory, learning outcomes, metrics, mistakes, extrospection, cron jobs, agents, skills, sessions, swarm buses, …). Each file is owned by a different module and each release can add keys or change shape. A user upgrading gem install pwn between two versions could therefore hit KeyError, NoMethodError for nil, or a silent empty-fallback because their on-disk state predates the new loader.

PWN::Migrate closes that gap. It is called from:

* `pwn setup --migrate`

It works by:

1. Stamping `~/.pwn/.schema` with `{ schema:, pwn_version:, at: }`
 the first time it runs.  Any release that changes the on-disk
 shape of a `~/.pwn` file bumps `SCHEMA_VERSION` here and appends
 an idempotent lambda to `MIGRATIONS`.
2. Declaratively verifying every state file against its OWNING
 module's own loader (`STATE_FILES`) — no re-implemented parsers.
 If the raw file has bytes but the owner returned its
 empty-fallback, the file is flagged incompatible.
3. Autofixing:  timestamped backup → ordered schema migrations →
 per-file repair (quarantine corrupt / re-seed missing) →
 `pwn.yaml` deep-key backfill (missing keys from the current
 `PWN::Config.env_template` are merged in-place under the user's
 values, then the vault is re-encrypted with the SAME key/iv).

Everything is idempotent, dry-run capable, and never overwrites a user-set value.

Constant Summary collapse

PWN_ROOT = 
File.join(Dir.home, '.pwn')
SCHEMA_FILE = 
File.join(PWN_ROOT, '.schema')
BACKUP_ROOT = 
File.join(PWN_ROOT, 'backup')
QUARANTINE_DIR = 
File.join(PWN_ROOT, 'quarantine')
SCHEMA_VERSION =

Bump this whenever the shape of any file under ~/.pwn changes in a way that requires a one-time transform. Add the transform as an entry in MIGRATIONS keyed by the NEW schema number.

7
OK = 
"\e[32mok\e[0m"
BAD = 
"\e[31mFAIL\e[0m"
WARN = 
"\e[33mdrift\e[0m"
STATE_FILES =

────────────────────────────────────────────────────────────────── Declarative registry of every persisted ~/.pwn artefact.

Each entry knows how to (a) locate itself, (b) ask its owning module to load it, and (c) decide whether the loaded shape is what this pwn version expects. Verifiers are intentionally SHALLOW — they detect "loader gave up and returned the fallback" rather than re-validating every key, so ownership stays with the module.

:fix values:

:quarantine

──────────────────────────────────────────────────────────────────

{
  'pwn.yaml' => {
    owner: 'PWN::Config',
    kind: :vault,
    fix: :vault,
    verify: lambda { |p, _raw|
      return { ok: false, why: 'missing' } unless File.file?(p)

      enc = PWN::Plugins::Vault.file_encrypted?(file: p)
      # Shape (missing keys vs env_template) is checked separately in
      # .vault_drift because it requires the decryptor.
      { ok: enc, why: enc ? nil : 'not encrypted (run PWN::Plugins::Vault.create)' }
    }
  },
  'memory.json' => {
    owner: 'PWN::Memory',
    kind: :json,
    fix: :quarantine,
    verify: lambda { |_p, raw|
      m = PWN::Memory.load
      bad = m.reject { |_k, v| v.is_a?(Hash) && v.key?(:value) }
      { ok: !(m.empty? && raw.to_s.length > 4) && bad.empty?,
        why: bad.any? ? "#{bad.length} entries missing :value" : nil }
    }
  },
  'memory.idx' => {
    owner: 'PWN::MemoryIndex',
    kind: :blob,
    fix: :quarantine,
    verify: ->(_p, _raw) { { ok: true } }
  },
  'learning.jsonl' => {
    owner: 'PWN::AI::Agent::Learning',
    kind: :jsonl,
    fix: :jsonl,
    verify: :jsonl
  },
  'preferences.jsonl' => {
    owner: 'PWN::AI::Agent::Reward',
    kind: :jsonl,
    fix: :jsonl,
    verify: :jsonl
  },
  'metrics.json' => {
    owner: 'PWN::AI::Agent::Metrics',
    kind: :json,
    fix: :quarantine,
    verify: lambda { |_p, raw|
      m = PWN::AI::Agent::Metrics.load
      fell_back = m == { tools: {}, updated_at: nil } && raw.to_s.length > 4
      { ok: m.is_a?(Hash) && m.key?(:tools) && !fell_back,
        why: fell_back ? 'loader fell back to empty (unparsable / wrong shape)' : nil }
    }
  },
  'mistakes.json' => {
    owner: 'PWN::AI::Agent::Mistakes',
    kind: :json,
    fix: :quarantine,
    verify: lambda { |_p, raw|
      m = PWN::AI::Agent::Mistakes.load
      fell_back = m.empty? && raw.to_s.length > 4
      { ok: m.is_a?(Hash) && !fell_back, why: fell_back ? 'loader fell back to {}' : nil }
    }
  },
  'extrospection.json' => {
    owner: 'PWN::AI::Agent::Extrospection',
    kind: :json,
    fix: :quarantine,
    verify: lambda { |_p, raw|
      m = PWN::AI::Agent::Extrospection.load
      shape = m.is_a?(Hash) && m.key?(:observations) && m[:observations].is_a?(Array)
      fell_back = shape && m[:snapshot].to_h.empty? && m[:observations].empty? && raw.to_s.length > 8
      { ok: shape && !fell_back, why: shape ? nil : 'missing :observations array' }
    }
  },
  'reward_sentinel.json' => {
    owner: 'PWN::AI::Agent::Reward',
    kind: :json,
    fix: :quarantine,
    verify: ->(_p, _raw) { { ok: true } }
  },
  'agents.yml' => {
    owner: 'PWN::AI::Agent::Swarm',
    kind: :yaml,
    fix: :quarantine,
    verify: lambda { |_p, raw|
      m = PWN::AI::Agent::Swarm.personas
      fell_back = m.empty? && raw.to_s.strip.length > 4
      bad = m.reject { |_k, v| v.is_a?(Hash) && v[:role] }
      { ok: !fell_back && bad.empty?,
        why: (fell_back && 'loader fell back to {}') || (bad.any? && "#{bad.length} personas missing :role") || nil }
    }
  },
  'cron/jobs.yml' => {
    owner: 'PWN::Cron',
    kind: :yaml,
    fix: :seed,
    verify: lambda { |_p, raw|
      j = PWN::Cron.list
      fell_back = j.empty? && raw.to_s.strip.length > 8
      bad = j.reject { |_id, v| v.is_a?(Hash) && v[:schedule] }
      { ok: !fell_back && bad.empty?,
        why: (fell_back && 'loader fell back to {}') || (bad.any? && "#{bad.length} jobs missing :schedule") || nil }
    }
  },
  'skills/' => {
    owner: 'PWN::Config',
    kind: :dir,
    fix: :skills,
    verify: lambda { |p, _raw|
      return { ok: true } unless File.directory?(p)

      legacy = Dir.glob(File.join(p, '*.{rb,md,txt,skill,yml,yaml}')).length
      { ok: legacy.zero?, why: legacy.zero? ? nil : "#{legacy} legacy flat skill file(s)" }
    }
  },
  'sessions/' => { owner: 'PWN::Sessions', kind: :dir, fix: :seed, verify: ->(_p, _r) { { ok: true } } },
  'swarm/' => { owner: 'PWN::AI::Agent::Swarm', kind: :dir, fix: :seed, verify: ->(_p, _r) { { ok: true } } },
  'curriculum/' => { owner: 'PWN::AI::Agent::Curriculum', kind: :dir, fix: :seed, verify: ->(_p, _r) { { ok: true } } },
  'finetune/' => { owner: 'PWN::AI::Agent::Curriculum', kind: :dir, fix: :seed, verify: ->(_p, _r) { { ok: true } } },
  'extrospection/' => { owner: 'PWN::AI::Agent::Extrospection', kind: :dir, fix: :seed, verify: ->(_p, _r) { { ok: true } } }
}.freeze
MIGRATIONS =

Ordered, idempotent one-shot transforms. Key == the schema number a ~/.pwn tree is AT after the lambda runs. Add a new entry every time SCHEMA_VERSION is bumped. Each lambda receives (root, io).

{
  1 => lambda { |root, io|
    # v0 → v1 : agentskills.io directory layout + seeded RL cron jobs
    #           + ensure all state subdirs exist.
    %w[skills sessions swarm cron curriculum finetune extrospection].each do |d|
      FileUtils.mkdir_p(File.join(root, d))
    end
    r = begin
      PWN::Config.migrate_legacy_skills
    rescue StandardError
      { migrated: 0 }
    end
    io.puts "    · migrate_legacy_skills → #{r[:migrated]} converted" if r[:migrated].to_i.positive?
    PWN::Config.install_default_skills if defined?(PWN::Config)
    PWN::Cron.install_defaults if defined?(PWN::Cron)
  },
  2 => lambda { |root, io|
    path = File.join(root, 'agents.yml')
    r = if defined?(PWN::AI::Agent::Swarm) && PWN::AI::Agent::Swarm.respond_to?(:migrate_personas)
          PWN::AI::Agent::Swarm.migrate_personas(path: path)
        else
          { changed: false }
        end
    io.puts "    · agents.yml #{Array(r[:patched]).join(', ')}" if r[:changed]
  },
  3 => lambda { |root, io|
    path = File.join(root, 'agents.yml')
    next unless File.file?(path)

    raw = YAML.safe_load_file(path, permitted_classes: [Symbol], aliases: true)
    next unless raw.is_a?(Hash)

    patched = 0
    raw.each_value do |persona|
      next unless persona.is_a?(Hash)
      next if persona.key?('model') || persona.key?(:model)

      persona['model'] = nil
      patched += 1
    end
    next if patched.zero?

    File.write(path, YAML.dump(raw))
    io.puts "    · agents.yml added default model to #{patched} personas"
  },
  4 => lambda { |_root, io|
    io.puts '    · pwn.yaml ai_sandbox, ai_router, model_routes task classes'
    PWN::Migrate.backfill_vault(io: io)
  },
  5 => lambda { |_root, io|
    io.puts '    · pwn.yaml ai.agent.skill_review'
    PWN::Migrate.backfill_vault(io: io)
  },
  6 => lambda { |_root, io|
    io.puts '    · pwn.yaml ai.tui.theme'
    PWN::Migrate.backfill_vault(io: io)
  },
  7 => lambda { |_root, io|
    io.puts '    · pwn.yaml ai.tui.theme.title'
    PWN::Migrate.backfill_vault(io: io)
  }
}.freeze

Class Method Summary collapse

Class Method Details

.authors ⇒ Object

Author(s)

0day Inc. [email protected]



717
718
719
720
721
 
# File 'lib/pwn/migrate.rb', line 717

public_class_method def self.authors
  "AUTHOR(S):
    0day Inc. <[email protected]>
  "
end

.backfill_vault(opts = {}) ⇒ Object

Supported Method Parameters

r = PWN::Migrate.backfill_vault( key: 'optional', iv: 'optional', dry_run: false, io: $stdout )

Decrypt → deep-merge template UNDER user (never overwrites) → re-encrypt with the SAME key/iv. This is the fix for the #1 upgrade break: NoMethodError: undefined method '[]' for nil when a release added env[:ai][:new_engine][:…] and the user's vault predates it.



460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
 
# File 'lib/pwn/migrate.rb', line 460

public_class_method def self.backfill_vault(opts = {})
  io      = opts[:io] || $stdout
  dry_run = opts[:dry_run] ? true : false
  d = vault_drift(key: opts[:key], iv: opts[:iv])
  unless d
    io.puts '  vault    : skipped (no decryptor available — run with --pwn-dec or set key:/iv:)'
    return nil
  end
  if d[:missing].empty?
    io.puts "  vault    : #{OK} (no missing keys vs PWN::Config.env_template)"
    return { rel: 'pwn.yaml', added: 0, missing: [] }
  end

  io.puts "  vault    : #{WARN} #{d[:missing].length} missing key(s) — backfilling under user values"
  d[:missing].first(8).each { |p| io.puts "             + #{p}" }
  io.puts "             … (+#{d[:missing].length - 8} more)" if d[:missing].length > 8
  return { rel: 'pwn.yaml', added: d[:missing].length, missing: d[:missing], dry_run: true } if dry_run

  merged = deep_defaults(user: d[:user], tmpl: d[:template])
  yaml   = File.join(PWN_ROOT, 'pwn.yaml')
  key, iv = decryptor(opts)
  # Same colon-stripping default_env uses so the file stays diff-clean.
  File.write(yaml, YAML.dump(merged).gsub(/^(\s*):/, '\1'))
  File.chmod(0o600, yaml)
  PWN::Plugins::Vault.encrypt(file: yaml, key: key, iv: iv)

  { rel: 'pwn.yaml', added: d[:missing].length, missing: d[:missing] }
rescue StandardError => e
  io.puts "  vault    : #{BAD} #{e.class}: #{e.message}"
  nil
end

.check(opts = {}) ⇒ Object

Supported Method Parameters

PWN::Migrate.check( io: 'optional - IO to write the report to (default $stdout)' )

Human-readable ~/.pwn state doctor. Read-only. Called from PWN::Setup.check so pwn setup shows this alongside the native-gem / toolchain report.



332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
 
# File 'lib/pwn/migrate.rb', line 332

public_class_method def self.check(opts = {})
  io = opts[:io] || $stdout
  st = status

  io.puts "~/.pwn state         schema #{st[:schema_installed]} → #{st[:schema_current]} " \
          "#{st[:needs_migration] ? "(#{WARN} — run `pwn setup --migrate`)" : "(#{OK})"}"
  st[:files].each do |f|
    mark = f[:ok] ? OK : BAD
    note = f[:ok] ? '' : "→ #{f[:why]}  [fix: #{f[:fix]}]"
    sz   = f[:exists] ? human_size(bytes: f[:size]) : '—'
    io.puts "  #{f[:rel].ljust(22)} #{mark}   #{sz.to_s.ljust(8)} #{f[:owner].to_s.ljust(30)} #{note}"
  end
  unless st[:incompatible].empty?
    io.puts
    io.puts "#{st[:incompatible].length} file(s) incompatible with pwn #{st[:pwn_version]} — " \
            'run `pwn setup --migrate --fix` to autofix (a timestamped backup is taken first).'
  end
  st
rescue StandardError => e
  raise e
end

.env_template ⇒ Object

Supported Method Parameters

tmpl = PWN::Migrate.env_template

The canonical current-release pwn.yaml shape, WITHOUT the I/O side-effects of PWN::Config.default_env. Prefers PWN::Config.env_template when that refactor is present.



499
500
501
502
503
504
505
 
# File 'lib/pwn/migrate.rb', line 499

public_class_method def self.env_template
  return PWN::Config.env_template if PWN::Config.respond_to?(:env_template)

  # Fallback: intercept default_env's write path.  Never reached once
  # PWN::Config.env_template ships, but keeps Migrate self-sufficient.
  {}
end

.help ⇒ Object

Display Usage for this Module



725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
 
# File 'lib/pwn/migrate.rb', line 725

public_class_method def self.help
  puts "USAGE:
    # Run installed schema and return its result
    #{self}.installed_schema

    # Run needed and return its result
    #{self}.needed?

    # Run status and return its result
    #{self}.status

    # Human-readable ~/.pwn state doctor.  Read-only.  Called from
    #{self}.check(
      io: 'optional - IO to write the report to (default $stdout)'
    )

    # Run run and return its result
    #{self}.run(
      fix: 'optional - also autofix incompatible files (default: schema-migrations only)',
      backup: 'optional - take timestamped backup of ~/.pwn first (default true)',
      dry_run: 'optional - report what WOULD happen, write nothing (default false)',
      key: 'optional - vault key   (default: read ~/.pwn/pwn.yaml.decryptor)',
      iv: 'optional - vault iv    (default: read ~/.pwn/pwn.yaml.decryptor)',
      io: 'optional - IO to write to (default $stdout)'
    )

    # Run vault drift and return its result
    #{self}.vault_drift(
      key: 'optional - vault key, iv: optional - vault iv'
    )

    # Run backfill vault and return its result
    #{self}.backfill_vault(
      key: 'optional - optional, iv: optional, dry_run: false, io: $stdout',
      io: 'optional - io value consumed by #backfill_vault',
      dry_run: 'optional - dry run value consumed by #backfill_vault',
      iv: 'optional - iv value consumed by #backfill_vault'
    )

    # Run env template and return its result
    #{self}.env_template

    # Print the AUTHOR(S) string for this module.
    #{self}.authors
  "
  constants.sort
end

.installed_schema ⇒ Object

Supported Method Parameters

schema = PWN::Migrate.installed_schema

→ Integer schema number stamped in ~/.pwn/.schema (0 when absent).



272
273
274
275
276
277
278
 
# File 'lib/pwn/migrate.rb', line 272

public_class_method def self.installed_schema
  return 0 unless File.file?(SCHEMA_FILE)

  JSON.parse(File.read(SCHEMA_FILE))['schema'].to_i
rescue StandardError
  0
end

.needed? ⇒ Boolean

Supported Method Parameters

bool = PWN::Migrate.needed?

Cheap predicate for PWN::Config.refresh_env — schema stamp only, no per-file probes, so it adds ~0 to launch time.

Returns:

  • (Boolean) 


286
287
288
289
290
 
# File 'lib/pwn/migrate.rb', line 286

public_class_method def self.needed?
  File.directory?(PWN_ROOT) && installed_schema < SCHEMA_VERSION
rescue StandardError
  false
end

.run(opts = {}) ⇒ Object

Supported Method Parameters

report = PWN::Migrate.run( fix: 'optional - also autofix incompatible files (default: schema-migrations only)', backup: 'optional - take timestamped backup of ~/.pwn first (default true)', dry_run: 'optional - report what WOULD happen, write nothing (default false)', key: 'optional - vault key (default: read ~/.pwn/pwn.yaml.decryptor)', iv: 'optional - vault iv (default: read ~/.pwn/pwn.yaml.decryptor)', io: 'optional - IO to write to (default $stdout)' )

  1. backup 2. schema migrations installed→current 3. per-file autofix (when fix:true) 4. pwn.yaml key backfill (when fix:true and decryptor available) 5. stamp .schema.


368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
 
# File 'lib/pwn/migrate.rb', line 368

public_class_method def self.run(opts = {})
  fix     = opts[:fix] ? true : false
  dry_run = opts[:dry_run] ? true : false
  backup  = opts.fetch(:backup, true)
  io      = opts[:io] || $stdout

  FileUtils.mkdir_p(PWN_ROOT)
  before = status
  io.puts "PWN::Migrate — pwn #{before[:pwn_version]} · ~/.pwn schema " \
          "#{before[:schema_installed]} → #{before[:schema_current]}" \
          "#{'  (dry-run)' if dry_run}"

  bpath = nil
  if backup && !dry_run && File.directory?(PWN_ROOT)
    bpath = backup!
    io.puts "  backup   : #{bpath}"
  end

  # ── ordered schema migrations ────────────────────────────────────
  applied = []
  ((before[:schema_installed] + 1)..SCHEMA_VERSION).each do |n|
    m = MIGRATIONS[n]
    next unless m

    io.puts "  migrate  : v#{n - 1} → v#{n}"
    m.call(PWN_ROOT, io) unless dry_run
    applied << n
  end

  # ── per-file autofix ─────────────────────────────────────────────
  fixed = []
  if fix
    before[:incompatible].each do |f|
      io.puts "  fix      : #{f[:rel].ljust(22)} [#{f[:fix]}]  #{f[:why]}"
      next if dry_run

      fixed << f.merge(result: apply_fix(file: f))
    end

    # pwn.yaml key backfill runs even when the vault verifier passed —
    # "encrypted" ≠ "has every key this release added".
    vd = backfill_vault(key: opts[:key], iv: opts[:iv], dry_run: dry_run, io: io)
    fixed << vd if vd && vd[:added].to_i.positive?
  end

  stamp! unless dry_run

  after = status
  io.puts
  io.puts "  result   : #{after[:incompatible].length} incompatible remaining " \
          "(was #{before[:incompatible].length}) · schema now #{after[:schema_installed]}"

  { backup: bpath, applied_migrations: applied, fixed: fixed,
    before: before, after: after, dry_run: dry_run }
rescue StandardError => e
  raise e
end

.status ⇒ Object

Supported Method Parameters

rows = PWN::Migrate.status

Per-file compatibility report — the machine-readable half of .check. Never writes. Never raises (each verifier is rescued).



298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
 
# File 'lib/pwn/migrate.rb', line 298

public_class_method def self.status
  files = STATE_FILES.map do |rel, meta|
    path = File.join(PWN_ROOT, rel)
    exists = meta[:kind] == :dir ? File.directory?(path) : File.file?(path)
    raw = exists && meta[:kind] != :dir ? safe_read(path: path) : nil
    v = verify_one(path: path, raw: raw, meta: meta)
    {
      rel: rel, path: path, owner: meta[:owner], kind: meta[:kind],
      exists: exists, size: exists ? File.size?(path) : nil,
      ok: v[:ok], why: v[:why], fix: meta[:fix]
    }
  end
  {
    root: PWN_ROOT,
    schema_installed: installed_schema,
    schema_current: SCHEMA_VERSION,
    pwn_version: (defined?(PWN::VERSION) ? PWN::VERSION : nil),
    needs_migration: needed?,
    files: files,
    incompatible: files.reject { |f| f[:ok] }
  }
rescue StandardError => e
  raise e
end

.vault_drift(opts = {}) ⇒ Object

Supported Method Parameters

drift = PWN::Migrate.vault_drift( key: 'optional - vault key', iv: 'optional - vault iv' )

Deep-diff the user's decrypted pwn.yaml against the current PWN::Config.env_template. → { missing: [dotted.paths], user: … } Returns nil when the decryptor is unavailable (never prompts).



435
436
437
438
439
440
441
442
443
444
445
446
447
 
# File 'lib/pwn/migrate.rb', line 435

public_class_method def self.vault_drift(opts = {})
  key, iv = decryptor(opts)
  return nil unless key && iv

  yaml = File.join(PWN_ROOT, 'pwn.yaml')
  return nil unless File.file?(yaml) && PWN::Plugins::Vault.file_encrypted?(file: yaml)

  user = PWN::Plugins::Vault.dump(file: yaml, key: key, iv: iv)
  tmpl = env_template
  { missing: missing_paths(tmpl: tmpl, user: user), user: user, template: tmpl }
rescue StandardError
  nil
end