Manifest and Project Layout

The manifest is the contract that tells goalseek what the provider can see, what it can modify, and how results are verified.

Project layout

goalseek project init creates a scaffold like this:

<project_name>/
  .git/
  .gitignore
  manifest.yaml
  program.md
  setup.py
  validate_results.py
  experiment.py
  data/
  context/
  hidden/
  config/
    project.yaml
  runs/
  logs/

Important directories

• context/: visible reference material for the provider
• data/: read-only project data when declared in the manifest
• hidden/: files used by setup or verification that should not be visible to the provider
• runs/: iteration artifacts
• logs/: resumable state and append-only result history

Key files

File	Default mode	Purpose
program.md	read_only	Reusable project instructions for planning and implementation.
experiment.py	writable	Main experiment implementation the provider may modify.
validate_results.py	hidden	Verifier harness run by manifest commands, not shown to the provider.

Scope modes

The manifest separates files into four modes:

Mode	Meaning
read_only	visible to the provider but not writable
writable	visible and editable
generated	artifacts written by the loop such as runs/** and logs/**
hidden	intentionally excluded from provider context

Metric extraction

Verification commands decide whether a candidate is valid. Metric extraction then turns verifier output into a scalar value the loop can compare over time.

JSON file

Use a verifier that writes structured output, then point the metric extractor at the JSON location.

stdout regex

Capture a metric directly from standard output when the verifier prints a stable line format.

stderr regex

Useful when tools report the metric on standard error instead of standard output.

Minimal manifest shape

version: 1
project:
  name: goalseek_demo

files:
  - path: manifest.yaml
    mode: read_only
  - path: program.md
    mode: read_only
  - path: setup.py
    mode: read_only
  - path: validate_results.py
    mode: hidden
  - path: experiment.py
    mode: writable
  - path: data/**
    mode: read_only
  - path: context/**
    mode: read_only
  - path: hidden/**
    mode: hidden
  - path: config/**
    mode: read_only
  - path: runs/**
    mode: generated
  - path: logs/**
    mode: generated

verification:
  commands:
    - name: train
      run: python3 experiment.py
      cwd: .
      timeout_sec: 1200
    - name: evaluate
      run: python3 validate_results.py --evaluate --output runs/latest/results.json
      cwd: .

metric:
  name: score
  direction: maximize
  extractor:
    type: json_file
    path: runs/latest/results.json
    json_pointer: /metric

execution:
  target: local

Hidden files are a real boundary

hidden/ is meant for data the provider should not see directly, such as sensitive inputs, evaluation logic, or ground truth the agent could otherwise game.

Defaults worth knowing

• runs/ and logs/ are git-ignored in scaffolded projects.
• program.md is read-only context; experiment.py is writable.
• validate_results.py is hidden and should be called only through verification commands.
• The working tree must be clean before run applies a change.
• Rejected changes are reverted with git revert, not discarded with history rewriting.