# PR: Add Config-Driven Experiment Runner, File Inputs, and Unified Non-Interactive CLI Support

## Title

runner: add config-driven experiment execution, file-based inputs, and unified batch support for GrowNet CLI

---

## Summary

This PR adds the **remaining non-interactive runner capabilities** so GrowNet can be driven by configuration files and real inputs, not just hardcoded scenarios or interactive key presses.

The goal is to evolve the CLI into a real **GrowNet experiment runner** that supports:

- initial scaffold definition through config
- 1D / 2D / 3D input-space declaration
- initial neuron / layer / region counts
- file-based input sources
- sequence / tensor inputs as first-class types
- image / audio inputs as adapters into tensor-like inputs
- shared config usage for:
  - batch mode
  - interactive mode
  - trace mode
  - benchmark mode

This PR should create a clean foundation for experimentation without forcing Python to become Java-like.

---

## Important architectural rule

Respect GrowNet's language-reference model:

- **Java** = core semantic / architectural reference
- **Python** = Python-language-style reference
- **Mojo** should track Python almost one-to-one
- **C++** follows Java as much as practical

For this PR:

- implement the config/input experiment runner in **Python first**
- keep Python explicit and readable
- do **not** force Python internals into Java structure
- design the config model so Mojo can later follow Python closely

---

## Main idea

The CLI should evolve from a simple scenario launcher into a **config-driven experiment runner**.

That means these should both make sense:

```bash
./grownet.sh --lang python --config experiments/single_scalar.json
./grownet.sh --lang python --interactive --config experiments/dual_scalar_probe.json
```

And eventually:

```bash
./grownet.sh --lang python --trace --config experiments/image_test.json
./grownet.sh --lang python --benchmark --config experiments/growth_bench.json
```

So the config becomes the common description of:

- initial scaffold
- input-space shape
- data source
- run mode defaults
- output settings

---

## Required reading before coding

Read these before implementation:

1. `docs/READ_ORDER.md`
2. `docs/LANGUAGE_REFERENCE_POLICY.md`
3. `docs/CODING_STYLE_MUST_READ.md`
4. `docs/GOLDEN_RULE.md`
5. `docs/GROWTH.md`
6. `docs/CREATION_AND_GROWTH_POINTS.md`
7. `docs/contracts/grownet.contract.v5.json`
8. `docs/GrowNet_Design_Spec_V5.md`
9. `docs/TRACE_RUNNER.md`
10. `docs/TRACE_PARITY.md`
11. `docs/PARITY_DECISION_CHECKLIST.md`
12. `docs/INTERACTIVE_RUNNER.md`
13. `docs/BENCHMARK_RUNNER.md` if present

---

## Coding style requirements

These are mandatory:

- Follow `docs/CODING_STYLE_MUST_READ.md`
- No single- or double-character identifiers anywhere, including loops
- Keep Python explicit and readable
- Avoid clever one-liners
- Prefer small, obvious helper modules
- Preserve determinism
- Preserve two-phase tick discipline
- Do not change runtime semantics just to make config handling easier
- Do not introduce speculative redesigns outside runner/input plumbing

---

## Scope

### Implement in this PR

1. config-driven non-interactive execution
2. unified experiment config schema
3. file-based inputs
4. sequence and tensor input support
5. image and audio input adapters
6. explicit initial scaffold definition
7. 1D / 2D / 3D input-space declaration
8. integration hooks for trace / benchmark / interactive modes
9. docs and tests

### Do not implement in this PR

- Java full config-driven runtime unless trivial
- Mojo full config-driven runtime
- C++ full config-driven runtime
- Blender integration
- rich visualization UI
- arbitrary live 3D editor
- large runtime architecture rewrites unrelated to runner/input support

---

# Phase 1 — CLI surface

## 1. Add config-driven runner flags

Extend the CLI to support at least:

```text
--config path/to/experiment.json
--input path/to/input_file
--input-type sequence|tensor|image|audio
--output path/to/output_file_or_dir
--mode run|trace|benchmark|interactive
```

### Notes

- `--config` is the primary way to define the experiment
- `--input` can override the input path in the config
- `--input-type` can override or clarify the input source type
- `--mode` may be optional if existing flags like `--trace`, `--benchmark`, `--interactive` remain the top-level selectors

### Preferred behavior

The runner should support both:

```bash
./grownet.sh --lang python --config experiments/foo.json
```

and

```bash
./grownet.sh --lang python --trace --config experiments/foo.json
```

### Important

Do not break the existing scenario-based path immediately.
This PR should be additive.

---

# Phase 2 — experiment config model

## 2. Introduce a unified experiment config schema

Create a clean JSON-based schema that describes:

- experiment metadata
- runtime mode defaults
- initial scaffold
- input-space shape
- input source
- output settings

### Example experiment config

```json
{
  "name": "dual_scalar_probe",
  "runtime": {
    "ticks": 10,
    "trace": false,
    "trace_format": "jsonl",
    "benchmark": false,
    "interactive": false
  },
  "input_space": {
    "rank": 1,
    "shape": [2],
    "ports": [
      {"name": "a"},
      {"name": "b"}
    ]
  },
  "network": {
    "regions": [
      {
        "name": "main",
        "layers": [
          {
            "excitatory_count": 20,
            "inhibitory_count": 4,
            "modulatory_count": 2
          }
        ]
      }
    ]
  },
  "input": {
    "type": "sequence",
    "values": [0.1, 0.2, 0.3]
  },
  "output": {
    "trace_file": "out/trace.jsonl",
    "metrics_file": "out/metrics.json"
  }
}
```

### Important rule

This config defines the **initial scaffold**, not a fixed final architecture.
GrowNet still remains free to grow.

---

# Phase 3 — initial scaffold support

## 3. Support scaffold definition in config

The config should let the user define at least:

- region count
- layer count per region
- excitatory / inhibitory / modulatory counts
- input-space rank and shape
- optional growth-policy overrides if already supported by the runtime

### Required initial support

At minimum, support:

- one region
- one or more layers
- per-layer neuron counts by type
- input-space rank:
  - `1`
  - `2`
  - `3`

### Important

This is **initial scaffold only**.
Do not block growth afterwards.

---

# Phase 4 — input types

## 4. Implement `sequence` input

This should be the simplest and most important first-class input.

### Examples

Inline values:

```json
{
  "input": {
    "type": "sequence",
    "values": [0.1, 0.2, 0.3, 0.4]
  }
}
```

File-backed values:

```json
{
  "input": {
    "type": "sequence",
    "path": "inputs/sequence.json"
  }
}
```

### Sequence semantics

One item corresponds to one tick for scalar mode unless the config says otherwise.

## 5. Implement `tensor` input

This becomes the general-purpose structured input type.

### Examples

```json
{
  "input": {
    "type": "tensor",
    "path": "inputs/frame.json"
  }
}
```

This should support:
- 1D arrays
- 2D arrays
- 3D arrays

### Important

`tensor` is the universal bridge for:
- images
- audio windows
- generic 3D volumes

## 6. Implement `image` input adapter

Image support should be implemented as an adapter into tensor-like input.

### Examples

```json
{
  "input": {
    "type": "image",
    "path": "inputs/sample.png",
    "mode": "grayscale",
    "resize": [64, 64]
  }
}
```

### Important

Do not build image support as a completely separate world.
Convert it into the same internal structured input path used by tensor handling.

## 7. Implement `audio` input adapter

Audio support should also be implemented as an adapter.

### Examples

```json
{
  "input": {
    "type": "audio",
    "path": "inputs/sample.wav",
    "sample_rate": 16000,
    "frame_size": 512,
    "hop_size": 256
  }
}
```

### Important

Audio should be windowed/framed and then passed into a structured input path.
Do not make this a one-off special execution path if that can be avoided.

---

# Phase 5 — 1D / 2D / 3D clarity

## 8. Make dimensionality explicit

Support clear declaration of input dimensionality through config.

Preferred model:

```json
{
  "input_space": {
    "rank": 3,
    "shape": [32, 32, 32]
  }
}
```

or for 2D:

```json
{
  "input_space": {
    "rank": 2,
    "shape": [64, 64]
  }
}
```

### Important

This should affect:
- validation
- loader expectations
- signal routing setup
- trace expectations
- future interactive/profile support

---

# Phase 6 — runner architecture

## 9. Add a Python experiment-runner layer

Create a small, explicit module structure, for example:

```text
src/python/runner/
    __init__.py
    experiment_config.py
    experiment_loader.py
    scaffold_builder.py
    input_loaders.py
    run_mode_dispatch.py
```

### Responsibilities

- `experiment_config.py` — config dataclasses / validation
- `experiment_loader.py` — parse JSON config and apply overrides
- `scaffold_builder.py` — build initial region/layer scaffold
- `input_loaders.py` — sequence/tensor/image/audio adapters
- `run_mode_dispatch.py` — route into run / trace / benchmark / interactive flows

Keep files small and obvious.

## 10. Reuse this from interactive mode

Interactive mode should eventually consume the same config/scaffold machinery.

That means this PR should not build a second unrelated config stack.
Instead, it should create reusable pieces that interactive mode can also call.

---

# Phase 7 — run semantics

## 11. Define non-interactive execution semantics clearly

For batch run mode, define exactly how input is consumed.

### Sequence input
- one scalar per tick
- or one vector per tick if input-space rank/shape requires it

### Tensor input
- one full tensor per tick
- or list-of-tensors means one tensor per tick

### Image input
- single image = one tick unless config says otherwise

### Audio input
- framed windows = one frame per tick

### Important

Document this clearly.
Do not leave ambiguity.

---

# Phase 8 — output and persistence

## 12. Add optional output persistence

Support optional output targets such as:

- trace output file
- metrics summary file
- benchmark output file

### Example

```json
{
  "output": {
    "trace_file": "out/trace.jsonl",
    "metrics_file": "out/metrics.json"
  }
}
```

### Important

Keep output persistence optional and simple.

---

# Phase 9 — docs

## 13. Add a dedicated experiment-runner doc

Create:

```text
docs/EXPERIMENT_RUNNER.md
```

Include:

- what config-driven execution is
- example configs
- sequence / tensor / image / audio inputs
- initial scaffold definition
- 1D / 2D / 3D declaration
- how config interacts with trace / benchmark / interactive modes
- example commands

## 14. Update related docs

Update:
- `docs/TRACE_RUNNER.md`
- `docs/INTERACTIVE_RUNNER.md`
- `docs/BENCHMARK_RUNNER.md` if needed

Make it clear that all modes can increasingly share the same config model.

---

# Phase 10 — tests

## 15. Add focused tests

At minimum, test:

- config parsing
- scaffold building
- sequence input loading
- tensor input loading
- image adapter plumbing
- audio adapter plumbing
- 1D / 2D / 3D validation
- config override behavior from CLI
- batch run mode dispatch

### Important

Do not write tests that depend on huge real media files.
Use tiny fixtures or mocked adapters where reasonable.

---

# Phase 11 — incremental rollout rules

## 16. Keep rollout honest

If some parts are not fully mature yet:

- mark them clearly as provisional
- especially image/audio if the downstream runtime path is still limited

For example:
- `sequence` and `tensor` can be fully supported
- `image` and `audio` may initially be adapters into the closest supported structured path

That is acceptable as long as it is **documented honestly**

---

## Acceptance criteria

This PR is complete when:

1. `--config` works in non-interactive Python runner mode
2. initial scaffold can be defined through config
3. the runner supports at least:
   - `sequence`
   - `tensor`
4. `image` and `audio` have clear adapter support or honest provisional support
5. 1D / 2D / 3D input-space declaration is explicit
6. batch run mode can execute from config without hardcoded scenarios
7. docs explain the experiment-runner model clearly
8. tests cover config, scaffold, and input loading
9. no runtime semantics were changed just to simplify config/file support

---

## Explicit non-goals

Do not do these in this PR:

- Java full config-driven runner unless trivial
- Mojo full config-driven runner
- C++ full config-driven runner
- Blender integration
- rich visualization UI
- arbitrary live 3D editing
- broad runtime redesign unrelated to experiment/config execution

---

## Suggested commit structure

1. add experiment config and loader
2. add scaffold builder and batch dispatch
3. add sequence/tensor input support
4. add image/audio adapters
5. add docs and tests

---

## Final response expected from Codex

At the end, provide:

1. summary of what changed
2. files changed
3. example config files
4. example commands for:
   - batch run
   - trace with config
   - benchmark with config
   - interactive with config
5. which input types are fully supported vs provisional
6. tests run
7. any follow-up ideas for Java/Mojo/C++ support
