File System Capability Technical Design

1. Overview

The file system capability is an infrastructure enhancement layered on top of the Planning module in harness9, addressing three core problems:

Problem	Solution
Oversized tool output blows up the context window	OffloadHook — automatically writes output exceeding a threshold to a file, keeping only a summary reference in context
The Agent's execution plan exists only in memory and is lost on process restart	FilePlanWriter — writes a markdown plan file in sync with every todo_write call
Offload files remain on disk after a session is deleted	Manager.DeleteSession cascades cleanup of ~/.harness9/tool_results/{sessionID}/

These three problems are solved via the hook mechanism (Hooks), interface injection (PlanWriter), and the options pattern (ManagerOption), each of which can be independently enabled or disabled without changing the engine's core logic.

---

2. Architecture: Hooks Interception Layer

2.1 The ToolHook Interface

// internal/hooks/hook.go
type ToolHook interface {
    BeforeExecute(ctx context.Context, tc schema.ToolCall) (context.Context, error)
    AfterExecute(ctx context.Context, tc schema.ToolCall, result schema.ToolResult) schema.ToolResult
}

• When BeforeExecute returns an error, it short-circuits the entire call chain, returning a ToolResult with IsError: true without executing the inner tool
• AfterExecute can modify result, executing in reverse order (onion model)

2.2 HookRegistry

HookRegistry implements the tools.Registry interface, wrapping the original Registry:

User input → HookRegistry.Execute
           ├─ BeforeExecute (forward): hook[0] → hook[1] → …
           ├─ inner.Execute (original tool)
           └─ AfterExecute (reverse): … → hook[1] → hook[0]

With zero hooks, behavior is fully identical to the original Registry — the engine is unaware of whether a Hook layer exists.

// Assembly example (main.go)
// The first argument to OffloadHook is workDir; files are written to workDir/.harness9/tool_results/{sessionID}/
offloadHook := hooks.NewOffloadHook(workDir, sess.SessionID())
hookReg := hooks.NewHookRegistry(registry, offloadHook)
eng := engine.NewAgentEngine(llm, hookReg, workDir, ...)

---

3. Context Offload (Oversized Output Offloading)

3.1 Design Motivation

When tool output has no upper bound (e.g., bash running grep -r across a large codebase), a single output may consume tens of thousands of tokens, causing:

• The context window to be exhausted within one or two turns
• The LLM to be unable to process subsequent instructions
• A sudden spike in compaction pressure, with semantic loss in summaries

The core idea of Offload: move the data out of context, leaving a "pointer" in context. The LLM can retrieve it on demand via read_file + offset/limit.

3.2 Implementation: OffloadHook

// internal/hooks/offload.go
type OffloadHook struct {
    workDir      string  // Agent workspace root directory; offload files are written to its .harness9 subdirectory
    sessionID    string
    threshold    int     // default 10000 characters
    previewLines int     // default 20 lines
}

Trigger conditions:

• len(result.Output) > threshold (default 10,000 characters)
• The tool is not in the exclusion list {read_file, write_file, edit_file} (to avoid read/write loops)

Execution flow:

1. os.MkdirAll({workDir}/.harness9/tool_results/{sessionID}/, 0700)
2. os.WriteFile({dir}/{toolCallID}.txt, full output, 0600)
3. Replace result.Output with a summary reference (using a path relative to workDir, which the LLM can pass directly to read_file):

[Output saved to .harness9/tool_results/{sessionID}/{id}.txt, 847 lines / 32416 bytes total.
Can be read in pages via the read_file tool with offset/limit parameters.

Preview (first 20 lines):
...(first 20 lines of content)...
...(truncated)]

4. Fail-open: if os.MkdirAll or os.WriteFile fails, the original result is returned unchanged, without interrupting the agent loop

3.3 File Naming Convention

{workDir}/.harness9/
└── tool_results/
    └── {sessionID}/
        ├── {toolCallID-1}.txt   # first output exceeding the threshold
        ├── {toolCallID-2}.txt   # second one
        └── ...

toolCallID is generated by the engine on each tool call (UUID), corresponding one-to-one with the reference path in context.

3.4 read_file Pagination Extension

To support the LLM retrieving offload files in segments, the read_file tool adds offset / limit parameters:

{
  "path": "relative or absolute path",
  "offset": 4096,   // starting byte (optional, default 0)
  "limit": 4096     // number of bytes to read (optional, default 4096)
}

Boundary handling:

• offset >= totalSize: returns [offset=N exceeds file size (T bytes), nothing to read.] without an error
• When the amount read exceeds limit (detected by reading one extra byte): a truncation notice "to continue reading use offset=N" is appended, allowing the LLM to auto-continue reading

Sandbox restriction: offset-based reads still go through safePath() validation, so path traversal attacks are ineffective.

3.5 System Prompt Integration

PromptBuilder.WithOffloadEnabled(true) injects retrieval guidance into the System Prompt:

## Large Output File Retrieval

When a tool's output exceeds the threshold, the full content is automatically saved to the file system, and only a path reference and preview are shown in context.
To view the full output, use the read_file tool with offset/limit parameters for paginated reading:
- offset: starting byte position (default 0)
- limit: number of bytes to read (default 4096)
Example: read_file({"path": "/path/to/offload/file.txt", "offset": 4096, "limit": 4096})

---

4. Plan Persistence (FilePlanWriter)

4.1 Design Motivation

The Planning module's TodoStore keeps the task list in memory (restored from SQLite each time a session starts). However, users often want the plan saved to the project directory in a human-readable format, to make it convenient to:

• View the current execution progress in an IDE or text editor
• Track the AI's task execution history in a git repository

4.2 The PlanWriter Interface (Decoupled Design)

// internal/planning/plan_writer.go
type PlanWriter interface {
    Write(todos []TodoItem) error
}

The interface is defined in the planning package (the consumer side), avoiding a tools → hooks → tools circular import:

tools.TodoWriteTool
  └─ planning.PlanWriter (interface)
       └─ hooks.FilePlanWriter (implementation)

4.3 FilePlanWriter Implementation

// internal/hooks/plan_writer.go
type FilePlanWriter struct {
    path      string  // absolute path of the plan file, determined at construction time and unchanged thereafter
    sessionID string
}

Path selection strategy:

• Detect whether workDir/.git exists
  • Git project: write to {workDir}/.harness9/plans/{timestamp}-{sessionID[:8]}.md
  • Non-Git directory: write to {homeDir}/.harness9/plans/{timestamp}-{sessionID[:8]}.md

Fail fast at construction: if os.MkdirAll fails, an error is returned immediately (rather than lazy creation), ensuring permission issues are discovered at startup.

File content format:

# Execution Plan

session: abc12345-...
updated: 2026-05-22T15:30:00+08:00

## Task List

- [ ] Create directory structure
- [>] Initialize go.mod
- [x] Implement main.go
- [-] Delete old file (cancelled)

Status marker mapping:

TodoStatus	Marker
pending	[ ]
in_progress	[>]
completed	[x]
cancelled	[-]

Fail-open: when Write fails, the todo_write tool only logs it, without interrupting the agent loop:

// tools/todo_write.go
if err := t.planWriter.Write(current); err != nil {
    log.Print(logfmt.FormatMsg("todo_write", fmt.Sprintf("failed to write plan file: %v", err)))
}

4.4 Injection Method

Injected via the options pattern; skipped (no-op) when nil:

tools.NewTodoWriteTool(todoStore, tools.WithPlanWriter(planWriter))

---

5. Session GC (Cascading Cleanup of Offload Files)

5.1 Problem

When a user deletes a session via /new or another mechanism, the corresponding offload files (~/.harness9/tool_results/{sessionID}/) need to be cleaned up in sync, to avoid long-term accumulation of disk usage.

5.2 Implementation

memory.Manager accepts the offload root directory via the WithToolResultsDir option:

mgr, err := memory.NewManager(
    filepath.Join(homeDir, ".harness9", "sessions.db"),
    memory.WithToolResultsDir(toolResultsDir),
)

After deleting the SQLite record, DeleteSession cascades cleanup of the corresponding directory:

func (m *Manager) DeleteSession(ctx context.Context, id string) error {
    _, err := m.db.ExecContext(ctx, `DELETE FROM sessions WHERE id = ?`, id)
    if err != nil {
        return fmt.Errorf("delete session: %w", err)
    }
    if m.toolResultsDir != "" {
        _ = os.RemoveAll(filepath.Join(m.toolResultsDir, id))
    }
    return nil
}

Errors from os.RemoveAll are silently ignored (a file system GC failure should not affect session deletion semantics).

---

6. Data Flow Overview

User input → engine.runLoop
         └─ hookReg.Execute(toolCall)
              ├─ OffloadHook.BeforeExecute (no-op)
              ├─ inner.Registry.Execute (actual tool execution)
              │    └─ (bash / read_file / write_file / ...)
              └─ OffloadHook.AfterExecute
                   ├─ len(output) ≤ threshold → returned unchanged
                   └─ len(output) > threshold
                        ├─ os.WriteFile(~/.harness9/tool_results/{sid}/{id}.txt)
                        └─ result.Output = summary reference + preview

When the LLM needs the full content:
    read_file({path, offset, limit}) → returns file content in pages

Each time todo_write writes:
    TodoStore.Write → planWriter.Write
                      └─ os.WriteFile({workDir}/.harness9/plans/{ts}-{sid}.md)

When a session is deleted:
    Manager.DeleteSession
        ├─ SQL DELETE (cascades deletion of messages, todos)
        └─ os.RemoveAll(~/.harness9/tool_results/{sessionID}/)

---

7. File System Directory Structure

~/.harness9/
├── sessions.db                          # SQLite session database
├── tool_results/
│   ├── {sessionID-1}/
│   │   ├── {toolCallID-a}.txt           # oversized output from a bash call
│   │   └── {toolCallID-b}.txt
│   └── {sessionID-2}/
│       └── ...
└── plans/                               # plan storage location for non-git directories
    └── {timestamp}-{sessionID[:8]}.md

{workDir}/.harness9/
└── plans/                               # plan storage location for git projects
    └── {timestamp}-{sessionID[:8]}.md

---

8. Configuration Parameters

Parameter	Location	Default	Description
threshold	OffloadHook	10,000 characters	Offload is triggered above this length
previewLines	OffloadHook	20 lines	Number of preview lines retained in context
maxReadLen	read_file	4,096 bytes	Single-read upper bound when limit is not specified
toolResultsDir	Manager	~/.harness9/tool_results	Offload root directory; GC is disabled when empty string

---

9. Extension: Custom Hooks

Implementing the ToolHook interface allows injecting new behavior, e.g., adding audit logging:

type AuditHook struct{ log *slog.Logger }

func (h *AuditHook) BeforeExecute(ctx context.Context, tc schema.ToolCall) (context.Context, error) {
    h.log.Info("tool start", "name", tc.Name, "id", tc.ID)
    return ctx, nil
}

func (h *AuditHook) AfterExecute(ctx context.Context, tc schema.ToolCall, result schema.ToolResult) schema.ToolResult {
    h.log.Info("tool done", "name", tc.Name, "is_error", result.IsError)
    return result
}

// Insert into HookRegistry during assembly
hookReg := hooks.NewHookRegistry(registry, offloadHook, &AuditHook{log: logger})

The execution order of multiple hooks follows the onion model: BeforeExecute forward, AfterExecute reverse.