Development

Guide for building Choir from source and understanding the codebase.

Building from Source

Prerequisites

• Go 1.21+ (module at go 1.25.6)
• Docker (for container builds and testing)
• git

Full Build

# Download dependencies
go mod download

# Build all three binaries
bash build.sh

This produces:

Agent-Only Build

To build only the container runtime binary (useful for image rebuilds):

bash build.sh --agent

Build Flags

The build script sets version metadata via ldflags:

Project Structure

choir/
  choird/              # Host daemon entry point
  choirctl/            # CLI client entry point
  choir-agent/         # Container runtime entry point
  internal/            # Shared internal packages
    protocol/          # RPC verbs and event types
    config/            # Configuration parsing
    docker/            # Container management
    logging/           # Structured logging
    models/            # LLM provider abstractions
    transport/         # UDS and HTTP transport
  bin/                 # Build output (gitignored)
  build.sh             # Build script
  install.sh           # Reconcile/update host install
  go.mod               # Go module definition
  DESIGN.md            # Complete design specification
  docs/                # Documentation

Monorepo Layout

Choir is a single Go module with three entry points (choird/, choirctl/, choir-agent/) and shared internal packages. This keeps all components versioned together and sharing the same type definitions.

• choird/ – the host daemon. Manages containers, Telegram gateway, memory, approvals, and all control plane logic.
• choirctl/ – stateless CLI. Communicates with a running choird instance.
• choir-agent/ – the in-container runtime. Built as a static linux/amd64 binary for Docker images.
• internal/ – shared packages used by multiple entry points.

Key Interfaces

Transport Interfaces

Agent-side (choir-agent connects to choird):

// ClientTransport is what choir-agent uses to talk to choird.
type ClientTransport interface {
    Send(ctx context.Context, verb Verb, payload []byte) ([]byte, error)
    OpenEventStream(ctx context.Context, sessionID string) (<-chan Event, error)
    Close() error
}

Server-side (choird listens for agent connections):

// ServerTransport is what choird exposes to choir-agent.
type ServerTransport interface {
    Listen(ctx context.Context) error
    Close() error
}

Control Plane Interface

Agent-side (high-level API over transport):

type ControlPlane interface {
    Heartbeat(ctx context.Context, req HeartbeatReq) (HeartbeatResp, error)
    RequestApproval(ctx context.Context, req ApprovalReq) (ApprovalResp, error)
    GetSecrets(ctx context.Context, req SecretReq) (SecretResp, error)
    Terminate(ctx context.Context, req TerminateReq) error
}

Server-side handler:

type ControlPlaneHandler interface {
    HandleHeartbeat(ctx context.Context, req HeartbeatReq) (HeartbeatResp, error)
    HandleApproval(ctx context.Context, req ApprovalReq) (ApprovalResp, error)
    HandleSecrets(ctx context.Context, req SecretReq) (SecretResp, error)
    HandleTerminate(ctx context.Context, req TerminateReq) error
}

Both interfaces are transport-agnostic. Transport selection happens once at startup; business logic never branches on transport type.

Tool Interface

type Tool interface {
    Name() string
    Schema() JSONSchema
    Execute(ctx context.Context, input json.RawMessage) (json.RawMessage, error)
}

Shared by both built-in tools (Go implementations) and external tools (manifest + executable).

RPC Protocol

JSON-over-HTTP (or UDS). No protobuf or gRPC in v1. Schema source of truth is Go struct definitions with JSON tags.

Common envelope for all requests:

{
  "request_id": "uuid",
  "session_id": "...",
  "lease_token": "...",
  "verb": "HEARTBEAT",
  "payload": { ... }
}

8 RPC verbs: INIT_HELLO, GET_SECRETS, HEARTBEAT, REQUEST_APPROVAL, REPORT_STATUS, TERMINATE_SELF, FETCH_DYNAMIC_CONFIG, EXECUTE_HOST_TOOL.

See internal/protocol/verbs.go for verb constants and internal/protocol/events.go for event types.

Dependencies

Choir aims for a minimal dependency set:

External runtime dependencies:

Testing Approach

Unit Tests

Individual components (lock manager, arbiter, skill engine, config parser) are tested in isolation with Go’s standard testing package.

Integration Tests

Test the full RPC protocol over both UDS and HTTP transports. Validate that all 8 verbs produce identical behavior regardless of transport.

Container Tests

Build a test agent image and run it through lifecycle operations: start, heartbeat, tool execution, graceful stop, crash recovery.

Running Tests

go test ./...

Implementation Phases

The project is built in 8 phases:

1. Control Plane Protocol – Transport abstraction, UDS, container lifecycle, handshake, heartbeat.
2. Single-Lane Tool Loop – Structured tool calling, lock manager, tool execution, skill engine.
3. Gateway & User Interface – Telegram gateway, DM routing, file transfer, TTS, email tools.
4. TCP/HTTP Transport – HTTP implementation, TLS, lease-token auth.
5. Core Lane Async – Core lane, injection/cancel, event streaming.
6. Crash Recovery – Heartbeat replication, ack protocol, snapshots, recovery handshake.
7. Memory Integration – Postgres/pgvector schema, embedding pipeline, hybrid search, compaction.
8. Self-Evolution & Hardening – Approval workflows, tool-builder/skill-builder, logging, security.

For the full design specification, see DESIGN.md.