# Worker Pool

> **Platform Reference** — describes implemented, production behavior.

The `WorkerPool` class adds distributed job processing to Acequia groups. Peers declare capabilities, and submitters dispatch jobs directly to capable workers over WebRTC — no coordinator, no leader election.

```javascript
import acequia from '/acequia.js'

// Worker: handles jobs
const pool = new acequia.groups.WorkerPool('my-pool', {
  capabilities: ['thumbnail'],
  concurrency: 2,
  handler: async (job) => await makeThumbnail(job.payload),
})

// Submitter: dispatches jobs
const pool = new acequia.groups.WorkerPool('my-pool')
const ticket = pool.submit({ type: 'thumbnail', payload: { path: '/photo.jpg' } })
const result = await ticket.result
```

---

## When to Use WorkerPool

| Use WorkerPool when... | Use [Routes](creating-acequia-apps.md#route-registration) when... | Use [Shared State](group-shared-state.md) when... | Use [Event Bus](event-bus.md) when... |
|---|---|---|---|
| Multiple peers handle the same work type | One canonical handler per URL pattern | All peers need the same data | Events are ephemeral |
| You need load distribution | You need `fetch()` semantics | Updates should reach ALL members | Fire-and-forget messaging |
| Jobs need retry, timeout, progress | Simple request/response is enough | Data is small metadata/config | Topic-based filtering |
| Workers may come and go dynamically | A stable handler is expected | Last-write-wins convergence | High-frequency updates |

---

## Quick Start

### Minimal Worker

```javascript
const pool = new acequia.groups.WorkerPool('my-pool', {
  capabilities: ['thumbnail'],
  concurrency: 2,
  handler: async (job) => await makeThumbnail(job.payload),
})
```

A peer with a `handler` is a worker. It joins the group, advertises its capabilities, and processes jobs dispatched to it.

### Minimal Submitter

```javascript
const pool = new acequia.groups.WorkerPool('my-pool')
const ticket = pool.submit({ type: 'thumbnail', payload: { path: '/photo.jpg' } })
const result = await ticket.result
```

A peer without a `handler` is a submitter. It joins the same group, discovers available workers, and dispatches jobs to them.

### Combined Worker + Submitter

A single peer can be both. The `handler` makes it a worker; `submit()` makes it a submitter:

```javascript
const pool = new acequia.groups.WorkerPool('media-pool', {
  capabilities: ['thumbnail', 'ocr'],
  concurrency: 3,
  handler: async (job, ctx) => {
    ctx.progress({ status: 'starting' })
    if (job.type === 'thumbnail') return await makeThumbnail(job.payload)
    if (job.type === 'ocr') return await runOCR(job.payload)
  },
})

// Submit a job (may be handled by this peer or another)
const ticket = pool.submit({
  type: 'ocr',
  payload: { path: '/document.pdf' },
  timeout: 60000,
  retries: 2,
})

ticket.onProgress(data => console.log(data.status))
ticket.onComplete(result => console.log('Done:', result))
ticket.onFailed(err => console.error('Failed:', err))
```

---

## Dispatch Strategies

```javascript
new acequia.groups.WorkerPool('pool', { dispatch: 'least-loaded' })
```

| Strategy | Behavior |
|----------|----------|
| `round-robin` (default) | Rotate through capable peers in sorted order |
| `least-loaded` | Pick peer with lowest `activeJobs/concurrency` ratio |
| `random` | Random capable peer |
| `affinity` | Prefer the peer that last handled this job type (cache locality) |
| `(workers, job) => id` | Custom function returning a worker instanceId |

---

## Job Lifecycle

A submitted job moves through these states:

```
queued → dispatched → (progress) → completed / failed / timed_out / lost
```

- **queued** — waiting for a capable worker
- **dispatched** — sent to a worker, awaiting result
- **progress** — worker sent an intermediate update (optional)
- **completed** — worker returned a result
- **failed** — worker threw an error or explicitly failed
- **timed_out** — no response within timeout (default 30s)
- **lost** — worker disconnected before completing

Failed jobs with `retries > 0` are automatically resubmitted to another worker.

---

## API

### WorkerPool Constructor

```javascript
const pool = new acequia.groups.WorkerPool(groupIdOrGroup, options?)
```

| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `capabilities` | `string[]` | `[]` | Job types this worker handles |
| `concurrency` | `number` | `1` | Max simultaneous jobs |
| `handler` | `async (job, ctx) => any` | — | Job handler (makes this peer a worker) |
| `dispatch` | `string \| Function` | `'round-robin'` | Dispatch strategy |
| `displayName` | `string` | — | Display name in group |

The first argument can be a group ID string (creates a new Group internally) or an existing `Group` instance.

### Submitting Jobs

```javascript
const ticket = pool.submit({
  type: string,           // must match a worker capability
  payload: any,           // serializable data
  timeout: number,        // ms (default 30000)
  retries: number,        // retry count (default 0)
  retryOnDisconnect: boolean, // resubmit if worker drops (default true)
})
```

### JobTicket

| Property | Type | Description |
|----------|------|-------------|
| `id` | `string` | Unique job ID (`crypto.randomUUID()`) |
| `status` | `string` | `'pending'` \| `'offered'` \| `'completed'` \| `'failed'` \| `'timed_out'` \| `'lost'` |
| `worker` | `string` | Assigned worker's instanceId |
| `result` | `Promise` | Resolves on complete, rejects on fail/timeout |

| Method | Description |
|--------|-------------|
| `onComplete(fn)` | Callback when job completes successfully |
| `onFailed(fn)` | Callback when job fails |
| `onProgress(fn)` | Callback for intermediate progress updates |
| `cancel()` | Cancel the job if not yet completed |

### Pool State

| Property / Method | Type | Description |
|-------------------|------|-------------|
| `pool.workers` | `Map<instanceId, peerInfo>` | Connected workers |
| `pool.activeJobs` | `number` | This worker's active job count |
| `pool.completedJobs` | `number` | This worker's completed count |
| `pool.group` | `Group` | Underlying Group instance |
| `pool.getTicket(id)` | `JobTicket` | Look up ticket by ID |

### Events

| Method | Description |
|--------|-------------|
| `pool.onWorkerJoin(fn)` | Worker joined the pool |
| `pool.onWorkerLeave(fn)` | Worker left the pool |
| `pool.onJobEvent(fn)` | Job lifecycle event (for observers/dashboards) |

### Lifecycle

```javascript
await pool.close()  // stop accepting, finish active jobs, clean up
```

---

## Patterns

### Route-Backed Workers

A registered route (single entry point, SW-interceptable) can delegate to a worker pool (distributed execution). Callers get `fetch()` ergonomics while work is distributed across peers:

```javascript
const pool = new acequia.groups.WorkerPool('media-pool', {
  capabilities: ['thumbnail'],
  concurrency: 2,
  handler: async (job) => makeThumbnail(job.payload),
})

pool.group.registerRoute('/_process', async (request) => {
  const ticket = pool.submit({ type: request.body.jobType, payload: request.body.payload })
  try {
    const result = await ticket.result
    return { status: 200, body: JSON.stringify({ result }) }
  } catch (err) {
    return { status: 500, body: JSON.stringify({ error: err.message }) }
  }
})

// Now any page can call: fetch('/groups/media-pool/_process', { ... })
```

### Observer Pattern

Join the group passively to receive job lifecycle events for monitoring:

```javascript
const observer = new acequia.groups.WorkerPool('media-pool', {
  capabilities: ['pool-observer'],
})

observer.onJobEvent(event => {
  // event.event: 'job:started', 'job:completed', 'job:failed', etc.
  console.log(`${event.event}: ${event.jobType} on ${event.worker}`)
})
```

### Dashboard Component

The `<worker-pool-dashboard>` web component renders live worker stats:

```html
<script type="module" src="/components/worker-pool-dashboard/worker-pool-dashboard.js"></script>

<!-- Attach to an existing pool -->
<worker-pool-dashboard id="dashboard"></worker-pool-dashboard>
<script type="module">
  document.getElementById('dashboard').pool = myPool
</script>

<!-- Or standalone observer (auto-joins the group) -->
<worker-pool-dashboard group="my-pool-group"></worker-pool-dashboard>
```

---

## Gotchas

**Self-dispatch.** A WorkerPool with a `handler` includes itself in the workers list. When the selected worker is the local peer, the handler runs directly — no WebRTC round-trip. A single tab can test job dispatch this way.

**Two data channels.** Stats go via the discovery server (`peersList` broadcasts); job payloads and results go via WebRTC data channels. Don't conflate them.

**Reuse peer connections.** The WorkerPool caches peers internally. If building custom peer-to-peer features alongside it, check `getPeers()` before calling `connectToPeer()`.

**State-leader capability.** Workers automatically get the `state-leader` capability, enabling [Group Shared State](group-shared-state.md) without extra configuration.

**Service worker caching.** After rebuilding `acequia.js`, hard-refresh the page or the browser may serve a stale cached version.

For the full set of patterns and gotchas, see [App Patterns & Field Notes](app-patterns-field-notes.md#worker-pool--distributed-job-processing).