> ## Documentation Index
> Fetch the complete documentation index at: https://docs.standardagents.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# @standardagents/sip

> Ultra memory-efficient image processing for edge computing - process 100MP+ images with less than 1MB peak memory

## Overview

The `@standardagents/sip` package (**S**mall **I**mage **P**rocessor) provides memory-efficient image processing designed for memory-constrained edge computing environments.

> [card] Key Features
>
> * Process 100MP+ images with less than 1MB peak memory
> * DCT-based scaling for JPEG (decode at reduced resolution)
> * Scanline-by-scanline processing (never holds full image in memory)
> * Native WASM codecs (libjpeg-turbo, libspng)
> * Works in edge runtimes, browsers, and Node.js

## Why sip?

Edge computing environments typically have strict memory limits (e.g., 128MB). Traditional image processing libraries decode the entire image into memory:

| Image Size | Decoded Memory |
| - | - |
| 12MP (4000×3000) | \~36MB |
| 25MP (6000×4166) | \~75MB |
| 50MP (8688×5792) | \~151MB |
| 100MP (11375×8992) | \~307MB |

> [!WARNING]
>
> A single 25MP image upload can crash your edge function without sip's streaming approach.

**sip's solution:**

1. **DCT Scaling** - Decode JPEG at 1/2, 1/4, or 1/8 scale during decompression
2. **Scanline Streaming** - Process one row at a time (\~50KB peak for any image size)
3. **Native WASM** - libjpeg-turbo and libspng compiled to WebAssembly

## Installation

```bash npm theme={null}
npm install @standardagents/sip
```

```bash pnpm theme={null}
pnpm add @standardagents/sip
```

```bash yarn theme={null}
yarn add @standardagents/sip
```

> [!WARNING]
>
> JPEG and PNG processing requires the WASM module to be built. See [WASM Build](#wasm-build) below.

## Quick Start

```typescript theme={null}
import { sip, probe } from "@standardagents/sip"
import { readFile, writeFile } from "fs/promises"

// Read an image file
const imageBuffer = await readFile("my-image.jpg")

// Get image info without decoding
const info = probe(imageBuffer)
console.log(info)
// { format: 'jpeg', width: 4000, height: 3000, hasAlpha: false }

// Process image: resize and convert to JPEG
const result = await sip.process(imageBuffer, {
  maxWidth: 2048,
  maxHeight: 2048,
  quality: 85,
})

console.log(result)
// {
//   data: ArrayBuffer,
//   width: 2048,
//   height: 1536,
//   mimeType: 'image/jpeg',
//   originalFormat: 'jpeg'
// }

// Save the resized image
await writeFile("my-image-resized.jpg", Buffer.from(result.data))
```

## Functions

### probe

Detect image format and dimensions by reading only the header bytes. No full decode occurs.

```typescript theme={null}
import { probe } from "@standardagents/sip"

const info = probe(imageBuffer)
// { format: 'jpeg', width: 4000, height: 3000, hasAlpha: false }
```

**Parameters:**

| Parameter | Type | Description |
| - | - | - |
| `data` | `ArrayBuffer \| Uint8Array` | Image data |

**Returns:** `ProbeResult`

```typescript theme={null}
interface ProbeResult {
  format: "jpeg" | "png" | "webp" | "avif" | "unknown"
  width: number   // 0 if unknown
  height: number  // 0 if unknown
  hasAlpha: boolean
}
```

**Format Detection:**

| Format | Magic Bytes |
| - | - |
| JPEG | `FF D8 FF` |
| PNG | `89 50 4E 47 0D 0A 1A 0A` |
| WebP | `RIFF....WEBP` |
| AVIF | `....ftyp` with `avif`/`avis` brand |

### sip.process

Process an image: decode, resize, and encode to JPEG.

```typescript theme={null}
import { sip } from "@standardagents/sip"

const result = await sip.process(imageBuffer, {
  maxWidth: 2048,
  maxHeight: 2048,
  maxBytes: 1.5 * 1024 * 1024,
  quality: 85,
})
```

**Parameters:**

| Parameter | Type | Description |
| - | - | - |
| `data` | `ArrayBuffer` | Input image data |
| `options` | `ProcessOptions` | Processing configuration |

**Options:**

| Option | Type | Default | Description |
| - | - | - | - |
| `maxWidth` | `number` | `4096` | Maximum output width in pixels |
| `maxHeight` | `number` | `4096` | Maximum output height in pixels |
| `maxBytes` | `number` | `1572864` (1.5MB) | Target maximum file size |
| `quality` | `number` | `85` | JPEG quality (1-100) |

**Returns:** `Promise<ProcessResult>`

```typescript theme={null}
interface ProcessResult {
  data: ArrayBuffer        // Output JPEG data
  width: number            // Output width
  height: number           // Output height
  mimeType: "image/jpeg"   // Always JPEG
  originalFormat: ImageFormat  // Input format detected
}
```

**Behavior:**

* **Aspect Ratio**: Always preserved. Output fits within `maxWidth×maxHeight` box
* **Quality Reduction**: If output exceeds `maxBytes`, retries with lower quality
* **Size Reduction**: If still over `maxBytes` at minimum quality, resizes smaller

### initStreaming

Initialize the WASM module. Optional but recommended for reducing first-call latency.

```typescript theme={null}
import { initStreaming } from "@standardagents/sip"

// Call early in your application startup
const available = await initStreaming()
console.log(`WASM available: ${available}`)
```

**Returns:** `Promise<boolean>` - `true` if WASM loaded successfully.

## Format Support

### Input Formats

| Format | Decoder | Method | Memory Usage |
| - | - | - | - |
| **JPEG** | libjpeg-turbo (WASM) | DCT scaling + scanline | Minimal |
| **PNG** | libspng (WASM) | Progressive scanline | Minimal |
| **WebP** | @jsquash/webp | Full decode | Higher |
| **AVIF** | @jsquash/avif | Full decode | Higher |

### Output Format

> [!NOTE]
>
> Output is always **JPEG**. This simplifies the encoder and provides universal browser compatibility.

## Memory Model

### DCT Scaling (JPEG only)

JPEG uses Discrete Cosine Transform (DCT) for compression. libjpeg-turbo can decode at reduced resolution **during decompression**:

| Scale | Output Size | Memory Savings |
| - | - | - |
| 1/1 | Full size | None |
| 1/2 | 50% each dimension | 4× less |
| 1/4 | 25% each dimension | 16× less |
| 1/8 | 12.5% each dimension | 64× less |

**Example:** 6800×4500 image (30.6MP)

| Scale | Output | Decode Memory |
| - | - | - |
| 1/1 | 6800×4500 | \~92MB |
| 1/2 | 3400×2250 | \~23MB |
| 1/4 | 1700×1125 | \~5.7MB |
| 1/8 | 850×563 | \~1.4MB |

sip automatically selects the optimal scale based on target dimensions.

### Scanline Streaming

Instead of decoding the entire image, sip processes row-by-row:

```
Input Image (any size)
        ↓
Decode Row 0 → Resize → Encode  ← Only these rows in memory
Decode Row 1 → Resize → Encode
        ↓
Output JPEG (streamed)
```

**Memory per row** (2000px width): \~24KB total, constant regardless of image height.

## WASM Build

### Prerequisites

Install the Emscripten SDK:

```bash theme={null}
# Clone emsdk
git clone https://github.com/emscripten-core/emsdk.git
cd emsdk

# Install and activate latest version
./emsdk install latest
./emsdk activate latest

# Add to current shell
source ./emsdk_env.sh
```

### Building

```bash theme={null}
git clone git@github.com:standardagents/sip.git
cd sip
pnpm install
pnpm build:wasm
```

This will:

1. Download libjpeg-turbo, libspng, and miniz to `wasm/libs/`
2. Compile with Emscripten
3. Output `dist/sip.js` and `dist/sip.wasm`

> [!WARNING]
>
> The WASM files are not committed to git. Your CI/CD pipeline must run `pnpm build:wasm`.

## Configuration Examples

**Recommended Settings by Use Case:**

| Use Case | maxWidth | maxHeight | maxBytes | quality |
| - | - | - | - | - |
| Thumbnails | 400 | 400 | 50KB | 80 |
| Preview | 1024 | 1024 | 200KB | 82 |
| Standard | 2048 | 2048 | 1.5MB | 85 |
| High Quality | 4096 | 4096 | 5MB | 90 |

## Examples

### Thumbnail Generation

```typescript theme={null}
import { sip } from "@standardagents/sip"

async function createThumbnail(buffer: ArrayBuffer): Promise<ArrayBuffer> {
  const result = await sip.process(buffer, {
    maxWidth: 200,
    maxHeight: 200,
    maxBytes: 30 * 1024, // 30KB max
    quality: 80,
  })
  return result.data
}
```

### Upload Handler with Validation

```typescript theme={null}
import { sip, probe } from "@standardagents/sip"

const ALLOWED_FORMATS = ["jpeg", "png", "webp", "avif"]
const MAX_INPUT_SIZE = 50 * 1024 * 1024 // 50MB

async function handleImageUpload(buffer: ArrayBuffer) {
  // Validate input size
  if (buffer.byteLength > MAX_INPUT_SIZE) {
    throw new Error(`Image too large: ${buffer.byteLength} bytes`)
  }

  // Validate format
  const info = probe(buffer)
  if (!ALLOWED_FORMATS.includes(info.format)) {
    throw new Error(`Unsupported format: ${info.format}`)
  }

  // Process
  return await sip.process(buffer, {
    maxWidth: 4096,
    maxHeight: 4096,
    maxBytes: 2 * 1024 * 1024,
    quality: 85,
  })
}
```

### Edge Function Integration

```typescript theme={null}
import { sip, probe, initStreaming } from "@standardagents/sip"

// Warm up WASM on startup
let wasmReady = initStreaming()

export default {
  async fetch(request: Request): Promise<Response> {
    await wasmReady

    if (request.method !== "POST") {
      return new Response("Method not allowed", { status: 405 })
    }

    try {
      const buffer = await request.arrayBuffer()
      const result = await sip.process(buffer, {
        maxWidth: 2048,
        maxHeight: 2048,
        quality: 85,
      })

      return new Response(result.data, {
        headers: {
          "Content-Type": "image/jpeg",
          "Content-Length": String(result.data.byteLength),
        },
      })
    } catch (error) {
      return new Response(`Processing failed: ${error.message}`, { status: 500 })
    }
  },
}
```

### Multiple Size Generation

```typescript theme={null}
import { sip } from "@standardagents/sip"

async function generateSizes(buffer: ArrayBuffer) {
  const [thumbnail, preview, full] = await Promise.all([
    sip.process(buffer, { maxWidth: 150, maxHeight: 150, quality: 80 }),
    sip.process(buffer, { maxWidth: 800, maxHeight: 800, quality: 82 }),
    sip.process(buffer, { maxWidth: 2048, maxHeight: 2048, quality: 85 }),
  ])

  return {
    thumbnail: thumbnail.data,
    preview: preview.data,
    full: full.data,
  }
}
```

## TypeScript Support

The package includes full TypeScript definitions:

```typescript theme={null}
import type {
  ProbeResult,
  ProcessOptions,
  ProcessResult,
  ImageFormat,
} from "@standardagents/sip"
```

## Limitations

> [accordion] Output is always JPEG
>
> No PNG, WebP, or AVIF output. All processed images become JPEG.

> [accordion] No alpha channel preservation
>
> Transparency is discarded. PNG/WebP with alpha become opaque JPEG.

> [accordion] WASM required for JPEG/PNG
>
> Without WASM, JPEG and PNG processing throws an error. WebP and AVIF still work but use more memory.

> [accordion] maxBytes is best-effort
>
> The output may slightly exceed `maxBytes`. The algorithm tries quality reduction first, then dimension reduction.

> [accordion] No EXIF rotation
>
> EXIF orientation is not applied. Handle EXIF separately if needed.

## Error Handling

```typescript theme={null}
try {
  const result = await sip.process(imageBuffer, options)
} catch (error) {
  if (error.message.includes("WASM module not available")) {
    // WASM not built - run `pnpm build:wasm`
    console.error("Run: pnpm build:wasm")
  } else if (error.message.includes("Unknown image format")) {
    // Unsupported format
    console.error("Unsupported image format")
  } else {
    // Other error (corrupt image, etc.)
    console.error("Processing failed:", error.message)
  }
}
```

## Comparison with Alternatives

| Library | 100MP JPEG Memory | Edge Compatible | Output Formats |
| - | - | - | - |
| **sip** | \~50KB | Yes | JPEG |
| sharp (Node) | \~300MB | No | Many |
| jimp | \~300MB | No | Many |
| @cf-wasm/photon | \~300MB | Limited\* | Many |

\*@cf-wasm/photon requires paid bindings for large images in some edge environments.

## Next Steps

> [card] [React SDK](https://docs.standardagents.ai/user-interfaces/react.md)
>
> Build UIs that connect to agent threads

> [card] [Tools](https://docs.standardagents.ai/core-concepts/tools.md)
>
> Create tools that process images


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.