docs: readme image section
This commit is contained in:
@@ -16,6 +16,10 @@ Unified LLM API with automatic model discovery, provider configuration, token an
|
||||
- [Validating Tool Arguments](#validating-tool-arguments)
|
||||
- [Complete Event Reference](#complete-event-reference)
|
||||
- [Image Input](#image-input)
|
||||
- [Image Generation](#image-generation)
|
||||
- [Basic Image Generation](#basic-image-generation)
|
||||
- [Streaming Image Events](#streaming-image-events)
|
||||
- [Notes and Limitations](#notes-and-limitations)
|
||||
- [Thinking/Reasoning](#thinkingreasoning)
|
||||
- [Unified Interface](#unified-interface-streamsimplecompletesimple)
|
||||
- [Provider-Specific Options](#provider-specific-options-streamcomplete)
|
||||
@@ -419,6 +423,117 @@ for (const block of response.content) {
|
||||
}
|
||||
```
|
||||
|
||||
## Image Generation
|
||||
|
||||
Image generation uses a separate API surface from text/chat generation. Use `getImageModel()` / `getImageModels()` / `getImageProviders()` to discover image-generation models, `images()` to stream events, and `generateImages()` to get the final result.
|
||||
|
||||
Do not use `stream()` or `complete()` for image generation. Request options, error handling, and event/result flow follow the same overall pattern as the streaming APIs, but use image-specific types and events.
|
||||
|
||||
### Basic Image Generation
|
||||
|
||||
```typescript
|
||||
import { getImageModel, generateImages } from '@mariozechner/pi-ai';
|
||||
|
||||
const model = getImageModel('openrouter', 'google/gemini-2.5-flash-image');
|
||||
|
||||
const result = await generateImages(model, {
|
||||
input: [{ type: 'text', text: 'Generate a red circle on a plain white background.' }]
|
||||
}, {
|
||||
apiKey: process.env.OPENROUTER_API_KEY
|
||||
});
|
||||
|
||||
for (const block of result.output) {
|
||||
if (block.type === 'text') {
|
||||
console.log(block.text);
|
||||
} else if (block.type === 'image') {
|
||||
console.log(block.mimeType);
|
||||
console.log(block.data.substring(0, 32));
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Some models also support image input:
|
||||
|
||||
```typescript
|
||||
import { readFileSync } from 'fs';
|
||||
|
||||
const imageBuffer = readFileSync('input.png');
|
||||
const result = await generateImages(model, {
|
||||
input: [
|
||||
{ type: 'text', text: 'Create a variation of this image with a blue background.' },
|
||||
{ type: 'image', data: imageBuffer.toString('base64'), mimeType: 'image/png' }
|
||||
]
|
||||
}, {
|
||||
apiKey: process.env.OPENROUTER_API_KEY
|
||||
});
|
||||
```
|
||||
|
||||
Check capabilities on the model metadata:
|
||||
|
||||
```typescript
|
||||
console.log(model.input); // ['text', 'image']
|
||||
console.log(model.output); // ['image'] or ['image', 'text']
|
||||
```
|
||||
|
||||
### Streaming Image Events
|
||||
|
||||
`images()` returns an `AssistantImagesEventStream` with image-specific events:
|
||||
|
||||
```typescript
|
||||
import { getImageModel, images } from '@mariozechner/pi-ai';
|
||||
|
||||
const model = getImageModel('openrouter', 'google/gemini-2.5-flash-image');
|
||||
const s = images(model, {
|
||||
input: [{ type: 'text', text: 'Generate a simple green triangle.' }]
|
||||
}, {
|
||||
apiKey: process.env.OPENROUTER_API_KEY
|
||||
});
|
||||
|
||||
for await (const event of s) {
|
||||
switch (event.type) {
|
||||
case 'start':
|
||||
console.log(`Starting image generation with ${event.partial.model}`);
|
||||
break;
|
||||
case 'image_start':
|
||||
console.log(`Image block started at index ${event.contentIndex}`);
|
||||
break;
|
||||
case 'image_end':
|
||||
console.log(`Image block complete: ${event.image.mimeType}`);
|
||||
break;
|
||||
case 'done':
|
||||
console.log(`Finished: ${event.reason}`);
|
||||
break;
|
||||
case 'error':
|
||||
console.error(`Error: ${event.error.errorMessage}`);
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
const finalResult = await s.result();
|
||||
```
|
||||
|
||||
| Event Type | Description | Key Properties |
|
||||
|------------|-------------|----------------|
|
||||
| `start` | Stream begins | `partial`: Initial `AssistantImages` structure |
|
||||
| `image_start` | Image block starts | `contentIndex`: Position in output array |
|
||||
| `image_end` | Image block complete | `image`: Final `ImageContent`, `contentIndex`: Position |
|
||||
| `done` | Stream complete | `reason`: Stop reason (`"stop"`), `images`: Final `AssistantImages` |
|
||||
| `error` | Error occurred | `reason`: Error type (`"error"` or `"aborted"`), `error`: Final `AssistantImages` |
|
||||
|
||||
Text output, if the model returns any, is available on the final `AssistantImages.output` array.
|
||||
|
||||
### Notes and Limitations
|
||||
|
||||
- Use `getImageModel(...)`, not `getModel(...)`.
|
||||
- Use `images()` / `generateImages()`, not `stream()` / `complete()`.
|
||||
- Image-generation models do not participate in tool calling.
|
||||
- Outputs are returned as base64-encoded `ImageContent` blocks in `AssistantImages.output`.
|
||||
- Some models return only images, others return images plus text. Check `model.output`.
|
||||
- Some models accept image input, others are text-to-image only. Check `model.input`.
|
||||
- Like the streaming APIs, image generation supports options such as `apiKey`, `signal`, `headers`, `onPayload`, and `onResponse`, and results may include `stopReason`, `responseId`, and `usage`.
|
||||
- If you want a model to analyze images in a conversation or call tools, use the regular `stream()` / `complete()` APIs with a model that supports image input.
|
||||
- At the moment, image generation is available through only one provider, OpenRouter.
|
||||
|
||||
## Thinking/Reasoning
|
||||
|
||||
Many models support thinking/reasoning capabilities where they can show their internal thought process. You can check if a model supports reasoning via the `reasoning` property. If you pass reasoning options to a non-reasoning model, they are silently ignored.
|
||||
|
||||
Reference in New Issue
Block a user