Expert guide for building AI agents with Scrapybara TypeScript SDK. Handles remote desktop instances, browser automation, computer use tools, and structured AI agent workflows.
You are an expert at working with Scrapybara, a TypeScript SDK for deploying and managing remote desktop instances for AI agents. This skill provides comprehensive guidance on properly interacting with the SDK to build computer use agents.
This skill helps you with:
```typescript
import { ScrapybaraClient } from "scrapybara";
const client = new ScrapybaraClient({ apiKey: "YOUR_API_KEY" });
```
Always import the appropriate types for your use case:
```typescript
// Core types
import { ScrapybaraClient, UbuntuInstance, BrowserInstance, WindowsInstance } from "scrapybara";
// Tool types
import { bashTool, computerTool, editTool } from "scrapybara/tools";
// Model types
import { anthropic } from "scrapybara/anthropic";
// System prompts
import { UBUNTU_SYSTEM_PROMPT, BROWSER_SYSTEM_PROMPT, WINDOWS_SYSTEM_PROMPT } from "scrapybara/prompts";
// Validation
import { z } from "zod";
// Error handling
import { ApiError } from "scrapybara/core";
```
Choose the appropriate instance type based on tool requirements:
```typescript
// Ubuntu instance - supports bash, computer, edit, browser
const ubuntuInstance = await client.startUbuntu({ timeoutHours: 1 });
// Browser instance - supports computer, browser
const browserInstance = await client.startBrowser({ timeoutHours: 1 });
// Windows instance - supports computer
const windowsInstance = await client.startWindows({ timeoutHours: 1 });
```
```typescript
// Pause to save resources
await instance.pause();
// Resume work
await instance.resume({ timeoutHours: 1 });
// Terminate and clean up (ALWAYS do this to prevent billing)
await instance.stop();
```
The Act SDK is the **primary focus** for building computer use agents. It provides a unified interface for LLM integration and tool execution.
```typescript
const instance = await client.startUbuntu();
const { messages, steps, text, output, usage } = await client.act({
model: anthropic(), // Uses Scrapybara's API key by default
tools: [
bashTool(instance),
computerTool(instance),
editTool(instance),
],
system: UBUNTU_SYSTEM_PROMPT,
prompt: "Your task description here",
onStep: (step) => {
console.log(`Step: ${step.text}`);
console.log(`Tokens: ${step.usage?.totalTokens ?? 'N/A'}`);
}
});
await instance.stop();
```
Use **either** `prompt` OR `messages`, never both:
```typescript
// Option 1: Simple prompt
await client.act({
model: anthropic(),
tools,
system: UBUNTU_SYSTEM_PROMPT,
prompt: "Navigate to example.com and extract the title"
});
// Option 2: Message history
await client.act({
model: anthropic(),
tools,
system: UBUNTU_SYSTEM_PROMPT,
messages: [
{ role: "user", content: [{ type: "text", text: "First task" }] },
{ role: "assistant", content: [{ type: "text", text: "Response" }] },
{ role: "user", content: [{ type: "text", text: "Follow-up task" }] }
]
});
```
Use Zod schemas to get typed, validated data from the agent:
```typescript
const schema = z.object({
posts: z.array(z.object({
title: z.string(),
url: z.string(),
points: z.number(),
})),
});
const { output } = await client.act({
model: anthropic(),
tools: [bashTool(instance), computerTool(instance)],
schema,
system: UBUNTU_SYSTEM_PROMPT,
prompt: "Get the top 10 posts on Hacker News",
});
// output.posts is fully typed
const posts = output.posts;
```
```typescript
const { base64Image } = await instance.screenshot();
```
```typescript
// Move mouse
await instance.computer({ action: "move_mouse", coordinates: [x, y] });
// Click (left, right, middle)
await instance.computer({ action: "click_mouse", button: "left", coordinates: [x, y] });
// Drag
await instance.computer({ action: "drag_mouse", path: [[x1, y1], [x2, y2]] });
// Scroll
await instance.computer({ action: "scroll", coordinates: [x, y], delta_x: 0, delta_y: -100 });
// Get cursor position
await instance.computer({ action: "get_cursor_position" });
```
```typescript
// Press specific keys
await instance.computer({ action: "press_key", keys: ["Control", "c"] });
// Type text
await instance.computer({ action: "type_text", text: "Hello world" });
```
```typescript
// Wait for duration (seconds)
await instance.computer({ action: "wait", duration: 3 });
```
```typescript
const result = await instance.bash({ command: "ls -la /tmp" });
console.log(result.output);
```
**Best Practice:** Prefer bash commands over GUI interactions for launching applications.
```typescript
const instance = await client.startUbuntu();
const { cdpUrl } = await instance.browser.start();
// Use browserTool after starting
const tools = [browserTool(instance), computerTool(instance)];
```
```typescript
// Save authentication state
const { authStateId } = await instance.browser.saveAuth({ name: "default" });
// Reuse saved authentication
await instance.browser.authenticate({ authStateId });
```
```typescript
await instance.browser.stop();
await instance.stop();
```
```typescript
// Write file
await instance.file.write({
path: "/tmp/data.txt",
content: "file content here"
});
// Read file
const { content } = await instance.file.read({ path: "/tmp/data.txt" });
```
```typescript
// Set variables
await instance.env.set({ API_KEY: "secret_value", DEBUG: "true" });
// Get all variables
const { variables } = await instance.env.get();
// Delete variables
await instance.env.delete(["API_KEY", "DEBUG"]);
```
```typescript
const handleStep = (step: Step) => {
// Text output from the model
console.log(`Text: ${step.text}`);
// Tool calls made by the model
if (step.toolCalls) {
for (const call of step.toolCalls) {
console.log(`Calling ${call.toolName} with args:`, call.args);
}
}
// Tool execution results
if (step.toolResults) {
for (const result of step.toolResults) {
console.log(`${result.toolName} result:`, result.result);
if (result.isError) {
console.error(`Error in ${result.toolName}`);
}
}
}
// Token usage tracking
if (step.usage) {
console.log(`Tokens - Prompt: ${step.usage.promptTokens}, Completion: ${step.usage.completionTokens}, Total: ${step.usage.totalTokens}`);
}
};
```
Messages contain typed content parts:
```typescript
// Text
{ type: "text", text: "content" }
// Image (base64 or URL)
{ type: "image", image: "base64...", mimeType: "image/png" }
// Reasoning (extended thinking)
{ type: "reasoning", id: "id", reasoning: "thinking process", signature: "...", instructions: "..." }
// Tool call
{ type: "tool-call", toolCallId: "id", toolName: "bash", args: { command: "ls" } }
// Tool result
{ type: "tool-result", toolCallId: "id", toolName: "bash", result: "output", isError: false }
```
Always wrap Scrapybara operations in try/catch blocks:
```typescript
import { ApiError } from "scrapybara/core";
try {
const instance = await client.startUbuntu();
// ... operations
await instance.stop();
} catch (e) {
if (e instanceof ApiError) {
console.error(`API Error ${e.statusCode}: ${e.body}`);
} else {
console.error("Unexpected error:", e);
}
}
```
1. **Always stop instances** - Prevents unnecessary billing
2. **Use async/await** - All operations are asynchronous
3. **Handle errors** - Wrap operations in try/catch
4. **Default timeout** - 60s default; customize with `timeout` or `requestOptions`
5. **Auto-termination** - Instances auto-terminate after 1 hour by default
6. **Browser workflow** - Always start browser before using browserTool
7. **Prefer bash** - Use bash commands over GUI interactions when possible
8. **Own API key** - Use `anthropic({ apiKey: "YOUR_KEY" })` to use your own Anthropic key
```typescript
import { ScrapybaraClient } from "scrapybara";
import { anthropic } from "scrapybara/anthropic";
import { UBUNTU_SYSTEM_PROMPT } from "scrapybara/prompts";
import { bashTool, computerTool, editTool } from "scrapybara/tools";
import { z } from "zod";
const client = new ScrapybaraClient({ apiKey: process.env.SCRAPYBARA_API_KEY });
async function main() {
const instance = await client.startUbuntu({ timeoutHours: 1 });
try {
await instance.browser.start();
const schema = z.object({
title: z.string(),
html: z.string(),
});
const { output, usage } = await client.act({
model: anthropic(),
tools: [
bashTool(instance),
computerTool(instance),
editTool(instance),
],
schema,
system: UBUNTU_SYSTEM_PROMPT,
prompt: "Go to the YC website and fetch the HTML",
onStep: (step) => console.log(`Step: ${step.text}\n`),
});
console.log("Title:", output.title);
console.log("Total tokens:", usage.totalTokens);
await instance.browser.stop();
} finally {
await instance.stop();
}
}
main().catch(console.error);
```
Track token usage at both step and response level:
```typescript
const { steps, usage } = await client.act({...});
// Per-step usage
steps.forEach(step => {
console.log(`Step tokens: ${step.usage?.totalTokens}`);
});
// Total usage
console.log(`Total - Prompt: ${usage.promptTokens}, Completion: ${usage.completionTokens}`);
```
Leave a review
No reviews yet. Be the first to review this skill!
# Download SKILL.md from killerskills.ai/api/skills/scrapybara-sdk-assistant/raw