Expert guidance for building AI agents with Scrapybara TypeScript SDK - remote desktop automation, browser control, and computer use patterns with Claude integration.
Expert guidance for building AI agents using the Scrapybara TypeScript SDK. Scrapybara provides remote desktop instances (Ubuntu, Windows, Browser) that AI agents can control through computer use tools.
This skill provides expertise in:
Always start by importing and initializing the Scrapybara client:
```typescript
import { ScrapybaraClient } from "scrapybara";
const client = new ScrapybaraClient({ apiKey: "YOUR_API_KEY" });
```
**Choose the appropriate instance type:**
**Manage instance lifecycle properly:**
```typescript
const instance = await client.startUbuntu({ timeoutHours: 1 });
// ... perform work ...
await instance.pause(); // Save resources when idle
await instance.resume({ timeoutHours: 1 }); // Resume work
await instance.stop(); // Always terminate when done
```
**Critical:** Always call `instance.stop()` to prevent unnecessary billing. Instance auto-terminates after 1 hour by default.
The ACT SDK is the **primary interface** for building agents. It combines three components:
**Component 1: Model (LLM Integration)**
```typescript
import { anthropic } from "scrapybara/anthropic";
const model = anthropic(); // Uses Scrapybara's key
// OR
const model = anthropic({ apiKey: "YOUR_KEY" }); // Uses your own key
```
**Component 2: Tools**
```typescript
import { bashTool, computerTool, editTool } from "scrapybara/tools";
const tools = [
bashTool(instance), // Shell command execution
computerTool(instance), // Mouse/keyboard control
editTool(instance), // File operations
];
```
**Component 3: Prompts and Execution**
```typescript
import { UBUNTU_SYSTEM_PROMPT } from "scrapybara/prompts";
const { messages, steps, text, output, usage } = await client.act({
model: anthropic(),
tools,
system: UBUNTU_SYSTEM_PROMPT, // Use BROWSER_SYSTEM_PROMPT or WINDOWS_SYSTEM_PROMPT as needed
prompt: "Your task description",
onStep: (step) => console.log(`Step: ${step.text}`)
});
```
**Important:** Provide either `prompt` (simple string) OR `messages` (conversation history), never both.
**Screenshots:**
```typescript
const { base64Image } = await instance.screenshot();
```
**Mouse operations:**
```typescript
// Move cursor
await instance.computer({ action: "move_mouse", coordinates: [x, y] });
// Click (left/right/middle)
await instance.computer({
action: "click_mouse",
button: "right",
coordinates: [x, y]
});
// Drag
await instance.computer({
action: "drag_mouse",
path: [[x1, y1], [x2, y2]]
});
// Scroll
await instance.computer({
action: "scroll",
coordinates: [x, y],
delta_x: 0,
delta_y: -100
});
// Get cursor position
const position = await instance.computer({ action: "get_cursor_position" });
```
**Keyboard operations:**
```typescript
// Press specific keys
await instance.computer({ action: "press_key", keys: ["Control", "c"] });
// Type text
await instance.computer({ action: "type_text", text: "Hello world" });
```
**Wait/delay:**
```typescript
await instance.computer({ action: "wait", duration: 3 }); // seconds
```
```typescript
await instance.bash({ command: "ls -la" });
await instance.bash({ command: "npm install" });
```
**Best practice:** Prefer bash commands over GUI interactions for launching applications.
```typescript
// Write file
await instance.file.write({
path: "/tmp/data.txt",
content: "file content"
});
// Read file
const { content } = await instance.file.read({ path: "/tmp/data.txt" });
```
```typescript
// Start browser
const { cdpUrl } = await instance.browser.start();
// Save authentication state
const { authStateId } = await instance.browser.saveAuth({ name: "default" });
// Reuse authentication
await instance.browser.authenticate({ authStateId });
// Stop browser
await instance.browser.stop();
```
**Important:** Always start browser before using browserTool.
```typescript
// Set variables
await instance.env.set({ API_KEY: "value", TOKEN: "secret" });
// Get all variables
const { variables } = await instance.env.get();
// Delete variables
await instance.env.delete(["VAR_NAME"]);
```
Use Zod schemas to extract typed data:
```typescript
import { z } from "zod";
const schema = z.object({
posts: z.array(z.object({
title: z.string(),
url: z.string(),
points: z.number(),
})),
});
const { output } = await client.act({
model: anthropic(),
tools,
schema,
system: UBUNTU_SYSTEM_PROMPT,
prompt: "Get the top 10 posts on Hacker News",
});
// output is fully typed based on schema
const posts = output.posts;
```
**Understanding message structure:**
Messages contain typed content parts:
**Step handler pattern:**
```typescript
const handleStep = (step: Step) => {
console.log(`Text: ${step.text}`);
if (step.toolCalls) {
for (const call of step.toolCalls) {
console.log(`Tool: ${call.toolName} - ${JSON.stringify(call.args)}`);
}
}
if (step.toolResults) {
for (const result of step.toolResults) {
console.log(`Result: ${result.result}`);
}
}
console.log(`Tokens: ${step.usage?.totalTokens ?? 'N/A'}`);
};
```
Track LLM token consumption:
```typescript
const { usage } = await client.act({...});
console.log(`Prompt tokens: ${usage.promptTokens}`);
console.log(`Completion tokens: ${usage.completionTokens}`);
console.log(`Total tokens: ${usage.totalTokens}`);
```
Token usage is also available in individual `Step` objects.
```typescript
import { ApiError } from "scrapybara/core";
try {
await client.startUbuntu();
} catch (e) {
if (e instanceof ApiError) {
console.error(`Error ${e.statusCode}: ${e.body}`);
}
}
```
```typescript
// Core types
import { ScrapybaraClient, UbuntuInstance, BrowserInstance, WindowsInstance } from "scrapybara";
// Tool types
import { bashTool, computerTool, editTool } from "scrapybara/tools";
// Model types
import { anthropic } from "scrapybara/anthropic";
// System prompts
import { UBUNTU_SYSTEM_PROMPT, BROWSER_SYSTEM_PROMPT, WINDOWS_SYSTEM_PROMPT } from "scrapybara/prompts";
// Schema validation
import { z } from "zod";
// Error types
import { ApiError } from "scrapybara/core";
// Request/Response types (namespace)
import { Scrapybara } from "scrapybara";
```
```typescript
import { ScrapybaraClient } from "scrapybara";
import { anthropic } from "scrapybara/anthropic";
import { UBUNTU_SYSTEM_PROMPT } from "scrapybara/prompts";
import { bashTool, computerTool, editTool } from "scrapybara/tools";
const client = new ScrapybaraClient();
const instance = await client.startUbuntu();
await instance.browser.start();
const { messages, steps, text, output, usage } = await client.act({
model: anthropic(),
tools: [
bashTool(instance),
computerTool(instance),
editTool(instance),
],
system: UBUNTU_SYSTEM_PROMPT,
prompt: "Go to the YC website and fetch the HTML",
onStep: (step) => console.log(`${step}\n`),
});
console.log(`Total tokens used: ${usage.totalTokens}`);
await instance.browser.stop();
await instance.stop();
```
1. **Always clean up:** Call `instance.stop()` after completing work to prevent billing charges
2. **Use async/await:** All operations are asynchronous
3. **Error handling:** Wrap API calls in try/catch blocks
4. **Timeout management:** Default timeout is 60s; customize via `timeout` parameter or `requestOptions`
5. **Browser initialization:** Always call `instance.browser.start()` before using browser tools
6. **Prefer bash:** Use shell commands instead of GUI interactions when possible for better reliability
7. **Resource management:** Use `pause()`/`resume()` for long-running tasks with idle periods
8. **System prompts:** Use appropriate system prompts (`UBUNTU_SYSTEM_PROMPT`, `BROWSER_SYSTEM_PROMPT`, `WINDOWS_SYSTEM_PROMPT`)
Leave a review
No reviews yet. Be the first to review this skill!
# Download SKILL.md from killerskills.ai/api/skills/scrapybara-sdk-agent/raw