Menu
Platform
AI
App store purchases
Database
Flags
Jobs and cron
Localization
Monitoring
Notifications
Payments
Queues
Sandboxes
Webhooks
Getting Started
Authentication
KV Store
Deploy & Infrastructure
Reference
Act a lease
Performs computer actions on a DESKTOP, BROWSER, or ANDROID lease, in order, and returns each action's result and, when `screenshot` is set,…
Performs computer actions on a DESKTOP, BROWSER, or ANDROID lease, in order, and returns each action's result and, when screenshot is set, the screenshot taken after the last action. The actions are the Anthropic computer tool's and OpenAI computer use's, one-to-one. Refused CONTROL_HELD_BY_HUMAN while a human holds control.
- Path
POST https://api.sylphx.com/v1/orgs/acme/projects/shop/envs/production/leases/lease:act - Scope
sandboxes:exec - Effect
write— a successful call changes state. - Collection leases
#Request
| Field | Type | What it is |
|---|---|---|
name | string | The name of the lease. Required. |
actions | ComputerAction[] | The actions, performed in order; at most 64. An empty list with screenshot set only takes a screenshot. |
screenshot | ScreenshotOptions | Take a screenshot after the last action and return it. |
lease_generation | int64 | Refuse STALE_GENERATION unless the lease is at this generation; 0 skips the check. |
control_epoch | int64 | Act under this AGENT control epoch; 0 acts only while nobody holds control. |
#ComputerAction
| Field | Type | What it is |
|---|---|---|
take_screenshot | ScreenshotAction | Capture the display (Anthropic screenshot, OpenAI screenshot). One of the action group. |
zoom | ZoomAction | Capture a region at full resolution (Anthropic zoom). One of the action group. |
move | PointerMove | Move the pointer (Anthropic mouse_move, OpenAI move). One of the action group. |
click | ClickAction | Click (Anthropic left_click, right_click, middle_click, double_click, triple_click; OpenAI click, double_click; Android tap). One of the action group. |
mouse_down | ButtonAction | Press a button without releasing it (Anthropic left_mouse_down). One of the action group. |
mouse_up | ButtonAction | Release a button (Anthropic left_mouse_up). One of the action group. |
drag | DragAction | Press, move along a path, release (Anthropic left_click_drag, OpenAI drag; Android swipe). One of the action group. |
scroll | ScrollAction | Scroll (Anthropic scroll, OpenAI scroll). One of the action group. |
key | KeyAction | Press a key combination (Anthropic key, OpenAI keypress; Android key events). One of the action group. |
hold_key | HoldKeyAction | Hold a key combination for a duration (Anthropic hold_key). One of the action group. |
type | TypeAction | Type text (Anthropic type, OpenAI type). One of the action group. |
cursor_position | CursorPositionAction | Report the pointer position (Anthropic cursor_position). One of the action group. |
wait | WaitAction | Wait (Anthropic wait, OpenAI wait). One of the action group. |
#ScreenshotOptions
| Field | Type | What it is |
|---|---|---|
format | ImageFormat | Default PNG. One of png, jpeg, webp. |
quality | int32 | JPEG and WebP quality 1 to 100; default 80. |
max_width | int32 | Downscale to at most this width, keeping the aspect ratio; 0 keeps the display size. |
settle | duration | Wait this long after the last action before capturing, so the screen settles; default 300 ms, at most 5 seconds. |
#ScreenshotAction
| Field | Type | What it is |
|---|---|---|
options | ScreenshotOptions | How to encode it. |
#ZoomAction
| Field | Type | What it is |
|---|---|---|
region | Region | The region, in display pixels. Required. |
options | ScreenshotOptions | How to encode it; the region is scaled to options.max_width when set. |
#PointerMove
| Field | Type | What it is |
|---|---|---|
position | Point | Where to. Required. |
#ClickAction
| Field | Type | What it is |
|---|---|---|
position | Point | Where; unset clicks at the current pointer position. |
button | MouseButton | Default LEFT. One of left, right, middle, back, forward. |
count | int32 | 1 (default), 2 (double), or 3 (triple). |
modifiers | string[] | Keys held during the click, xdotool names (shift, ctrl, alt, super). |
#ButtonAction
| Field | Type | What it is |
|---|---|---|
position | Point | Where; unset acts at the current pointer position. |
button | MouseButton | Default LEFT. One of left, right, middle, back, forward. |
#DragAction
| Field | Type | What it is |
|---|---|---|
path | Point[] | At least two points. Required. |
button | MouseButton | Default LEFT. One of left, right, middle, back, forward. |
modifiers | string[] | Keys held during the drag, xdotool names. |
duration | duration | Total time of the move; default 300 ms (Android swipe duration). |
#ScrollAction
| Field | Type | What it is |
|---|---|---|
position | Point | Where; unset scrolls at the current pointer position. |
delta_x | int32 | Horizontal amount in unit. |
delta_y | int32 | Vertical amount in unit. |
unit | ScrollUnit | Default CLICKS (wheel notches, Anthropic scroll_amount); PIXELS (OpenAI scroll_x/scroll_y) are converted at 100 pixels per notch on desktops. One of clicks, pixels. |
modifiers | string[] | Keys held during the scroll, xdotool names. |
#KeyAction
| Field | Type | What it is |
|---|---|---|
combo | string | The combination. Required. |
repeat | int32 | Press it this many times; default 1. |
#HoldKeyAction
| Field | Type | What it is |
|---|---|---|
combo | string | The combination, xdotool syntax. Required. |
duration | duration | How long; at most 10 seconds. Required. |
#TypeAction
| Field | Type | What it is |
|---|---|---|
text | string | The text; at most 64 KiB. Required. |
keystroke_delay | duration | Delay between keystrokes; default 12 ms. |
#WaitAction
| Field | Type | What it is |
|---|---|---|
duration | duration | How long; at most 30 seconds. Required. |
#Region
| Field | Type | What it is |
|---|---|---|
x | int32 | Left edge. Required. |
y | int32 | Top edge. Required. |
width | int32 | Width. Required. |
height | int32 | Height. Required. |
#Point
| Field | Type | What it is |
|---|---|---|
x | int32 | Pixels from the left edge. Required. |
y | int32 | Pixels from the top edge. Required. |
#Response
| Field | Type | What it is |
|---|---|---|
results | ActionResult[] | One result per action, in order. |
screenshot | Screenshot | The screenshot after the last action, when requested. |
lease_generation | int64 | The lease generation the actions ran under. |
control | ControlState | The control the actions ran under. |
#ActionResult
| Field | Type | What it is |
|---|---|---|
cursor | Point | The pointer position after the action (and the answer of cursor_position). |
image | Screenshot | For take_screenshot and zoom. |
#Screenshot
| Field | Type | What it is |
|---|---|---|
data | bytes | The encoded image. |
media_type | string | image/png, image/jpeg, or image/webp. |
width | int32 | Image width in pixels. |
height | int32 | Image height in pixels. |
display_width | int32 | The display's logical width; divide to map image pixels to action coordinates. |
display_height | int32 | The display's logical height. |
capture_time | timestamp | When it was captured. |
#ControlState
| Field | Type | What it is |
|---|---|---|
holder | ControlHolder | Who holds control; UNSPECIFIED when nobody does (agent actions allowed). Output only. One of agent, human. |
epoch | int64 | Increases on every acquire; release names it. Output only. |
principal | string | The Access principal that acquired control. Output only. |
holder_label | string | The caller's label for the person or agent, for example a user id. Output only. |
acquire_time | timestamp | When control was acquired. Output only. |
expire_time | timestamp | When control lapses unless re-acquired. Output only. |
#Point
| Field | Type | What it is |
|---|---|---|
x | int32 | Pixels from the left edge. Required. |
y | int32 | Pixels from the top edge. Required. |
#Errors
UNAUTHENTICATED— No valid key or token was presented.PERMISSION_DENIED— The key lacks the method's permission.RESOURCE_NOT_FOUND— The named Resource does not exist or is not visible.INVALID_FIELD— A field failed validation.INVALID_STATE— The Resource is in a state that forbids the call.STALE_GENERATION— The call named a lease generation or control epoch that is no longer current.CONTROL_HELD— Another holder has input control of the display.CONTROL_HELD_BY_HUMAN— A human has taken over the display; agent input and screenshots are refused until control is released or expires.
Every error arrives in the body Errors describes.
#Examples
curl -X POST "https://api.sylphx.com/v1/orgs/acme/projects/shop/envs/production/leases/lease:act" \
-H "Authorization: Bearer $SYLPHX_API_KEY" \
-H "Content-Type: application/json" \
-d '{}'