Actions API - PinchTab

Overview

The Actions API enables browser automation by simulating user interactions:

Click elements
Type text
Fill inputs
Press keys
Hover elements
Scroll pages
Select dropdowns
Focus elements

Single Action

Execute one action on a tab.

POST /tabs/{id}/action

curl -X POST http://localhost:9867/tabs/tab_abc123/action \
  -H "Content-Type: application/json" \
  -d '{"kind": "click", "ref": "e5"}'

Request Body

kind

string

required

Action type: click, type, fill, press, hover, focus, scroll, select

ref

string

Element reference from snapshot (e.g., e5, e12)

selector

string

CSS selector (alternative to ref)

text

string

Text to type (for type and fill actions)

value

string

Value for select dropdowns

key

string

Key name for press action (e.g., Enter, Tab)

pixels

integer

Scroll amount in pixels (for scroll action)

Response (200 OK)

{
  "success": true,
  "result": {}
}

Action Types

click

Click an element (button, link, checkbox, etc.).

{
  "kind": "click",
  "ref": "e5"
}

type

Type text into an input field (simulates keyboard events).

{
  "kind": "type",
  "ref": "e1",
  "text": "user@example.com"
}

type simulates actual keystrokes. Use fill for instant value setting.

fill

Set input value directly without triggering keyboard events (faster).

{
  "kind": "fill",
  "ref": "e1",
  "text": "instant value"
}

press

Press a keyboard key.

{
  "kind": "press",
  "key": "Enter"
}

Supported keys: Enter, Tab, Escape, Backspace, Delete, ArrowUp, ArrowDown, ArrowLeft, ArrowRight, Space, Home, End, PageUp, PageDown

hover

Move mouse over an element (triggers hover effects).

{
  "kind": "hover",
  "ref": "e5"
}

focus

Focus an input element.

{
  "kind": "focus",
  "ref": "e5"
}

scroll

Scroll to an element or by pixels.

// Scroll to element
{
  "kind": "scroll",
  "ref": "e5"
}

// Scroll by pixels
{
  "kind": "scroll",
  "pixels": 500
}

select

Select a dropdown option.

{
  "kind": "select",
  "ref": "e7",
  "value": "Option 2"
}

Multiple Actions

Execute multiple actions in sequence.

POST /tabs/{id}/actions

curl -X POST http://localhost:9867/tabs/tab_abc123/actions \
  -H "Content-Type: application/json" \
  -d '{
    "actions": [
      {"kind": "type", "ref": "e1", "text": "user@example.com"},
      {"kind": "type", "ref": "e2", "text": "password123"},
      {"kind": "click", "ref": "e3"}
    ],
    "stopOnError": true
  }'

Request Body

actions

array

required

Array of action objects

stopOnError

boolean

default:"false"

Stop execution on first error

Response (200 OK)

{
  "results": [
    {"index": 0, "success": true, "result": {}},
    {"index": 1, "success": true, "result": {}},
    {"index": 2, "success": true, "result": {}}
  ],
  "total": 3,
  "successful": 3,
  "failed": 0
}

Element References

Before executing actions, take a snapshot to get element references:

# 1. Get snapshot
curl "http://localhost:9867/tabs/tab_abc123/snapshot?filter=interactive" | jq .

# Response shows elements with refs
{
  "nodes": [
    {"ref": "e0", "role": "heading", "name": "Sign In"},
    {"ref": "e1", "role": "textbox", "name": "Email"},
    {"ref": "e2", "role": "textbox", "name": "Password"},
    {"ref": "e3", "role": "button", "name": "Sign in"}
  ]
}

# 2. Use refs in actions
curl -X POST http://localhost:9867/tabs/tab_abc123/actions \
  -d '{
    "actions": [
      {"kind": "type", "ref": "e1", "text": "user@example.com"},
      {"kind": "type", "ref": "e2", "text": "password"},
      {"kind": "click", "ref": "e3"}
    ]
  }'

Element refs (e0, e1, etc.) are cached from snapshots. Take a new snapshot after page navigation.

import requests
import time

BASE = "http://localhost:9867"

# Start instance
inst = requests.post(f"{BASE}/instances/start",
    json={"mode": "headed"}).json()

# Open tab
tab = requests.post(f"{BASE}/instances/{inst['id']}/tabs/open",
    json={"url": "https://example.com/login"}).json()

tab_id = tab["tabId"]

# Wait for page load
time.sleep(2)

# Get page structure
snapshot = requests.get(
    f"{BASE}/tabs/{tab_id}/snapshot?filter=interactive"
).json()

print("Interactive elements:")
for node in snapshot["nodes"]:
    print(f"  {node['ref']}: {node['role']} - {node['name']}")

# Perform login
requests.post(f"{BASE}/tabs/{tab_id}/actions", json={
    "actions": [
        {"kind": "type", "ref": "e1", "text": "user@example.com"},
        {"kind": "type", "ref": "e2", "text": "password123"},
        {"kind": "click", "ref": "e3"}
    ],
    "stopOnError": True
})

# Wait for navigation
time.sleep(3)

# Verify login
result = requests.get(f"{BASE}/tabs/{tab_id}").json()
print(f"Current URL: {result['url']}")

# Cleanup
requests.post(f"{BASE}/instances/{inst['id']}/stop")

Form Filling Example

#!/bin/bash

BASE="http://localhost:9867"
TAB="tab_abc123"

# Navigate to form
curl -X POST "$BASE/tabs/$TAB/navigate" \
  -d '{"url":"https://example.com/form"}'

sleep 2

# Get form elements
curl -s "$BASE/tabs/$TAB/snapshot?filter=interactive" | jq .

# Fill form
curl -X POST "$BASE/tabs/$TAB/actions" \
  -H "Content-Type: application/json" \
  -d '{
    "actions": [
      {"kind": "fill", "ref": "e1", "text": "John Doe"},
      {"kind": "fill", "ref": "e2", "text": "john@example.com"},
      {"kind": "select", "ref": "e3", "value": "United States"},
      {"kind": "click", "ref": "e4"},
      {"kind": "click", "ref": "e5"}
    ],
    "stopOnError": true
  }'

Error Handling

Ref Not Found (404)

curl -X POST http://localhost:9867/tabs/tab_abc123/action \
  -d '{"kind":"click","ref":"e999"}'

Response:

{
  "error": "ref e999 not found - take a /snapshot first",
  "statusCode": 404
}

Solution: Take a snapshot before using refs.

Unknown Action Kind (400)

curl -X POST http://localhost:9867/tabs/tab_abc123/action \
  -d '{"kind":"unknown","ref":"e1"}'

Response:

{
  "error": "unknown action kind: unknown - valid values: click, type, fill, press, hover, focus, scroll, select",
  "statusCode": 400
}

Tab Locked (423)

curl -X POST http://localhost:9867/tabs/tab_abc123/action \
  -H "X-Owner: agent-2" \
  -d '{"kind":"click","ref":"e5"}'

Response (if locked by agent-1):

{
  "error": "tab tab_abc123 is locked by agent-1",
  "code": "tab_locked",
  "statusCode": 423
}

Best Practices

Always Take Snapshots First

# 1. Navigate
requests.post(f"{BASE}/tabs/{tab_id}/navigate",
    json={"url": "https://example.com"})

# 2. Get snapshot (REQUIRED before actions)
snapshot = requests.get(
    f"{BASE}/tabs/{tab_id}/snapshot?filter=interactive"
).json()

# 3. Use refs from snapshot
requests.post(f"{BASE}/tabs/{tab_id}/action",
    json={"kind": "click", "ref": "e5"})

Use fill for Speed, type for Realism

// Fast (instant value set)
{"kind": "fill", "ref": "e1", "text": "value"}

// Realistic (simulates typing)
{"kind": "type", "ref": "e1", "text": "value"}

Batch Actions for Efficiency

Instead of multiple requests:

# ❌ Slow
requests.post(f"{BASE}/tabs/{tab_id}/action", json={"kind": "type", "ref": "e1", "text": "email"})
requests.post(f"{BASE}/tabs/{tab_id}/action", json={"kind": "type", "ref": "e2", "text": "pass"})
requests.post(f"{BASE}/tabs/{tab_id}/action", json={"kind": "click", "ref": "e3"})

Use batch:

# ✅ Fast
requests.post(f"{BASE}/tabs/{tab_id}/actions", json={
    "actions": [
        {"kind": "type", "ref": "e1", "text": "email"},
        {"kind": "type", "ref": "e2", "text": "pass"},
        {"kind": "click", "ref": "e3"}
    ]
})

Handle Stale References

If an action fails with “node not found”, refresh the snapshot:

try:
    requests.post(f"{BASE}/tabs/{tab_id}/action",
        json={"kind": "click", "ref": "e5"})
except:
    # Refresh snapshot
    snapshot = requests.get(
        f"{BASE}/tabs/{tab_id}/snapshot?filter=interactive"
    ).json()
    # Retry action with new ref

Next Steps

Snapshot API

Get element refs for actions

Evaluation

Execute custom JavaScript

Screenshots

Capture page state

Navigation

Navigate to pages

Documentation Index

​Overview

​Single Action

​Request Body

​Response (200 OK)

​Action Types

​click

​type

​fill

​press

​hover

​focus

​scroll

​select

​Multiple Actions

​Request Body

​Response (200 OK)

​Element References

​Complete Login Example

​Form Filling Example

​Error Handling

​Ref Not Found (404)

​Unknown Action Kind (400)

​Tab Locked (423)

​Best Practices

​Always Take Snapshots First

​Use fill for Speed, type for Realism

​Batch Actions for Efficiency

​Handle Stale References

​Next Steps

Snapshot API

Evaluation

Screenshots

Navigation

Overview

Single Action

Request Body

Response (200 OK)

Action Types

click

type

fill

press

hover

focus

scroll

select

Multiple Actions

Request Body

Response (200 OK)

Element References

Complete Login Example

Form Filling Example

Error Handling

Ref Not Found (404)

Unknown Action Kind (400)

Tab Locked (423)

Best Practices

Always Take Snapshots First

Use fill for Speed, type for Realism

Batch Actions for Efficiency

Handle Stale References

Next Steps