mirror of
https://github.com/moltbot/moltbot.git
synced 2026-03-08 06:54:24 +00:00
docs: sync android node docs with current pairing and capabilities
This commit is contained in:
Ayaan Zaidi
14
README.md
14
README.md
@@ -128,7 +128,7 @@ Run `openclaw doctor` to surface risky/misconfigured DM policies.
|
||||
- **[Local-first Gateway](https://docs.openclaw.ai/gateway)** — single control plane for sessions, channels, tools, and events.
|
||||
- **[Multi-channel inbox](https://docs.openclaw.ai/channels)** — WhatsApp, Telegram, Slack, Discord, Google Chat, Signal, BlueBubbles (iMessage), iMessage (legacy), Microsoft Teams, Matrix, Zalo, Zalo Personal, WebChat, macOS, iOS/Android.
|
||||
- **[Multi-agent routing](https://docs.openclaw.ai/gateway/configuration)** — route inbound channels/accounts/peers to isolated agents (workspaces + per-agent sessions).
|
||||
- **[Voice Wake](https://docs.openclaw.ai/nodes/voicewake) + [Talk Mode](https://docs.openclaw.ai/nodes/talk)** — always-on speech for macOS/iOS/Android with ElevenLabs.
|
||||
- **[Voice Wake](https://docs.openclaw.ai/nodes/voicewake) + [Talk Mode](https://docs.openclaw.ai/nodes/talk)** — wake words on macOS/iOS and continuous voice on Android (ElevenLabs + system TTS fallback).
|
||||
- **[Live Canvas](https://docs.openclaw.ai/platforms/mac/canvas)** — agent-driven visual workspace with [A2UI](https://docs.openclaw.ai/platforms/mac/canvas#canvas-a2ui).
|
||||
- **[First-class tools](https://docs.openclaw.ai/tools)** — browser, canvas, nodes, cron, sessions, and Discord/Slack actions.
|
||||
- **[Companion apps](https://docs.openclaw.ai/platforms/macos)** — macOS menu bar app + iOS/Android [nodes](https://docs.openclaw.ai/nodes).
|
||||
@@ -156,8 +156,8 @@ Run `openclaw doctor` to surface risky/misconfigured DM policies.
|
||||
### Apps + nodes
|
||||
|
||||
- [macOS app](https://docs.openclaw.ai/platforms/macos): menu bar control plane, [Voice Wake](https://docs.openclaw.ai/nodes/voicewake)/PTT, [Talk Mode](https://docs.openclaw.ai/nodes/talk) overlay, [WebChat](https://docs.openclaw.ai/web/webchat), debug tools, [remote gateway](https://docs.openclaw.ai/gateway/remote) control.
|
||||
- [iOS node](https://docs.openclaw.ai/platforms/ios): [Canvas](https://docs.openclaw.ai/platforms/mac/canvas), [Voice Wake](https://docs.openclaw.ai/nodes/voicewake), [Talk Mode](https://docs.openclaw.ai/nodes/talk), camera, screen recording, Bonjour pairing.
|
||||
- [Android node](https://docs.openclaw.ai/platforms/android): [Canvas](https://docs.openclaw.ai/platforms/mac/canvas), [Talk Mode](https://docs.openclaw.ai/nodes/talk), camera, screen recording, optional SMS.
|
||||
- [iOS node](https://docs.openclaw.ai/platforms/ios): [Canvas](https://docs.openclaw.ai/platforms/mac/canvas), [Voice Wake](https://docs.openclaw.ai/nodes/voicewake), [Talk Mode](https://docs.openclaw.ai/nodes/talk), camera, screen recording, Bonjour + device pairing.
|
||||
- [Android node](https://docs.openclaw.ai/platforms/android): Connect tab (setup code/manual), chat sessions, voice tab, [Canvas](https://docs.openclaw.ai/platforms/mac/canvas), camera/screen recording, and Android device commands (notifications/location/SMS/photos/contacts/calendar/motion/app update).
|
||||
- [macOS node mode](https://docs.openclaw.ai/nodes): system.run/notify + canvas/camera exposure.
|
||||
|
||||
### Tools + automation
|
||||
@@ -207,7 +207,7 @@ WhatsApp / Telegram / Slack / Discord / Google Chat / Signal / iMessage / BlueBu
|
||||
- **[Tailscale exposure](https://docs.openclaw.ai/gateway/tailscale)** — Serve/Funnel for the Gateway dashboard + WS (remote access: [Remote](https://docs.openclaw.ai/gateway/remote)).
|
||||
- **[Browser control](https://docs.openclaw.ai/tools/browser)** — openclaw‑managed Chrome/Chromium with CDP control.
|
||||
- **[Canvas + A2UI](https://docs.openclaw.ai/platforms/mac/canvas)** — agent‑driven visual workspace (A2UI host: [Canvas/A2UI](https://docs.openclaw.ai/platforms/mac/canvas#canvas-a2ui)).
|
||||
- **[Voice Wake](https://docs.openclaw.ai/nodes/voicewake) + [Talk Mode](https://docs.openclaw.ai/nodes/talk)** — always‑on speech and continuous conversation.
|
||||
- **[Voice Wake](https://docs.openclaw.ai/nodes/voicewake) + [Talk Mode](https://docs.openclaw.ai/nodes/talk)** — wake words on macOS/iOS plus continuous voice on Android.
|
||||
- **[Nodes](https://docs.openclaw.ai/nodes)** — Canvas, camera snap/clip, screen record, `location.get`, notifications, plus macOS‑only `system.run`/`system.notify`.
|
||||
|
||||
## Tailscale access (Gateway dashboard)
|
||||
@@ -297,7 +297,7 @@ Note: signed builds required for macOS permissions to stick across rebuilds (see
|
||||
|
||||
### iOS node (optional)
|
||||
|
||||
- Pairs as a node via the Bridge.
|
||||
- Pairs as a node over the Gateway WebSocket (device pairing).
|
||||
- Voice trigger forwarding + Canvas surface.
|
||||
- Controlled via `openclaw nodes …`.
|
||||
|
||||
@@ -305,8 +305,8 @@ Runbook: [iOS connect](https://docs.openclaw.ai/platforms/ios).
|
||||
|
||||
### Android node (optional)
|
||||
|
||||
- Pairs via the same Bridge + pairing flow as iOS.
|
||||
- Exposes Canvas, Camera, and Screen capture commands.
|
||||
- Pairs as a WS node via device pairing (`openclaw devices ...`).
|
||||
- Exposes Connect/Chat/Voice tabs plus Canvas, Camera, Screen capture, and Android device command families.
|
||||
- Runbook: [Android connect](https://docs.openclaw.ai/platforms/android).
|
||||
|
||||
## Agent workspace + skills
|
||||
|
||||
@@ -156,8 +156,8 @@ pnpm openclaw gateway --port 18789 --verbose
|
||||
3) Approve pairing (on the gateway machine):
|
||||
|
||||
```bash
|
||||
openclaw nodes pending
|
||||
openclaw nodes approve <requestId>
|
||||
openclaw devices list
|
||||
openclaw devices approve <requestId>
|
||||
```
|
||||
|
||||
More details: `docs/platforms/android.md`.
|
||||
|
||||
@@ -92,12 +92,12 @@ Service commands accept `--json` for machine-readable output.
|
||||
|
||||
## Pairing
|
||||
|
||||
The first connection creates a pending node pair request on the Gateway.
|
||||
The first connection creates a pending device pairing request (`role: node`) on the Gateway.
|
||||
Approve it via:
|
||||
|
||||
```bash
|
||||
openclaw nodes pending
|
||||
openclaw nodes approve <requestId>
|
||||
openclaw devices list
|
||||
openclaw devices approve <requestId>
|
||||
```
|
||||
|
||||
The node host stores its node id, token, display name, and gateway connection info in
|
||||
|
||||
@@ -24,7 +24,7 @@ title: "Features"
|
||||
Web Control UI and macOS companion app.
|
||||
</Card>
|
||||
<Card title="Mobile nodes" icon="smartphone">
|
||||
iOS and Android nodes with Canvas support.
|
||||
iOS and Android nodes with pairing, voice/chat, and rich device commands.
|
||||
</Card>
|
||||
</Columns>
|
||||
|
||||
@@ -44,8 +44,8 @@ title: "Features"
|
||||
- Media support for images, audio, and documents
|
||||
- Optional voice note transcription hook
|
||||
- WebChat and macOS menu bar app
|
||||
- iOS node with pairing and Canvas surface
|
||||
- Android node with pairing, Canvas, chat, and camera
|
||||
- iOS node with pairing, Canvas, camera, screen recording, location, and voice features
|
||||
- Android node with pairing, Connect tab, chat sessions, voice tab, Canvas/camera/screen, plus device, notifications, contacts/calendar, motion, photos, SMS, and app update commands
|
||||
|
||||
<Note>
|
||||
Legacy Claude, Codex, Gemini, and Opencode paths have been removed. Pi is the only
|
||||
|
||||
@@ -1527,8 +1527,8 @@ Typical setup:
|
||||
5. Approve the node on the Gateway:
|
||||
|
||||
```bash
|
||||
openclaw nodes pending
|
||||
openclaw nodes approve <requestId>
|
||||
openclaw devices list
|
||||
openclaw devices approve <requestId>
|
||||
```
|
||||
|
||||
No separate TCP bridge is required; nodes connect over the Gateway WebSocket.
|
||||
@@ -1697,8 +1697,8 @@ Recommended setup:
|
||||
3. **Approve the node** on the gateway:
|
||||
|
||||
```bash
|
||||
openclaw nodes pending
|
||||
openclaw nodes approve <requestId>
|
||||
openclaw devices list
|
||||
openclaw devices approve <requestId>
|
||||
```
|
||||
|
||||
Docs: [Gateway protocol](/gateway/protocol), [Discovery](/gateway/discovery), [macOS remote mode](/platforms/mac/remote).
|
||||
|
||||
@@ -89,7 +89,7 @@ The Gateway is the single source of truth for sessions, routing, and channel con
|
||||
Browser dashboard for chat, config, sessions, and nodes.
|
||||
</Card>
|
||||
<Card title="Mobile nodes" icon="smartphone">
|
||||
Pair iOS and Android nodes with Canvas support.
|
||||
Pair iOS and Android nodes for Canvas, camera/screen, and voice-enabled workflows.
|
||||
</Card>
|
||||
</Columns>
|
||||
|
||||
@@ -164,7 +164,7 @@ Example:
|
||||
Channel-specific setup for WhatsApp, Telegram, Discord, and more.
|
||||
</Card>
|
||||
<Card title="Nodes" href="/nodes" icon="smartphone">
|
||||
iOS and Android nodes with pairing and Canvas.
|
||||
iOS and Android nodes with pairing, Canvas, camera/screen, and device actions.
|
||||
</Card>
|
||||
<Card title="Help" href="/help" icon="life-buoy">
|
||||
Common fixes and troubleshooting entry point.
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
summary: "Camera capture (iOS node + macOS app) for agent use: photos (jpg) and short video clips (mp4)"
|
||||
summary: "Camera capture (iOS/Android nodes + macOS app) for agent use: photos (jpg) and short video clips (mp4)"
|
||||
read_when:
|
||||
- Adding or modifying camera capture on iOS nodes or macOS
|
||||
- Adding or modifying camera capture on iOS/Android nodes or macOS
|
||||
- Extending agent-accessible MEDIA temp-file workflows
|
||||
title: "Camera Capture"
|
||||
---
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
summary: "Nodes: pairing, capabilities, permissions, and CLI helpers for canvas/camera/screen/system"
|
||||
summary: "Nodes: pairing, capabilities, permissions, and CLI helpers for canvas/camera/screen/device/notifications/system"
|
||||
read_when:
|
||||
- Pairing iOS/Android nodes to a gateway
|
||||
- Using node canvas/camera for agent context
|
||||
@@ -9,7 +9,7 @@ title: "Nodes"
|
||||
|
||||
# Nodes
|
||||
|
||||
A **node** is a companion device (macOS/iOS/Android/headless) that connects to the Gateway **WebSocket** (same port as operators) with `role: "node"` and exposes a command surface (e.g. `canvas.*`, `camera.*`, `system.*`) via `node.invoke`. Protocol details: [Gateway protocol](/gateway/protocol).
|
||||
A **node** is a companion device (macOS/iOS/Android/headless) that connects to the Gateway **WebSocket** (same port as operators) with `role: "node"` and exposes a command surface (e.g. `canvas.*`, `camera.*`, `device.*`, `notifications.*`, `system.*`) via `node.invoke`. Protocol details: [Gateway protocol](/gateway/protocol).
|
||||
|
||||
Legacy transport: [Bridge protocol](/gateway/bridge-protocol) (TCP JSONL; deprecated/removed for current nodes).
|
||||
|
||||
@@ -96,9 +96,9 @@ openclaw node restart
|
||||
On the gateway host:
|
||||
|
||||
```bash
|
||||
openclaw nodes pending
|
||||
openclaw nodes approve <requestId>
|
||||
openclaw nodes list
|
||||
openclaw devices list
|
||||
openclaw devices approve <requestId>
|
||||
openclaw nodes status
|
||||
```
|
||||
|
||||
Naming options:
|
||||
@@ -261,6 +261,33 @@ Notes:
|
||||
- The permission prompt must be accepted on the Android device before the capability is advertised.
|
||||
- Wi-Fi-only devices without telephony will not advertise `sms.send`.
|
||||
|
||||
## Android device + personal data commands
|
||||
|
||||
Android nodes can advertise additional command families when the corresponding capabilities are enabled.
|
||||
|
||||
Available families:
|
||||
|
||||
- `device.status`, `device.info`, `device.permissions`, `device.health`
|
||||
- `notifications.list`, `notifications.actions`
|
||||
- `photos.latest`
|
||||
- `contacts.search`, `contacts.add`
|
||||
- `calendar.events`, `calendar.add`
|
||||
- `motion.activity`, `motion.pedometer`
|
||||
- `app.update`
|
||||
|
||||
Example invokes:
|
||||
|
||||
```bash
|
||||
openclaw nodes invoke --node <idOrNameOrIp> --command device.status --params '{}'
|
||||
openclaw nodes invoke --node <idOrNameOrIp> --command notifications.list --params '{}'
|
||||
openclaw nodes invoke --node <idOrNameOrIp> --command photos.latest --params '{"limit":1}'
|
||||
```
|
||||
|
||||
Notes:
|
||||
|
||||
- Motion commands are capability-gated by available sensors.
|
||||
- `app.update` is permission + policy gated by the node runtime.
|
||||
|
||||
## System commands (node host / mac node)
|
||||
|
||||
The macOS node exposes `system.run`, `system.notify`, and `system.execApprovals.get/set`.
|
||||
@@ -331,7 +358,7 @@ openclaw node run --host <gateway-host> --port 18789
|
||||
|
||||
Notes:
|
||||
|
||||
- Pairing is still required (the Gateway will show a node approval prompt).
|
||||
- Pairing is still required (the Gateway will show a device pairing prompt).
|
||||
- The node host stores its node id, token, display name, and gateway connection info in `~/.openclaw/node.json`.
|
||||
- Exec approvals are enforced locally via `~/.openclaw/exec-approvals.json`
|
||||
(see [Exec approvals](/tools/exec-approvals)).
|
||||
|
||||
@@ -12,7 +12,8 @@ OpenClaw treats **wake words as a single global list** owned by the **Gateway**.
|
||||
|
||||
- There are **no per-node custom wake words**.
|
||||
- **Any node/app UI may edit** the list; changes are persisted by the Gateway and broadcast to everyone.
|
||||
- Each device still keeps its own **Voice Wake enabled/disabled** toggle (local UX + permissions differ).
|
||||
- macOS and iOS keep local **Voice Wake enabled/disabled** toggles (local UX + permissions differ).
|
||||
- Android currently keeps Voice Wake off and uses a manual mic flow in the Voice tab.
|
||||
|
||||
## Storage (Gateway host)
|
||||
|
||||
@@ -61,5 +62,5 @@ Who receives it:
|
||||
|
||||
### Android node
|
||||
|
||||
- Exposes a Wake Words editor in Settings.
|
||||
- Calls `voicewake.set` over the Gateway WS so edits sync everywhere.
|
||||
- Voice Wake is currently disabled in Android runtime/Settings.
|
||||
- Android voice uses manual mic capture in the Voice tab instead of wake-word triggers.
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
summary: "Android app (node): connection runbook + Canvas/Chat/Camera"
|
||||
summary: "Android app (node): connection runbook + Connect/Chat/Voice/Canvas command surface"
|
||||
read_when:
|
||||
- Pairing or reconnecting the Android node
|
||||
- Debugging Android gateway discovery or auth
|
||||
@@ -13,7 +13,7 @@ title: "Android App"
|
||||
|
||||
- Role: companion node app (Android does not host the Gateway).
|
||||
- Gateway required: yes (run it on macOS, Linux, or Windows via WSL2).
|
||||
- Install: [Getting Started](/start/getting-started) + [Pairing](/gateway/pairing).
|
||||
- Install: [Getting Started](/start/getting-started) + [Pairing](/channels/pairing).
|
||||
- Gateway: [Runbook](/gateway) + [Configuration](/gateway/configuration).
|
||||
- Protocols: [Gateway protocol](/gateway/protocol) (nodes + control plane).
|
||||
|
||||
@@ -25,7 +25,7 @@ System control (launchd/systemd) lives on the Gateway host. See [Gateway](/gatew
|
||||
|
||||
Android node app ⇄ (mDNS/NSD + WebSocket) ⇄ **Gateway**
|
||||
|
||||
Android connects directly to the Gateway WebSocket (default `ws://<host>:18789`) and uses Gateway-owned pairing.
|
||||
Android connects directly to the Gateway WebSocket (default `ws://<host>:18789`) and uses device pairing (`role: node`).
|
||||
|
||||
### Prerequisites
|
||||
|
||||
@@ -75,9 +75,9 @@ Details and example CoreDNS config: [Bonjour](/gateway/bonjour).
|
||||
In the Android app:
|
||||
|
||||
- The app keeps its gateway connection alive via a **foreground service** (persistent notification).
|
||||
- Open **Settings**.
|
||||
- Under **Discovered Gateways**, select your gateway and hit **Connect**.
|
||||
- If mDNS is blocked, use **Advanced → Manual Gateway** (host + port) and **Connect (Manual)**.
|
||||
- Open the **Connect** tab.
|
||||
- Use **Setup Code** or **Manual** mode.
|
||||
- If discovery is blocked, use manual host/port (and TLS/token/password when required) in **Advanced controls**.
|
||||
|
||||
After the first successful pairing, Android auto-reconnects on launch:
|
||||
|
||||
@@ -89,11 +89,12 @@ After the first successful pairing, Android auto-reconnects on launch:
|
||||
On the gateway machine:
|
||||
|
||||
```bash
|
||||
openclaw nodes pending
|
||||
openclaw nodes approve <requestId>
|
||||
openclaw devices list
|
||||
openclaw devices approve <requestId>
|
||||
openclaw devices reject <requestId>
|
||||
```
|
||||
|
||||
Pairing details: [Gateway pairing](/gateway/pairing).
|
||||
Pairing details: [Pairing](/channels/pairing).
|
||||
|
||||
### 5) Verify the node is connected
|
||||
|
||||
@@ -111,13 +112,13 @@ Pairing details: [Gateway pairing](/gateway/pairing).
|
||||
|
||||
### 6) Chat + history
|
||||
|
||||
The Android node’s Chat sheet uses the gateway’s **primary session key** (`main`), so history and replies are shared with WebChat and other clients:
|
||||
The Android Chat tab supports session selection (default `main`, plus other existing sessions):
|
||||
|
||||
- History: `chat.history`
|
||||
- Send: `chat.send`
|
||||
- Push updates (best-effort): `chat.subscribe` → `event:"chat"`
|
||||
|
||||
### 7) Canvas + camera
|
||||
### 7) Canvas + screen + camera
|
||||
|
||||
#### Gateway Canvas Host (recommended for web content)
|
||||
|
||||
@@ -149,3 +150,20 @@ Camera commands (foreground only; permission-gated):
|
||||
- `camera.clip` (mp4)
|
||||
|
||||
See [Camera node](/nodes/camera) for parameters and CLI helpers.
|
||||
|
||||
Screen commands:
|
||||
|
||||
- `screen.record` (mp4; foreground only)
|
||||
|
||||
### 8) Voice + expanded Android command surface
|
||||
|
||||
- Voice: Android uses a single mic on/off flow in the Voice tab with transcript capture and TTS playback (ElevenLabs when configured, system TTS fallback).
|
||||
- Voice wake/talk-mode toggles are currently removed from Android UX/runtime.
|
||||
- Additional Android command families (availability depends on device + permissions):
|
||||
- `device.status`, `device.info`, `device.permissions`, `device.health`
|
||||
- `notifications.list`, `notifications.actions`
|
||||
- `photos.latest`
|
||||
- `contacts.search`, `contacts.add`
|
||||
- `calendar.events`, `calendar.add`
|
||||
- `motion.activity`, `motion.pedometer`
|
||||
- `app.update`
|
||||
|
||||
@@ -38,8 +38,8 @@ openclaw gateway --port 18789
|
||||
3. Approve the pairing request on the gateway host:
|
||||
|
||||
```bash
|
||||
openclaw nodes pending
|
||||
openclaw nodes approve <requestId>
|
||||
openclaw devices list
|
||||
openclaw devices approve <requestId>
|
||||
```
|
||||
|
||||
4. Verify connection:
|
||||
@@ -98,11 +98,11 @@ openclaw nodes invoke --node "iOS Node" --command canvas.snapshot --params '{"ma
|
||||
|
||||
- `NODE_BACKGROUND_UNAVAILABLE`: bring the iOS app to the foreground (canvas/camera/screen commands require it).
|
||||
- `A2UI_HOST_NOT_CONFIGURED`: the Gateway did not advertise a canvas host URL; check `canvasHost` in [Gateway configuration](/gateway/configuration).
|
||||
- Pairing prompt never appears: run `openclaw nodes pending` and approve manually.
|
||||
- Pairing prompt never appears: run `openclaw devices list` and approve manually.
|
||||
- Reconnect fails after reinstall: the Keychain pairing token was cleared; re-pair the node.
|
||||
|
||||
## Related docs
|
||||
|
||||
- [Pairing](/gateway/pairing)
|
||||
- [Pairing](/channels/pairing)
|
||||
- [Discovery](/gateway/discovery)
|
||||
- [Bonjour](/gateway/bonjour)
|
||||
|
||||
Reference in New Issue
Block a user