diff --git a/README.md b/README.md index b15cabfbbe9..8a7ba4ef4f3 100644 --- a/README.md +++ b/README.md @@ -128,7 +128,7 @@ Run `openclaw doctor` to surface risky/misconfigured DM policies. - **[Local-first Gateway](https://docs.openclaw.ai/gateway)** — single control plane for sessions, channels, tools, and events. - **[Multi-channel inbox](https://docs.openclaw.ai/channels)** — WhatsApp, Telegram, Slack, Discord, Google Chat, Signal, BlueBubbles (iMessage), iMessage (legacy), Microsoft Teams, Matrix, Zalo, Zalo Personal, WebChat, macOS, iOS/Android. - **[Multi-agent routing](https://docs.openclaw.ai/gateway/configuration)** — route inbound channels/accounts/peers to isolated agents (workspaces + per-agent sessions). -- **[Voice Wake](https://docs.openclaw.ai/nodes/voicewake) + [Talk Mode](https://docs.openclaw.ai/nodes/talk)** — always-on speech for macOS/iOS/Android with ElevenLabs. +- **[Voice Wake](https://docs.openclaw.ai/nodes/voicewake) + [Talk Mode](https://docs.openclaw.ai/nodes/talk)** — wake words on macOS/iOS and continuous voice on Android (ElevenLabs + system TTS fallback). - **[Live Canvas](https://docs.openclaw.ai/platforms/mac/canvas)** — agent-driven visual workspace with [A2UI](https://docs.openclaw.ai/platforms/mac/canvas#canvas-a2ui). - **[First-class tools](https://docs.openclaw.ai/tools)** — browser, canvas, nodes, cron, sessions, and Discord/Slack actions. - **[Companion apps](https://docs.openclaw.ai/platforms/macos)** — macOS menu bar app + iOS/Android [nodes](https://docs.openclaw.ai/nodes). @@ -156,8 +156,8 @@ Run `openclaw doctor` to surface risky/misconfigured DM policies. ### Apps + nodes - [macOS app](https://docs.openclaw.ai/platforms/macos): menu bar control plane, [Voice Wake](https://docs.openclaw.ai/nodes/voicewake)/PTT, [Talk Mode](https://docs.openclaw.ai/nodes/talk) overlay, [WebChat](https://docs.openclaw.ai/web/webchat), debug tools, [remote gateway](https://docs.openclaw.ai/gateway/remote) control. -- [iOS node](https://docs.openclaw.ai/platforms/ios): [Canvas](https://docs.openclaw.ai/platforms/mac/canvas), [Voice Wake](https://docs.openclaw.ai/nodes/voicewake), [Talk Mode](https://docs.openclaw.ai/nodes/talk), camera, screen recording, Bonjour pairing. -- [Android node](https://docs.openclaw.ai/platforms/android): [Canvas](https://docs.openclaw.ai/platforms/mac/canvas), [Talk Mode](https://docs.openclaw.ai/nodes/talk), camera, screen recording, optional SMS. +- [iOS node](https://docs.openclaw.ai/platforms/ios): [Canvas](https://docs.openclaw.ai/platforms/mac/canvas), [Voice Wake](https://docs.openclaw.ai/nodes/voicewake), [Talk Mode](https://docs.openclaw.ai/nodes/talk), camera, screen recording, Bonjour + device pairing. +- [Android node](https://docs.openclaw.ai/platforms/android): Connect tab (setup code/manual), chat sessions, voice tab, [Canvas](https://docs.openclaw.ai/platforms/mac/canvas), camera/screen recording, and Android device commands (notifications/location/SMS/photos/contacts/calendar/motion/app update). - [macOS node mode](https://docs.openclaw.ai/nodes): system.run/notify + canvas/camera exposure. ### Tools + automation @@ -207,7 +207,7 @@ WhatsApp / Telegram / Slack / Discord / Google Chat / Signal / iMessage / BlueBu - **[Tailscale exposure](https://docs.openclaw.ai/gateway/tailscale)** — Serve/Funnel for the Gateway dashboard + WS (remote access: [Remote](https://docs.openclaw.ai/gateway/remote)). - **[Browser control](https://docs.openclaw.ai/tools/browser)** — openclaw‑managed Chrome/Chromium with CDP control. - **[Canvas + A2UI](https://docs.openclaw.ai/platforms/mac/canvas)** — agent‑driven visual workspace (A2UI host: [Canvas/A2UI](https://docs.openclaw.ai/platforms/mac/canvas#canvas-a2ui)). -- **[Voice Wake](https://docs.openclaw.ai/nodes/voicewake) + [Talk Mode](https://docs.openclaw.ai/nodes/talk)** — always‑on speech and continuous conversation. +- **[Voice Wake](https://docs.openclaw.ai/nodes/voicewake) + [Talk Mode](https://docs.openclaw.ai/nodes/talk)** — wake words on macOS/iOS plus continuous voice on Android. - **[Nodes](https://docs.openclaw.ai/nodes)** — Canvas, camera snap/clip, screen record, `location.get`, notifications, plus macOS‑only `system.run`/`system.notify`. ## Tailscale access (Gateway dashboard) @@ -297,7 +297,7 @@ Note: signed builds required for macOS permissions to stick across rebuilds (see ### iOS node (optional) -- Pairs as a node via the Bridge. +- Pairs as a node over the Gateway WebSocket (device pairing). - Voice trigger forwarding + Canvas surface. - Controlled via `openclaw nodes …`. @@ -305,8 +305,8 @@ Runbook: [iOS connect](https://docs.openclaw.ai/platforms/ios). ### Android node (optional) -- Pairs via the same Bridge + pairing flow as iOS. -- Exposes Canvas, Camera, and Screen capture commands. +- Pairs as a WS node via device pairing (`openclaw devices ...`). +- Exposes Connect/Chat/Voice tabs plus Canvas, Camera, Screen capture, and Android device command families. - Runbook: [Android connect](https://docs.openclaw.ai/platforms/android). ## Agent workspace + skills diff --git a/apps/android/README.md b/apps/android/README.md index f10c7fcede4..50704e63d0b 100644 --- a/apps/android/README.md +++ b/apps/android/README.md @@ -156,8 +156,8 @@ pnpm openclaw gateway --port 18789 --verbose 3) Approve pairing (on the gateway machine): ```bash -openclaw nodes pending -openclaw nodes approve +openclaw devices list +openclaw devices approve ``` More details: `docs/platforms/android.md`. diff --git a/docs/cli/node.md b/docs/cli/node.md index fb731cefedc..af07e61ba22 100644 --- a/docs/cli/node.md +++ b/docs/cli/node.md @@ -92,12 +92,12 @@ Service commands accept `--json` for machine-readable output. ## Pairing -The first connection creates a pending node pair request on the Gateway. +The first connection creates a pending device pairing request (`role: node`) on the Gateway. Approve it via: ```bash -openclaw nodes pending -openclaw nodes approve +openclaw devices list +openclaw devices approve ``` The node host stores its node id, token, display name, and gateway connection info in diff --git a/docs/concepts/features.md b/docs/concepts/features.md index 5eecd2153ef..55f0b2bcd12 100644 --- a/docs/concepts/features.md +++ b/docs/concepts/features.md @@ -24,7 +24,7 @@ title: "Features" Web Control UI and macOS companion app. - iOS and Android nodes with Canvas support. + iOS and Android nodes with pairing, voice/chat, and rich device commands. @@ -44,8 +44,8 @@ title: "Features" - Media support for images, audio, and documents - Optional voice note transcription hook - WebChat and macOS menu bar app -- iOS node with pairing and Canvas surface -- Android node with pairing, Canvas, chat, and camera +- iOS node with pairing, Canvas, camera, screen recording, location, and voice features +- Android node with pairing, Connect tab, chat sessions, voice tab, Canvas/camera/screen, plus device, notifications, contacts/calendar, motion, photos, SMS, and app update commands Legacy Claude, Codex, Gemini, and Opencode paths have been removed. Pi is the only diff --git a/docs/help/faq.md b/docs/help/faq.md index 10009ba1b7a..0a81714eeff 100644 --- a/docs/help/faq.md +++ b/docs/help/faq.md @@ -1527,8 +1527,8 @@ Typical setup: 5. Approve the node on the Gateway: ```bash - openclaw nodes pending - openclaw nodes approve + openclaw devices list + openclaw devices approve ``` No separate TCP bridge is required; nodes connect over the Gateway WebSocket. @@ -1697,8 +1697,8 @@ Recommended setup: 3. **Approve the node** on the gateway: ```bash - openclaw nodes pending - openclaw nodes approve + openclaw devices list + openclaw devices approve ``` Docs: [Gateway protocol](/gateway/protocol), [Discovery](/gateway/discovery), [macOS remote mode](/platforms/mac/remote). diff --git a/docs/index.md b/docs/index.md index 60c59bb7fa4..661bd4e92f1 100644 --- a/docs/index.md +++ b/docs/index.md @@ -89,7 +89,7 @@ The Gateway is the single source of truth for sessions, routing, and channel con Browser dashboard for chat, config, sessions, and nodes. - Pair iOS and Android nodes with Canvas support. + Pair iOS and Android nodes for Canvas, camera/screen, and voice-enabled workflows. @@ -164,7 +164,7 @@ Example: Channel-specific setup for WhatsApp, Telegram, Discord, and more. - iOS and Android nodes with pairing and Canvas. + iOS and Android nodes with pairing, Canvas, camera/screen, and device actions. Common fixes and troubleshooting entry point. diff --git a/docs/nodes/camera.md b/docs/nodes/camera.md index 2be8025ffa0..a8e952d9cb2 100644 --- a/docs/nodes/camera.md +++ b/docs/nodes/camera.md @@ -1,7 +1,7 @@ --- -summary: "Camera capture (iOS node + macOS app) for agent use: photos (jpg) and short video clips (mp4)" +summary: "Camera capture (iOS/Android nodes + macOS app) for agent use: photos (jpg) and short video clips (mp4)" read_when: - - Adding or modifying camera capture on iOS nodes or macOS + - Adding or modifying camera capture on iOS/Android nodes or macOS - Extending agent-accessible MEDIA temp-file workflows title: "Camera Capture" --- diff --git a/docs/nodes/index.md b/docs/nodes/index.md index 8d5e599419c..c58cd247a6c 100644 --- a/docs/nodes/index.md +++ b/docs/nodes/index.md @@ -1,5 +1,5 @@ --- -summary: "Nodes: pairing, capabilities, permissions, and CLI helpers for canvas/camera/screen/system" +summary: "Nodes: pairing, capabilities, permissions, and CLI helpers for canvas/camera/screen/device/notifications/system" read_when: - Pairing iOS/Android nodes to a gateway - Using node canvas/camera for agent context @@ -9,7 +9,7 @@ title: "Nodes" # Nodes -A **node** is a companion device (macOS/iOS/Android/headless) that connects to the Gateway **WebSocket** (same port as operators) with `role: "node"` and exposes a command surface (e.g. `canvas.*`, `camera.*`, `system.*`) via `node.invoke`. Protocol details: [Gateway protocol](/gateway/protocol). +A **node** is a companion device (macOS/iOS/Android/headless) that connects to the Gateway **WebSocket** (same port as operators) with `role: "node"` and exposes a command surface (e.g. `canvas.*`, `camera.*`, `device.*`, `notifications.*`, `system.*`) via `node.invoke`. Protocol details: [Gateway protocol](/gateway/protocol). Legacy transport: [Bridge protocol](/gateway/bridge-protocol) (TCP JSONL; deprecated/removed for current nodes). @@ -96,9 +96,9 @@ openclaw node restart On the gateway host: ```bash -openclaw nodes pending -openclaw nodes approve -openclaw nodes list +openclaw devices list +openclaw devices approve +openclaw nodes status ``` Naming options: @@ -261,6 +261,33 @@ Notes: - The permission prompt must be accepted on the Android device before the capability is advertised. - Wi-Fi-only devices without telephony will not advertise `sms.send`. +## Android device + personal data commands + +Android nodes can advertise additional command families when the corresponding capabilities are enabled. + +Available families: + +- `device.status`, `device.info`, `device.permissions`, `device.health` +- `notifications.list`, `notifications.actions` +- `photos.latest` +- `contacts.search`, `contacts.add` +- `calendar.events`, `calendar.add` +- `motion.activity`, `motion.pedometer` +- `app.update` + +Example invokes: + +```bash +openclaw nodes invoke --node --command device.status --params '{}' +openclaw nodes invoke --node --command notifications.list --params '{}' +openclaw nodes invoke --node --command photos.latest --params '{"limit":1}' +``` + +Notes: + +- Motion commands are capability-gated by available sensors. +- `app.update` is permission + policy gated by the node runtime. + ## System commands (node host / mac node) The macOS node exposes `system.run`, `system.notify`, and `system.execApprovals.get/set`. @@ -331,7 +358,7 @@ openclaw node run --host --port 18789 Notes: -- Pairing is still required (the Gateway will show a node approval prompt). +- Pairing is still required (the Gateway will show a device pairing prompt). - The node host stores its node id, token, display name, and gateway connection info in `~/.openclaw/node.json`. - Exec approvals are enforced locally via `~/.openclaw/exec-approvals.json` (see [Exec approvals](/tools/exec-approvals)). diff --git a/docs/nodes/voicewake.md b/docs/nodes/voicewake.md index fe7e2aa6a05..b188ffaff9d 100644 --- a/docs/nodes/voicewake.md +++ b/docs/nodes/voicewake.md @@ -12,7 +12,8 @@ OpenClaw treats **wake words as a single global list** owned by the **Gateway**. - There are **no per-node custom wake words**. - **Any node/app UI may edit** the list; changes are persisted by the Gateway and broadcast to everyone. -- Each device still keeps its own **Voice Wake enabled/disabled** toggle (local UX + permissions differ). +- macOS and iOS keep local **Voice Wake enabled/disabled** toggles (local UX + permissions differ). +- Android currently keeps Voice Wake off and uses a manual mic flow in the Voice tab. ## Storage (Gateway host) @@ -61,5 +62,5 @@ Who receives it: ### Android node -- Exposes a Wake Words editor in Settings. -- Calls `voicewake.set` over the Gateway WS so edits sync everywhere. +- Voice Wake is currently disabled in Android runtime/Settings. +- Android voice uses manual mic capture in the Voice tab instead of wake-word triggers. diff --git a/docs/platforms/android.md b/docs/platforms/android.md index 39f5aa12ae0..fe1683abdbf 100644 --- a/docs/platforms/android.md +++ b/docs/platforms/android.md @@ -1,5 +1,5 @@ --- -summary: "Android app (node): connection runbook + Canvas/Chat/Camera" +summary: "Android app (node): connection runbook + Connect/Chat/Voice/Canvas command surface" read_when: - Pairing or reconnecting the Android node - Debugging Android gateway discovery or auth @@ -13,7 +13,7 @@ title: "Android App" - Role: companion node app (Android does not host the Gateway). - Gateway required: yes (run it on macOS, Linux, or Windows via WSL2). -- Install: [Getting Started](/start/getting-started) + [Pairing](/gateway/pairing). +- Install: [Getting Started](/start/getting-started) + [Pairing](/channels/pairing). - Gateway: [Runbook](/gateway) + [Configuration](/gateway/configuration). - Protocols: [Gateway protocol](/gateway/protocol) (nodes + control plane). @@ -25,7 +25,7 @@ System control (launchd/systemd) lives on the Gateway host. See [Gateway](/gatew Android node app ⇄ (mDNS/NSD + WebSocket) ⇄ **Gateway** -Android connects directly to the Gateway WebSocket (default `ws://:18789`) and uses Gateway-owned pairing. +Android connects directly to the Gateway WebSocket (default `ws://:18789`) and uses device pairing (`role: node`). ### Prerequisites @@ -75,9 +75,9 @@ Details and example CoreDNS config: [Bonjour](/gateway/bonjour). In the Android app: - The app keeps its gateway connection alive via a **foreground service** (persistent notification). -- Open **Settings**. -- Under **Discovered Gateways**, select your gateway and hit **Connect**. -- If mDNS is blocked, use **Advanced → Manual Gateway** (host + port) and **Connect (Manual)**. +- Open the **Connect** tab. +- Use **Setup Code** or **Manual** mode. +- If discovery is blocked, use manual host/port (and TLS/token/password when required) in **Advanced controls**. After the first successful pairing, Android auto-reconnects on launch: @@ -89,11 +89,12 @@ After the first successful pairing, Android auto-reconnects on launch: On the gateway machine: ```bash -openclaw nodes pending -openclaw nodes approve +openclaw devices list +openclaw devices approve +openclaw devices reject ``` -Pairing details: [Gateway pairing](/gateway/pairing). +Pairing details: [Pairing](/channels/pairing). ### 5) Verify the node is connected @@ -111,13 +112,13 @@ Pairing details: [Gateway pairing](/gateway/pairing). ### 6) Chat + history -The Android node’s Chat sheet uses the gateway’s **primary session key** (`main`), so history and replies are shared with WebChat and other clients: +The Android Chat tab supports session selection (default `main`, plus other existing sessions): - History: `chat.history` - Send: `chat.send` - Push updates (best-effort): `chat.subscribe` → `event:"chat"` -### 7) Canvas + camera +### 7) Canvas + screen + camera #### Gateway Canvas Host (recommended for web content) @@ -149,3 +150,20 @@ Camera commands (foreground only; permission-gated): - `camera.clip` (mp4) See [Camera node](/nodes/camera) for parameters and CLI helpers. + +Screen commands: + +- `screen.record` (mp4; foreground only) + +### 8) Voice + expanded Android command surface + +- Voice: Android uses a single mic on/off flow in the Voice tab with transcript capture and TTS playback (ElevenLabs when configured, system TTS fallback). +- Voice wake/talk-mode toggles are currently removed from Android UX/runtime. +- Additional Android command families (availability depends on device + permissions): + - `device.status`, `device.info`, `device.permissions`, `device.health` + - `notifications.list`, `notifications.actions` + - `photos.latest` + - `contacts.search`, `contacts.add` + - `calendar.events`, `calendar.add` + - `motion.activity`, `motion.pedometer` + - `app.update` diff --git a/docs/platforms/ios.md b/docs/platforms/ios.md index e56f7e192a4..0a2eb5abae5 100644 --- a/docs/platforms/ios.md +++ b/docs/platforms/ios.md @@ -38,8 +38,8 @@ openclaw gateway --port 18789 3. Approve the pairing request on the gateway host: ```bash -openclaw nodes pending -openclaw nodes approve +openclaw devices list +openclaw devices approve ``` 4. Verify connection: @@ -98,11 +98,11 @@ openclaw nodes invoke --node "iOS Node" --command canvas.snapshot --params '{"ma - `NODE_BACKGROUND_UNAVAILABLE`: bring the iOS app to the foreground (canvas/camera/screen commands require it). - `A2UI_HOST_NOT_CONFIGURED`: the Gateway did not advertise a canvas host URL; check `canvasHost` in [Gateway configuration](/gateway/configuration). -- Pairing prompt never appears: run `openclaw nodes pending` and approve manually. +- Pairing prompt never appears: run `openclaw devices list` and approve manually. - Reconnect fails after reinstall: the Keychain pairing token was cleared; re-pair the node. ## Related docs -- [Pairing](/gateway/pairing) +- [Pairing](/channels/pairing) - [Discovery](/gateway/discovery) - [Bonjour](/gateway/bonjour)