refactor: store memory wiki digests in sqlite

This commit is contained in:
Peter Steinberger
2026-05-08 19:47:14 +01:00
parent 7a954b6d61
commit cdfd09554f
22 changed files with 463 additions and 106 deletions

View File

@@ -232,7 +232,7 @@ iMessage catchup is now available as an opt-in feature on the bundled plugin. On
There is no supported BlueBubbles runtime to switch back to. If iMessage verification fails, set `channels.imessage.enabled: false`, restart the Gateway, fix the `imsg` blocker, and retry the cutover.
The reply cache lives at `~/.openclaw/state/imessage/reply-cache.jsonl` (mode `0600`, parent dir `0700`). It is safe to delete if you want a clean slate.
The reply cache lives in SQLite plugin state under `~/.openclaw/state/openclaw.sqlite`. Run `openclaw doctor --fix` after updating if an older `imessage/reply-cache.jsonl` file is still present.
## Related

View File

@@ -646,9 +646,9 @@ curl "https://api.telegram.org/bot<bot_token>/getUpdates"
- `Sticker.fileUniqueId`
- `Sticker.cachedDescription`
Sticker cache file:
Sticker cache storage:
- `~/.openclaw/telegram/sticker-cache.json`
- SQLite plugin state in `~/.openclaw/state/openclaw.sqlite`
Stickers are described once (when possible) and cached to reduce repeated vision calls.
@@ -773,7 +773,7 @@ curl "https://api.telegram.org/bot<bot_token>/getUpdates"
- `channels.telegram.timeoutSeconds` overrides Telegram API client timeout (if unset, grammY default applies). Bot clients clamp configured values below the 60-second outbound text/typing request guard so grammY does not abort visible reply delivery before OpenClaw's transport guard and fallback can run. Long polling still uses a 45-second `getUpdates` request guard so idle polls are not abandoned indefinitely.
- `channels.telegram.pollingStallThresholdMs` defaults to `120000`; tune between `30000` and `600000` only for false-positive polling-stall restarts.
- group context history uses `channels.telegram.historyLimit` or `messages.groupChat.historyLimit` (default 50); `0` disables.
- reply/quote/forward supplemental context is normalized into a nearest-first reply chain when the gateway has observed the parent messages; the observed-message cache is persisted beside the session store. Telegram only includes one shallow `reply_to_message` in updates, so chains older than the cache are limited to Telegram's current update payload.
- reply/quote/forward supplemental context is normalized into a nearest-first reply chain when the gateway has observed the parent messages; the observed-message cache is persisted in SQLite plugin state. Telegram only includes one shallow `reply_to_message` in updates, so chains older than the cache are limited to Telegram's current update payload.
- Telegram allowlists primarily gate who can trigger the agent, not a full supplemental-context redaction boundary.
- DM history controls:
- `channels.telegram.dmHistoryLimit`

View File

@@ -106,12 +106,10 @@ Notes:
### `wiki compile`
Rebuild indexes, related blocks, dashboards, and compiled digests.
Rebuild indexes, related blocks, dashboards, and SQLite-backed compiled digests.
This writes stable machine-facing artifacts under:
- `.openclaw-wiki/cache/agent-digest.json`
- `.openclaw-wiki/cache/claims.jsonl`
The stable machine-facing digests live in OpenClaw's SQLite plugin state so
agents and runtime code do not have to scrape Markdown pages.
If `render.createDashboards` is enabled, compile also refreshes report pages.

View File

@@ -236,14 +236,9 @@ claims:
## Compile pipeline
The compile step reads wiki pages, normalizes summaries, and emits stable
machine-facing artifacts under:
- `.openclaw-wiki/cache/agent-digest.json`
- `.openclaw-wiki/cache/claims.jsonl`
These digests exist so agents and runtime code do not have to scrape Markdown
pages.
The compile step reads wiki pages, normalizes summaries, and stores stable
machine-facing digests in SQLite plugin state. These digests exist so agents
and runtime code do not have to scrape Markdown pages.
Compiled output also powers:
@@ -353,7 +348,7 @@ plugin supports corpus selection.
## Prompt and context behavior
When `context.includeCompiledDigestPrompt` is enabled, memory prompt sections
append a compact compiled snapshot from `agent-digest.json`.
append a compact compiled snapshot from SQLite plugin state.
That snapshot is intentionally small and high-signal:

View File

@@ -302,8 +302,8 @@ The remaining cleanup is mostly consolidation and deletion:
than OpenClaw runtime cache rows.
- Memory Wiki activity logs now use SQLite plugin state instead of
`.openclaw-wiki/log.jsonl`. The Memory Wiki migration provider imports old
JSONL logs; wiki markdown, generated digest files, and user vault content stay
file-backed as workspace content.
JSONL logs; wiki markdown and user vault content stay file-backed as
workspace content.
- Crestodian audit entries now use core SQLite plugin state instead of
`audit/crestodian.jsonl`. Doctor imports the legacy JSONL audit log and
removes it after successful import.
@@ -428,6 +428,10 @@ The remaining cleanup is mostly consolidation and deletion:
per vault/run id instead of writing `.openclaw-wiki/import-runs/*.json`.
Rollback snapshots remain explicit vault files until import-run snapshot
archival is moved into blob storage.
- Memory Wiki compiled digests now store SQLite plugin blob rows instead of
writing `.openclaw-wiki/cache/agent-digest.json` and
`.openclaw-wiki/cache/claims.jsonl`. The migration provider imports old cache
files and removes the cache directory when it becomes empty.
- ClawHub skill install tracking now stores one SQLite plugin-state row per
workspace/skill instead of writing or reading `.clawhub/lock.json` and
`.clawhub/origin.json` sidecars at runtime. Doctor/migrate imports the legacy
@@ -1077,6 +1081,8 @@ Add a repo check that fails new runtime writes to legacy state paths:
- Memory Wiki `.openclaw-wiki/log.jsonl`
- Memory Wiki `.openclaw-wiki/source-sync.json`
- Memory Wiki `.openclaw-wiki/import-runs/*.json`
- Memory Wiki `.openclaw-wiki/cache/agent-digest.json`
- Memory Wiki `.openclaw-wiki/cache/claims.jsonl`
- ClawHub `.clawhub/lock.json`
- ClawHub `.clawhub/origin.json`

View File

@@ -199,12 +199,11 @@ Imported themes are stored only in the current browser profile. They are not wri
The Control UI ships a `manifest.webmanifest` and a service worker, so modern browsers can install it as a standalone PWA. Web Push lets the Gateway wake the installed PWA with notifications even when the tab or browser window is not open.
| Surface | What it does |
| ----------------------------------------------------- | ------------------------------------------------------------------ |
| `ui/public/manifest.webmanifest` | PWA manifest. Browsers offer "Install app" once it is reachable. |
| `ui/public/sw.js` | Service worker that handles `push` events and notification clicks. |
| `push/vapid-keys.json` (under the OpenClaw state dir) | Auto-generated VAPID keypair used to sign Web Push payloads. |
| `push/web-push-subscriptions.json` | Persisted browser subscription endpoints. |
| Surface | What it does |
| -------------------------------- | ------------------------------------------------------------------ |
| `ui/public/manifest.webmanifest` | PWA manifest. Browsers offer "Install app" once it is reachable. |
| `ui/public/sw.js` | Service worker that handles `push` events and notification clicks. |
| `state/openclaw.sqlite` | SQLite-backed VAPID keys and browser subscription endpoints. |
Override the VAPID keypair through env vars on the Gateway process when you want to pin keys (for multi-host deployments, secrets rotation, or tests):