LLM/moltbot
1
0
Fork 0
mirror of https://github.com/moltbot/moltbot.git synced 2026-05-10 12:32:27 +00:00
Code Issues Packages Projects Releases Wiki Activity

refactor: use sqlite model catalog at runtime

Browse Source
This commit is contained in:
Peter Steinberger
2026-05-09 22:58:40 +01:00
parent ab01130dcc
commit 29fe040cda
23 changed files with 74 additions and 67 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
docs/auth-credential-semantics.md
View File

@@ -86,8 +86,8 @@ removes the marker from the credential store.
## Probe target resolution
- Probe targets can come from auth profiles, environment credentials, or
`models.json`.
- Probe targets can come from auth profiles, environment credentials, or the
stored model catalog.
- If a provider has credentials but OpenClaw cannot resolve a probeable model
candidate for it, `models status --probe` reports `status: no_model` with
`reasonCode: no_model`.

2
docs/cli/agent.md
View File

@@ -60,7 +60,7 @@ openclaw agent --agent ops --message "Run locally" --local
- `--json` keeps stdout reserved for the JSON response. Gateway, plugin, and embedded-fallback diagnostics are routed to stderr so scripts can parse stdout directly.
- Embedded fallback JSON includes `meta.transport: "embedded"` and `meta.fallbackFrom: "gateway"` so scripts can distinguish fallback runs from Gateway runs.
- If the Gateway accepts an agent run but the CLI times out waiting for the final reply, embedded fallback uses a fresh explicit `gateway-fallback-*` session/run id and reports `meta.fallbackReason: "gateway_timeout"` plus the fallback session fields. This avoids racing the Gateway-owned transcript lock or silently replacing the original routed conversation session.
- When this command triggers `models.json` regeneration, SecretRef-managed provider credentials are persisted as non-secret markers (for example env var names, `secretref-env:ENV_VAR_NAME`, or `secretref-managed`), not resolved secret plaintext.
- When this command materializes the stored model catalog, SecretRef-managed provider credentials are persisted as non-secret markers (for example env var names, `secretref-env:ENV_VAR_NAME`, or `secretref-managed`), not resolved secret plaintext.
- Marker writes are source-authoritative: OpenClaw persists markers from the active source config snapshot, not from resolved runtime secret values.
## Related

6
docs/cli/models.md
View File

@@ -39,7 +39,7 @@ Probes are real requests (may consume tokens and trigger rate limits).
Use `--agent <id>` to inspect a configured agent's model/auth state. When omitted,
the command uses `OPENCLAW_AGENT_DIR`/`PI_CODING_AGENT_DIR` if set, otherwise the
configured default agent.
Probe rows can come from auth profiles, env credentials, or `models.json`.
Probe rows can come from auth profiles, env credentials, or the stored model catalog.
For Codex OAuth troubleshooting, `openclaw models status`,
`openclaw models auth list --provider openai-codex`, and
`openclaw config get agents.defaults.model --json` are the quickest way to
@@ -50,8 +50,8 @@ Notes:
- `models set <model-or-alias>` accepts `provider/model` or an alias.
- `models list` is read-only: it reads config, auth profiles, existing catalog
state, and provider-owned catalog rows, but it does not rewrite
`models.json`.
state, and provider-owned catalog rows, but it does not rewrite the stored
model catalog.
- The `Auth` column is provider-level and read-only. It is computed from local
auth profile metadata, env markers, configured provider keys, local-provider
markers, AWS Bedrock env/profile markers, and plugin synthetic-auth metadata;

2
docs/cli/secrets.md
View File

@@ -72,7 +72,7 @@ Scan OpenClaw state for:
- plaintext secret storage
- unresolved refs
- precedence drift (`auth-profiles.json` credentials shadowing `openclaw.json` refs)
- generated `agents/*/agent/models.json` residues (provider `apiKey` values and sensitive provider headers)
- stored model catalog residues (provider `apiKey` values and sensitive provider headers)
- legacy residues (legacy auth store entries, OAuth reminders)
Header residue note:

2
docs/concepts/model-providers.md
View File

@@ -342,7 +342,7 @@ See [/providers/kilocode](/providers/kilocode) for setup details.
## Providers via `models.providers` (custom/base URL)
Use `models.providers` (or `models.json`) to add **custom** providers or OpenAI/Anthropic-compatible proxies.
Use `models.providers` to add **custom** providers or OpenAI/Anthropic-compatible proxies. Older `models.json` files are imported by `openclaw doctor --fix`.
Many of the bundled provider plugins below already publish a default catalog. Use explicit `models.providers.<id>` entries only when you want to override the default base URL, headers, or model list.

4
docs/gateway/authentication.md
View File

@@ -108,7 +108,7 @@ openclaw models auth paste-token --provider openrouter
}
```
OpenClaw expects the canonical `version` + `profiles` shape at runtime. If an older install still has a flat file such as `{ "openrouter": { "apiKey": "..." } }`, run `openclaw doctor --fix` to rewrite it as an `openrouter:default` API-key profile; doctor keeps a `.legacy-flat.*.bak` copy beside the original. Endpoint details such as `baseUrl`, `api`, model ids, headers, and timeouts belong under `models.providers.<id>` in `openclaw.json` or `models.json`, not in `auth-profiles.json`.
OpenClaw expects the canonical `version` + `profiles` shape at runtime. If an older install still has a flat file such as `{ "openrouter": { "apiKey": "..." } }`, run `openclaw doctor --fix` to rewrite it as an `openrouter:default` API-key profile; doctor keeps a `.legacy-flat.*.bak` copy beside the original. Endpoint details such as `baseUrl`, `api`, model ids, headers, and timeouts belong under `models.providers.<id>` in `openclaw.json` or the stored model catalog, not in `auth-profiles.json`.
External auth routes such as Bedrock `auth: "aws-sdk"` are also not credentials. If you want a named Bedrock route, put `auth.profiles.<id>.mode: "aws-sdk"` in `openclaw.json`; do not write `type: "aws-sdk"` into `auth-profiles.json`. `openclaw doctor --fix` moves legacy AWS SDK markers from the credential store into config metadata.
@@ -132,7 +132,7 @@ openclaw models status --probe
Notes:
- Probe rows can come from auth profiles, env credentials, or `models.json`.
- Probe rows can come from auth profiles, env credentials, or the stored model catalog.
- If explicit `auth.order.<provider>` omits a stored profile, probe reports
`excluded_by_auth_order` for that profile instead of trying it.
- If auth exists but OpenClaw cannot resolve a probeable model candidate for

6
docs/gateway/config-tools.md
View File

@@ -387,7 +387,7 @@ Experimental built-in tool flags. Default off unless a strict-agentic GPT-5 auto
## Custom providers and base URLs
OpenClaw uses the built-in model catalog. Add custom providers via `models.providers` in config or `~/.openclaw/agents/<agentId>/agent/models.json`.
OpenClaw uses the built-in model catalog. Add custom providers via `models.providers` in config; doctor imports old `~/.openclaw/agents/<agentId>/agent/models.json` files into the stored model catalog.
```json5
{
@@ -421,14 +421,14 @@ OpenClaw uses the built-in model catalog. Add custom providers via `models.provi
- Use `authHeader: true` + `headers` for custom auth needs.
- Override agent config root with `OPENCLAW_AGENT_DIR` (or `PI_CODING_AGENT_DIR`, a legacy environment variable alias).
- Merge precedence for matching provider IDs:
- Non-empty agent `models.json` `baseUrl` values win.
- Non-empty stored agent catalog `baseUrl` values win.
- Non-empty agent `apiKey` values win only when that provider is not SecretRef-managed in current config/auth-profile context.
- SecretRef-managed provider `apiKey` values are refreshed from source markers (`ENV_VAR_NAME` for env refs, `secretref-managed` for file/exec refs) instead of persisting resolved secrets.
- SecretRef-managed provider header values are refreshed from source markers (`secretref-env:ENV_VAR_NAME` for env refs, `secretref-managed` for file/exec refs).
- Empty or missing agent `apiKey`/`baseUrl` fall back to `models.providers` in config.
- Matching model `contextWindow`/`maxTokens` use the higher value between explicit config and implicit catalog values.
- Matching model `contextTokens` preserves an explicit runtime cap when present; use it to limit effective context without changing native model metadata.
- Use `models.mode: "replace"` when you want config to fully rewrite `models.json`.
- Use `models.mode: "replace"` when you want config to fully rewrite the stored model catalog.
- Marker persistence is source-authoritative: markers are written from the active source config snapshot (pre-resolution), not from resolved runtime secret values.
</Accordion>

4
docs/gateway/secrets.md
View File

@@ -469,8 +469,8 @@ Default operator flow:
<Accordion title="secrets audit">
Findings include:
- plaintext values at rest (`openclaw.json`, `auth-profiles.json`, `.env`, and generated `agents/*/agent/models.json`)
- plaintext sensitive provider header residues in generated `models.json` entries
- plaintext values at rest (`openclaw.json`, `auth-profiles.json`, `.env`, and the stored model catalog)
- plaintext sensitive provider header residues in stored model catalog entries
- unresolved refs
- precedence shadowing (`auth-profiles.json` taking priority over `openclaw.json` refs)
- legacy residues (`auth.json`, OAuth reminders)

2
docs/pi.md
View File

@@ -96,7 +96,7 @@ src/agents/
├── model-auth.ts # Auth profile resolution
├── auth-profiles.ts # Profile store, cooldown, failover
├── model-selection.ts # Default model resolution
├── models-config.ts # models.json generation
├── models-config.ts # SQLite model catalog materialization
├── model-catalog.ts # Model catalog cache
├── context-window-guard.ts # Context window validation
├── failover-error.ts # FailoverError class

2
docs/plugins/architecture-internals.md
View File

@@ -256,7 +256,7 @@ listed here.
| # | Hook | What it does | When to use |
| --- | --------------------------------- | -------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- |
| 1 | `catalog` | Publish provider config into `models.providers` during `models.json` generation | Provider owns a catalog or base URL defaults |
| 1 | `catalog` | Publish provider config into `models.providers` during model catalog materialization | Provider owns a catalog or base URL defaults |
| 2 | `applyConfigDefaults` | Apply provider-owned global config defaults during config materialization | Defaults depend on auth mode, env, or provider model-family semantics |
| -- | _(built-in model lookup)_ | OpenClaw tries the normal registry/catalog path first | _(not a plugin hook)_ |
| 3 | `normalizeModelId` | Normalize legacy or preview model-id aliases before lookup | Provider owns alias cleanup before canonical model resolution |

2
docs/providers/huggingface.md
View File

@@ -126,7 +126,7 @@ You can append `:fastest` or `:cheapest` to any model id. Set your default order
You can add these as separate entries in `models.providers.huggingface.models` or set `model.primary` with the suffix. You can also set your default provider order in [Inference Provider settings](https://hf.co/settings/inference-providers) (no suffix = use that order).
- **Config merge:** Existing entries in `models.providers.huggingface.models` (e.g. in `models.json`) are kept when config is merged. So any custom `name`, `alias`, or model options you set there are preserved.
- **Config merge:** Existing entries in `models.providers.huggingface.models` and the stored model catalog are kept when config is merged. So any custom `name`, `alias`, or model options you set there are preserved.
</Accordion>

2
docs/providers/minimax.md
View File

@@ -441,7 +441,7 @@ See [MiniMax Search](/tools/minimax-search) for full web search configuration an
- Alternate chat model: `MiniMax-M2.7-highspeed`
- Onboarding and direct API-key setup write text-only model definitions for both M2.7 variants
- Image understanding uses the plugin-owned `MiniMax-VL-01` media provider
- Update pricing values in `models.json` if you need exact cost tracking
- Update pricing values in `models.providers` if you need exact cost tracking
- Use `openclaw models list` to confirm the current provider id, then switch with `openclaw models set minimax/MiniMax-M2.7` or `openclaw models set minimax-portal/MiniMax-M2.7`
<Tip>

2
docs/providers/ollama.md
View File

@@ -190,7 +190,7 @@ When you set `OLLAMA_API_KEY` (or an auth profile) and **do not** define `models
| Token limits | Sets `maxTokens` to the default Ollama max-token cap used by OpenClaw |
| Costs | Sets all costs to `0` |
This avoids manual model entries while keeping the catalog aligned with the local Ollama instance. You can use a full ref such as `ollama/<pulled-model>:latest` in local `infer model run`; OpenClaw resolves that installed model from Ollama's live catalog without requiring a hand-written `models.json` entry.
This avoids manual model entries while keeping the catalog aligned with the local Ollama instance. You can use a full ref such as `ollama/<pulled-model>:latest` in local `infer model run`; OpenClaw resolves that installed model from Ollama's live catalog without requiring a hand-written model catalog entry.
For signed-in Ollama hosts, some `:cloud` models may be usable through `/api/chat`
and `/api/show` before they appear in `/api/tags`. When you explicitly select a

2
docs/reference/secretref-credential-surface.md
View File

@@ -122,7 +122,7 @@ Notes:
- Auth-profile refs are included in runtime resolution and audit coverage.
- In `openclaw.json`, SecretRefs must use structured objects such as `{"source":"env","provider":"default","id":"DISCORD_BOT_TOKEN"}`. Legacy `secretref-env:<ENV_VAR>` marker strings are rejected on SecretRef credential paths; run `openclaw doctor --fix` to migrate valid markers.
- OAuth policy guard: `auth.profiles.<id>.mode = "oauth"` cannot be combined with SecretRef inputs for that profile. Startup/reload and auth-profile resolution fail fast when this policy is violated.
- For SecretRef-managed model providers, generated `agents/*/agent/models.json` entries persist non-secret markers (not resolved secret values) for `apiKey`/header surfaces.
- For SecretRef-managed model providers, stored model catalog entries persist non-secret markers (not resolved secret values) for `apiKey`/header surfaces.
- Marker persistence is source-authoritative: OpenClaw writes markers from the active source config snapshot (pre-resolution), not from resolved runtime secret values.
- For web search:
- In explicit provider mode (`tools.web.search.provider` set), only the selected provider key is active.

2
src/config/schema.help.ts
View File

@@ -868,7 +868,7 @@ export const FIELD_HELP: Record<string, string> = {
models:
"Model catalog root for provider definitions, merge/replace behavior, and optional Bedrock discovery integration. Keep provider definitions explicit and validated before relying on production failover paths.",
"models.mode":
'Controls provider catalog behavior: "merge" keeps built-ins and overlays your custom providers, while "replace" uses only your configured providers. In "merge", matching provider IDs preserve non-empty agent models.json baseUrl values, while apiKey values are preserved only when the provider is not SecretRef-managed in current config/auth-profile context; SecretRef-managed providers refresh apiKey from current source markers, and matching model contextWindow/maxTokens use the higher value between explicit and implicit entries.',
'Controls provider catalog behavior: "merge" keeps built-ins and overlays your custom providers, while "replace" uses only your configured providers. In "merge", matching provider IDs preserve non-empty stored agent catalog baseUrl values, while apiKey values are preserved only when the provider is not SecretRef-managed in current config/auth-profile context; SecretRef-managed providers refresh apiKey from current source markers, and matching model contextWindow/maxTokens use the higher value between explicit and implicit entries.',
"models.providers":
"Provider map keyed by provider ID containing connection/auth settings and concrete model definitions. Use stable provider keys so references from agents and tooling remain portable across environments.",
"models.pricing":

27
src/gateway/gateway-models.profiles.live.test.ts
View File

@@ -31,7 +31,11 @@ import { isLiveProfileKeyModeEnabled, isLiveTestEnabled } from "../agents/live-t
import { getApiKeyForModel, resolveEnvApiKey } from "../agents/model-auth.js";
import { normalizeProviderId } from "../agents/model-selection.js";
import { shouldSuppressBuiltInModel } from "../agents/model-suppression.js";
import { ensureOpenClawModelsJson } from "../agents/models-config.js";
import {
readStoredModelsConfigRaw,
writeStoredModelsConfigRaw,
} from "../agents/models-config-store.js";
import { ensureOpenClawModelCatalog } from "../agents/models-config.js";
import {
clampThinkingLevel,
type Api,
@@ -1557,10 +1561,13 @@ async function loadProviderScopedConfiguredModels(params: {
agentDir: string;
providerList: readonly string[];
}): Promise<Array<Model<Api>>> {
const modelsPath = path.join(params.agentDir, "models.json");
let parsed: { providers?: Record<string, ModelProviderConfig> };
try {
parsed = JSON.parse(await fs.readFile(modelsPath, "utf8")) as {
const stored = readStoredModelsConfigRaw(params.agentDir);
if (!stored) {
return [];
}
parsed = JSON.parse(stored.raw) as {
providers?: Record<string, ModelProviderConfig>;
};
} catch {
@@ -1942,9 +1949,11 @@ async function runGatewayModelSuite(params: GatewayModelSuiteParams) {
const liveProviders = nextCfg.models?.providers;
if (liveProviders && Object.keys(liveProviders).length > 0) {
const modelsPath = path.join(tempAgentDir, "models.json");
await fs.mkdir(tempAgentDir, { recursive: true });
await fs.writeFile(modelsPath, `${JSON.stringify({ providers: liveProviders }, null, 2)}\n`);
writeStoredModelsConfigRaw(
tempAgentDir,
`${JSON.stringify({ providers: liveProviders }, null, 2)}\n`,
);
}
// Keep the broad live Docker suite on the impl entrypoint. The lazy public
@@ -2639,14 +2648,14 @@ describeLive("gateway live (dev agent, profile keys)", () => {
"[all-models] load config",
);
const workspaceDir = resolveAgentWorkspaceDir(cfg, DEFAULT_AGENT_ID);
logProgress("[all-models] preparing models.json");
logProgress("[all-models] preparing model catalog");
await withGatewayLiveSetupTimeout(
ensureOpenClawModelsJson(cfg, undefined, {
ensureOpenClawModelCatalog(cfg, undefined, {
workspaceDir,
...(providerList ? { providerDiscoveryProviderIds: providerList } : {}),
providerDiscoveryEntriesOnly: true,
}),
"[all-models] prepare models.json",
"[all-models] prepare model catalog",
);
const agentDir = resolveDefaultAgentDir(cfg);
@@ -2865,7 +2874,7 @@ describeLive("gateway live (dev agent, profile keys)", () => {
process.env.OPENCLAW_GATEWAY_TOKEN = token;
const cfg = getRuntimeConfig();
await ensureOpenClawModelsJson(cfg);
await ensureOpenClawModelCatalog(cfg);
const agentDir = resolveDefaultAgentDir(cfg);
const authStorage = discoverAuthStorage(agentDir);

14
src/gateway/server-startup-post-attach.test.ts
View File

@@ -45,7 +45,7 @@ const hoisted = vi.hoisted(() => {
model: "gpt-5.4",
}));
const resolveEmbeddedAgentRuntime = vi.fn(() => "pi");
const ensureOpenClawModelsJson = vi.fn(async () => undefined);
const ensureOpenClawModelCatalog = vi.fn(async () => undefined);
return {
startPluginServices,
startGmailWatcherWithLogs,
@@ -71,7 +71,7 @@ const hoisted = vi.hoisted(() => {
isCliProvider,
resolveConfiguredModelRef,
resolveEmbeddedAgentRuntime,
ensureOpenClawModelsJson,
ensureOpenClawModelCatalog,
};
});
@@ -170,7 +170,7 @@ vi.mock("../agents/pi-embedded-runner/runtime.js", () => ({
}));
vi.mock("../agents/models-config.js", () => ({
ensureOpenClawModelsJson: hoisted.ensureOpenClawModelsJson,
ensureOpenClawModelCatalog: hoisted.ensureOpenClawModelCatalog,
}));
vi.mock("./server-tailscale.js", () => ({
@@ -218,8 +218,8 @@ describe("startGatewayPostAttachRuntime", () => {
hoisted.resolveConfiguredModelRef.mockClear();
hoisted.resolveEmbeddedAgentRuntime.mockReset();
hoisted.resolveEmbeddedAgentRuntime.mockReturnValue("pi");
hoisted.ensureOpenClawModelsJson.mockReset();
hoisted.ensureOpenClawModelsJson.mockResolvedValue(undefined);
hoisted.ensureOpenClawModelCatalog.mockReset();
hoisted.ensureOpenClawModelCatalog.mockResolvedValue(undefined);
});
afterEach(() => {
@@ -551,7 +551,7 @@ describe("startGatewayPostAttachRuntime", () => {
}
});
it("prewarms models.json in the configured default agent dir", async () => {
it("prewarms the model catalog in the configured default agent dir", async () => {
const cfg = {
agents: {
defaults: { model: "openai/gpt-5.4" },
@@ -568,7 +568,7 @@ describe("startGatewayPostAttachRuntime", () => {
});
expect(hoisted.resolveDefaultAgentDir).toHaveBeenCalledWith(cfg);
expect(hoisted.ensureOpenClawModelsJson).toHaveBeenCalledWith(
expect(hoisted.ensureOpenClawModelCatalog).toHaveBeenCalledWith(
cfg,
"/tmp/openclaw-state/agents/ops/agent",
expect.objectContaining({

4
src/gateway/server-startup-post-attach.ts
View File

@@ -251,12 +251,12 @@ async function prewarmConfiguredPrimaryModel(params: {
return;
}
// Keep startup prewarm metadata-only; resolving models can import provider runtimes and block readiness.
const { ensureOpenClawModelsJson } = await import("../agents/models-config.js");
const { ensureOpenClawModelCatalog } = await import("../agents/models-config.js");
const agentDir = resolveDefaultAgentDir(params.cfg);
const workspaceDir =
params.workspaceDir ?? resolveAgentWorkspaceDir(params.cfg, resolveDefaultAgentId(params.cfg));
try {
await ensureOpenClawModelsJson(params.cfg, agentDir, {
await ensureOpenClawModelCatalog(params.cfg, agentDir, {
workspaceDir,
providerDiscoveryProviderIds: [provider],
providerDiscoveryTimeoutMs: STARTUP_PROVIDER_DISCOVERY_TIMEOUT_MS,

22
src/gateway/server-startup.test.ts
View File

@@ -1,7 +1,7 @@
import { beforeAll, beforeEach, describe, expect, it, vi } from "vitest";
import type { OpenClawConfig } from "../config/config.js";
const ensureOpenClawModelsJsonMock = vi.fn<
const ensureOpenClawModelCatalogMock = vi.fn<
(
config: unknown,
agentDir: unknown,
@@ -18,8 +18,8 @@ vi.mock("../agents/agent-scope.js", () => ({
}));
vi.mock("../agents/models-config.js", () => ({
ensureOpenClawModelsJson: (config: unknown, agentDir: unknown, options?: unknown) =>
ensureOpenClawModelsJsonMock(config, agentDir, options),
ensureOpenClawModelCatalog: (config: unknown, agentDir: unknown, options?: unknown) =>
ensureOpenClawModelCatalogMock(config, agentDir, options),
}));
vi.mock("../agents/pi-embedded-runner/model.js", () => {
@@ -44,7 +44,7 @@ describe("gateway startup primary model warmup", () => {
});
beforeEach(() => {
ensureOpenClawModelsJsonMock.mockClear();
ensureOpenClawModelCatalogMock.mockClear();
piModelModuleLoadedMock.mockClear();
resolveEmbeddedAgentRuntimeMock.mockClear();
resolveEmbeddedAgentRuntimeMock.mockReturnValue("auto");
@@ -66,7 +66,7 @@ describe("gateway startup primary model warmup", () => {
log: { warn: vi.fn() },
});
expect(ensureOpenClawModelsJsonMock).toHaveBeenCalledWith(
expect(ensureOpenClawModelCatalogMock).toHaveBeenCalledWith(
cfg,
"/tmp/agent",
expect.objectContaining({
@@ -85,7 +85,7 @@ describe("gateway startup primary model warmup", () => {
log: { warn: vi.fn() },
});
expect(ensureOpenClawModelsJsonMock).not.toHaveBeenCalled();
expect(ensureOpenClawModelCatalogMock).not.toHaveBeenCalled();
expect(piModelModuleLoadedMock).not.toHaveBeenCalled();
});
@@ -123,7 +123,7 @@ describe("gateway startup primary model warmup", () => {
log: { warn: vi.fn() },
});
expect(ensureOpenClawModelsJsonMock).not.toHaveBeenCalled();
expect(ensureOpenClawModelCatalogMock).not.toHaveBeenCalled();
expect(piModelModuleLoadedMock).not.toHaveBeenCalled();
});
@@ -142,7 +142,7 @@ describe("gateway startup primary model warmup", () => {
log: { warn: vi.fn() },
});
expect(ensureOpenClawModelsJsonMock).not.toHaveBeenCalled();
expect(ensureOpenClawModelCatalogMock).not.toHaveBeenCalled();
expect(piModelModuleLoadedMock).not.toHaveBeenCalled();
});
@@ -163,7 +163,7 @@ describe("gateway startup primary model warmup", () => {
log: { warn: vi.fn() },
});
expect(ensureOpenClawModelsJsonMock).toHaveBeenCalledWith(
expect(ensureOpenClawModelCatalogMock).toHaveBeenCalledWith(
cfg,
"/tmp/agent",
expect.objectContaining({
@@ -176,8 +176,8 @@ describe("gateway startup primary model warmup", () => {
expect(piModelModuleLoadedMock).not.toHaveBeenCalled();
});
it("warns when scoped models.json preparation fails", async () => {
ensureOpenClawModelsJsonMock.mockRejectedValueOnce(new Error("models write failed"));
it("warns when scoped model catalog preparation fails", async () => {
ensureOpenClawModelCatalogMock.mockRejectedValueOnce(new Error("models write failed"));
const warn = vi.fn();
await prewarmConfiguredPrimaryModel({

14
src/gateway/test-helpers.mocks.ts
View File

@@ -1,4 +1,3 @@
import path from "node:path";
import { vi } from "vitest";
import {
getTestPluginRegistry,
@@ -99,11 +98,10 @@ vi.mock("../agents/pi-model-discovery.js", async () => {
);
const createActualRegistry = (...args: Parameters<typeof actual.discoverModels>) => {
const modelsFile = path.join(args[1], "models.json");
const Registry = actual.ModelRegistry as unknown as {
create?: (
authStorage: unknown,
modelsFile: string,
agentDir?: string,
) => {
getAll: () => Array<{ provider?: string; id?: string }>;
getAvailable: () => Array<{ provider?: string; id?: string }>;
@@ -111,7 +109,7 @@ vi.mock("../agents/pi-model-discovery.js", async () => {
};
new (
authStorage: unknown,
modelsFile: string,
agentDir?: string,
): {
getAll: () => Array<{ provider?: string; id?: string }>;
getAvailable: () => Array<{ provider?: string; id?: string }>;
@@ -119,17 +117,17 @@ vi.mock("../agents/pi-model-discovery.js", async () => {
};
};
if (typeof Registry.create === "function") {
return Registry.create(args[0], modelsFile);
return Registry.create(args[0], args[1]);
}
return new Registry(args[0], modelsFile);
return new Registry(args[0], args[1]);
};
class MockModelRegistry {
private readonly actualRegistry?: ReturnType<typeof createActualRegistry>;
constructor(authStorage: unknown, modelsFile: string) {
constructor(authStorage: unknown, agentDir: string) {
if (!piSdkMock.enabled) {
this.actualRegistry = createActualRegistry(authStorage as never, path.dirname(modelsFile));
this.actualRegistry = createActualRegistry(authStorage as never, agentDir);
}
}

10
src/media-understanding/image.test.ts
View File

@@ -2,7 +2,7 @@ import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
const hoisted = vi.hoisted(() => ({
completeMock: vi.fn(),
ensureOpenClawModelsJsonMock: vi.fn(async () => {}),
ensureOpenClawModelCatalogMock: vi.fn(async () => {}),
getApiKeyForModelMock: vi.fn(async () => ({
apiKey: "oauth-test", // pragma: allowlist secret
source: "test",
@@ -24,7 +24,7 @@ const hoisted = vi.hoisted(() => ({
}));
const {
completeMock,
ensureOpenClawModelsJsonMock,
ensureOpenClawModelCatalogMock,
getApiKeyForModelMock,
resolveApiKeyForProviderMock,
requireApiKeyMock,
@@ -60,7 +60,7 @@ vi.mock("../agents/models-config.js", async () => ({
...(await vi.importActual<typeof import("../agents/models-config.js")>(
"../agents/models-config.js",
)),
ensureOpenClawModelsJson: ensureOpenClawModelsJsonMock,
ensureOpenClawModelCatalog: ensureOpenClawModelCatalogMock,
}));
vi.mock("../agents/model-auth.js", () => ({
@@ -174,7 +174,7 @@ describe("describeImageWithModel", () => {
text: "portal ok",
model: "MiniMax-VL-01",
});
expect(ensureOpenClawModelsJsonMock).toHaveBeenCalled();
expect(ensureOpenClawModelCatalogMock).toHaveBeenCalled();
const authRequest = getApiKeyForModelCall();
expect(authRequest?.store).toBe(authStore);
expect(requireApiKeyMock).toHaveBeenCalled();
@@ -635,7 +635,7 @@ describe("describeImageWithModel", () => {
it("rejects when image runtime setup exceeds the request timeout", async () => {
vi.useFakeTimers();
ensureOpenClawModelsJsonMock.mockImplementationOnce(() => new Promise(() => {}));
ensureOpenClawModelCatalogMock.mockImplementationOnce(() => new Promise(() => {}));
const result = describeImageWithModel({
cfg: {},

4
src/media-understanding/image.ts
View File

@@ -5,7 +5,7 @@ import {
resolveApiKeyForProvider,
} from "../agents/model-auth.js";
import { findNormalizedProviderValue, normalizeModelRef } from "../agents/model-selection.js";
import { ensureOpenClawModelsJson } from "../agents/models-config.js";
import { ensureOpenClawModelCatalog } from "../agents/models-config.js";
import type { Api, Context, Model, ProviderStreamOptions } from "../agents/pi-ai-contract.js";
import { complete } from "../agents/pi-ai-contract.js";
import { resolveModelWithRegistry } from "../agents/pi-embedded-runner/model.js";
@@ -142,7 +142,7 @@ async function resolveImageRuntime(params: {
preferredProfile?: string;
authStore?: ImageDescriptionRequest["authStore"];
}): Promise<{ apiKey: string; model: Model<Api> }> {
await ensureOpenClawModelsJson(params.cfg, params.agentDir);
await ensureOpenClawModelCatalog(params.cfg, params.agentDir);
const { discoverAuthStorage, discoverModels } = await loadPiModelDiscoveryRuntime();
const authStorage = discoverAuthStorage(params.agentDir);
const modelRegistry = discoverModels(authStorage, params.agentDir);

2
src/media-understanding/runner.entries.ts
View File

@@ -79,7 +79,7 @@ function sanitizeProviderHeaders(
}
// Intentionally preserve marker-shaped values here. This path handles
// explicit config/runtime provider headers, where literal values may
// legitimately match marker patterns; discovered models.json entries are
// legitimately match marker patterns; discovered model catalog entries are
// sanitized separately in the model registry path.
next[key] = value;
}