LLM/moltbot
1
0
Fork 0
mirror of https://github.com/moltbot/moltbot.git synced 2026-04-26 07:57:40 +00:00
Code Issues Packages Projects Releases Wiki Activity

test: add shared media live harness

Browse Source
This commit is contained in:
Peter Steinberger
2026-04-06 18:49:31 +01:00
parent dd978bf975
commit 41ea5316aa
7 changed files with 487 additions and 20 deletions
Download Patch File Download Diff File Expand all files Collapse all files

25
docs/help/testing.md
View File

@@ -454,6 +454,7 @@ If you want to rely on env keys (e.g. exported in your `~/.profile`), run local
- Test: `src/image-generation/runtime.live.test.ts`
- Command: `pnpm test:live src/image-generation/runtime.live.test.ts`
- Harness: `pnpm test:live:media image`
- Scope:
- Enumerates every registered image-generation provider plugin
- Loads missing provider env vars from your login shell (`~/.profile`) before probing
@@ -478,6 +479,7 @@ If you want to rely on env keys (e.g. exported in your `~/.profile`), run local
- Test: `extensions/music-generation-providers.live.test.ts`
- Enable: `OPENCLAW_LIVE_TEST=1 pnpm test:live -- extensions/music-generation-providers.live.test.ts`
- Harness: `pnpm test:live:media music`
- Scope:
- Exercises the shared bundled music-generation provider path
- Currently covers Google and MiniMax
@@ -501,6 +503,7 @@ If you want to rely on env keys (e.g. exported in your `~/.profile`), run local
- Test: `extensions/video-generation-providers.live.test.ts`
- Enable: `OPENCLAW_LIVE_TEST=1 pnpm test:live -- extensions/video-generation-providers.live.test.ts`
- Harness: `pnpm test:live:media video`
- Scope:
- Exercises the shared bundled video-generation provider path
- Loads provider env vars from your login shell (`~/.profile`) before probing
@@ -508,20 +511,36 @@ If you want to rely on env keys (e.g. exported in your `~/.profile`), run local
- Skips providers with no usable auth/profile/model
- Runs both declared runtime modes when available:
- `generate` with prompt-only input
- `imageToVideo` when the provider declares `capabilities.imageToVideo.enabled`
- `imageToVideo` when the provider declares `capabilities.imageToVideo.enabled` and the selected provider/model accepts buffer-backed local image input in the shared sweep
- `videoToVideo` when the provider declares `capabilities.videoToVideo.enabled` and the selected provider/model accepts buffer-backed local video input in the shared sweep
- Current declared-but-skipped `imageToVideo` providers in the shared sweep:
- `vydra` because bundled `veo3` is text-only and bundled `kling` requires a remote image URL
- Current `videoToVideo` live coverage:
- `google`
- `openai`
- `runway` only when the selected model is `runway/gen4_aleph`
- Current declared-but-skipped `videoToVideo` providers in the shared sweep:
- `alibaba`, `qwen`, `xai` because those paths currently require remote `http(s)` / MP4 reference URLs
- `google` because the current shared Gemini/Veo lane uses local buffer-backed input and that path is not accepted in the shared sweep
- `openai` because the current shared lane lacks org-specific video inpaint/remix access guarantees
- Optional narrowing:
- `OPENCLAW_LIVE_VIDEO_GENERATION_PROVIDERS="google,openai,runway"`
- `OPENCLAW_LIVE_VIDEO_GENERATION_MODELS="google/veo-3.1-fast-generate-preview,openai/sora-2,runway/gen4_aleph"`
- Optional auth behavior:
- `OPENCLAW_LIVE_REQUIRE_PROFILE_KEYS=1` to force profile-store auth and ignore env-only overrides
## Media live harness
- Command: `pnpm test:live:media`
- Purpose:
- Runs the shared image, music, and video live suites through one repo-native entrypoint
- Auto-loads missing provider env vars from `~/.profile`
- Auto-narrows each suite to providers that currently have usable auth by default
- Reuses `scripts/test-live.mjs`, so heartbeat and quiet-mode behavior stay consistent
- Examples:
- `pnpm test:live:media`
- `pnpm test:live:media image video --providers openai,google,minimax`
- `pnpm test:live:media video --video-providers openai,runway --all-providers`
- `pnpm test:live:media music --quiet`
## Docker runners (optional "works in Linux" checks)
These Docker runners split into two buckets:

6
docs/tools/music-generation.md
View File

@@ -248,6 +248,12 @@ Opt-in live coverage for the shared bundled providers:
OPENCLAW_LIVE_TEST=1 pnpm test:live -- extensions/music-generation-providers.live.test.ts
```
Repo wrapper:
```bash
pnpm test:live:media music
```
This live file loads missing provider env vars from `~/.profile`, prefers
live/env API keys ahead of stored auth profiles by default, and runs both
`generate` and declared `edit` coverage when the provider enables edit mode.

38
docs/tools/video-generation.md
View File

@@ -103,20 +103,20 @@ runtime modes at runtime.
This is the explicit mode contract used by `video_generate`, contract tests,
and the shared live sweep.
| Provider | `generate` | `imageToVideo` | `videoToVideo` | Shared live lanes today |
| -------- | ---------- | -------------- | -------------- | ---------------------------------------------------------------------------------------------------------- |
| Alibaba | Yes | Yes | Yes | `generate`, `imageToVideo`; `videoToVideo` skipped because this provider needs remote `http(s)` video URLs |
| BytePlus | Yes | Yes | No | `generate`, `imageToVideo` |
| ComfyUI | Yes | Yes | No | Not in the shared sweep; workflow-specific coverage lives with Comfy tests |
| fal | Yes | Yes | No | `generate`, `imageToVideo` |
| Google | Yes | Yes | Yes | `generate`, `imageToVideo`, `videoToVideo` |
| MiniMax | Yes | Yes | No | `generate`, `imageToVideo` |
| OpenAI | Yes | Yes | Yes | `generate`, `imageToVideo`, `videoToVideo` |
| Qwen | Yes | Yes | Yes | `generate`, `imageToVideo`; `videoToVideo` skipped because this provider needs remote `http(s)` video URLs |
| Runway | Yes | Yes | Yes | `generate`, `imageToVideo`; `videoToVideo` runs only when the selected model is `runway/gen4_aleph` |
| Together | Yes | Yes | No | `generate`, `imageToVideo` |
| Vydra | Yes | Yes | No | `generate`, `imageToVideo` |
| xAI | Yes | Yes | Yes | `generate`, `imageToVideo`; `videoToVideo` skipped because this provider currently needs a remote MP4 URL |
| Provider | `generate` | `imageToVideo` | `videoToVideo` | Shared live lanes today |
| -------- | ---------- | -------------- | -------------- | ---------------------------------------------------------------------------------------------------------------------------------------- |
| Alibaba | Yes | Yes | Yes | `generate`, `imageToVideo`; `videoToVideo` skipped because this provider needs remote `http(s)` video URLs |
| BytePlus | Yes | Yes | No | `generate`, `imageToVideo` |
| ComfyUI | Yes | Yes | No | Not in the shared sweep; workflow-specific coverage lives with Comfy tests |
| fal | Yes | Yes | No | `generate`, `imageToVideo` |
| Google | Yes | Yes | Yes | `generate`, `imageToVideo`; shared `videoToVideo` skipped because the current buffer-backed Gemini/Veo sweep does not accept that input |
| MiniMax | Yes | Yes | No | `generate`, `imageToVideo` |
| OpenAI | Yes | Yes | Yes | `generate`, `imageToVideo`; shared `videoToVideo` skipped because this org/input path currently needs provider-side inpaint/remix access |
| Qwen | Yes | Yes | Yes | `generate`, `imageToVideo`; `videoToVideo` skipped because this provider needs remote `http(s)` video URLs |
| Runway | Yes | Yes | Yes | `generate`, `imageToVideo`; `videoToVideo` runs only when the selected model is `runway/gen4_aleph` |
| Together | Yes | Yes | No | `generate`, `imageToVideo` |
| Vydra | Yes | Yes | No | `generate`; shared `imageToVideo` skipped because bundled `veo3` is text-only and bundled `kling` requires a remote image URL |
| xAI | Yes | Yes | Yes | `generate`, `imageToVideo`; `videoToVideo` skipped because this provider currently needs a remote MP4 URL |
## Tool parameters
@@ -140,7 +140,7 @@ and the shared live sweep.
| Parameter | Type | Description |
| ----------------- | ------- | ------------------------------------------------------------------------ |
| `aspectRatio` | string | `1:1`, `2:3`, `3:2`, `3:4`, `4:3`, `4:5`, `5:4`, `9:16`, `16:9`, `21:9` |
| `resolution` | string | `480P`, `720P`, or `1080P` |
| `resolution` | string | `480P`, `720P`, `768P`, or `1080P` |
| `durationSeconds` | number | Target duration in seconds (rounded to nearest provider-supported value) |
| `size` | string | Size hint when the provider supports it |
| `audio` | boolean | Enable generated audio when supported |
@@ -254,6 +254,12 @@ Opt-in live coverage for the shared bundled providers:
OPENCLAW_LIVE_TEST=1 pnpm test:live -- extensions/video-generation-providers.live.test.ts
```
Repo wrapper:
```bash
pnpm test:live:media video
```
This live file loads missing provider env vars from `~/.profile`, prefers
live/env API keys ahead of stored auth profiles by default, and runs the
declared modes it can exercise safely with local media:
@@ -265,8 +271,6 @@ declared modes it can exercise safely with local media:
Today the shared `videoToVideo` live lane covers:
- `google`
- `openai`
- `runway` only when you select `runway/gen4_aleph`
## Configuration