DocsGPT/tests/QA_TESTS.md

Human QA Checklist (~10 min)

Four compound flows. Each chains 5–10 breakable features into one tester journey — if anything regressed, it surfaces without a dedicated test. Run in order; state carries between flows.

Prereqs

• Backend (Flask 7091 + Celery) and frontend dev server running
• LLM + embeddings keys in .env
• Fresh browser profile / incognito (no stale authToken)
• Ready: a small PDF, a public URL, an agent-tool credential (MCP endpoint or Brave/search key)

---

Flow A — First-run journey: auth → settings → source → chat → share (~3 min)

Covers: session bootstrap, settings persistence + i18n, local + URL ingestion, streaming chat, abort, edit/retry, citations, feedback, attachments, markdown rendering, share link.

• Load app fresh → localStorage.authToken set (or JWT modal accepts pasted token); no console errors
• Settings → flip theme, change language (EN→JP→EN), bump chunk count, pick default model → reload → all four persist
• Sources → upload local PDF and ingest a URL in parallel; both finish with token counts
• New chat → ask a question the PDF answers → tokens stream → hit Stop mid-stream → cleanly aborts
• Edit the question, retry → new answer renders below; thumbs up it
• Open Sources popup on the answer → snippets map to the uploaded doc
• Attach a file inline in a follow-up → referenced in the response
• Ask for a reply with a code block, a table, and a mermaid diagram → all render
• Share conversation → open link in incognito → read-only view loads with full history
• Back in app, hard-refresh → conversation, thumbs, settings all rehydrate

Flow B — Agent lifecycle: build → tool-use → publish → organize (~3 min)

Covers: agent create/edit/draft/publish, system prompt, model swap, source + tool attach, tool approval + denial, MCP config, pin, folders, public share, delete.

• Tools tab → add MCP server using https://docs.mcp.cloudflare.com/mcp → connects / saves; toggle it on
• Create classic agent with custom system prompt, model, the source from Flow A, and that tool → Save draft
• Reload → draft still there → publish it
• Chat with agent → ask "how do I use AI workers on Cloudflare?" → MCP tool approval prompt appears → approve, tool runs, answer cites docs content
• Ask again → this time deny approval → agent handles gracefully (no crash, no stuck spinner)
• Edit agent: swap model, add a second source, edit tool with blank secret field → save → existing credentials preserved, new config applied
• Pin the agent → appears in sidebar; create a folder, move it in, rename folder
• Open agent's public share URL in incognito → works unauthenticated
• Try a Research agent type on a broad question → research-progress steps render
• Delete the agent → gone from sidebar, folder, and pins

Flow C — Admin, regression & cascade (~2 min)

Covers: analytics + filters, agent-scoped analytics, logs, delete cascades, conversation bulk delete, session rehydrate, responsive layout, chunk editing, source deletion.

• Analytics → messages / tokens / feedback charts load; toggle 1h / 7d / 30d — each returns sane data
• Agent-scoped analytics (on remaining agent) — numbers are a subset of global
• Logs tab → recent chats + tool calls listed; expand a row
• Open a source → edit one chunk, delete one chunk → persists
• Start a streaming chat, then delete the conversation mid-stream → no ghost spinner, sidebar updates
• Delete the original source → any chat still open degrades gracefully (no white screen)
• Settings → Delete all conversations → sidebar empties
• Sign out / clear token → sign back in → theme, language, pins, prompts, tools all rehydrate
• Resize to ~400px wide → sidebar collapses, chat layout doesn't break

Flow D — API + connector spot-check (~1 min, skip items without creds)

Covers: agent API key, v1 streaming + non-streaming, inbound webhook, GitHub/Drive connectors, manual sync.

Set once:

export API="http://localhost:7091"
export KEY="a197be6b-969d-44c0-ba4d-af2f972a03df"

• Non-streaming — expect a JSON body with choices[0].message.content:

  curl -sS "$API/v1/chat/completions" \
    -H "Authorization: Bearer $KEY" \
    -H "Content-Type: application/json" \
    -d '{"messages":[{"role":"user","content":"how do I use AI workers"}],"stream":false}'
  

• Streaming — expect data: {...} SSE chunks ending in data: [DONE]:

  curl -N -sS "$API/v1/chat/completions" \
    -H "Authorization: Bearer $KEY" \
    -H "Content-Type: application/json" \
    -d '{"messages":[{"role":"user","content":"What tools do you have?"}],"stream":true}'
  

• Models list — expect the agent's model:

  curl -sS "$API/v1/models" -H "Authorization: Bearer $KEY"
  

• Inbound webhook — copy the webhook URL from the agent page, then fire it; confirm the run appears in Logs:

  curl -sS -X POST "<webhook_url>" \
    -H "Content-Type: application/json" \
    -d '{"message":"What tools do you have?"}'
  

• GitHub or Drive connector → auth → pick item → ingest → manual Sync → status updates; change sync frequency and reload

---