docs: expand add-new-service guide with comprehensive patterns

add multi-service profiles, gpu variants, and yaml anchors document caddyfile patterns for conditional auth and special protocols expand secret generation docs with SERVICES_NEEDING_HASH importance add configure_services.sh, quick start, and final report sections include changelog and update scripts documentation enhance checklist with logging, multi-container, and first-run items
2026-03-07 22:33:11 +00:00 · 2025-12-21 16:01:03 -07:00
parent f4319321a3
commit 5d9d592267
1 changed files with 271 additions and 8 deletions
--- a/.cursor/rules/add-new-service.mdc
+++ b/.cursor/rules/add-new-service.mdc
@@ -20,9 +20,14 @@ Minimal example:
 ```yaml
  myservice:
    image: yourorg/myservice:latest
-    container_name: myservice
+    container_name: myservice              # Required - used in scripts and Caddyfile
    profiles: ["myservice"]
    restart: unless-stopped
+    logging:                               # Recommended - prevents log bloat
+      driver: "json-file"
+      options:
+        max-size: "1m"
+        max-file: "1"
    # command: ...
    # healthcheck: { test: ["CMD-SHELL", "curl -fsS http://localhost:8080/health || exit 1"], interval: 30s, timeout: 10s, retries: 5 }
 ```
@@ -93,11 +98,90 @@ Common dependencies:
 - `minio` - S3-compatible object storage
 - `clickhouse` - Analytics database (for Langfuse)

+## 1.8) Multi-service Profiles
+
+Some services consist of multiple containers that should be enabled together. Use the same profile for all related containers:
+
+```yaml
+  myservice-worker:
+    image: yourorg/myservice-worker:latest
+    container_name: myservice-worker
+    profiles: ["myservice"]  # Same profile as main service
+    # ...
+
+  myservice-web:
+    image: yourorg/myservice-web:latest
+    container_name: myservice-web
+    profiles: ["myservice"]  # Same profile as main service
+    # ...
+```
+
+For shared configuration, use YAML anchors:
+```yaml
+# At the top of docker-compose.yml (x- prefix = extension, ignored by Docker)
+x-myservice-common: &myservice-common
+  depends_on:
+    postgres:
+      condition: service_healthy
+    redis:
+      condition: service_healthy
+
+# In service definitions:
+  myservice-worker:
+    <<: *myservice-common
+    # ... rest of config
+
+  myservice-web:
+    <<: *myservice-common
+    # ... rest of config
+```
+
+Examples in this project:
+- `langfuse` → langfuse-worker + langfuse-web + clickhouse + minio
+- `ragflow` → ragflow + ragflow-mysql + ragflow-redis + ragflow-minio + ragflow-elasticsearch
+- `monitoring` → prometheus + grafana + cadvisor + node-exporter
+
+## 1.9) Hardware/GPU Profiles
+
+For services with multiple hardware variants (CPU, NVIDIA GPU, AMD GPU), use mutually exclusive profiles:
+
+```yaml
+  # CPU variant
+  myservice-cpu:
+    <<: *service-myservice
+    profiles: ["cpu"]
+
+  # NVIDIA GPU variant
+  myservice-gpu:
+    <<: *service-myservice
+    profiles: ["gpu-nvidia"]
+    deploy:
+      resources:
+        reservations:
+          devices:
+            - driver: nvidia
+              count: 1
+              capabilities: [gpu]
+
+  # AMD GPU variant
+  myservice-gpu-amd:
+    <<: *service-myservice
+    image: yourorg/myservice:rocm
+    profiles: ["gpu-amd"]
+    devices:
+      - "/dev/kfd"
+      - "/dev/dri"
+```
+
+The wizard (`scripts/04_wizard.sh`) handles mutual exclusion - only one hardware profile can be active.
+
+Example in this project: Ollama (ollama-cpu, ollama-gpu, ollama-gpu-amd)
+
 ## 2) Caddyfile
 - Add a site block for the service hostname if it should be reachable externally:
 - Ask users whether the service needs Basic Auth via Caddy; if yes, add `basic_auth` with env-based credentials.

-Example:
+### Basic Example
 ```caddyfile
 {$MYSERVICE_HOSTNAME} {
    # Optional. Ask the user if we should protect this endpoint via Basic Auth
@@ -108,6 +192,39 @@ Example:
 }
 ```

+### Conditional Basic Auth (allow internal networks without auth)
+```caddyfile
+{$MYSERVICE_HOSTNAME} {
+    @protected not remote_ip 127.0.0.0/8 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 100.64.0.0/10
+
+    basic_auth @protected {
+        {$MYSERVICE_USERNAME} {$MYSERVICE_PASSWORD_HASH}
+    }
+    reverse_proxy myservice:8080
+}
+```
+
+### Special Protocols (e.g., Neo4j Bolt on non-standard port)
+```caddyfile
+# Neo4j Bolt Protocol on port 7687
+https://{$NEO4J_HOSTNAME}:7687 {
+    reverse_proxy neo4j:7687
+}
+```
+Note: Non-standard ports must be exposed in Caddy's `ports:` section in docker-compose.yml.
+
+### Static File Serving (e.g., Welcome Page)
+```caddyfile
+{$WELCOME_HOSTNAME} {
+    basic_auth {
+        {$WELCOME_USERNAME} {$WELCOME_PASSWORD_HASH}
+    }
+    root * /srv/welcome
+    file_server
+    try_files {path} /index.html  # SPA fallback
+}
+```
+
 Notes:
 - Keep using env placeholders (e.g., `{$MYSERVICE_HOSTNAME}`), supplied by the `caddy` service environment in `docker-compose.yml`.

@@ -171,21 +288,74 @@ EMAIL_VARS=(
 )
 ```

-3. If service needs Caddy basic auth, add to `SERVICES_NEEDING_HASH` array (~line 520):
+### SERVICES_NEEDING_HASH (Critical for Basic Auth)
+
+**⚠️ IMPORTANT:** If your service uses Caddy basic auth, you MUST add it to this array. Missing this step will cause authentication to fail!
+
+Add to `SERVICES_NEEDING_HASH` array (~line 520):
 ```bash
 SERVICES_NEEDING_HASH=("PROMETHEUS" "SEARXNG" ... "MYSERVICE")
 ```
+
 This automatically generates bcrypt hash using: `docker exec caddy caddy hash-password`

+The script will:
+1. Read `MYSERVICE_PASSWORD` from `.env`
+2. Generate bcrypt hash via Caddy container
+3. Write hash to `MYSERVICE_PASSWORD_HASH` in `.env`
+
 **Note:** Hash generation requires the Caddy container to be running.

 ## 5) scripts/04_wizard.sh
- Add the service to the selectable profiles list so users can opt-in during installation:
+
+Add the service to the selectable profiles list so users can opt-in during installation:
 ```bash
-# base_services_data+=
+# In base_services_data array:
 "myservice" "MyService (Short description)"
 ```

+### Special Wizard Patterns
+
+**Services requiring additional prompts** (like GOST asking for upstream proxy URL):
+- Add custom whiptail dialog in the wizard
+- See GOST implementation (~lines 206-242) as example
+
+**Mutually exclusive services** (like Dify and Supabase):
+- Add logic to remove conflicting service from selection
+- See Dify/Supabase exclusion (~lines 139-156) as example
+
+For complex service-specific configuration, see section 5.5.
+
+## 5.5) scripts/05_configure_services.sh
+
+For services that need configuration beyond what the wizard can handle, add logic to this script.
+
+Use cases:
+- **Token-based authentication** (e.g., Cloudflare Tunnel token)
+- **Mutual exclusion logic** (e.g., Supabase vs Dify - both use same ports)
+- **Dynamic configuration files** (e.g., generating config from template)
+
+Example pattern:
+```bash
+# Configure MyService if active
+if is_profile_active "myservice"; then
+    log_info "Configuring MyService..."
+
+    # Example: Generate config file from template
+    envsubst < ./myservice/config.template.yml > ./myservice/config.yml
+
+    # Example: Handle mutual exclusion
+    if is_profile_active "conflicting-service"; then
+        log_warn "MyService and ConflictingService cannot run together"
+        # Remove conflicting profile from COMPOSE_PROFILES
+    fi
+fi
+```
+
+See existing implementations:
+- Cloudflare Tunnel token handling (~lines 132-153)
+- Supabase/Dify mutual exclusion (~lines 159-173)
+
 ## 6) scripts/generate_welcome_page.sh

 This script is called automatically from `scripts/07_final_report.sh` during installation.
@@ -252,6 +422,38 @@ fi

 **Important:** Always use `json_escape "$VAR"` to prevent JSON breaking from special characters.

+### Quick Start Steps (QUICK_START_ARRAY)
+
+If your service requires first-run setup (e.g., initial configuration, API key setup), add a Quick Start step:
+
+```bash
+# MyService Quick Start
+if is_profile_active "myservice"; then
+    QUICK_START_ARRAY+=("    \"myservice\": {
+      \"title\": \"Configure MyService\",
+      \"description\": \"Complete initial setup\",
+      \"action\": \"Visit https://\${MYSERVICE_HOSTNAME} to configure\"
+    }")
+fi
+```
+
+This appears in the Welcome Page's "Quick Start" section to guide users through post-installation steps.
+
+## 6.5) scripts/07_final_report.sh
+
+For services with first-run instructions that should appear in the terminal after installation:
+
+```bash
+if is_profile_active "myservice"; then
+    echo -e "     ${GREEN}*${NC} ${WHITE}MyService${NC}: Visit dashboard to complete initial setup"
+fi
+```
+
+This provides immediate guidance in the terminal after installation completes. Use for:
+- First-time configuration steps
+- Important security notices
+- Links to documentation
+
 ## 7) welcome/app.js

 Add service metadata to the `SERVICE_METADATA` object (located around line 145):
@@ -296,6 +498,23 @@ color: 'bg-lime-500'      // Another named color
    - **MyService:** `myservice.yourdomain.com` (Brief description)
 ```

+## 8.5) CHANGELOG.md
+
+Add an entry when introducing a new service:
+
+```markdown
+## [Unreleased]
+
+### Added
+- **MyService** - Brief description of what the service provides
+```
+
+Follow [Keep a Changelog](https://keepachangelog.com/) format:
+- `Added` - for new features
+- `Changed` - for changes in existing functionality
+- `Fixed` - for bug fixes
+- `Removed` - for removed features
+
 ## 9) Ask about Basic Auth (important)
 When adding any new public-facing service, explicitly ask the user whether they want to protect the service with Basic Auth via Caddy. If yes, add:
 - Credentials section to `.env.example`
@@ -320,20 +539,64 @@ docker compose -p localai logs -f --tail=200 myservice | cat
 docker compose -p localai logs -f --tail=200 caddy | cat
 ```

+## 10.5) scripts/update_preview.sh (optional)
+
+If your service uses a Docker image that should be tracked for updates, add version checking logic:
+
+```bash
+# Check MyService version
+if is_profile_active "myservice"; then
+    check_image_update "yourorg/myservice" "MYSERVICE"
+fi
+```
+
+This allows `make update` to show available updates for your service.
+
+## 10.6) scripts/apply_update.sh (for complex services)
+
+For services with their own docker-compose files (like Supabase, Dify), add handling logic:
+
+```bash
+# Update MyService
+if is_profile_active "myservice"; then
+    log_info "Updating MyService..."
+    # Pull latest compose file or run service-specific update
+    docker compose -f docker-compose.myservice.yml pull
+fi
+```
+
+Most services don't need this - only use for services with external compose files or complex update procedures.
+
 ## 11) Quick checklist

 ### Core setup
- [ ] Service added to `docker-compose.yml` with a profile (no external ports exposed)
- [ ] Hostname and (optional) credentials added to `.env.example`
- [ ] Secret + hash generation added to `scripts/03_generate_secrets.sh`
+- [ ] Service added to `docker-compose.yml` with profile (no external ports)
+- [ ] `container_name: servicename` specified (required)
+- [ ] `logging:` driver json-file configured (recommended)
+- [ ] Hostname and credentials added to `.env.example`
+- [ ] Password added to `VARS_TO_GENERATE` in `scripts/03_generate_secrets.sh`
+- [ ] If basic auth: added to `SERVICES_NEEDING_HASH` array
 - [ ] Exposed via `Caddyfile` with `reverse_proxy` (+ `basic_auth` if desired)
 - [ ] Service selectable in `scripts/04_wizard.sh`
 - [ ] Service data added to `scripts/generate_welcome_page.sh`
 - [ ] Service metadata added to `welcome/app.js` (`SERVICE_METADATA`)
 - [ ] One-line description added to `README.md`
+- [ ] Entry added to `CHANGELOG.md`

 ### If service needs outbound proxy (AI API calls)
 - [ ] Added `<<: *proxy-env` to service environment in `docker-compose.yml`
 - [ ] Added service name to `GOST_NO_PROXY` list in `.env.example`
 - [ ] Healthcheck bypasses proxy: `http_proxy= https_proxy= ... wget ...`

+### If service has first-run setup
+- [ ] Added to `QUICK_START_ARRAY` in `scripts/generate_welcome_page.sh`
+- [ ] Added instructions in `scripts/07_final_report.sh`
+
+### If service needs special configuration
+- [ ] Added prompts in `scripts/04_wizard.sh`
+- [ ] Added logic in `scripts/05_configure_services.sh`
+
+### For complex multi-container services
+- [ ] All related containers use same profile
+- [ ] Consider YAML anchors for shared dependencies (see section 1.8)
+