n8n-install/CLAUDE.md

# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project Overview

This is the n8n-installer: a Docker Compose-based installer that creates a comprehensive self-hosted AI automation environment. It includes n8n, Flowise, Open WebUI, Supabase, Qdrant, Langfuse, Ollama, and 20+ other services that can be selectively deployed.

## Essential Commands

### Installation and Updates
```bash
# Main installation (from project root)
sudo bash ./scripts/install.sh

# Update all services to latest versions
sudo bash ./scripts/update.sh

# Clean up unused Docker resources
sudo bash ./scripts/docker_cleanup.sh
```

### Docker Operations
```bash
# View running services
docker compose ps

# View logs for specific service
docker compose logs [service-name]

# Restart services with specific profiles
docker compose --profile n8n,flowise up -d

# Stop all services
docker compose down
```

### Python Helper Script
```bash
# Start services with automatic profile detection
python3 start_services.py
```

### Service Management
```bash
# Apply configuration updates to running services
sudo bash ./scripts/apply_update.sh

# View all available Docker Compose profiles
grep -E '^\s*profiles:' docker-compose.yml

# Check service health status
docker compose ps --format "table {{.Name}}\t{{.Status}}\t{{.Ports}}"
```

## Architecture

### Core Installation Flow
The installation follows a strict 6-step sequence managed by `scripts/install.sh`:
1. **01_system_preparation.sh** - Updates system, installs dependencies, configures security
2. **02_install_docker.sh** - Installs Docker Engine and Docker Compose
3. **03_generate_secrets.sh** - Creates .env file with secure passwords and keys
4. **04_wizard.sh** - Interactive service selection with whiptail UI
5. **05_run_services.sh** - Deploys selected services using Docker Compose profiles
6. **06_final_report.sh** - Displays access URLs and credentials

### Docker Compose Architecture
- **Profiles**: Services are organized into profiles (n8n, flowise, monitoring, langfuse, ollama, etc.)
- **Environment**: All configuration through `.env` file generated in step 3
- **Volumes**: Named volumes for data persistence across container rebuilds
- **Networks**: All services on default Docker network with internal service discovery

### Key Service Dependencies
- **n8n**: Requires postgres, redis. Runs in queue mode with configurable worker count
- **Supabase**: Full stack with postgres, auth, storage, analytics, edge functions
- **Langfuse**: Requires postgres, redis, clickhouse, minio for LLM observability
- **Open WebUI**: Can integrate with Ollama for local LLMs

## Important Implementation Details

### Environment Variable System
- Generated by `03_generate_secrets.sh` using `openssl rand -hex`
- Contains service hostnames, passwords, API keys
- Used by both Docker Compose and Caddy for routing

### Service Selection Wizard
- `04_wizard.sh` uses whiptail to create interactive checklist
- Updates `COMPOSE_PROFILES` in `.env` based on user selections
- Some services have dependencies (e.g., langfuse requires clickhouse)

### Shared File Access
- `./shared/` directory is mounted to `/data/shared` inside n8n containers
- Use this path in n8n workflows to read/write files on the host system

### Custom n8n Configuration
- Pre-installs Node.js libraries: cheerio, axios, moment, lodash
- Runs in production mode with PostgreSQL backend
- Enables queue mode for parallel workflow processing with configurable worker count
- Community packages and AI runners enabled
- Configured for tool usage and external function calls
- Metrics enabled for monitoring integration

### Network Architecture
- Caddy handles HTTPS/TLS termination and reverse proxy
- Services exposed via subdomains: n8n.domain.com, flowise.domain.com, grafana.domain.com, etc.
- Internal services (redis, postgres) not exposed externally
- Supports Cloudflare Tunnel as alternative to port exposure

## Common Development Tasks

### Testing Profile Changes
1. Update `COMPOSE_PROFILES` in `.env`
2. Run `docker compose --profile [profiles] up -d`
3. Check `docker compose ps` to verify services started

### Adding New Services
1. Define service in `docker-compose.yml` with appropriate profile
2. Add hostname variables to Caddy environment section
3. Update `04_wizard.sh` to include in service selection
4. Add Caddyfile routing if service needs web access
5. Consider service dependencies and health checks

### Backup/Restore Workflows
- n8n workflows stored in `./n8n/backup/workflows/`
- Credentials stored in `./n8n/backup/credentials/`
- Import script: `./n8n/n8n_import_script.sh`

### Troubleshooting Service Issues
1. Check service logs: `docker compose logs [service-name]`
2. Verify environment variables in `.env`
3. Check DNS configuration for domain-based services
4. Monitor resource usage: `docker stats`

## Security Considerations

- All services require authentication (configured during setup)
- Firewall configured to only allow SSH, HTTP, HTTPS
- Fail2Ban enabled for brute-force protection
- SSL certificates managed by Caddy with Let's Encrypt
- Sensitive data in `.env` file (never commit to git)

## File Structure Notes
- `memory-bank/` - Project documentation and development notes
- `flowise/` - Flowise workflow templates and custom tools
- `n8n-tool-workflows/` - n8n workflow tools for integration
- `grafana/` - Monitoring dashboards and provisioning configs
- `scripts/` - Installation and utility scripts (all bash)
- `prometheus/` - Prometheus monitoring configuration
- `searxng/` - SearXNG search engine settings
- `caddy-addon/` - Additional Caddy configuration files
- `python-runner/` - Optional Python execution environment

## Critical Implementation Notes

- Never modify the installation script sequence (01-06) without understanding dependencies
- Always use logging functions from `utils.sh` in new scripts
- The `.env` file contains all secrets and must never be committed
- Services use Docker health checks - respect dependency conditions
- Profile-based deployment allows selective service activation
- All bash scripts should be executable and follow the established patterns
- When modifying `docker-compose.yml`, maintain the x-templates pattern for service definitions
- Community workflows are imported during installation - see `n8n/backup/workflows/` for examples