n8n-install/ai-docs/basic-structure.md

# Project Structure: n8n Ecosystem Installer

This document outlines the basic structure of the n8n Ecosystem Installer project, focusing on key directories and files and their purposes.

## 1. Overview

The project is designed to deploy a comprehensive development environment centered around n8n, using Docker Compose for service orchestration. It includes n8n, Flowise, Open WebUI, Supabase, Qdrant, Langfuse, SearXNG, Prometheus, Grafana, Crawl4ai, and Caddy for reverse proxying and SSL.

## 2. Root Directory Key Files

- **`README.md`**: Provides a high-level overview of the project, its components, installation instructions, and usage guidelines.
- **`docker-compose.yml`**: The primary Docker Compose file. It defines most of the services (n8n, Flowise, Open WebUI, Qdrant, Langfuse, Caddy, SearXNG, monitoring tools, Crawl4ai, etc.), their configurations, volumes, and networks. Note that the full Supabase stack is managed by a separate Docker Compose file (see Section 4.8).
- **`.env` (Generated)**: This critical file is generated by the installation script (`scripts/03_generate_secrets.sh`). It stores all necessary environment variables, including API keys, domain names, user credentials, and other configuration parameters for the various services. **It should not be committed to version control.**
- **`Caddyfile`**: Configuration file for the Caddy web server. It defines how incoming traffic is routed to the appropriate backend services (n8n, Flowise, Supabase, etc.) and handles automatic HTTPS/SSL certificate management via Let's Encrypt.
- **`start_services.py`**: A Python script that offers an alternative method for starting services. It notably handles the cloning of the Supabase Docker configuration from its official repository into a local `supabase/docker/` directory, prepares its `.env` file, and starts Supabase services before the main stack. It also includes logic for SearXNG secret key generation and initial setup.

## 3. Core Directories

### 3.1. `scripts/`

This directory contains shell scripts essential for the installation, configuration, and maintenance of the ecosystem.

- **`install.sh`**: The main entry point for setting up the entire environment. It orchestrates the execution of other scripts in a specific sequence.
- **`01_system_preparation.sh`**: Prepares the host system (e.g., updates, firewall configuration).
- **`02_install_docker.sh`**: Installs Docker and Docker Compose if not already present.
- **`03_generate_secrets.sh`**: Prompts the user for necessary information (domain, email, API keys) and generates the `.env` file with all required secrets and configurations.
- **`04_run_services.sh`**: Starts all the services defined in `docker-compose.yml` using `docker compose up`.
- **`05_final_report.sh`**: Displays a summary of access URLs and credentials for the deployed services after successful installation.
- **`update.sh`**: Script to pull the latest changes from the Git repository and update the Docker images for all services.
- **`utils.sh`**: Contains helper functions used by the other shell scripts.

### 3.2. `n8n/`

Configuration and data related to the n8n service.

- **`backup/`**: This subdirectory is used to store n8n workflow JSON files. Workflows placed here can be automatically imported into the n8n instance during the initial setup or via the `n8n-import` service defined in `docker-compose.yml`.
- **`n8n_import_script.sh`**: A shell script executed by the `n8n-import` Docker service to process and import workflows from the `backup/` directory into n8n.

### 3.3. `shared/`

This directory (created if it doesn't exist) is intended to be mounted as a shared volume into n8n containers (typically at `/data/shared` inside the container). Its purpose is to facilitate file exchange between the host machine and n8n workflows (e.g., reading local files, writing outputs).

### 3.4. `searxng/`

Configuration for the SearXNG private metasearch engine.

- **`settings-base.yml`**: A base template for SearXNG's `settings.yml` configuration file. The `start_services.py` script or a similar setup step copies this to `settings.yml` and populates it, for instance, with a generated secret key. The `docker-compose.yml` mounts this directory into the SearXNG container.

### 3.5. `prometheus/`

Configuration for the Prometheus monitoring system.

- **`prometheus.yml`**: The main configuration file for Prometheus, defining scrape targets and rules. This file is mounted directly into the Prometheus container.

### 3.6. `grafana/`

Configuration and provisioning files for the Grafana observability platform.

- **`dashboards/`**: Contains JSON files defining pre-configured Grafana dashboards (e.g., `1860_rev39.json`).
- **`provisioning/`**:
  - `datasources/main.yml`: Defines data sources to be automatically provisioned in Grafana (e.g., connecting to the Prometheus service).
  - `dashboards/` (subdirectory): Can contain YAML files to provision dashboards from the `grafana/dashboards` directory.

### 3.7. `flowise/` (Project Root Directory)

This directory in the project root contains example/template files for Flowise.

- **`*.json` files**: These are typically exported Flowise chatflow definitions (e.g., `Web Search + n8n Agent Chatflow.json`) and custom tool configurations.
- **Note**: The actual runtime data and configuration for the Flowise service (defined in `docker-compose.yml`) are stored in a volume mapped from `~/.flowise` on the host machine to `/root/.flowise` inside the container. The files in this project directory serve as starting points or examples.

### 3.8. `supabase/` (Dynamically Created)

This directory is **not** present in the initial project checkout. It is created by the `start_services.py` script (or a similar process within the main installer) by cloning parts of the official Supabase GitHub repository.

- **`docker/`**: Contains Supabase's own Docker setup.
  - `docker-compose.yml`: A Docker Compose file specific to Supabase services (Postgres, Kong, GoTrue, etc.).
  - `.env`: Supabase-specific environment variables, typically populated by copying relevant values from the main project's `.env` file.

The Supabase stack is started and managed somewhat independently but is integrated into the overall project (e.g., accessible via Caddy and its database can be used by other services like n8n).

## 4. Service Data and Volumes

Most services persist their data in Docker named volumes, which are defined at the beginning of the main `docker-compose.yml` file (e.g., `n8n_storage`, `qdrant_storage`, `langfuse_postgres_data`, etc.). This ensures data is retained across container restarts.