Add initial documentation files for n8n-installer project

- Created multiple foundational documents including activeContext.md, productContext.md, progress.md, projectbrief.md, systemPatterns.md, tasks.md, techContext.md to establish a comprehensive overview of the project.
- Each document outlines key aspects such as project status, user personas, technical stack, architectural patterns, and future development opportunities, ensuring clarity and direction for ongoing and future work.
- The project is now fully initialized and ready for development across various modes.
This commit is contained in:
Yury Kossakovsky
2025-08-06 11:06:01 -06:00
parent 43cedb8082
commit ae79e1b88f
7 changed files with 706 additions and 0 deletions

View File

@@ -0,0 +1,77 @@
# Active Context - n8n-installer Project
## Current Session Status
- **Mode**: VAN (Initialization)
- **Date**: Current session
- **Branch**: develop (up to date with origin)
- **Working Tree**: Clean
## Project State Assessment
- **Memory Bank**: ✅ Newly initialized during VAN mode
- **Core Documentation**: ✅ Created (projectbrief.md, techContext.md, productContext.md, systemPatterns.md)
- **Active Tasks**: None currently defined
- **Repository Status**: Clean working tree, no pending changes
## Current Focus Areas
### Immediate Context
The project is in **VAN mode initialization** - setting up the Memory Bank system for future development work. This is a mature, production-ready n8n-installer project that provides comprehensive Docker Compose templates for self-hosted AI and automation environments.
### Project Readiness Assessment
- **Codebase**: Stable and complete
- **Documentation**: Comprehensive README with installation guides
- **Architecture**: Well-defined microservices pattern with Docker Compose
- **Community**: Active with 300+ workflow templates available
### Available Development Areas
1. **Installation Scripts**: Shell-based automation in `/scripts/` directory
2. **Service Configurations**: Docker Compose and service-specific configs
3. **Workflow Templates**: Community-contributed n8n workflows
4. **Monitoring Setup**: Prometheus/Grafana dashboard configurations
5. **Backup Systems**: Automated backup and restore functionality
## Technical Environment
- **Platform**: macOS (darwin 24.6.0)
- **Shell**: /bin/zsh
- **Working Directory**: /Users/kossakovsky/projects/n8n-installer
- **Project Structure**: Complete with all service configurations and scripts
## Recently Completed
- ✅ Memory Bank directory structure creation
- ✅ Project brief documentation based on README analysis
- ✅ Technical context documentation covering technology stack
- ✅ Product context covering user value and market positioning
- ✅ System patterns documentation covering architectural patterns
## Next Steps Considerations
Based on VAN mode initialization, the project is now ready for:
1. **PLAN mode**: For planning new features or enhancements
2. **CREATIVE mode**: For designing new components or improvements
3. **IMPLEMENT mode**: For Level 1 quick fixes or specific implementations
4. **QA mode**: For validation and testing of components
## Key Project Characteristics
- **Maturity**: Production-ready with active community usage
- **Scope**: Comprehensive AI/automation platform installer
- **Architecture**: Microservices with Docker orchestration
- **Target Users**: AI developers, automation engineers, self-hosters
- **Value Proposition**: Complete data sovereignty with enterprise capabilities
## Memory Bank Files Status
- **projectbrief.md**: ✅ Comprehensive project overview
- **techContext.md**: ✅ Complete technology stack documentation
- **productContext.md**: ✅ User value and market positioning
- **systemPatterns.md**: ✅ Architectural and operational patterns
- **activeContext.md**: ✅ Current file (this document)
- **tasks.md**: ⏳ Ready for task-specific initialization
- **progress.md**: ⏳ Ready for implementation tracking
## Project Health Indicators
- **Repository**: Clean, up-to-date with develop branch
- **Dependencies**: Docker-based, well-managed
- **Community**: Active with ongoing contributions
- **Documentation**: Comprehensive and current
- **Testing**: Installation scripts with validation
- **Monitoring**: Built-in observability patterns
The project is fully initialized and ready for development work in any of the supported modes.

View File

@@ -0,0 +1,123 @@
# n8n-installer Product Context
## Product Vision
Create a comprehensive, self-hosted AI workshop that democratizes access to powerful automation and AI tools, giving users complete control over their data and workflows while maintaining enterprise-grade capabilities.
## Target Users
### Primary Audiences
1. **AI Developers**: Building and testing AI agents, RAG systems, and LLM applications
2. **Automation Engineers**: Creating complex workflow automations for business processes
3. **DevOps Teams**: Setting up self-hosted alternatives to cloud AI services
4. **Small/Medium Businesses**: Implementing AI-powered automation without vendor lock-in
5. **Researchers**: Experimenting with AI workflows and data processing pipelines
6. **Privacy-Conscious Organizations**: Requiring complete data sovereignty
### User Personas
- **The AI Experimenter**: Wants to test different AI models and agents locally
- **The Automation Builder**: Needs to connect multiple services and automate workflows
- **The Self-Hoster**: Prefers running own infrastructure over cloud dependencies
- **The Privacy Advocate**: Requires complete control over data and processing
## Product Value Propositions
### Core Benefits
1. **Comprehensive Toolkit**: 15+ integrated AI and automation tools in one package
2. **Data Sovereignty**: Complete control over data, processing, and storage
3. **Cost Efficiency**: Self-hosted alternative to expensive cloud AI services
4. **Rapid Deployment**: Single command installation with interactive configuration
5. **Community Resources**: 300+ pre-built workflows for immediate productivity
6. **Scalable Architecture**: From single-user setups to multi-worker production systems
### Competitive Advantages
- **Integration Depth**: Pre-configured tool interoperability vs. manual setup
- **Workflow Library**: Extensive community-contributed automation templates
- **Update Automation**: Seamless upgrade path for all components
- **Security Focus**: Built-in HTTPS, firewall configuration, and security enhancements
- **Resource Flexibility**: Configurable resource allocation based on use case
## Product Features
### Installation Experience
- **One-Command Setup**: Single script handles entire deployment
- **Interactive Wizard**: Guided service selection and configuration
- **Automatic Dependencies**: Docker, SSL certificates, networking handled automatically
- **Validation Checks**: Pre-flight verification of requirements and DNS
- **Progress Reporting**: Clear feedback during installation process
### Core Functionality
- **n8n Workflow Engine**: Visual automation builder with 400+ integrations
- **AI Agent Development**: Multiple platforms (Flowise, Letta) for agent creation
- **Vector Storage**: Choice of Qdrant, Supabase, or Weaviate for embeddings
- **Local LLM Hosting**: Ollama integration for private model deployment
- **Web Interface**: Open WebUI for ChatGPT-like interaction
### Monitoring & Operations
- **Performance Dashboards**: Grafana visualizations with n8n-specific metrics
- **AI Observability**: Langfuse for tracking model performance and costs
- **Health Monitoring**: Prometheus metrics for all services
- **Update Management**: Automated update process with rollback capabilities
- **Cleanup Tools**: Docker maintenance and space management utilities
### Developer Experience
- **Pre-installed Libraries**: Common JavaScript libraries available in n8n
- **File System Access**: Shared volumes for workflow data processing
- **Custom Tools**: Example integrations (Slack, Google Docs, Postgres)
- **Backup System**: Automated workflow and credential backup to Google Drive
- **Documentation**: Comprehensive guides and troubleshooting resources
## Market Positioning
### Alternative To
- **Cloud AI Platforms**: OpenAI, Anthropic, Google AI (for privacy-sensitive use cases)
- **Workflow Tools**: Zapier, Microsoft Power Automate (for self-hosted requirements)
- **AI Development Platforms**: LangChain Cloud, Flowise Cloud (for cost control)
- **Vector Databases**: Pinecone, Weaviate Cloud (for data sovereignty)
### Unique Market Position
- **"AI Workshop in a Box"**: Complete self-hosted AI development environment
- **Enterprise Privacy**: Cloud capabilities without cloud dependencies
- **Community-Driven**: Open source with active contribution ecosystem
- **Educational Platform**: Ideal for learning AI development and automation
## Success Metrics
### Adoption Indicators
- GitHub stars and repository forks
- Community forum engagement and support requests
- Workflow template downloads and usage
- Update script execution frequency
### User Satisfaction
- Installation success rate and time-to-first-workflow
- Service uptime and performance metrics
- Community contribution rate (workflows, tools, documentation)
- Issue resolution time and user feedback sentiment
### Technical Performance
- Multi-service deployment success rate
- Resource utilization efficiency
- SSL certificate acquisition success rate
- Service interdependency reliability
## Roadmap Considerations
### Immediate Opportunities
- Enhanced monitoring dashboards for AI-specific metrics
- Additional vector database integrations
- Improved backup and restore functionality
- Performance optimization for resource-constrained environments
### Strategic Directions
- Kubernetes deployment option for enterprise users
- CI/CD integration for automated workflow testing
- Enhanced security features (RBAC, audit logging)
- Multi-tenant support for organizational deployments
- Integration marketplace for community-contributed tools
### Community Growth
- Video tutorial series and educational content
- Template gallery with categorization and search
- Contribution guidelines and developer onboarding
- Partner integrations with AI model providers
- Conference presentations and community meetups

107
memory-bank/progress.md Normal file
View File

@@ -0,0 +1,107 @@
# Progress - n8n-installer Project
## Project Status Overview
- **Current Phase**: Initialized and Ready for Development
- **Overall Health**: ✅ Excellent - Production Ready
- **Last Updated**: Current VAN session
## Implementation Status
### Core Infrastructure: ✅ COMPLETE
- **Docker Compose**: Fully configured multi-service orchestration
- **Installation Scripts**: Complete automation pipeline (01-06 numbered scripts)
- **Service Configurations**: All major AI/automation tools pre-configured
- **Security Setup**: Firewall, HTTPS, and credential management
- **Update Mechanisms**: Automated update and cleanup scripts
### Service Integration: ✅ COMPLETE
- **n8n Core**: Queue mode with worker scaling
- **Database Layer**: PostgreSQL with Redis caching
- **Reverse Proxy**: Caddy with automatic SSL
- **Monitoring Stack**: Prometheus + Grafana dashboards
- **AI Services**: Flowise, Open WebUI, Ollama support
- **Vector Stores**: Qdrant, Supabase, Weaviate options
### Community Resources: ✅ COMPLETE
- **Workflow Library**: 300+ community-contributed templates
- **Documentation**: Comprehensive README and troubleshooting guides
- **Integration Examples**: Custom tools for Slack, Google Docs, Postgres
- **Backup System**: Automated workflow and credential backup
### Quality Assurance: ✅ COMPLETE
- **Installation Testing**: Validated deployment process
- **Service Health Checks**: Automated monitoring and alerts
- **Error Handling**: Graceful failure management
- **Recovery Mechanisms**: Rollback and cleanup capabilities
## Development Metrics
### Code Quality
- **Repository Status**: Clean working tree, up-to-date
- **Architecture**: Well-structured microservices pattern
- **Documentation**: Comprehensive and current
- **Testing**: Installation validation and health checks
### Performance Characteristics
- **Scalability**: Configurable worker count for parallel processing
- **Resource Efficiency**: Minimal setup (4GB RAM) to full deployment (8GB RAM)
- **Monitoring**: Built-in performance tracking and visualization
- **Update Speed**: Automated update process with minimal downtime
### Community Health
- **Active Development**: Regular updates and improvements
- **Community Support**: Forum and GitHub issue tracking
- **Contribution Path**: Clear guidelines for community involvement
- **Template Ecosystem**: Extensive workflow library with ongoing additions
## Recent Accomplishments
### VAN Mode Initialization (Current Session)
- ✅ Memory Bank directory structure established
- ✅ Project analysis completed and documented
- ✅ Core context files created (projectbrief, techContext, productContext, systemPatterns)
- ✅ Active context and progress tracking initialized
- ✅ Task management system prepared
## Current Capabilities
### Deployment Options
- **Single Command Installation**: Fully automated setup process
- **Service Selection**: Interactive wizard for component selection
- **Resource Configuration**: Flexible resource allocation
- **Domain Management**: Automatic SSL and subdomain routing
### AI/Automation Features
- **Workflow Engine**: Visual automation with 400+ integrations
- **AI Agent Development**: Multiple platforms for agent creation
- **Vector Processing**: High-performance embedding storage and retrieval
- **Local LLM Support**: Private model hosting with Ollama
- **Monitoring**: AI-specific performance tracking
### Operations and Maintenance
- **Automated Updates**: Version management with rollback capability
- **Health Monitoring**: Service availability and performance tracking
- **Backup Management**: Automated data protection and recovery
- **Resource Cleanup**: Docker maintenance and space management
## Next Development Opportunities
### Enhancement Potential
- **Advanced Monitoring**: AI workflow performance metrics
- **Security Features**: Enhanced authentication and audit logging
- **User Experience**: Improved installation feedback and error recovery
- **Community Tools**: Workflow validation and quality assurance
### Strategic Directions
- **Enterprise Features**: Multi-tenant support and RBAC
- **Integration Expansion**: Additional AI model providers and services
- **Performance Optimization**: Resource efficiency and scaling improvements
- **Educational Content**: Tutorials and best practice guides
## Success Indicators
- **Installation Success Rate**: High reliability across different environments
- **Community Adoption**: Active usage and contribution patterns
- **Service Reliability**: Minimal downtime and quick recovery
- **User Satisfaction**: Positive feedback and continued usage
The project is in excellent condition and ready for continued development in any area that aligns with user needs and strategic objectives.

View File

@@ -0,0 +1,66 @@
# n8n-installer Project Brief
## Project Overview
The **n8n-installer** is an open-source Docker Compose template designed to significantly simplify setting up a comprehensive, self-hosted environment for n8n and Flowise. It bundles essential supporting tools for AI development, automation, and monitoring.
## Project Purpose
This installer helps users create their own powerful, private AI workshop with capabilities to:
- Automate repetitive tasks
- Build smart assistants tailored to specific needs
- Analyze information and gain insights
- Generate creative content
## Core Architecture
- **Foundation**: Docker Compose orchestration
- **Core Services**: n8n, Caddy, Postgres, Redis (always included)
- **Scalable Design**: n8n runs in queue mode with Redis for task management
- **Configurable Workers**: Dynamic specification of n8n workers for parallel processing
## Available Services Suite
### Always Included
- **n8n**: Low-code platform with 400+ integrations and AI components
- **Caddy**: Web proxy with automatic HTTPS/TLS
- **Postgres**: Database storage
- **Redis**: Caching and task queue management
### Optional Services (Wizard Selection)
- **Supabase**: Open-source Firebase alternative (database, auth, vectors)
- **Open WebUI**: ChatGPT-like interface for AI models and n8n agents
- **Flowise**: No-code/low-code AI agent builder
- **Qdrant**: High-performance vector store for AI
- **SearXNG**: Private metasearch engine
- **Langfuse**: AI agent performance monitoring
- **Crawl4ai**: Flexible web crawler for AI
- **Letta**: Open-source agent server and SDK
- **Weaviate**: AI-native vector database
- **Neo4j**: Graph database
- **Ollama**: Local LLM hosting
- **Prometheus/Grafana**: Monitoring and visualization
## Key Features
- **Rich Toolset**: Curated collection of open-source AI/automation tools
- **Full Control**: Self-hosted with complete data ownership
- **300+ Community Workflows**: Optional import of ready-made automation templates
- **Pre-installed Libraries**: cheerio, axios, moment, lodash for n8n custom JavaScript
- **Managed HTTPS**: Automatic SSL certificate handling via Caddy
- **Comprehensive Monitoring**: Built-in performance tracking capabilities
## Installation Requirements
- **Domain**: Registered domain with wildcard DNS A-record configured
- **Server**: Ubuntu 24.04 LTS, 64-bit minimum
- **Resources**:
- Full setup: 8GB RAM / 4 CPU cores / 60GB disk
- Minimal (n8n + Flowise): 4GB RAM / 2 CPU cores / 30GB disk
## Current Status
- **Repository**: Active development on `develop` branch
- **State**: Clean working tree, up to date with origin
- **Architecture**: Mature, production-ready template system
- **Community**: Active with forum support and contribution tracking
## Technical Foundation
- **Platform**: Docker/Docker Compose
- **Languages**: Shell scripting, Python utilities, JSON configurations
- **Deployment**: Automated installation with interactive wizard
- **Updates**: Automated update mechanism with version management
- **Cleanup**: Built-in Docker maintenance utilities

View File

@@ -0,0 +1,176 @@
# n8n-installer System Patterns
## Architectural Patterns
### Container Orchestration Pattern
- **Docker Compose**: Declarative service definition and management
- **Service Discovery**: Named containers for inter-service communication
- **Network Isolation**: Custom Docker networks for security boundaries
- **Volume Management**: Persistent data storage with named volumes
- **Environment Configuration**: Centralized secrets and settings via .env
### Reverse Proxy Pattern
- **Caddy as Gateway**: Single entry point for all services
- **Automatic SSL**: Let's Encrypt integration for certificate management
- **Subdomain Routing**: Service-specific subdomain mapping
- **Load Balancing**: Built-in support for service scaling
- **Static File Serving**: Efficient asset delivery
### Queue-Based Processing Pattern
- **Redis Queue**: Decoupled task execution in n8n
- **Worker Scaling**: Configurable parallel processing capacity
- **Job Distribution**: Load balancing across multiple workers
- **Persistence**: Task state management and recovery
- **Monitoring**: Queue depth and worker utilization tracking
### Configuration Management Patterns
### Environment-Based Configuration
```bash
# Central configuration in .env file
DOMAIN=yourdomain.com
N8N_WORKERS=2
POSTGRES_PASSWORD=secure_password
OPENAI_API_KEY=optional_key
```
### Service Selection Pattern
- **Interactive Wizard**: Runtime service selection during installation
- **Conditional Deployment**: Docker Compose service activation based on choices
- **Dependency Management**: Automatic inclusion of required supporting services
- **Resource Optimization**: Only deploy selected services to conserve resources
### Security Patterns
### Defense in Depth
1. **Network Level**: Firewall configuration and port management
2. **Application Level**: Service-specific authentication and authorization
3. **Transport Level**: Automatic HTTPS/TLS encryption
4. **Data Level**: Database password security and secret management
### Credential Management
- **Generated Secrets**: Automatic secure password creation
- **Environment Isolation**: Secrets stored in environment variables
- **Service Accounts**: Dedicated credentials for inter-service communication
- **Backup Security**: Encrypted credential storage in backup systems
## Installation and Deployment Patterns
### Progressive Installation Pattern
```bash
# Sequential script execution
01_system_preparation.sh # System updates and security
02_install_docker.sh # Container runtime
03_generate_secrets.sh # Security credentials
04_wizard.sh # Interactive configuration
05_run_services.sh # Service deployment
06_final_report.sh # Success confirmation
```
### Idempotent Operations
- **State Checking**: Verify current system state before modifications
- **Conditional Execution**: Skip already-completed installation steps
- **Error Recovery**: Resume installation from failure points
- **Rollback Capability**: Undo changes if deployment fails
### Update and Maintenance Patterns
### Rolling Update Pattern
1. **Backup Current State**: Preserve existing data and configurations
2. **Fetch Updates**: Pull latest code and Docker images
3. **Service Replacement**: Replace containers with minimal downtime
4. **Health Verification**: Confirm all services operational post-update
5. **Rollback on Failure**: Restore previous state if issues detected
### Cleanup Pattern
- **Resource Identification**: Scan for unused Docker resources
- **Safe Removal**: Delete only genuinely unused containers/images
- **Space Recovery**: Reclaim disk space without affecting running services
- **User Confirmation**: Require explicit approval for destructive operations
## Data Management Patterns
### Shared Storage Pattern
```
/data/shared/ # Host filesystem
/data/shared/ # n8n container access path
```
- **File Exchange**: Common area for workflow file operations
- **Cross-Service Data**: Shared data access across multiple containers
- **Backup Inclusion**: Shared data included in backup processes
### Database Pattern
- **Shared Postgres**: Single database instance for multiple services
- **Schema Isolation**: Service-specific database schemas
- **Connection Pooling**: Efficient database connection management
- **Backup Strategy**: Regular automated database backups
### Vector Storage Pattern
- **Multiple Options**: Qdrant, Supabase, Weaviate for different use cases
- **Embedding Management**: Centralized vector storage and retrieval
- **Search Capabilities**: Semantic search across stored embeddings
- **Scaling Strategy**: Performance optimization for large datasets
## Monitoring and Observability Patterns
### Metrics Collection Pattern
```
Application Metrics → Prometheus → Grafana Dashboards
```
- **Service Metrics**: Individual container performance data
- **System Metrics**: Host resource utilization
- **Custom Metrics**: n8n workflow execution statistics
- **Alert Configuration**: Threshold-based monitoring alerts
### Logging Pattern
- **Container Logs**: Docker native log collection
- **Log Aggregation**: Centralized log management
- **Error Tracking**: Exception monitoring and alerting
- **Performance Logs**: Execution time and resource usage tracking
### Health Check Pattern
- **Service Health**: Individual container health verification
- **Dependency Health**: Inter-service connectivity testing
- **External Health**: Domain resolution and certificate validation
- **Automated Recovery**: Service restart on health check failure
## Integration Patterns
### API Gateway Pattern
- **Unified Interface**: Single API endpoint for external integrations
- **Authentication**: Centralized auth for API access
- **Rate Limiting**: API usage control and throttling
- **Version Management**: API versioning for backward compatibility
### Webhook Pattern
- **Event-Driven**: Trigger workflows based on external events
- **Secure Endpoints**: HTTPS webhook receivers
- **Payload Validation**: Input sanitization and verification
- **Error Handling**: Graceful failure management for webhook failures
### File Processing Pattern
- **Watch Folders**: Monitor directories for new files
- **Processing Pipelines**: Multi-step file transformation workflows
- **Format Conversion**: Support for multiple input/output formats
- **Error Recovery**: Handle corrupted or invalid files gracefully
## Development and Testing Patterns
### Local Development Pattern
- **Development Environment**: Local Docker setup for testing
- **Hot Reload**: Development container with live code updates
- **Debug Access**: Direct container access for troubleshooting
- **Test Data**: Sample datasets for development workflows
### Workflow Testing Pattern
- **Version Control**: Git-based workflow versioning
- **Testing Environment**: Isolated testing infrastructure
- **Automated Testing**: CI/CD integration for workflow validation
- **Performance Testing**: Load testing for production workflows
### Community Contribution Pattern
- **Template Sharing**: Standardized workflow export/import
- **Documentation**: Inline workflow documentation standards
- **Quality Assurance**: Community review process for shared workflows
- **Categorization**: Organized template library with search capabilities

60
memory-bank/tasks.md Normal file
View File

@@ -0,0 +1,60 @@
# Tasks - n8n-installer Project
## Current Task Status
**No active tasks** - Project initialized and ready for new work.
## Task History
- **VAN Mode Initialization** (Current session): ✅ COMPLETED
- Created Memory Bank directory structure
- Initialized core documentation files
- Project analysis and context establishment
## Available for Development
### Potential Enhancement Areas
1. **Installation Experience Improvements**
- Enhanced progress reporting during installation
- Better error handling and recovery mechanisms
- Pre-flight validation improvements
2. **Monitoring and Observability**
- Enhanced Grafana dashboards for AI-specific metrics
- Custom n8n workflow performance tracking
- Service health monitoring improvements
3. **Security Enhancements**
- Advanced firewall configuration options
- Enhanced credential management
- Audit logging capabilities
4. **Community Workflow Management**
- Workflow categorization and search improvements
- Template validation and quality assurance
- Automated workflow testing framework
5. **Documentation and User Experience**
- Video tutorials and walkthrough guides
- Troubleshooting automation
- Performance optimization guides
## Next Steps
The project is ready for mode transitions based on user requirements:
- **PLAN**: For planning new features or major enhancements
- **CREATIVE**: For designing new components or architectural improvements
- **IMPLEMENT**: For direct implementation of specific features or fixes
- **QA**: For validation, testing, and quality assurance work
## Task Tracking Template
```
### Task: [Task Name]
- **Type**: [Level 1-4 / Bug Fix / Enhancement / Feature]
- **Priority**: [High / Medium / Low]
- **Status**: [Planning / In Progress / Testing / Complete]
- **Components**: [List of affected components]
- **Checklist**:
- [ ] Task item 1
- [ ] Task item 2
- [ ] Task item 3
```
*This file will be updated with specific task details when active development begins.*

View File

@@ -0,0 +1,97 @@
# n8n-installer Technical Context
## Technology Stack
### Core Infrastructure
- **Docker Compose**: Service orchestration and container management
- **Caddy Server**: HTTP/2 web server with automatic HTTPS
- **PostgreSQL**: Primary database for n8n and optional services
- **Redis**: Caching layer and n8n queue management
### Programming Languages
- **Shell Scripts**: Primary automation and installation logic
- **Python**: Utility scripts (n8n_pipe.py, start_services.py)
- **JavaScript/Node.js**: n8n workflows and custom code nodes
- **JSON**: Configuration and workflow definitions
### Service Architecture
- **n8n Platform**: Queue mode with worker scaling
- **Microservices**: Each tool runs as isolated Docker container
- **Reverse Proxy**: Caddy handles SSL termination and routing
- **Database Layer**: Postgres with optional vector capabilities
### AI/ML Integration
- **Vector Stores**: Qdrant, Supabase pgvector, Weaviate
- **LLM Hosting**: Ollama for local models
- **AI Frameworks**: Support for OpenAI, Anthropic, Gemini, Claude
- **Agent Platforms**: Flowise, Letta, n8n AI nodes
### Development Tools
- **Monitoring**: Prometheus metrics collection, Grafana visualization
- **Debugging**: Langfuse for AI performance tracking
- **Search**: SearXNG for private web search
- **Crawling**: Crawl4ai for web data extraction
### Security & Networking
- **SSL/TLS**: Automatic certificate management via Let's Encrypt
- **Domain Routing**: Wildcard subdomain configuration
- **Firewall**: Basic security enhancements during installation
- **Authentication**: Service-specific login systems
### File Structure
```
n8n-installer/
├── scripts/ # Installation and maintenance scripts
├── n8n/ # n8n configuration and backups
├── flowise/ # Flowise custom tools and workflows
├── grafana/ # Monitoring dashboards and configuration
├── prometheus/ # Metrics collection configuration
├── caddy-addon/ # Additional Caddy configurations
├── searxng/ # Search engine settings
└── docker-compose.yml # Main orchestration file
```
### Configuration Management
- **Environment Variables**: `.env` file for secrets and settings
- **Service Discovery**: Docker network with named containers
- **Volume Management**: Persistent data storage configuration
- **Port Mapping**: Internal service communication patterns
### Development Libraries (Pre-installed in n8n)
- **cheerio**: HTML/XML parsing and manipulation
- **axios**: HTTP client for API requests
- **moment**: Date/time manipulation
- **lodash**: Utility functions for JavaScript
### Deployment Pipeline
1. **System Preparation**: Updates, firewall, security enhancements
2. **Docker Installation**: Container runtime setup
3. **Secret Generation**: Secure password and key creation
4. **Interactive Wizard**: Service selection and configuration
5. **Service Launch**: Orchestrated container startup
6. **Health Verification**: Service availability confirmation
### Update Mechanism
- **Git-based Updates**: Fetch latest installer changes
- **Image Updates**: Pull newest Docker images
- **Service Restart**: Coordinated rolling updates
- **Backup Integration**: Optional workflow re-import
### Resource Management
- **Scaling**: Configurable n8n worker count
- **Memory**: Service-specific memory allocation
- **Storage**: Volume management for persistent data
- **Network**: Container-to-container communication
### Integration Points
- **API Connectivity**: RESTful interfaces between services
- **Database Sharing**: Common Postgres instance for multiple services
- **Event Triggers**: Webhook-based workflow activation
- **File System**: Shared volume for data exchange (`/data/shared`)
### Monitoring & Observability
- **Metrics**: Prometheus data collection
- **Dashboards**: Grafana visualization panels
- **Logging**: Container-level log aggregation
- **Health Checks**: Service availability monitoring
- **Performance**: AI model execution tracking via Langfuse