LLM/local-llm-stack
1
0
Fork 0
mirror of https://github.com/dalekurt/local-llm-stack.git synced 2026-01-28 07:40:24 +00:00
Code Issues Packages Projects Releases Wiki Activity

Add Open WebUI Pipelines service and configure local data storage

Browse Source
This commit is contained in:
Dale-Kurt Murray
2025-05-30 11:21:32 -04:00
parent c7702f6ba8
commit 94d543f9e2
2 changed files with 202 additions and 96 deletions
Download Patch File Download Diff File Expand all files Collapse all files

236
README.md
View File

@@ -1,97 +1,121 @@
# Docker Local AI LLM Environment
# Local LLM Stack: Your Complete AI Workbench
This repository contains a Docker Compose setup for running a local AI and automation environment with multiple services.
*A comprehensive Docker-based environment for working with Large Language Models locally*
## Overview
![Local LLM Stack Banner](https://example.com/banner.jpg)
The Local LLM Stack is a comprehensive, Docker-based environment designed to provide a complete ecosystem for working with Large Language Models (LLMs) locally. It integrates several powerful open-source tools that work together to provide document management, workflow automation, vector search, and LLM inference capabilities.
## Introduction
This stack is designed with flexibility in mind, allowing you to use either a locally installed Ollama instance or run Ollama as a containerized service. All components are configured to work together out of the box, with sensible defaults that can be customized to suit your specific needs.
In today's rapidly evolving AI landscape, having a reliable local environment for experimenting with Large Language Models (LLMs) has become essential for developers, researchers, and AI enthusiasts. The Local LLM Stack provides exactly that - a complete ecosystem for document management, workflow automation, vector search, and LLM inference capabilities, all running locally on your machine.
Key features of this stack include:
- Document processing and management with AnythingLLM
- AI workflow automation with Flowise
- Direct LLM interaction through Open WebUI
- Workflow automation with n8n
- Vector storage with Qdrant
- Seamless integration with Ollama for LLM inference
Inspired by setups like [Peter Nhan's Ollama-AnythingLLM integration](https://peter-nhan.github.io/posts/Ollama-AnythingLLM/), this stack takes the concept further by incorporating additional services and enhanced configuration options to create a more robust and versatile environment.
## Services
## What Makes This Stack Special?
### Flowise
- AI workflow automation tool
- Port: 3001
- Web interface: http://localhost:3001
The Local LLM Stack integrates several powerful open-source tools that work together seamlessly:
- **Document processing and RAG capabilities** with AnythingLLM
- **AI workflow automation** with Flowise
- **Direct LLM interaction** through Open WebUI
- **Text generation pipeline** with Open WebUI Pipelines
- **Advanced workflow automation** with n8n
- **Vector storage** with Qdrant
- **Seamless integration** with Ollama for LLM inference
Unlike simpler setups, this stack is designed with flexibility in mind, allowing you to use either a locally installed Ollama instance or run Ollama as a containerized service. All components are configured to work together out of the box, with sensible defaults that can be customized to suit your specific needs.
## The Services: A Closer Look
### AnythingLLM
- Document management and AI interaction platform
- Port: 3002
- Web interface: http://localhost:3002
- Uses Qdrant as vector database
- Connects to Ollama for LLM capabilities
AnythingLLM serves as your document management and AI interaction platform. It allows you to chat with any document, such as PDFs or Word files, using various LLMs. In our setup, AnythingLLM is configured to use:
- Qdrant as the vector database for efficient document embeddings
- Ollama for LLM capabilities
- Port 3002 for web access (http://localhost:3002)
### Flowise
Flowise provides a visual programming interface for creating AI workflows:
- Build complex AI applications without coding
- Connect to various AI services and data sources
- Port 3001 for web access (http://localhost:3001)
### Open WebUI
- Web interface for interacting with AI models
- Port: 11500
- Web interface: http://localhost:11500
Open WebUI offers a clean interface for direct interaction with AI models:
- Chat with models hosted on Ollama
- Manage and switch between different models
- Port 11500 for web access (http://localhost:11500)
### Open WebUI Pipelines
Pipelines provides optimized text generation capabilities:
- High-performance inference server for LLMs
- OpenAI-compatible API for easy integration
- Works seamlessly with Open WebUI
- Accessible through Open WebUI or directly via API
### n8n
- Workflow automation platform
- Port: 5678
- Web interface: http://localhost:5678
n8n is a powerful workflow automation platform:
- Create automated workflows connecting various services
- Includes automatic import of workflows and credentials from backup directory
- Port 5678 for web access (http://localhost:5678)
### Qdrant
- Vector database for AI applications
- Port: 6333
- API endpoint: http://localhost:6333
Qdrant serves as our vector database for AI applications:
- Stores and retrieves vector embeddings for semantic search
- Used by AnythingLLM for document embeddings
- Port 6333 for API access (http://localhost:6333)
### PostgreSQL
- Database used by n8n
- Port: 5432
PostgreSQL provides database services for n8n:
- Reliable and robust SQL database
- Port 5432 for database connections
### Ollama (Optional)
- LLM inference engine
- Port: 11434
- API endpoint: http://localhost:11434
Ollama is our LLM inference engine:
- Run models like Llama, Mistral, and others locally
- Can be run natively (default) or as a containerized service
- Used by AnythingLLM and can be used by Flowise
- Port 11434 for API access (http://localhost:11434)
## Prerequisites
## Getting Started
### Prerequisites
Before diving in, make sure you have:
- Docker and Docker Compose installed on your system
- Basic understanding of Docker containers and networking
- Ollama installed locally (or optionally run as a container by uncommenting the Ollama service in docker-compose.yml)
## Setup Instructions
### Setup Instructions
1. Clone this repository:
```
1. **Clone the repository:**
```bash
git clone https://github.com/yourusername/local-llm-stack.git
cd local-llm-stack
```
2. Create a `.env` file based on the provided `.env.sample` file:
```
2. **Create your environment file:**
```bash
cp .env.sample .env
```
3. Modify the `.env` file with your preferred credentials and secrets.
3. **Customize your environment:**
Open the `.env` file in your favorite editor and modify the credentials and settings to your liking.
4. Ollama Configuration:
- By default, the setup is configured to use a locally installed Ollama instance
- If you want to run Ollama as a container, uncomment the following in docker-compose.yml:
- The `ollama_storage` volume in the volumes section
- The entire `ollama` service definition
- Update the OLLAMA_BASE_PATH and EMBEDDING_BASE_PATH in .env to use http://ollama:11434
4. **Configure Ollama:**
By default, the setup is configured to use a locally installed Ollama instance. If you prefer to run Ollama as a container:
- Uncomment the `ollama_storage` volume in the volumes section of docker-compose.yml
- Uncomment the entire `ollama` service definition
- Update the OLLAMA_BASE_PATH and EMBEDDING_BASE_PATH in .env to use http://ollama:11434
5. Start the services:
5. **Create the data directory structure:**
```bash
mkdir -p ./data/{n8n,postgres,qdrant,openwebui,flowise,anythingllm,pipelines}
```
6. **Launch the stack:**
```bash
docker-compose up -d
```
6. Access the services through their respective ports:
7. **Access your services:**
- Flowise: http://localhost:3001
- AnythingLLM: http://localhost:3002
- Open WebUI: http://localhost:11500
@@ -100,34 +124,43 @@ Key features of this stack include:
## Data Persistence
The following Docker volumes are used for data persistence:
Your data is persisted in the local `./data` directory with separate subdirectories for each service:
- `n8n_storage`: n8n data
- `postgres_storage`: PostgreSQL database
- `qdrant_storage`: Qdrant vector database
- `open-webui`: Open WebUI data
- `flowise`: Flowise data
- `anythingllm_storage`: AnythingLLM data
- `ollama_storage`: Ollama data (when using containerized Ollama)
- `./data/n8n`: n8n data
- `./data/postgres`: PostgreSQL database
- `./data/qdrant`: Qdrant vector database
- `./data/openwebui`: Open WebUI data
- `./data/flowise`: Flowise data
- `./data/anythingllm`: AnythingLLM data
- `./data/pipelines`: Open WebUI Pipelines data
- `./data/ollama`: Ollama data (when using containerized Ollama)
This structure ensures your data persists across container restarts and makes it easy to backup or transfer your data.
## Network Configuration
All services are connected to the `ai-network` Docker network for internal communication.
All services are connected to the `ai-network` Docker network for internal communication, ensuring they can seamlessly work together while remaining isolated from your host network.
## Customization
You can customize the services by modifying the `docker-compose.yml` and `.env` files according to your needs.
## Customization Options
### AnythingLLM Configuration
AnythingLLM is configured to use:
- Qdrant as the vector database
AnythingLLM is highly configurable through environment variables. In our setup, it's configured to use:
- Qdrant as the vector database for document embeddings
- Ollama for LLM capabilities and embeddings
- The configuration can be adjusted in the .env file
- Various settings that can be adjusted in the .env file
### Open WebUI Pipelines
To connect Open WebUI with the Pipelines service:
1. In Open WebUI, go to Settings > Connections > OpenAI API
2. Set the API URL to `http://pipelines:9099`
3. Set the API key to `0p3n-w3bu!` (or your custom key if modified)
### Ollama Configuration
Ollama can be configured in two ways:
1. **Native Installation (Default)**:
- Install Ollama directly on your host machine
- The services are configured to access Ollama via host.docker.internal
@@ -139,23 +172,80 @@ Ollama can be configured in two ways:
- Update the OLLAMA_BASE_PATH and EMBEDDING_BASE_PATH in .env to http://ollama:11434
- No need to run Ollama separately on your host
## Use Cases
### Document Analysis with RAG
Upload documents to AnythingLLM and chat with them using the power of Retrieval-Augmented Generation (RAG). The system will:
1. Process and embed your documents using Ollama's embedding models
2. Store the embeddings in Qdrant
3. Retrieve relevant context when you ask questions
4. Generate responses using the LLM with the retrieved context
### Workflow Automation
Use n8n to create workflows that:
- Monitor folders for new documents
- Trigger document processing in AnythingLLM
- Send notifications when processing is complete
- Extract and process information from documents
### AI Application Development
Use Flowise to build AI applications with a visual interface:
- Create chatbots with specific knowledge domains
- Build document processing pipelines
- Develop custom AI assistants for specific tasks
### High-Performance Inference
Use Open WebUI Pipelines for optimized text generation:
- Faster response times compared to standard Ollama API
- OpenAI-compatible API for easy integration with various tools
- Connect directly from Open WebUI for an enhanced experience
## Troubleshooting
If you encounter any issues:
1. Check the logs of the specific service:
```
1. **Check the logs:**
```bash
docker-compose logs [service-name]
```
2. Ensure all required ports are available on your host system.
2. **Ensure port availability:**
Make sure all required ports are available on your host system.
3. Verify that the `.env` file contains all necessary variables.
3. **Verify environment variables:**
Check that the `.env` file contains all necessary variables.
4. If using native Ollama, ensure it's running on your host machine:
```
4. **Ollama status:**
If using native Ollama, ensure it's running on your host machine:
```bash
ollama serve
```
5. If you're having connection issues between containers and your host machine, check that the `extra_hosts` configuration is correctly set in docker-compose.yml.
5. **Connection issues:**
If you're having connection issues between containers and your host machine, check that the `extra_hosts` configuration is correctly set in docker-compose.yml.
6. **Permission issues with data directories:**
If you encounter permission errors with the data directories, you may need to adjust permissions:
```bash
sudo chmod -R 777 ./data # Use with caution in production environments
```
## Conclusion
The Local LLM Stack provides a powerful, flexible environment for working with AI and LLMs locally. By combining the best open-source tools available, it offers capabilities that were previously only available through cloud services, all while keeping your data private and under your control.
Whether you're a developer looking to build AI applications, a researcher experimenting with document analysis, or an enthusiast exploring the capabilities of LLMs, this stack provides the foundation you need to get started quickly and efficiently.
---
## References
- [Peter Nhan's Ollama-AnythingLLM Integration](https://peter-nhan.github.io/posts/Ollama-AnythingLLM/)
- [AnythingLLM Documentation](https://docs.anythingllm.com/)
- [Ollama GitHub Repository](https://github.com/ollama/ollama)
- [Qdrant Vector Database](https://qdrant.tech/)
- [n8n Workflow Automation](https://n8n.io/)
- [Flowise AI](https://flowiseai.com/)
- [Open WebUI](https://github.com/open-webui/open-webui)
- [Open WebUI Pipelines](https://github.com/open-webui/pipelines)

62
docker-compose.yml
View File

@@ -1,11 +1,12 @@
volumes:
n8n_storage:
postgres_storage:
qdrant_storage:
open-webui:
flowise:
anythingllm_storage:
# ollama_storage: # Uncomment if using containerized Ollama
# All volumes are mapped directly to host paths in each service
n8n_storage: {}
postgres_storage: {}
qdrant_storage: {}
openwebui_storage: {}
flowise_storage: {}
anythingllm_storage: {}
pipelines_storage: {}
networks:
@@ -28,8 +29,6 @@ x-n8n: &service-n8n
- postgres
services:
# Ollama service - commented out by default since Ollama is installed natively
# Uncomment this service if you want to run Ollama in a container
# ollama:
# image: ollama/ollama:latest
# container_name: ollama
@@ -39,9 +38,9 @@ services:
# - "11434:11434"
# volumes:
# - ollama_storage:/root/.ollama
# # If using containerized Ollama, update OLLAMA_BASE_PATH in .env to:
# # OLLAMA_BASE_PATH=http://ollama:11434
# # EMBEDDING_BASE_PATH=http://ollama:11434
# Note: When using containerized Ollama, make sure to update in .env:
# OLLAMA_BASE_PATH=http://ollama:11434
# EMBEDDING_BASE_PATH=http://ollama:11434
flowise:
image: flowiseai/flowise
@@ -49,14 +48,13 @@ services:
restart: unless-stopped
container_name: flowise
environment:
- PORT=${FLOWISE_PORT:-3001}
- PORT=${FLOWISE_PORT:-3010}
ports:
- ${FLOWISE_PORT:-3001}:3001
- ${FLOWISE_PORT:-3001}:3010
extra_hosts:
- "host.docker.internal:host-gateway"
volumes:
- ~/.flowise:/root/.flowise
entrypoint: /bin/sh -c "sleep 3; flowise start"
- ./data/flowise:/root/.flowise
anythingllm:
image: mintplexlabs/anythingllm
@@ -85,8 +83,7 @@ services:
- TTS_PROVIDER=${TTS_PROVIDER}
- PASSWORDMINCHAR=${PASSWORDMINCHAR}
volumes:
- anythingllm_storage:/app/server/storage
# When using native Ollama, add host.docker.internal to allow container to access host
- ./data/anythingllm:/app/server/storage
extra_hosts:
- "host.docker.internal:host-gateway"
depends_on:
@@ -99,16 +96,26 @@ services:
container_name: open-webui
ports:
- "11500:8080"
# When using native Ollama, uncomment this to allow container to access host
extra_hosts:
- "host.docker.internal:host-gateway"
volumes:
- open-webui:/app/backend/data
- ./data/openwebui:/app/backend/data
pipelines:
image: ghcr.io/open-webui/pipelines:main
networks: [ 'ai-network' ]
restart: unless-stopped
container_name: pipelines
volumes:
- ./data/pipelines:/app/pipelines
environment:
- PIPELINES_API_KEY=0p3n-w3bu!
postgres:
image: postgres:16-alpine
networks: [ 'ai-network' ]
restart: unless-stopped
container_name: postgres
ports:
- 5432:5432
environment:
@@ -116,7 +123,7 @@ services:
- POSTGRES_PASSWORD
- POSTGRES_DB
volumes:
- postgres_storage:/var/lib/postgresql/data
- ./data/postgres:/var/lib/postgresql/data
healthcheck:
test: [ 'CMD-SHELL', 'pg_isready -h localhost -U ${POSTGRES_USER} -d ${POSTGRES_DB}' ]
interval: 5s
@@ -143,7 +150,7 @@ services:
ports:
- 5678:5678
volumes:
- n8n_storage:/home/node/.n8n
- ./data/n8n:/home/node/.n8n
- ./n8n/backup:/backup
- ./shared:/data/shared
- ./shared:/home/node/host_mount/shared_drive
@@ -161,4 +168,13 @@ services:
ports:
- 6333:6333
volumes:
- qdrant_storage:/qdrant/storage
- ./data/qdrant:/qdrant/storage
watchtower:
image: containrrr/watchtower
container_name: watchtower
networks: [ 'ai-network' ]
restart: unless-stopped
volumes:
- /var/run/docker.sock:/var/run/docker.sock
command: --interval 30 --cleanup --label-enable --monitor-only --label-filter=ai-network