Tactical Programming

Running OpenWebUI and Ollama in Docker Compose with GPU Support

Problem

We want to run OpenWebUI so we have an easy way to interact with, and test out, LLMs. We also want the convenience of Ollama for managing/running our models for us. We also want to make use of the GPU in our system.

Solution

Go the ~lazy~ simple route of running both inside containers. We'll coordinate the tech with docker-compose.

The file

Below is one successful docker-compose.yaml file for standing up both containers, getting them talking to each other, and also exposing both ports on the host network for easy access over your home network. This is highly insecure but isn't everything in your house like that?

Note the volumes for storing data across restarts. Also the deploy and runtime keys that configure access to our Nvidia GPU.

Also note that this is only one way to do it. There are certainly more/better methods for running this stack.

services:
  ollama:
    image: ollama/ollama
    container_name: ollama
    ports:
      - "11434:11434"
    volumes:
      - ollama_data:/root/.ollama
    deploy:
      resources:
        reservations:
          devices:
            - capabilities: [gpu]
    runtime: nvidia
    environment:
      - NVIDIA_VISIBLE_DEVICES=all
    networks:
      - ollama_net

  openwebui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: openwebui
    ports:
      - "3000:8080"  # WebUI exposed on port 3000
    environment:
      - OLLAMA_BASE_URL=http://ollama:11434  # Connect to Ollama by container name
    depends_on:
      - ollama
    networks:
      - ollama_net
    volumes:
      - ./openwebui:/app/backend/data

volumes:
  ollama_data:

networks:
  ollama_net:
    driver: bridge

Was this made with an LLM? Absolutely.

A quick docker-compose up and you're off to the races. Now go find a useful LLM and impress your friends & family.

Other articles

Running the Wiz MCP Server in WSL2

Wed 04 June 2025

Problem

We want to take advantage of the Wiz MCP Server on our Windows workstation running WSL2.

Solution

The long-term solution to this is to just host the MCP server in a central location and put something like the OpenWebUI OpenAI Proxy in front of it. But this is a …
read more
Troubleshooting Battery Drain Issues on the Garmin Venu 3

Tue 07 January 2025

Problem

Garmin Venu 3 watches will sometimes go into severe battery drain mode, measured by more than 1% per hour (normaly, a single charge will last over ten days). This is further complicated by the sheer opacity of the devices. There's no logs, no error indicators, and the only way …
read more
Self-Signed Certs with Subject Alt Names

Thu 11 May 2023

Problem

We need to create self-signed certs. But because browsers/tools are finally enforcing what they said they would for years, we have to put the CN into the list of SANs for the cert. This is non-negotiable.

That means we have to use the x509v3 extensions. Which is highly …
read more
Automatic Updates for Raspberry Pi's

Sun 15 May 2022

Problem

Raspberry Pi's are great but they don't automatically update themselves. And once we've figured out how to make them do whatever it is we wanted them to do, we're not likely to log back in just for updates.

We need to set the machine to update itself.

Solution

There …
read more
Using cert-manager in Kubernetes with Lets Encrypt

Wed 19 January 2022

Problem

We have Kubernetes. We need a cert. We want to use Let's Encrypt to create (and manage) that cert for us.

Solution

We can install cert-manager and, with three objects, do exactly that. At a high level, we create a ClusterIssuer, we configure it with our cloud credentials (in …
read more
Rotating Pinniped Credentials in Tanzu Kubernetes Grid

Thu 16 December 2021

Problem

We've successfully implemented user authentication in our TKG clusters. Now, we'd like to update the client ID/secret for our TKG Management Cluster.

Solution

TKG uses Pinniped to support authentication, including to OIDC providers. However, exactly which Kubernetes object we need to update/reconfigure is not immediately obvious. Fortunately …
read more
Cluster Node Autoscaling in Tanzu Kubernetes Grid (TKG)

Tue 19 October 2021

Problem

We need to reliably implement cluster autoscaling for our Kubernetes platform. How can we do that? And how can we know it's working?

Solution

TKG uses the ClusterAPI provider, part of the Cluster Autoscaler but makes it easy to configure. We'll ask TKG to enable Cluster Autoscaler for us …
read more
User Management in Tanzu Kubernetes Grid (TKG)
Mon 28 June 2021

Problem

User management is not part of Kubernetes. You have to tie Kubernetes to an external system. This gets.... complicated.

Solution

For TKG, we can hook our Kubernetes clusters up to any OIDC provider. For this excercise, we'll use Okta. At a high level, here's what we'll do.
- Create a …
read more
GPU-Enabled Kubernetes Clusters with Tanzu Kubernetes Grid
Mon 24 May 2021

Stated Goal

We want easy GPU access for Kubernetes workloads in our TKG clusters. This is done by:
- Installing GPU device drivers on our Kubernetes worker nodes
- Installing the device plugins on our Kubernetes worker nodes
- Applying the appropriate labels to our Kubernetes worker nodes so that GPU workloads can …
read more

Page 1 / 3 »

Other articles

Problem

Solution

Problem

Problem

Problem

Solution

Problem

Solution

Problem

Solution

Problem

Solution

Problem

Solution

Stated Goal

social