🎥Watch the Video Tutorial
💡Why LXC Instead of VM for Ollama?
Ollama doesn’t need a full virtual machine. It’s a user-space application — a binary that runs on top of an existing operating system. When you put it in a VM, you’re paying the overhead of a full kernel boot, system services, and RAM allocation for the VM itself.
An LXC container gives you the same result with less waste: it shares the host kernel, starts in seconds instead of minutes, uses less RAM, and still gives you your own IP address and SSH access. For a service like Ollama that doesn’t need kernel isolation, LXC is the more efficient choice.
By the end of this guide, you’ll have Ollama running in a Proxmox LXC container, a model pulled and ready to use from the command line, and accessible from other machines on your network via Tailscale. I include a link to set up Open WebUI in Docker separately — you’ll be able to chat with your models through a browser instead of the terminal.
ℹ️Note: Ollama runs fine on CPU. It’s slower than GPU, but for testing, learning, cron jobs, or overnight batch processing, CPU-only is perfectly functional. If you have a compatible NVIDIA or AMD GPU, Ollama detects it automatically.
🛠 What You’ll Need
- Proxmox host — any version, with LXC container support
- 8GB+ RAM recommended — shared with host kernel, so your container gets more effective RAM than a VM would
- 20GB+ storage for the LXC container — expandable later if you need more room for models
- Ubuntu Cloud Image template — download from Proxmox before creating the container
- Tailscale account — free tier, for remote access (optional but recommended)
- SSH client — to connect to the container from another machine
- Basic Linux familiarity — you should be comfortable with terminal commands
💡Tip: A full VM makes more sense if you’re running a different kernel (like Windows) or need complete isolation. For Ollama — a user-space application that doesn’t need kernel-level isolation — LXC is the more efficient choice.
📋Step-by-Step Setup
1. Download the Ubuntu Cloud Image Template
Open the Proxmox web interface. Click on your storage in the left sidebar, then look for CT Templates — it’s usually listed under the node. If you don’t see it there, you can also download the template directly in the Create CT wizard.
Click Templates
Select your Operating System
Click Download. This takes 1-2 minutes depending on your connection. The cloud image is a pre-built minimal image designed for containers — no desktop, no server installation fluff, just the bare essentials.
2. Create the LXC Container
Click Create CT at the top of the Proxmox interface.
VM ID: Use the default or any available number.
Name: “ollama” (or whatever you prefer).
Template: Select the Ubuntu cloud image you just downloaded.
Hostname: “ollama”.
Core settings:
- Memory: 8GB — In a VM you’d allocate 8GB plus the VM kernel overhead. In an LXC container, this 8GB is purely for your workload. The container adds almost no overhead, so you get more effective RAM for Ollama.
- CPU: 2 cores — Ollama supports multi-threading, so more cores means faster inference. 2 cores is fine for this demo.
- Disk: 20GB — Ollama models can be large, but they’re stored in the container’s filesystem. This gives you room for a few models. You can increase this later.
- Network: Use the default bridge (vmbr0). Set IP address to DHCP — the container will get an IP from your router like any other device.
- Features: Enable Nesting and Unprivileged container.
Click Next through the remaining screens — the defaults are fine.
Click Finish. The container is created.
3. Start the Container
Select the container in Proxmox, then click Start at the top.
The container starts almost instantly — that’s the benefit of LXC. No kernel boot, no system initialisation. It’s ready in seconds.
4. Log Into the Console
Click Console to open the container console.
The default username is root, and there’s no password set by default on these Ubuntu cloud images — so log in as root with no password.
Once you’re in, run ip addr to see what IP address the container has been assigned. Note it down — you’ll need it for SSH later.
ip addr
Also update the LXC config file, run nano /etc/pve/lxc/110.conf and add the following 2 lines at the end:
lxc.cgroup2.devices.allow: c 10:200 rwm
lxc.mount.entry: /dev/net dev/net none bind,create=dir
5. Set Up a User Account and SSH
I do everything through SSH because it lets me copy and paste commands from a website, which is much easier than typing them in the Proxmox console.
Create a user account. I use “ollama” for consistency with my other containers:
adduser ubuntu
Follow the prompts — set a password, fill in details or just skip through.
Give this user sudo access:
usermod -aG sudo ubuntu
Install SSH server (the cloud image doesn’t come with it by default — that’s intentional for security):
apt update && apt install openssh-server -y
Enable and start SSH:
systemctl enable ssh && systemctl start ssh
Then reboot
reboot
6. SSH Into the Container
From your main machine, SSH into the container using the IP you noted earlier:
ssh ubuntu@10.0.0.50
(Replace 10.0.0.50 with your container’s actual IP.)
The first time you connect, it’ll ask you to confirm the fingerprint. Type “yes” and press Enter.
You’re now signed in.
7. Install Prerequisites
Ollama needs curl. Install it:
sudo apt install curl zstd -y
That’s pretty much it — Ollama’s install script handles the rest. Ollama is a user-space binary, so it doesn’t need any special kernel modules or build tools.
8. Install Ollama
Run the official install script:
curl -fsSL https://ollama.com/install.sh | sudo sh
This downloads the Ollama binary, sets it up as a system service, and starts it. Because you’re in a container, it installs in the container’s filesystem, not on the host.
When it’s done, Ollama will be running in the background, listening on port 11434.
9. Verify Ollama Is Running
Check the service status:
sudo systemctl status ollama
You should see it active and running. Press q to quit.
You’ll notice it hasn’t detected any NVIDIA or AMD GPUs — that’s expected since you’re running CPU-only on this hardware.
10. Configure Ollama for Network Access
By default, Ollama only accepts connections from the local machine. You need to open it up so other machines on your network can access it.
Run:
sudo systemctl edit ollama
This opens an override file. Add the following in the [Service] section:
[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"
This tells Ollama to listen on all IP addresses. Save and exit (Ctrl+X, then Y, then Enter).
Reload the systemd daemon:
sudo systemctl daemon-reload
Restart Ollama:
sudo systemctl restart ollama
11. Install Tailscale for Remote Access
If you want to access Ollama from outside your local network, install Tailscale:
curl -fsSL https://tailscale.com/install.sh | sudo sh
Then start the Tailscale connection:
sudo tailscale up
Copy the URL it gives you, open it in a browser, log into your Tailscale account, and authorise the machine.
Once connected, you can see your Ollama machine in the Tailscale admin console. I recommend disabling key expiry on it so it doesn’t time out and lock you out later.
12. Test Ollama
From your browser test that Ollama is running over tailscale:
http://ollama.yourtailnet.ts.net:11434
(Replace ollama.yourtailnet.ts.net with your actual Tailscale hostname.)
If you get a response “Ollama is running”, it is working.
13. Pull and Run a Model
Pull a model. I’m using llama3.2 — it’s a small model that works well for demonstration:
ollama pull llama3.2
You can download and run multiple models. I’m just using this one for demonstration.
Run the model:
ollama run llama3.2
This drops you into an Ollama prompt. You can now ask questions:
In one sentence, explain what Proxmox is.
Ollama will respond. On CPU, it’s slow, you might need to grab a coffee while you wait, or run things overnight depending on how large your request is.
🔧Using Your Ollama Setup
Command Line: You can query Ollama directly from the terminal on the container or from any machine that can reach it over Tailscale. Use ollama run llama3.2 to start an interactive session, or ollama list to see what models you have installed.
Open WebUI (Browser-based): If you want a web interface instead of the terminal, set up Open WebUI in Docker. I have a separate video on that — link below. Open WebUI connects to your Ollama instance and gives you a ChatGPT-like interface in your browser.
Hermes Agent or n8n Integration: You can connect Ollama to automation tools. Hermes agent can use Ollama as a local LLM, and n8n workflows can call Ollama for AI-powered automation. Point them at your Ollama URL (the Tailscale hostname and port 11434).
Multiple Models: You can pull and run multiple models. Switch between them with ollama run modelname. Models are stored in the container’s filesystem, so they persist across restarts.
⚠️Important Considerations
No Built-in Authentication: Ollama has no authentication on its API by default. Because you’re routing through Tailscale rather than exposing it directly to your local network, access is limited to devices on your Tailscale network. That’s a reasonable security boundary for a home lab, but something you need to be mindful of. Don’t expose Ollama directly to the internet.
CPU Performance: Running Ollama on CPU is slow. On a home lab with modest hardware, you’re going to wait for responses. This is fine for learning, testing, cron jobs, or overnight batch processing. If you want interactive, responsive AI, you need a GPU. But CPU-only is still useful — it’s free, private, and offline.
Model Size Matters on CPU: Smaller models (like llama3.2) are more practical on CPU. Larger models will be very slow or may not fit in memory. Stick to smaller models for CPU-only setups.
LXC vs VM: LXC shares the host kernel — it doesn’t give you the same isolation as a full VM. For Ollama, this doesn’t matter because Ollama is a user-space application that doesn’t need kernel-level isolation. If you were running something that needs a different kernel (like Windows) or requires complete isolation, a VM would be the better choice.
Container Filesystem: Models are stored in the LXC container’s filesystem. If you delete the container, you lose the models. If you need to preserve models across container rebuilds, back up the Ollama data directory or use persistent storage.
🚀What’s Next?
- Set up Open WebUI in Docker — link below — for a browser-based chat interface
- Connect Ollama to Hermes agent for local AI-powered automation
- Integrate with n8n workflows for AI-powered automation
- Try different models — pull llama3.2, mistral, or other models available in Ollama
- Set up Ollama as a service in your home lab — accessible from any machine on your Tailscale network
- Experiment with cron jobs that use Ollama overnight for batch processing
✅Conclusion
Ollama in a Proxmox LXC container gives you a lightweight, efficient way to run local AI. It’s faster to start, uses less RAM than a VM, and gives you the isolation and network access you need.
The setup is straightforward: create an LXC container from an Ubuntu cloud image, install Ollama from the official script, configure systemd for network access, install Tailscale for remote access, pull a model, and start using it.
You don’t need a GPU. You don’t need API keys. You don’t need a subscription. Just your Proxmox host and an LXC container.
Your AI runs locally, privately, and offline. That’s the point.
Enjoyed this guide?
Subscribe to the channel for more homelab builds, Proxmox tutorials, and local AI guides.
If you found this helpful, like and subscribe to IT HomeLab Online on YouTube for more tutorials.
☕️Support the channel: Buy Me a Coffee