12 November 2026 · VRAMfarm team

Your own private ChatGPT: Open WebUI with Ollama or vLLM on a rented GPU

You want to paste a contract, a customer list or unreleased code into a chat, and you would rather it didn’t end up on an AI company’s servers. The answer is an open model running on a GPU you rent, behind Open WebUI, a chat interface that looks and works much like ChatGPT. Qwen3 8B needs a 12 GB card, Qwen3 32B a 24 GB card, a 70B model a 48 GB card. Our ready-made Private ChatGPT app sets up the first two, plus MiMo 9B, by itself; from pressing Launch to the first answer takes about 10 minutes.

The ready-made app: launch, wait, chat

Pick Private ChatGPT in the dashboard and one of three models:

• Qwen3 8B, from a 12 GB card. Fast, good for drafting, rewriting and questions about short texts.
• MiMo 9B (Xiaomi, MIT licence), from a 12 GB card. Similar size and speed, and it understands images: drop in a screenshot, a photo of a document or a chart and ask about it.
• Qwen3 32B, from a 24 GB card. Noticeably smarter: longer texts, reasoning, code. This is the one most people keep.

The server is ready in 2–5 minutes, and those minutes are free. Then the app installs Open WebUI and downloads the model, about 5 GB for the 8B and 20 GB for the 32B, which takes another 5–10 minutes and is billed; the downloads count as traffic. The dashboard shows when the chat is ready, and the Chat button opens it in your browser. Opening it earlier gives an error page, not a broken server.

The app comes tuned: 8k tokens of context with an 8-bit cache, so the 32B model stays entirely on a 24 GB card at about 30 tokens per second, and the model is loaded into memory as soon as it arrives, so the first answer doesn’t wait for it. The chat opens without a login, because the server is yours alone; see the privacy section before you share the link.

Qwen3 models think before they answer, and Open WebUI shows that reasoning folded above the reply. For quick questions, add /no_think to your message and the answer comes several times sooner. The Qwen3 page and the MiMo page have the memory figures and licences.

Which card for 8B, 32B and 70B

Speeds are planning ranges for 4-bit models from memory bandwidth and community reports. Anything above 8 tokens per second reads as fluently as you can follow.

For a 70B model, launch the ready-made app on a 48 GB card and swap the model in the terminal (Jupyter → Terminal):

ollama stop qwen3:8b
ollama pull llama3.3:70b

The 43 GB download is billed traffic and takes a while. Reload the chat page and pick llama3.3:70b from the model menu. The app’s 8k context fits a 70B on 48 GB; for long documents you want an 80 GB card, as explained in running a 70B model on one GPU.

Build it yourself: Open WebUI with Ollama or vLLM

On an empty PyTorch server you can set up the same thing in a few commands, with any model:

curl -fsSL https://ollama.com/install.sh | sh
OLLAMA_CONTEXT_LENGTH=8192 OLLAMA_FLASH_ATTENTION=1 OLLAMA_KV_CACHE_TYPE=q8_0 \
  nohup ollama serve > /workspace/ollama.log 2>&1 &
ollama pull qwen3:14b

pip install open-webui
DATA_DIR=/workspace/open-webui OLLAMA_BASE_URL=http://127.0.0.1:11434 \
  nohup open-webui serve --host 127.0.0.1 --port 3000 > /workspace/webui.log 2>&1 &

An empty server publishes only SSH and Jupyter, so open the chat through an SSH tunnel from your computer: ssh -p <port> -L 3000:localhost:3000 root@<address>, with the port and address from the server card, then go to http://localhost:3000. The SSH key guide sets up the key. Open WebUI keeps chats, users and settings in DATA_DIR, which is why it points into /workspace.

Ollama is the right engine for one person or a few. If many people will write at the same time, put vLLM behind Open WebUI instead. It batches parallel requests, so twenty people asking at once each get an answer at close to full speed, where Ollama answers a few and queues the rest. On a 48 GB card, where Qwen3 32B in 4-bit (about 19 GB) leaves cache room for roughly ten 8k conversations at once, unload Ollama’s model (ollama stop <model>), start vLLM with a key, then add it in Open WebUI under Admin Panel → Settings → Connections as an OpenAI API connection with the address http://127.0.0.1:8000/v1 and the same key:

pip install "vllm==0.11.0" "transformers>=4.56,<5"
nohup vllm serve Qwen/Qwen3-32B-AWQ --max-model-len 8192 --port 8000 \
  --api-key change-me > /workspace/vllm.log 2>&1 &

The pinned version is built for CUDA 12.8; a newer vLLM may need a newer driver than the machine has. The runtime comparison explains why, and when each engine fits.

What stays private, and what doesn’t

What stays on the server. The model runs on the GPU you rented. Your prompts, the files you attach and the answers are processed there, and no AI company’s API is involved. Document search in Open WebUI also runs locally, with a small embedding model downloaded to the server. Web search and connections to external model APIs exist in Open WebUI, and they stay off unless you add them.

Where the server is. It is a machine at one of our vetted hosts, not your own hardware. Its country is shown before launch, so you can choose one that suits your requirements. “Private” here means that no AI provider sees or stores your conversations, which is what most people need when they move off a public chatbot.

The link. The ready-made app’s chat address is a plain http link on a port of our address, and it opens without a login. Anyone who has the link, or tries ports on our address until one answers, can use the chat while the server runs. Don’t post it anywhere, switch on accounts before you share it (next section), and for confidential material open the chat through an SSH tunnel as above, which encrypts the connection.

After you stop. The server’s disk is deleted. Your chat history lives in /workspace/open-webui, so choose to save the folder when you stop if you want it back next time; it is restored on the next server, even a different card, for €0.03 per GB per month, and chat history is usually well under a gigabyte. The model itself is not kept and downloads again at the next launch. The article on files after a stop has the details.

Sharing the chat with a team

For a team, switch on Open WebUI’s accounts. The data folder the app created was set up without real accounts, so the clean way is to start Open WebUI with a new one:

pkill -f "open-webui serve"
DATA_DIR=/workspace/open-webui-team WEBUI_AUTH=True OLLAMA_BASE_URL=http://127.0.0.1:11434 \
  nohup open-webui serve --host 0.0.0.0 --port 3000 > /workspace/webui.log 2>&1 &

The first account you create becomes the administrator. Colleagues who sign up wait for your approval, or you create their accounts under Admin Panel → Users. Each person gets their own chat history; the administrator can share models, system prompts and document collections with everyone. Chats from the old folder stay there and don’t move over. If this process ever crashes, the app’s automatic repair starts the original no-login chat again, so after any restart check that the chat still asks for a login.

How much one server handles: a 24 GB card with Qwen3 32B through Ollama is comfortable for a handful of people asking now and then. With ten or more typing at the same time, answers start to queue; switch the engine to vLLM, and consider a 48 GB card so more conversations fit in the cache at once.

The cost works per server, not per seat. A 24 GB RTX 4090 for 9 hours on 22 working days is 198 hours, €116.82 a month for the whole team. A paid chat subscription costs about €20–25 per person per month (November 2026). For one person, the subscription is cheaper unless privacy is the point. Somewhere around six or more people, depending on today’s card price and your hours, one shared server costs less than a seat each, with the data on a machine you control.

Idle protection helps here: if nobody uses the chat, it emails you after 2 hours without connections or GPU load, and an hour later saves the folder and stops the server. For fixed office hours, a small script on your side (cron, for example) can start and stop the server through the API, with a token from Settings.