Guide

How to set up an AI chatbot with Ollama

You can run the ZChat AI agent on a local model with Ollama, so both your chat data and your AI inference stay on your own hardware. This guide covers installing a model, connecting it, grounding it in your knowledge base, and handing off to a human, five steps to a private support bot.

Plan for one more step: these steps configure the chatbot and make it available through ZChat's chatbot API. The widget you install talks to your human agents out of the box, so putting the AI in front of visitors is an integration you wire up - see what's turnkey and what you connect. The same model also powers agent reply suggestions, which need no wiring at all.

Download Free Evaluation About the AI Chatbot

Step 1. Install Ollama and a model

Install Ollama on a machine your chat server can reach, on the same box or elsewhere on your network, and pull a local model. Because inference runs on your own hardware, no customer message is sent to an outside AI service, which is the point of running the bot locally.

Step 2. Connect it to the chat server

In the dashboard, choose Ollama as the AI provider and point it at your Ollama endpoint and model name. Prefer a hosted model instead? The same setting also supports your own OpenAI or Anthropic keys, so you can switch providers without changing anything else.

Step 3. Ground it in your knowledge base

Add knowledge-base articles covering your product, policies, and common questions. The chatbot answers from that content rather than guessing, so its responses match how your team actually answers.

Step 4. Set human handoff

Decide when the bot should escalate to a person, for example when a visitor asks for a human or the question falls outside the knowledge base. On handoff the full transcript goes to the agent, so the customer never has to repeat themselves.

Step 5. Test and enable

Ask the bot several real questions, confirm the answers are accurate and the handoff works, then enable it on your live widget. The bot is optional and off by default, so nothing goes live until you switch it on.

Private by design

Keep the whole conversation in-house

With Ollama the model runs on your hardware and the chat server runs on your infrastructure, so a support conversation, and the AI that handles it, never leaves your control. Try the whole setup free on localhost first.

Download ZChat Add the widget

Owned customer support software

Deploy live chat on your own terms, not on someone else's pricing model.

ZChat gives you the installable server, web dashboard, website widget, and desktop agent tools in one self-hosted product you buy once and keep. Run it on infrastructure you trust and connect AI only if and how you want it.

Download ZChat See Product Demo

Deployment

Install on Windows or Linux, behind IIS or Nginx, in a VM, or in Docker if that fits your stack.

Commercial model

One-time purchase, perpetual license, and no monthly per-agent bill attached to growth.

AI Flexibility

Use Ollama locally or connect OpenAI and Anthropic with your own provider accounts.