AI Models Overview

The AI Models page lets you manage the language models available to users across the platform.

Model Types

Local Models (Ollama)

Local models run entirely within your Kubernetes cluster using Ollama. No data leaves your infrastructure.

• Model weights are downloaded to a persistent volume
• Inference runs on CPU by default; GPU acceleration is supported
• Supports all models available in the Ollama library (Llama 3, Mistral, Qwen, Gemma, etc.)

Remote Models

Remote models connect to external OpenAI-compatible inference APIs. You provide an endpoint URL and API key; the platform proxies requests through the backend.

Examples of supported providers:

• OpenAI
• Anthropic (via compatibility layer)
• Azure OpenAI
• Any OpenAI-compatible self-hosted endpoint

Inference Sidebar

The sidebar in the platform UI lists all available models. Local Ollama models appear under the Local section; remote models appear under Remote. Clicking a model opens a chat dialog for interactive inference.

Managing Models

From the AI Models page you can:

• Add a local model — enter a model name from the Ollama library and click Deploy
• Add a remote model — provide a name, API endpoint, and API key
• Delete a model — removes the deployment (local) or registration (remote)
• Refresh — reload the current model list from the cluster

See Local Models and Remote Models for detailed instructions.