Using AI Models

The platform exposes an OpenAI-compatible inference API that works for both locally deployed Ollama models and configured remote models.

Endpoint Overview

Method	Path	Description
GET	/api/v1/models	List all available models
POST	/api/v1/chat/completions	Chat completions (streaming and non-streaming)

The backend automatically routes each request to the right backend:

• Models with owned_by: "ollama" are served by the local Ollama deployment.
• All other models are proxied to their configured remote provider.

Authentication

Both endpoints accept two token formats in the Authorization: Bearer <token> header:

Token type	How to obtain	Restrictions
Project token	Tenancy → Projects → key icon	Limits MCP tool calls to the project’s allowlist
User JWT	Issued on login	Requires ollama:read permission; no MCP restrictions

Notebooks launched inside a project receive a project token automatically as FTN_API_KEY (see Environment Variables below).

---

From a Jupyter Notebook

Environment Variables in Notebooks

When you create a notebook inside a project, the following variables are injected into the pod automatically:

Variable	Example value	Purpose
FTN_BASE_URL	http://ftn-ai-backend.ftn-ai.svc.cluster.local:8081	In-cluster backend URL
FTN_API_KEY	a3f8...d2c1	Project-scoped token for authentication
FTN_PROJECT_ID	550e8400-e29b-41d4-a716...	UUID of the owning project

Notebooks not attached to a project will not have these variables set. You can still use the API by providing a user JWT manually.

The Sample Notebook

Every new notebook instance is seeded with Example.ipynb on first launch. Open it from the file browser — it is a self-contained walkthrough that covers all the steps below.

Step-by-Step Usage

1. Read the environment

import os

FTN_BASE_URL   = os.environ.get('FTN_BASE_URL',   'http://ftn-ai-backend.ftn-ai.svc.cluster.local:8081')
FTN_API_KEY    = os.environ.get('FTN_API_KEY',    '')
FTN_PROJECT_ID = os.environ.get('FTN_PROJECT_ID', '')

print(FTN_BASE_URL, FTN_PROJECT_ID)

2. Create the client

from openai import OpenAI

client = OpenAI(
    base_url=f'{FTN_BASE_URL}/api/v1',
    api_key=FTN_API_KEY or 'no-key',  # library requires a non-empty string
)

3. Discover available models

all_models    = client.models.list().data
ollama_models = [m for m in all_models if m.owned_by == 'ollama']
remote_models = [m for m in all_models if m.owned_by != 'ollama']

print('Local Ollama models:')
for m in ollama_models:
    print(f'  {m.id}')

print('Remote models:')
for m in remote_models:
    print(f'  {m.id}  ({m.owned_by})')

4. Set the model you want to use

# Copy an ID from the list printed above
MODEL_ID = "llama3:8b"

5. Send a chat request

response = client.chat.completions.create(
    model=MODEL_ID,
    messages=[
        {'role': 'system', 'content': 'You are a concise, helpful assistant.'},
        {'role': 'user',   'content': 'What is a Kubernetes namespace?'},
    ],
)
print(response.choices[0].message.content)

6. Streaming response

import sys

with client.chat.completions.create(
    model=MODEL_ID,
    messages=[{'role': 'user', 'content': 'List five Linux commands with a one-line description.'}],
    stream=True,
) as stream:
    for chunk in stream:
        sys.stdout.write(chunk.choices[0].delta.content or '')
        sys.stdout.flush()

7. Multi-turn conversation

history = [{'role': 'system', 'content': 'You are a Kubernetes expert. Be concise.'}]

def chat(user_message):
    history.append({'role': 'user', 'content': user_message})
    resp = client.chat.completions.create(model=MODEL_ID, messages=history)
    reply = resp.choices[0].message.content
    history.append({'role': 'assistant', 'content': reply})
    return reply

print(chat('What is a Deployment?'))
print(chat('How does it differ from a StatefulSet?'))

---

From an External Client

Any OpenAI-compatible client can reach the platform API from outside the cluster. Replace the in-cluster URL with the external hostname.

Python (openai library)

from openai import OpenAI

client = OpenAI(
    base_url='https://<your-domain>/api/v1',
    api_key='<your-project-token-or-jwt>',
)

models = client.models.list()
response = client.chat.completions.create(
    model='llama3:8b',
    messages=[{'role': 'user', 'content': 'Hello!'}],
)
print(response.choices[0].message.content)

curl

# List models
curl -s https://<your-domain>/api/v1/models \
  -H "Authorization: Bearer <token>" | jq .

# Chat completion
curl -s https://<your-domain>/api/v1/chat/completions \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama3:8b",
    "messages": [{"role": "user", "content": "Hello!"}]
  }' | jq .choices[0].message.content

# Streaming
curl -s https://<your-domain>/api/v1/chat/completions \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama3:8b",
    "messages": [{"role": "user", "content": "Count to 5."}],
    "stream": true
  }'

Any OpenAI-compatible tool

Because the API follows the OpenAI wire format, it works as a drop-in replacement in any tool that accepts a custom base_url:

Tool / Library	Configuration
LangChain	ChatOpenAI(base_url="...", openai_api_key="...")
LlamaIndex	OpenAI(api_base="...", api_key="...")
Cursor / VS Code Copilot	Set custom OpenAI endpoint in settings
Open WebUI	Add as a custom OpenAI connection
n8n	Use the OpenAI node with a custom base URL

---

MCP Tools from Notebooks

If the notebook is attached to a project, the project token automatically restricts which MCP tools can be called. Fetch the allowed tools and pass them directly to the model for function calling:

import requests, json

headers = {'Authorization': f'Bearer {FTN_API_KEY}'} if FTN_API_KEY else {}
tools_resp = requests.get(f'{FTN_BASE_URL}/api/mcp/tools', headers=headers, timeout=15)
mcp_tools = tools_resp.json().get('tools', [])

# Convert to OpenAI function-calling format
openai_tools = [
    {
        'type': 'function',
        'function': {
            'name': f"{t['server']}__{t['name']}",
            'description': t.get('description', ''),
            'parameters': t.get('inputSchema', {'type': 'object', 'properties': {}}),
        },
    }
    for t in mcp_tools
]

print(f'{len(openai_tools)} tools available to this project')

Manage the allowlist for a project under Tenancy → Projects → key icon → MCP Servers.

---

Obtaining a Token

Project token (recommended for notebooks)

1. Go to Tenancy → Projects
2. Click the key icon on the project
3. The token is shown in the Access Management dialog — copy it or rotate it

User JWT (for development / external scripts)

curl -s https://<your-domain>/api/auth/login \
  -H "Content-Type: application/json" \
  -d '{"username": "alice", "password": "..."}' | jq -r .token

Use the returned JWT as the Authorization: Bearer value. JWTs expire after the configured TTL (default 24 hours).