Submitting a Batch Job

Submit a notebook from the project’s shared workspace as a batch job to run it unattended on the cluster.

Prerequisites

• At least one Jupyter notebook deployed in the project
• The notebook you want to run must be in the project’s shared workspace (/home/jovyan/work/shared/)
• The shared workspace PVC will be created automatically on first submission

From the UI

1. Navigate to Batch Jobs in the sidebar
2. Click New Job
3. Fill in the form:

Field	Required	Description
Notebook Path	Yes	Path relative to the shared workspace (e.g. reports/monthly.ipynb)
Parameters	No	Key-value pairs injected into the notebook via Papermill
CPU	No	CPU cores to request (default: 1)
Memory	No	Memory to request (default: 2Gi)
GPU	No	Number of GPUs to allocate (optional)

4. Click Submit

The job appears in the list with pending status and transitions to running within seconds.

Parameters

Parameters are injected into the notebook using Papermill’s parameter cell. To accept parameters in your notebook:

1. Create a cell and tag it with parameters in JupyterLab (View → Cell Toolbar → Tags)
2. Define your variables with defaults:

# Parameters cell (tagged with "parameters")
start_date = "2024-01-01"
end_date = "2024-12-31"
model_name = "llama3:8b"

When you submit the job with start_date=2024-06-01, Papermill injects a new cell overriding the default, then executes the full notebook.

Submitting via API

You can also submit batch jobs programmatically using a project API token:

curl -X POST https://<domain>/api/batch/jobs \
  -H "X-API-Key: <project-token>" \
  -H "Content-Type: application/json" \
  -d '{
    "notebook_path": "reports/monthly.ipynb",
    "parameters": {
      "start_date": "2024-06-01",
      "end_date": "2024-06-30"
    },
    "resources": {
      "cpu": "2",
      "memory": "4Gi"
    }
  }'

Resubmitting a Job

To re-run a job with the same (or modified) parameters, open the job detail view and click Resubmit. This creates a new job record with a new output path.

Resource Considerations

Batch jobs share cluster resources with Jupyter notebooks and MCP servers. Set appropriate CPU/memory requests to avoid starving other workloads. Monitor cluster capacity on the Dashboard.