Batch Jobs

The Batch API lets you run large numbers of requests asynchronously without real-time responses. It is compatible with the OpenAI Batch API: same JSONL format, Files API for uploads, and batch create/retrieve/cancel/list endpoints. Use {apiBaseUrl} and your $8080 API key in place of OpenAI’s base URL and key.

Overview

1. Prepare a JSONL file — One line per request, each with custom_id, method, url, and body.
2. Upload the file — POST /v1/files with purpose: "batch".
3. Create the batch — POST /v1/batches with input_file_id, endpoint, and completion_window.
4. Check status — GET /v1/batches/{batch_id} until status is completed, failed, or cancelled.
5. Retrieve results — Download the output (and optionally error) file via GET /v1/files/{file_id}/content.

Batches complete within the chosen completion window (e.g. 24 hours). Input files can contain up to 50,000 requests and be up to 200 MB.

1. Prepare your JSONL file

Each line must be a single JSON object with:

Field	Type	Description
custom_id	string	Your unique ID for this request (used to match results).
method	string	HTTP method, e.g. "POST".
url	string	API path, e.g. "/v1/chat/completions".
body	object	Request body for that endpoint (same as the underlying API).

All requests in one file must target the same endpoint. The body must match what the endpoint expects (e.g. model, messages for chat completions).

Example: chat completions input file

Save as batch_input.jsonl:

{"custom_id": "req-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "8080/taalas/llama3.1-8b-instruct", "messages": [{"role": "user", "content": "Say hello in one word."}], "max_tokens": 50}}
{"custom_id": "req-2", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "8080/taalas/llama3.1-8b-instruct", "messages": [{"role": "user", "content": "What is 2+2?"}], "max_tokens": 50}}
{"custom_id": "req-3", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "8080/taalas/llama3.1-8b-instruct", "messages": [{"role": "user", "content": "Name a color."}], "max_tokens": 50}}

Each line is one request. Use custom_id later to map output lines back to your inputs (output order may not match input order).

2. Upload the file (Files API)

Upload the JSONL file with purpose batch so it can be used as batch input.

cURL:

curl https://api.8080.io/v1/files \
  -H "Authorization: Bearer $_8080_API_KEY" \
  -F purpose="batch" \
  -F file="@batch_input.jsonl"

Python:

import os
import requests

API_KEY = os.environ.get("_8080_API_KEY")
BASE = "https://api.8080.io"

with open("batch_input.jsonl", "rb") as f:
    r = requests.post(
        f"{BASE}/v1/files",
        headers={"Authorization": f"Bearer {API_KEY}"},
        files={"file": ("batch_input.jsonl", f, "application/jsonl")},
        data={"purpose": "batch"},
    )
r.raise_for_status()
file_id = r.json()["id"]
print("Uploaded file ID:", file_id)

The response includes an id (e.g. file-abc123). Use this as input_file_id when creating the batch.

3. Create the batch

Request: POST /v1/batches

Parameter	Type	Required	Description
input_file_id	string	Yes	File ID from the upload step.
endpoint	string	Yes	Endpoint for all requests in the file, e.g. "/v1/chat/completions".
completion_window	string	Yes	Time window to complete the batch; e.g. "24h" if supported.
metadata	object	No	Key-value pairs (e.g. for labeling).
output_expires_after	object	No	Expiration for output/error files (e.g. {"anchor": "created_at", "seconds": 86400}).

cURL:

curl https://api.8080.io/v1/batches \
  -H "Authorization: Bearer $_8080_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input_file_id": "file-abc123",
    "endpoint": "/v1/chat/completions",
    "completion_window": "24h"
  }'

Python:

r = requests.post(
    f"{BASE}/v1/batches",
    headers={
        "Authorization": f"Bearer {API_KEY}",
        "Content-Type": "application/json",
    },
    json={
        "input_file_id": file_id,
        "endpoint": "/v1/chat/completions",
        "completion_window": "24h",
        "metadata": {"job": "nightly-eval"},
    },
)
r.raise_for_status()
batch = r.json()
batch_id = batch["id"]
print("Batch ID:", batch_id)

The response is a batch object with id, status (e.g. validating), input_file_id, output_file_id, error_file_id (often null until the batch progresses), and timestamps.

4. Check batch status

Request: GET /v1/batches/{batch_id}

Poll this until status is a terminal state.

cURL:

curl https://api.8080.io/v1/batches/batch_abc123 \
  -H "Authorization: Bearer $_8080_API_KEY"

Python:

r = requests.get(
    f"{BASE}/v1/batches/{batch_id}",
    headers={"Authorization": f"Bearer {API_KEY}"},
)
r.raise_for_status()
batch = r.json()
print("Status:", batch["status"])
print("Request counts:", batch.get("request_counts"))

Status values

Status	Description
validating	Input file is being validated.
failed	Validation failed; see errors on the batch.
in_progress	Batch is running.
finalizing	Batch finished; results are being prepared.
completed	Done; use output_file_id (and optionally error_file_id) to download.
expired	Did not finish within the completion window.
cancelling	Cancel requested; waiting for in-flight work.
cancelled	Batch was cancelled.

When status is completed, output_file_id points to a JSONL file of successful results, and error_file_id (if set) points to a JSONL file of failed requests.

5. Retrieve results and errors

Request: GET /v1/files/{file_id}/content

Use the batch’s output_file_id and error_file_id from the completed batch object.

cURL:

# Output (successful requests)
curl https://api.8080.io/v1/files/FILE_ID/content \
  -H "Authorization: Bearer $_8080_API_KEY" \
  -o batch_output.jsonl

# Errors (failed requests)
curl https://api.8080.io/v1/files/ERROR_FILE_ID/content \
  -H "Authorization: Bearer $_8080_API_KEY" \
  -o batch_errors.jsonl

Python:

# After batch status is "completed"
output_file_id = batch["output_file_id"]
error_file_id = batch.get("error_file_id")

r = requests.get(
    f"{BASE}/v1/files/{output_file_id}/content",
    headers={"Authorization": f"Bearer {API_KEY}"},
)
r.raise_for_status()
with open("batch_output.jsonl", "w") as f:
    f.write(r.text)

if error_file_id:
    r = requests.get(
        f"{BASE}/v1/files/{error_file_id}/content",
        headers={"Authorization": f"Bearer {API_KEY}"},
    )
    r.raise_for_status()
    with open("batch_errors.jsonl", "w") as f:
        f.write(r.text)

Output file format (JSONL)

One line per successful request. Each line is a JSON object with:

Field	Description
id	Batch request ID.
custom_id	The custom_id from your input line.
response	Object with status_code, request_id, and body (the API response body).
error	null for success lines.

Example output line (chat completions):

{"id": "batch_req_abc", "custom_id": "req-1", "response": {"status_code": 200, "request_id": "req_xyz", "body": {"id": "chatcmpl-...", "object": "chat.completion", "choices": [{"index": 0, "message": {"role": "assistant", "content": "Hello."}, "finish_reason": "stop"}], "usage": {"prompt_tokens": 12, "completion_tokens": 1, "total_tokens": 13}}}, "error": null}

Match results to inputs using custom_id; do not rely on line order.

Error file format (JSONL)

One line per failed request. Each line has id, custom_id, response: null, and error with code and message, e.g.:

{"id": "batch_req_456", "custom_id": "req-2", "response": null, "error": {"code": "invalid_request", "message": "Invalid model."}}

Example: full Python script

import os
import time
import requests

API_KEY = os.environ.get("_8080_API_KEY")
BASE = "https://api.8080.io"
headers = {"Authorization": f"Bearer {API_KEY}"}

# 1. Upload
with open("batch_input.jsonl", "rb") as f:
    r = requests.post(f"{BASE}/v1/files", headers=headers, files={"file": f}, data={"purpose": "batch"})
r.raise_for_status()
file_id = r.json()["id"]

# 2. Create batch
r = requests.post(
    f"{BASE}/v1/batches",
    headers={**headers, "Content-Type": "application/json"},
    json={"input_file_id": file_id, "endpoint": "/v1/chat/completions", "completion_window": "24h"},
)
r.raise_for_status()
batch_id = r.json()["id"]

# 3. Poll until completed
while True:
    r = requests.get(f"{BASE}/v1/batches/{batch_id}", headers=headers)
    r.raise_for_status()
    batch = r.json()
    status = batch["status"]
    print("Status:", status)
    if status in ("completed", "failed", "expired", "cancelled"):
        break
    time.sleep(30)

# 4. Download output
if batch.get("output_file_id"):
    r = requests.get(f"{BASE}/v1/files/{batch['output_file_id']}/content", headers=headers)
    r.raise_for_status()
    with open("batch_output.jsonl", "w") as f:
        f.write(r.text)
    print("Wrote batch_output.jsonl")

List and cancel batches

List batches: GET /v1/batches (optional query: limit, after for pagination).

curl https://api.8080.io/v1/batches?limit=20 \
  -H "Authorization: Bearer $_8080_API_KEY"

Cancel a batch: POST /v1/batches/{batch_id}/cancel

curl https://api.8080.io/v1/batches/batch_abc123/cancel \
  -H "Authorization: Bearer $_8080_API_KEY" \
  -X POST

After cancelling, status moves to cancelling then cancelled (may take a few minutes).