Containers: Building Blocks for AI Applications

Containers provide the following main features:

Telemetry: When you run inference in a container, we automatically record inputs and outputs. This constructs a synthetic dataset that you can later use to optimize models for your specific use case.
Abstraction: When you initialize a container, you specify the initial_model, initial_system_prompt, and other inference configuration options. Running inference then infers these configurations from the container.
Optimization: As the container collects telemetry, we automatically optimize models on the synthetic dataset. Our optimization algorithms are heavily configurable and based on cutting edge research. Either use our default optimization pipelines or bring your own frameworks. Either way, we'll orchestrate experiments across GPUs. You retain full control over your data, the model weights, etc. We never train our own models on your data.
Routing: Once the optimized models begin outperforming the standard model, we automatically route inference requests to those models.

Creating your first container

Containers are initialized from the client library

import { Maniac } from "maniac-js"

const maniac = new Maniac()

const container = maniac.containers.create({
  label: "my-container",
  initial_model: "openai/gpt-5",
  initial_system_prompt: "You are a helpful assistant",
})

from maniac import Maniac

maniac = Maniac()

container = maniac.containers.create(
  label = "my-container",
  initial_model = "openai/gpt-5",
  intitial_system_prompt = "You are a helpful assistant"
)

Run inference with a container

import { Maniac } from "maniac-js"

const maniac = new Maniac()

const MyContainer = maniac.containers.get({label: "my-container"})

const response = maniac.chat.completions.create({
  container: MyContainer,
  messages: [{role: "user", content: "Hello!!"}]
})

from maniac import Maniac

maniac = Maniac()

my_container = maniac.containers.get(label = "my-container")

response = maniac.chat.completions.create(
  container = my_container,
  messages = [{role: "user", content: "Hello!!"}]
)

Your container will now collect telemetry on live production data.

Directly upload data to a container

In case you have data in another form (such as from another inference provider), you can upload that code directly to your container using our SDK.

import { Maniac } from "maniac-js"

const maniac = new Maniac()

const MyContainer = maniac.containers.get({label: "my-container"})

const response = maniac.chat.completions.register({
  container: MyContainer,
  dataset: [{
      input: {
          messages: [{
              role: "user",
              content: "hello"
          }]
      },
      output: {
          choices: [{
              message: {
                  role: "assistant",
                  content: "Hello! How can I help you today?"
              }
          }]
      },
  }]
})

from maniac import Maniac

maniac = Maniac()

my_container = maniac.containers.get(label = "my-container")

response = maniac.chat.completions.register(
  container = my_container,
  dataset = [{
      "input": {
          "messages": [
              {
                  "user": "user", 
                  "content": "Hello!!"
              }
          ]
      },
      "output": {
          "choices": [{
              "message": {
                  "role": "assistant",
                  "content": "Hello! How can I help you today?"
              }
          }]
      },

  }]
)

Optimization runs are triggered from the dashboard using the data collected through telemetry or directly uploaded. By default, we use LLM as a Judge. You can configure the exact pipeline your runs use when you create a container, or you can configure it on a per run basis from the dashboard.

Configuring LLM as a Judge evals

import { Maniac } from "maniac-js"

const maniac = new Maniac()

const MyContainer = maniac.containers.evals.create({
  label: "my-container",
  evals: {
      judge: {
          prompt: “Is response A better than response B at …”,
          model: "gpt-5"
      }
  }
})

from maniac import Maniac

maniac = Maniac()

my_container = maniac.containers.evals.create(
  label = "my-container",
  evals = {
      "judge": {
          "prompt": “Is response A better than response B at …”,
          "model": "gpt-5"
      }
  }
)

Adding a default optimization pipeline to your container

This example only includes SFT. By default, our optimization includes SFT, GEPA, and GRPO.

import { Maniac } from "maniac-js"

const maniac = new Maniac()

const MyContainer = maniac.containers.stages.create({
  label: "my-container",
  optimization_config: {
      stages: [{
          key: "sft",
          config: {
              lora_dim: [8, 16, 32, 64],
              lora_alpha: [2, 4, 8, 16, 32, 64],
              lora_targets: ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj", "lm_head"]
          }
      }],
      
  }
})

from maniac import Maniac

maniac = Maniac()

my_container = maniac.containers.stages.create(
  label = "my-container",
  optimization_config = {
      "stages": [{
          "key": "sft",
          "config": {
              "lora_dim": [8, 16, 32, 64],
              "lora_alpha": [2, 4, 8, 16, 32, 64],
              "lora_targets": ["q_proj", "k_proj", "v_proj", "o_proj", 'gate_proj', "up_proj", "down_proj", "lm_head"]
          }
      }],
      
  }
)

Containers