# Helix ML

> AI agent orchestration platform. Build AI agents with skills, knowledge, sandboxes, and integrations.

## Docs

- [Getting Started](https://helix.ml/docs): Documentation for Helix ML.
- [Projects & Spec Tasks](https://helix.ml/docs/projects): Projects and spec tasks are the core of Helix ML. This is how work is planned and executed.
- [Agents](https://helix.ml/docs/agents): Build AI agents with skills, knowledge, and integrations.
- [Sandboxes](https://helix.ml/docs/sandboxes): Isolated desktop environments for AI coding agents.
- [On-Prem deployment](https://helix.ml/docs/on-prem): Run Helix on your own infrastructure.

---

## What is Helix ML?

Helix is a batteries-included platform for building, running and managing AI agents. It can do everything
from working as a simple self-hosted ChatGPT alternative to working as a controlplane for your fleet of agents.

## Features

- Kanban style board to manage your project backlog, automatically picking up new work 
- Agent sandboxes with full desktop
- Integration with all major LLM providers (Anthropic, OpenAI, Google, etc.)
- Integration with all majors source control providers (GitHub, GitLab, Bitbucket)
- Full planning -> implementation -> review -> pull request -> merge workflow

## Use cases

- Accelerate development of any project
- Internal knowledge base for your team
- Automated customer support
- Closed self-improvement loop through support -> tickets -> spec tasks -> fixes

---

## Projects

Projects are spec-driven project management boards that organize how work gets planned and executed by AI agents. Each project acts as a workspace where you define what needs to be built, track progress across tasks, and review the results — all through a structured, approval-based workflow.

![Projects board showing spec tasks organized by status](/images/projects.png)

Project provide:
- Ability to collaborate with your team
- Manage spec tasks
- Setup Optimus, a Helix agent to help you plan and manage your project
- Setup recurring tasks such as reviewing your GitHub issues and preparing new spec tasks to work on

## Spec Tasks

A spec task is a discrete unit of work within a project. You describe what needs to happen — a feature to build, a bug to fix, a change to make — and an agent prepares an execution plan. Once you approve the plan, the agent carries out the work inside an isolated sandbox environment, producing a pull request for human review.

Spec tasks move through a clear lifecycle:

- **Define**: Write a spec describing the desired outcome
- **Plan**: An agent analyzes the codebase and proposes an execution plan
- **Approve** Review the plan and approve it before any work begins
- **Implementation**: The agent works inside a sandbox desktop with full access to browser, terminal, and file system
- **Review**: A pull request is created for human review and merge

![Spec task detail view showing the execution plan and status](/images/spec_tasks.png)

This human-in-the-loop approach keeps you in control. Agents never push code or merge changes without your explicit approval. You set the direction, agents handle the execution, and you review the output.


## Jump in at any step

You can always participate in the process at any step. You can review the plan, approve it, or even edit the spec to make changes. You can also review the implementation and make changes to the spec based on the implementation.

![Collaborate on the spec task](/images/spec_task_detail.png)

Interact with this desktop view directly, use the browser, IDE and terminals inside the sandbox.

---

Helix agents combine language models with skills, knowledge bases, and external integrations. An agent receives a user message, decides which tools to use, executes them, and synthesizes a response.

## How Agents Work

When a user sends a message, the agent:

1. Loads the assistant configuration (model, system prompt, skills)
2. Queries relevant knowledge bases for context
3. Calls the LLM with available tools exposed as OpenAI function calls
4. Executes any tools the LLM decides to use (in parallel if multiple)
5. Adds results to the conversation and repeats until done

Agents iterate up to a configurable maximum (default 10) before stopping to prevent runaway loops.

## Skills

Skills are capabilities you attach to an agent. Each skill exposes one or more tools the LLM can invoke.

### Built-in Skills

| Skill | Description |
|-------|-------------|
| **Web Search** | Search the internet and browse results |
| **Browser** | Open URLs and extract content as markdown |
| **Knowledge** | Query RAG sources for document retrieval |
| **API Calling** | Execute REST APIs defined with OpenAPI schemas |
| **Calculator** | Perform mathematical operations |
| **Memory** | Persist information across sessions |
| **MCP** | Connect to Model Context Protocol servers |

### Enabling Skills

Skills are enabled in the assistant configuration:

```yaml
assistants:
- name: Research Assistant
  model: qwen3:8b
  system_prompt: You help users research topics.

  web_search:
    enabled: true
    max_results: 10

  browser:
    enabled: true
    markdown_post_processing: true

  calculator:
    enabled: true
```

## Knowledge

Knowledge bases provide RAG (retrieval augmented generation) for your agents. When attached to an assistant, knowledge becomes a tool the agent can query.

```yaml
assistants:
- model: qwen3:8b
  knowledge:
  - name: product-docs
    description: Product documentation and API reference
    source:
      web:
        urls:
        - https://docs.example.com/
        crawler:
          enabled: true
    rag_settings:
      results_count: 8
      chunk_size: 2048
```

Knowledge sources can be:

- **Web URLs** with optional crawling
- **Uploaded files** (PDF, DOCX, PPTX)
- **Inline content** (text directly in config)
- **S3 buckets**
- **Google Drive**

Knowledge can refresh on a schedule:

```yaml
knowledge:
- name: news-feed
  refresh_enabled: true
  refresh_schedule: "0 */6 * * *"  # Every 6 hours
  source:
    web:
      urls:
      - https://news.example.com/feed
```

## API Integrations

Connect agents to REST APIs using OpenAPI schemas. The agent can call endpoints based on user requests.

```yaml
assistants:
- model: qwen3:8b
  apis:
  - name: Weather API
    description: Get current weather and forecasts
    url: https://api.weather.example.com
    schema: |
      openapi: 3.0.0
      info:
        title: Weather API
        version: 1.0.0
      paths:
        /current:
          get:
            summary: Get current weather
            parameters:
            - name: city
              in: query
              required: true
              schema:
                type: string
```

For APIs requiring authentication, use OAuth providers:

```yaml
apis:
- name: GitHub
  url: https://api.github.com
  schema: ./github-openapi.yaml
  oauth_provider: github
  oauth_scopes:
  - repo
  - read:user
```

## MCP Servers

Model Context Protocol (MCP) lets agents connect to external tool servers. Any MCP-compatible server exposes its tools to your agent.

```yaml
assistants:
- model: qwen3:8b
  mcps:
  - name: Database Tools
    description: Query and manage database records
    url: https://mcp.example.com/tools
    headers:
      Authorization: Bearer ${MCP_API_KEY}
```

MCP servers can also use OAuth for authentication:

```yaml
mcps:
- name: Salesforce
  url: https://mcp.salesforce.example.com
  oauth_provider: salesforce
  oauth_scopes:
  - api
  - refresh_token
```

## Triggers

Agents can be triggered from various sources beyond the chat UI:

### Discord

```yaml
triggers:
- discord:
    server_name: My Server
    channel_names:
    - support
    - general
```

### Slack

```yaml
triggers:
- slack:
    bot_token: ${SLACK_BOT_TOKEN}
    channels:
    - C01234567
```

### Scheduled (Cron)

```yaml
triggers:
- cron:
    schedule: "0 9 * * 1-5"  # 9am weekdays
    input: Generate the daily report
```

### Webhooks

Agents expose webhook endpoints that can be called by external systems to trigger conversations.

## Secrets

Store sensitive values as secrets rather than hardcoding them:

```yaml
assistants:
- model: qwen3:8b
  apis:
  - name: Internal API
    url: https://api.internal.example.com
    headers:
      X-API-Key: ${INTERNAL_API_KEY}
```

Secrets are configured in the Helix UI under the app settings, or via the `secrets` field in the app configuration.

## Agent Types

Helix supports different agent execution modes:

| Type | Description |
|------|-------------|
| **helix_basic** | Simple chat assistant without tool calling |
| **helix_agent** | Standard agent with skills and tool calling |
| **zed_external** | Runs in isolated desktop environment with Zed IDE (Helix Code) |

Set the agent type in the assistant configuration:

```yaml
assistants:
- model: qwen3:8b
  agent_type: helix_agent
  max_iterations: 20
```

For Helix Code sandboxes, use `zed_external`:

```yaml
assistants:
- model: claude-sonnet-4-20250514
  agent_type: zed_external
  generation_model_provider: anthropic
```

## Example: Full Agent Configuration

```yaml
name: Customer Support Agent
description: Handles customer inquiries with access to docs and CRM

assistants:
- name: Support
  model: qwen3:8b
  system_prompt: |
    You are a helpful customer support agent. Use the knowledge base
    to answer product questions and the CRM API to look up customer
    information when needed.

  temperature: 0.3
  max_iterations: 15

  knowledge:
  - name: help-docs
    description: Product documentation and FAQs
    source:
      web:
        urls:
        - https://help.example.com/
        crawler:
          enabled: true
          max_depth: 3

  apis:
  - name: CRM
    description: Look up customer records and order history
    url: https://crm.example.com/api
    schema: ./crm-openapi.yaml
    oauth_provider: internal-sso

  web_search:
    enabled: true

triggers:
- slack:
    bot_token: ${SLACK_BOT_TOKEN}
    channels:
    - support-requests
- cron:
    schedule: "0 8 * * 1"
    input: Generate weekly support summary
```

---

Helix Code provides isolated desktop environments where AI coding agents work autonomously. Each agent runs in its own containerized Linux desktop with a full IDE, and you watch their progress via real-time video streaming in your browser.

See also the [Coding Agents overview](/docs/agents) for how sandboxes fit into the broader agent workflow.

## How It Works

Each agent session receives its own containerized environment with:

- **Isolated Linux desktop** running on Wayland (Sway) or X11 (GNOME)
- **Zed IDE** - a Rust-based editor with native GPU rendering
- **AI agent** - powered by Claude, Qwen, or other LLMs via the Helix proxy
- **WebSocket streaming** - H.264 video streamed to your browser over standard HTTPS

The Helix control plane manages orchestration, knowledge sources, and conversation history through the UI.

## Architecture

Helix Code uses a three-tier isolation model for security and multi-tenancy:

```
+--------------------------------------------+
|                HOST MACHINE                |
|  +--------------------------------------+  |
|  |         Helix Control Plane          |  |
|  |   (API Server, Frontend, Postgres)   |  |
|  +-----------------+--------------------+  |
|                     |                      |
|  +-----------------v--------------------+  |
|  |       HELIX-SANDBOX CONTAINER        |  |
|  |          (Docker-in-Docker)          |  |
|  |  +--------------------------------+  |  |
|  |  |             HYDRA              |  |  |
|  |  |    (Multi-Tenant Isolation)    |  |  |
|  |  |   Isolated dockerd + network   |  |  |
|  |  +-------+----------------+-------+  |  |
|  |          |                |          |  |
|  |    +-----v----+     +-----v----+     |  |
|  |    | Ubuntu   |     | Ubuntu   |     |  |
|  |    | Session  |     | Session  | ... |  |
|  |    | with AI  |     | with AI  |     |  |
|  |    +----------+     +----------+     |  |
|  +--------------------------------------+  |
|                     |                      |
|           WebSocket Video Stream           |
|            (H.264 over HTTPS)              |
|                     v                      |
|                  Browser                   |
+--------------------------------------------+
```

**Hydra** manages multi-tenant isolation within the sandbox. Each concurrent session gets its own Docker daemon with isolated networking (separate bridge networks with non-overlapping subnets). Sessions cannot see each other's containers or network traffic.

## Features

### AI Agent Integration

Agents connect to Zed IDE via WebSocket and can use any LLM provider configured in Helix:

- Anthropic Claude (Sonnet, Opus)
- OpenAI GPT-4
- Local models via Helix runners (Qwen, Llama)
- Any OpenAI-compatible API

### Contextual Awareness

Integrate external knowledge sources into agent environments:

- PDFs and documents
- Jira and Confluence
- MCP (Model Context Protocol) servers
- Custom knowledge bases

### Knowledge Aggregation

Conversation histories across all coding sessions are searchable via RAG, enabling knowledge sharing across your team.

### Spec-Driven Workflows

Use Kanban boards to manage agent task specifications before implementation, ensuring agents work on well-defined tasks.

## Installation

### Requirements

- Linux x86_64 (Ubuntu 22.04+ recommended)
- Docker
- (Optional) NVIDIA GPU for hardware video encoding


macOS and Windows are not supported for running sandboxes. The sandbox requires Linux with Docker.


### Quick Start

Run the Helix installer:

```bash
curl -sL -O https://get.helixml.tech/install.sh
chmod +x install.sh
sudo ./install.sh
```

The installer will detect your hardware and configure the sandbox automatically. On systems with NVIDIA GPUs, hardware H.264 encoding provides 60 FPS streaming with minimal CPU overhead.

### Configure Inference Providers

After installation, configure AI providers through **Account** → **AI Providers**:

- Anthropic (Claude)
- OpenAI
- Together AI
- Any OpenAI-compatible API

Or connect a Helix runner with local models for complete data privacy.

## Desktop Environments

Three desktop options are available:

| Environment | Display Server | Best For |
|-------------|----------------|----------|
| **Sway** | Native Wayland | Lightweight, fast startup, lowest resource usage |
| **Ubuntu** | GNOME (Xwayland) | Full desktop experience, broader app compatibility |
| **Zorin** | GNOME (Xwayland) | User-friendly interface |

Each desktop includes:
- Zed IDE with AI agent integration
- Firefox browser
- Docker CLI
- Git

## Use Cases

### Autonomous Development

Agents work asynchronously on tasks while you focus on other work. Review results when ready rather than waiting for completions.

### Fleet Management

Manage multiple concurrent agent tasks across your team. Track progress, review outputs, and coordinate work through the Kanban board.

### Cloud Development Environments

Persistent development environments that survive disconnections. Pick up where you left off from any device with a browser.

### Security

Each agent runs in its own isolated container with:
- Separate filesystem (no access to host)
- Isolated network (no cross-session traffic)
- Dedicated Docker daemon (via Hydra)

Code execution is contained within the session. Even if an agent runs malicious commands, the blast radius is limited to that session's container.

## Getting Started

1. Install Helix with the installer script
2. Access the Helix UI at your deployment URL
3. Go to **Helix Code** and create a new session
4. Send a message to start the agent working
5. Watch the agent's desktop in real-time via the video stream

---

## Local Setup

To install Helix on your own Linux server with Nvidia/AMD GPU support, run the following command:

```bash
curl -sL -O https://get.helixml.tech/install.sh && bash install.sh
```

## Kubernetes Setup

To install Helix as a chart in your Kubernetes cluster you will first need to create secrets with the following contents:

```
# Create namespace and secrets
kubectl create namespace helix
kubectl config set-context --current --namespace=helix

# Create secrets with generated secure values
kubectl create secret generic postgresql-auth-secret \
  --from-literal=postgres-password="zuriOfPBBpOaUUDff32FGSAz" \
  --from-literal=username="helix" \
  --from-literal=password="" \
  --from-literal=database="helix"

kubectl create secret generic pgvector-auth-secret \
  --from-literal=username="postgres" \
  --from-literal=password="" \
  --from-literal=database="postgres"

kubectl create secret generic helix-pgvector-creds \
  --from-literal=dsn="postgresql://postgres:@my-helix-controlplane-pgvector:5432/postgres"

kubectl create secret generic helix-license \
  --from-literal=license-key="eyJkYXRhIjoie1wiaWRcIjpcImxpY18zNlpBNllzRGRjZlVDRlVCRzdadDU4TEF5TTVcIixcIm9yZ2FuaXphdGlvblwiOlwicXF3ZXFcIixcInZhbGlkXCI6dHJ1ZSxcImlzc3VlZFwiOlwiMjAyNS0xMi0wOFQxNDoxOTowNC4wMjM5MzQ0ODJaXCIsXCJ2YWxpZF91bnRpbFwiOlwiMjAyNi0xMi0wOFQxNDoxOTowNC4wMjM3OTA4NDRaXCIsXCJmZWF0dXJlc1wiOntcInVzZXJzXCI6dHJ1ZX0sXCJsaW1pdHNcIjp7XCJ1c2Vyc1wiOjAsXCJtYWNoaW5lc1wiOjB9fSIsInNpZ25hdHVyZSI6IkFtamZqR0EzL2p5d1N5bEpDa0trM0x4eDRDWG5malpjTFVjcFBiTGdXZ2l1d0JKRTU2M2JrUC9Ea21pL1VIZEtNckdmb25OT2R1V2hrOHJ5TmNZR1RBPT0ifQ=="

kubectl create secret generic helix-runner-secrets \
  --from-literal=api-token="CHANGE-ME"
```

Now, save this as values.yaml for the chart:

```yaml
# Production-ready configuration for helix-controlplane
# For more information: https://docs.helixml.tech/helix/private-deployment/manual-install/kubernetes/

global:
  serverUrl: http://localhost:8080

image:
  tag: ""

searxng:
  enabled: true

chrome:
  enabled: true
pgvector:
  enabled: true
  auth:
    existingSecret: "pgvector-auth-secret"
    usernameKey: "username"
    passwordKey: "password"
    databaseKey: "database"
  persistence:
    enabled: true
    size: 50Gi
    storageClass: ""
    annotations: {}
    accessModes:
      - ReadWriteOnce

controlplane:
  licenseKeyExistingSecret: "helix-license"
  licenseKeyExistingSecretKey: "license-key"

  runnerTokenExistingSecret: "helix-runner-secrets"
  runnerTokenExistingSecretKey: "api-token"

  admin:
    userSource: "env"
    userIds: "all"
  
  haystack:
    enabled: true
    existingSecret: "helix-pgvector-creds"
    existingSecretDsnKey: "dsn"
    embeddingsModel: "MrLight/dse-qwen2-2b-mrl-v1"
    embeddingsDim: "1536"
    chunkSize: "1000"
    chunkOverlap: "50"
    chunkUnit: "word"
  
  rag:
    defaultProvider: "haystack"
    embeddingsProvider: "helix"
  
  inference:
    defaultProvider: "helix"
  
  fineTuning:
    defaultProvider: "helix"
persistence:
  enabled: true
  size: 100Gi
  storageClass: ""
  accessModes:
    - ReadWriteOnce

volumes:
  - name: data

postgresql:
  enabled: true
  auth:
    existingSecret: "postgresql-auth-secret"
    postgresPasswordKey: "postgres-password"
    usernameKey: "username"
    passwordKey: "password"
    databaseKey: "database"
  architecture: standalone

tika:
  enabled: false

typesense:
  enabled: false
```

Now we are ready to deploy controlplane:

```bash
# Add Helix Helm repository
helm repo add helix https://charts.helixml.tech
helm repo update
# Install Control Plane with your generated values.yaml and secrets
export HELIX_VERSION="latest"
helm upgrade --install my-helix-controlplane helix/helix-controlplane \
  -f values.yaml \
  --set image.tag="${HELIX_VERSION}"
```

Now install the runner:

```bash
# Install Runner (after control plane is running and secrets are created)
export HELIX_VERSION="latest"
helm upgrade --install my-helix-runner helix/helix-runner \
  --set runner.host="http://my-helix-controlplane:8080" \
  --set runner.tokenExistingSecret="helix-runner-secrets" \
  --set runner.tokenExistingSecretKey="api-token" \
  --set replicaCount=1 \
  --set image.tag="${HELIX_VERSION}-small"
```

And to access it you can use the following command:

```bash
kubectl port-forward svc/helix-helix-controlplane 8080:80
```