Secrets

Let sandboxed code deploy, publish, push, and call your APIs without holding your long-lived keys.

Anything in a Sandbox can be read by the code running in it. So the question is not how to hide a key from that code, but how to avoid putting the key there at all.

Where the secret lives

TechniqueSandbox code can read itStored with the runLives until
ToolNoTool arguments and resultYour server only
Tool that runs a command with a tokenOne commandExit code and the output you returnThe token expires or is revoked
Per-command envThat commandNames only, values [redacted]The command, and anything it writes
Credential helper or *_BASE_URLA short-lived tokenThe tool resultThe token expires
Workspace secretsYes, for the Sandbox's whole lifeNames onlyThe Sandbox and its snapshots

Pick a technique

You want toUse
Open a PR, query a database, call a billing API, sign somethingA tool. The key never enters.
git push, gh, or another CLIA short-lived token, run from a tool or as per-command env
npm publish or a deploy CLIA short-lived, single-resource token as per-command env
Cloud SDKsShort-lived session credentials from your provider, as per-command env
Call an LLMA *_BASE_URL route on your server, else a workspace secret with a spend cap

Decide in this order:

  1. Can a tool do the work? Use it.
  2. Can the credential be short-lived and scoped to one resource? Mint it per use.
  3. Only then inject a long-lived key, with a workspace secret.

Let a tool do it

A tool runs in your function, with your credentials. The Sandbox calls it by name and gets the result.

import { tool } from "inngest/experimental";
import { z } from "zod";

const openPr = tool.step({
  description: "Open a PR for a branch",
  input: z.object({
    branch: z.string().regex(/^[\w./-]+$/),
    title: z.string().max(120),
  }),
  output: z.object({ url: z.string() }),
  run: async ({ branch, title }) => {
    const pr = await octokit.rest.pulls.create({
      owner: "acme",
      repo: "box",
      head: branch,
      base: "main",
      title,
    });

    return { url: pr.data.html_url };
  },
});

await sb.commands.run("agent", "node agent.js", { tools: { openPr } });
  • input is validated before run starts. Arguments come from untrusted code, so constrain them.
  • The result is stored with the run. Return ids and status, never tokens.
  • The code can only ask for what the tool allows. Its blast radius is the tool's own surface.

Pass a secret to one command v4.26.0+

environment on a step.sandbox command, or env on $, overrides the Sandbox's environment for that command. The SDK records the step with every value replaced by [redacted], then restores the real values before running it. A command run with tools records no environment at all.

const publish$ = sb.$({ env: { NPM_TOKEN: process.env.NPM_TOKEN! } });

await publish$.sh`pnpm publish`;

await sb.commands.run("publish", ["pnpm", "publish"], {
  environment: { NPM_TOKEN: process.env.NPM_TOKEN! },
});
  • The command can read and print the value. Redaction hides it from the trace, not from the code.
  • Read the value from your app's environment. If it comes from a step's return value, it is stored with the run.
  • Only environment values are redacted. The command text, step ID, and output are not.
  • Values are literal strings. A selected workspace secret cannot be passed by name per command, so pair per-command values with a short-lived token.

The same redaction applies to step.sandbox.create and processes.start. A Create environment is also stored as Sandbox configuration, so never put a secret there.

Mint, use, and revoke a token

A token that expires turns a leak into a small problem. The host keeps one credential that can mint tokens, creates a token for one resource and a few minutes, and revokes it when the command ends.

Do all three in one place. A token returned from step.run is stored with the run, so mint and use it inside one tool.

Run the command from a tool

A tool can mint a token, run the command in the Sandbox with it as environment, and revoke it in finally. The token exists only in that tool step's memory.

const sb = await step.sandbox.create("create", {
  name: `pr-${event.data.jobId}`,
  vcpu: 2,
  memoryMb: 2048,
});

const createPr = tool.step({
  input: z.object({ branch: z.string().regex(/^[\w./-]+$/) }),
  output: z.object({ ok: z.boolean(), tail: z.string() }),
  run: async ({ branch }) => {
    const { data } = await appOctokit.rest.apps.createInstallationAccessToken({
      installation_id: installationId,
      repositories: ["box"],
      permissions: { pull_requests: "write", contents: "read" },
    });

    try {
      const sandbox = await inngest.sandboxes.get(sb.id);
      const result = await sandbox!.commands.run(
        ["gh", "pr", "create", "--fill", "--head", branch],
        { cwd: "/work/box", environment: { GH_TOKEN: data.token }, timeout: "2m" },
      );

      return { ok: result.exitCode === 0, tail: result.stderr.slice(-500) };
    } finally {
      await new Octokit({ auth: data.token }).rest.apps.revokeInstallationAccessToken();
    }
  },
});

await sb.commands.run("agent", "node agent.js", {
  tools: { createPr },
  toolOptions: { toolTimeout: "5m" },
});
  • The code cannot pick the command. It supplies a validated branch name and nothing else.
  • The token is scoped. One repository, two permissions, and GitHub expires it after an hour if revoke never runs.
  • Return an exit code and a trimmed tail. Never return raw output.
  • Use the Sandbox your run created. Do not look one up from untrusted input.
  • Raise toolTimeout. It defaults to 30 seconds.
  • Make the work safe to retry. A tool that fails runs again. Tools receive an idempotencyKey as their second argument; pass it to APIs that deduplicate writes. See Tools.

inngest.sandboxes is the direct client. It has no step replay, which is why the token stays in the tool's memory.

Call home for a credential

Two patterns let the Sandbox use a credential without holding a long-lived one.

A credential helper asks a tool for a token on demand. Run the command with the tool, and the helper, a child of that command, can call it.

const gitToken = tool.step({
  output: z.object({ password: z.string() }),
  run: async () => {
    const { data } = await appOctokit.rest.apps.createInstallationAccessToken({
      installation_id: installationId,
      repositories: ["box"],
      permissions: { contents: "write" },
    });

    return { password: data.token };
  },
});

await sb.commands.run(
  "push",
  ["git", "-c", "credential.helper=!node /opt/git-credential.js", "push", "origin", "HEAD"],
  { tools: { gitToken } },
);
// /opt/git-credential.js
const { tools } = require("inngest_tools");

(async () => {
  if (process.argv[2] !== "get") {
    return;
  }

  const { password } = await tools.gitToken();

  console.log("username=x-access-token");
  console.log(`password=${password}`);
})();

Git also calls the helper with store and erase, so the script answers only get. Put the helper in your image or write it once with files.write; it holds no secret. The result of a tool call is stored with the run, so keep this token short-lived and scoped.

A *_BASE_URL points an SDK at a route on your server. The route adds the real key and forwards the request, so the key never enters the Sandbox.

await sb.$({
  env: {
    ANTHROPIC_BASE_URL: "https://app.example.com/llm",
    ANTHROPIC_API_KEY: runToken,
  },
}).sh`node agent.js`;

runToken is a per-run token that only your route accepts. The Sandbox can use the route while it is reachable, so expire the token, allowlist what it can call, and rate limit it.

Select workspace secrets

For keys that are cheap to rotate and cheap to leak, such as a capped LLM key or a read-only token, select existing workspace secrets by exact name when you create a fresh Sandbox. Inngest makes each value available as an environment variable with that name for the Sandbox's whole life.

Secret selection is in the open beta, and its API can change before launch.

Note Secret injection is not output redaction. Any code in the Sandbox can read selected values. A value printed to stdout, stderr, logs, or files can be returned by the Sandbox APIs.

Create the secrets in the same Inngest workspace that will create the Sandbox. The public Sandbox REST API and JavaScript SDK select existing secrets; they do not create, list, rotate, or archive secret values. Names are case-sensitive, so save a secret under the variable name your application expects, such as OPENAI_API_KEY.

Select secrets at Create

Direct client:

const sandbox = await inngest.sandboxes.create({
  name: `agent-${crypto.randomUUID()}`,
  vcpu: 1,
  memoryMb: 1024,
  environment: {
    APP_ENV: "beta",
  },
  secrets: ["OPENAI_API_KEY", "GITHUB_TOKEN"],
});

Middleware-backed client:

const sandbox = await step.sandbox.create("create-sandbox", {
  name: `agent-${event.data.jobId}`,
  vcpu: 1,
  memoryMb: 1024,
  secrets: ["OPENAI_API_KEY"],
});

REST:

{
  "name": "agent-job-42",
  "vcpu": 1,
  "memoryMb": 1024,
  "environment": {
    "APP_ENV": "beta"
  },
  "secrets": ["OPENAI_API_KEY", "GITHUB_TOKEN"]
}

The SDK and API send only the names. Values are injected under those same names and inherited by commands and managed processes. Selection is supported only for a fresh Sandbox: a snapshot clone restores the environment captured by its snapshot and cannot select new secrets.

Environment precedence

Environment is layered in this order:

  1. guest defaults;
  2. literal Sandbox environment values and selected secret values; and
  3. command- or process-specific environment values.

An operation-specific value can override a selected secret for that operation. A Create request cannot include the same key in both environment and secrets.

Rotation and lifecycle

Secret names resolve to stable secret identities when the Sandbox is created.

  • Selection order does not affect Create identity.
  • Rotating a secret changes the value fetched by a future launch, but does not replace a value already delivered to a running or paused guest.
  • Archiving a selected secret and creating another secret with the same name affects new Sandboxes. An existing Sandbox remains bound to the original identity.
  • Pause and Resume do not fetch secrets again.
  • A user snapshot captures guest memory, disk, and environment after delivery. Values can remain in the snapshot and its clones.
  • A clone does not fetch current values and cannot select additional secrets.

Rotation or archival cannot revoke a value already delivered to a guest or captured in a snapshot. Destroy Sandboxes and delete snapshots when their sensitive state is no longer needed.

Delivery and failure

Names resolve inside the authenticated account and workspace. Unknown, archived, duplicated, or colliding names reject Create as invalid input. If the secrets cannot be delivered at launch, the launch fails. See How secrets stay safe.

Validation and limits

  • Each name is exact and case-sensitive.
  • Each name must be non-empty and at most 256 UTF-8 bytes.
  • Names cannot have leading or trailing Unicode whitespace.
  • Names cannot contain =, carriage return, line feed, or NUL.
  • Names must be unique and cannot overlap literal environment keys.
  • Literal environment keys and selected names share a maximum of 256 entries.
  • Literal environment and selected names together are limited to 64 KiB.

How secrets stay safe

Workspace secrets are protected before they reach a Sandbox.

  • Encrypted at rest. Each value is envelope-encrypted and bound to your account and workspace.
  • Short-lived access. A launch can fetch its secrets only with an authorization that lasts one minute and is tied to that Sandbox and the machine launching it.
  • No cloud credentials on the machine. The machine running your Sandbox receives the values, never the keys that decrypt them.
  • Never fetched twice. Each Sandbox gets one fetch attempt. A failed or lost response is not replayed, and the launch fails.

Once a value is inside the Sandbox, code there can read it. Prefer a tool or a short-lived token for anything sensitive.

What not to do

const token = await step.run("mint", () => {
  return mintToken();
});

await sb.files.write("env", { path: "/work/.env", data: `TOKEN=${token}` });

await sb.$`curl -H ${`Authorization: Bearer ${token}`} https://api.example.com`;
  • A file's contents are stored with the run. files.write is a step, so the data is its input, up to about 2 MiB. It also stays on the Sandbox disk and in every snapshot.
  • Command text and the step ID are stored. Put the value in env.
  • A step's return value is stored. A token returned from step.run stays in step data until the run is purged.

What is stored with the run

ValueIn the trace
environment or env values on a step.sandbox callNames only, values [redacted]. A command run with tools records none
secretsNames only
Command text and step IDStored
Command outputStored, the last 2 MiB (512 KiB when timeout is over 5 minutes)
files.write dataStored as step input
Tool arguments and resultsStored
A value returned from step.runStored

Values in the Sandbox itself, on disk or in memory, go into any snapshot you take. Destroy Sandboxes and delete snapshots when their sensitive state is no longer required. See Managed lifecycle for cleanup that runs after a failure.