Production deployment¶

A hosted editor runs the Spec-Driven Agent in its own container, besser-wme-smartgen, next to the regular backend. Generation is the only path that executes LLM-authored code, so the worker gets the compilers, the shell sandbox and the LLM credentials, and nothing else; the backend keeps every other secret and has no compilers at all. This page lists what a production host needs for that split. The reference definitions are in docker-compose.prod.yml and .github/workflows/deploy-wme.yml.

Images¶

Both images are built from the root Dockerfile, one target each, and stamped with the commit they were built from (BESSER_BUILD_SHA).

The Dockerfile requires BuildKit (the default builder in current Docker releases; set DOCKER_BUILDKIT=1 on older ones): the proxy certificates are read through a RUN --mount=type=bind of ca-certs-extra/, so they never get an image layer of their own. The directory ships with a .gitkeep and must stay in the build context.

Both targets trust a build-time proxy CA only while building: put its certificates (*.crt) in ca-certs-extra/ and pass --build-arg TRUST_EXTRA_CAS=1. Before the image is finished the CA is removed again, and the build fails if a known TLS-inspection CA is still in the system trust store or, on the worker, in the JDK keystore, which is rebuilt from the cleaned store for that reason.

The worker service¶

Add this service to the host’s compose file, unchanged apart from the image registry if yours differs. It is the besser-wme-smartgen block of docker-compose.prod.yml:

  besser-wme-smartgen:
    image: artefacts.list.lu/besser/web_modeling_editor/smartgen_worker:latest
    container_name: besser-wme-smartgen
    ports:
      - "127.0.0.1:9001:9000"
    working_dir: /app/besser
    environment:
      - PYTHONPATH=/app
      - BESSER_LLM_MAX_CONCURRENT_RUNS=${BESSER_LLM_MAX_CONCURRENT_RUNS:-8}
      # The image carries tsc / cargo / kotlinc / javac; compiling generated
      # code catches errors the static checks miss.
      - BESSER_LLM_ENABLE_TOOLCHAIN_VALIDATION=true
      # Shell tools are enabled on this service only: it holds just the LLM
      # tokens, is off besser_network and is disposable. Being off the network
      # hides the other services' names, not ports they publish on 0.0.0.0.
      # Each run's shell is confined (security_opt below); the worker process
      # itself is not, so a compromise still reaches the LLM tokens.
      - BESSER_LLM_ENABLE_SHELL_TOOLS=true
      - BESSER_LLM_ALLOW_CUSTOM_BASE_URL=false
      # Each run's shell gets its own network namespace (pasta, /dev/net/tun
      # below): no shared localhost, no reaching this worker's own API.
      # private refuses commands when that cannot be set up.
      - BESSER_LLM_SHELL_NETWORK=${BESSER_LLM_SHELL_NETWORK:-private}
      - BESSER_LLM_SHELL_SESSION_IDLE_SECONDS=${BESSER_LLM_SHELL_SESSION_IDLE_SECONDS:-900}
      - BESSER_LLM_SHELL_SESSION_MAX=${BESSER_LLM_SHELL_SESSION_MAX:-10}
      # Shared (read-only) with the backend, which serves push-to-github.
      - BESSER_LLM_RUN_WORKSPACE_ROOT=/workspace/runs
      - BESSER_LLM_RUN_STORE_PATH=/workspace/runs/run_store.sqlite3
      # Budgets
      - BESSER_LLM_DEFAULT_MAX_COST_USD=${BESSER_LLM_DEFAULT_MAX_COST_USD:-5.0}
      - BESSER_LLM_DEFAULT_MAX_RUNTIME_SECONDS=${BESSER_LLM_DEFAULT_MAX_RUNTIME_SECONDS:-2400}
      - BESSER_LLM_MAX_RUNTIME_SECONDS_HARD_CAP=${BESSER_LLM_MAX_RUNTIME_SECONDS_HARD_CAP:-2400}
      # LLM credentials -- the only secrets this service is given.
      - BESSER_FREE_LLM_BASE_URL=${BESSER_FREE_LLM_BASE_URL:-}
      - BESSER_FREE_LLM_TOKEN=${BESSER_FREE_LLM_TOKEN:-}
      - BESSER_FREE_LLM_MODEL=${BESSER_FREE_LLM_MODEL:-}
      # The config endpoint is served by this container, so every variable it
      # reads must be in this allowlist or it reads as unset.
      - BESSER_FREE_LLM_ALT_MODELS=${BESSER_FREE_LLM_ALT_MODELS:-}
      # Keyless model pre-selected for an opt-in telemetry session.
      - BESSER_PILOT_LLM_MODEL=${BESSER_PILOT_LLM_MODEL:-}
      # Run telemetry is recorded where generation runs. The admin token is
      # not here: the report endpoint it guards stays on the backend.
      - BESSER_TELEMETRY_ENABLED=${BESSER_TELEMETRY_ENABLED:-}
      - BESSER_TELEMETRY_DIR=${BESSER_TELEMETRY_DIR:-/app/telemetry}
      - BESSER_FREE_LLM_FALLBACK_BASE_URL=${BESSER_FREE_LLM_FALLBACK_BASE_URL:-}
      - BESSER_FREE_LLM_FALLBACK_TOKEN=${BESSER_FREE_LLM_FALLBACK_TOKEN:-}
      - BESSER_FREE_LLM_FALLBACK_MODEL=${BESSER_FREE_LLM_FALLBACK_MODEL:-}
      - BESSER_SPONSORED_LLM_BASE_URL=${BESSER_SPONSORED_LLM_BASE_URL:-}
      - BESSER_SPONSORED_LLM_TOKEN=${BESSER_SPONSORED_LLM_TOKEN:-}
      - BESSER_SPONSORED_LLM_MODEL=${BESSER_SPONSORED_LLM_MODEL:-}
      # Secret authorising the sponsored tier (?demo= links). The gate runs
      # here and fails closed: without it every demo run is refused.
      - BESSER_DEMO_TOKEN=${BESSER_DEMO_TOKEN:-}
    volumes:
      - smartgen_workspace:/workspace
      - ./telemetry-logs:/app/telemetry
    # run_command wraps every model-authored command in bubblewrap, which needs
    # an unprivileged user namespace. Docker's default seccomp profile gates
    # unshare/mount/pivot_root behind CAP_SYS_ADMIN and the docker-default
    # AppArmor profile carries `deny mount`, so both step aside (cap_add
    # SYS_ADMIN alone still fails on pivot_root). No capability is added, and
    # each run's shell sees only its own workspace and PID namespace.
    # Without these two lines the worker fails closed: every run_command is
    # refused.
    security_opt:
      - seccomp=unconfined
      - apparmor=unconfined
      # Amazon Linux: Docker's masked /proc paths make the kernel refuse
      # bwrap's private /proc mount (measured on experimental 2026-09-29).
      - systempaths=unconfined
      - no-new-privileges:true
    # pasta's tap device for each shell session's private network.
    devices:
      - /dev/net/tun:/dev/net/tun
    # Reaps orphans (exited bwrap monitors), which python as PID 1 does not.
    init: true
    command: python -m besser.utilities.web_modeling_editor.backend.backend
    restart: unless-stopped
    # NOT on besser_network: generation has no reason to reach the backend, the
    # frontend or the modeling agent by name. It still has outbound internet for
    # the LLM endpoints via its own bridge.
    networks:
      - smartgen_network

and, at the top level, its volume and network:

volumes:
  smartgen_workspace:
    driver: local

networks:
  smartgen_network:
    name: smartgen_network
    driver: bridge

The backend mounts the same volume read-only (smartgen_workspace:/workspace:ro) so that push-to-GitHub and import-from-GitHub, which need the backend’s OAuth session, can read the generated files.

Environment¶

The backend loads the whole ./.env (env_file). The worker has no env_file: it receives only the variables listed in its environment block, whose values compose substitutes from the same ./.env.

Set in ./.env (read by the worker through substitution): the LLM credentials BESSER_FREE_LLM_BASE_URL / _TOKEN / _MODEL / _ALT_MODELS, the BESSER_FREE_LLM_FALLBACK_* and BESSER_SPONSORED_LLM_* triples, BESSER_DEMO_TOKEN, BESSER_PILOT_LLM_MODEL, BESSER_TELEMETRY_ENABLED and, optionally, the budget overrides (BESSER_LLM_DEFAULT_MAX_COST_USD, BESSER_LLM_DEFAULT_MAX_RUNTIME_SECONDS, BESSER_LLM_MAX_RUNTIME_SECONDS_HARD_CAP, BESSER_LLM_MAX_CONCURRENT_RUNS).

Never put in ./.env, because the backend would pick them up:

• BESSER_LLM_ENABLE_SHELL_TOOLS – shell tools belong to the worker only, which holds nothing but the LLM tokens. On the backend they would run model-authored commands next to the SMTP, GitHub OAuth and telemetry secrets. See Shell tools.

• BESSER_LLM_ENABLE_TOOLCHAIN_VALIDATION – the backend image has no compilers to run.

• BESSER_LLM_RUN_WORKSPACE_ROOT and BESSER_LLM_RUN_STORE_PATH – the backend’s /workspace is read-only, so runs created under it fail.

• BESSER_LLM_ALLOW_CUSTOM_BASE_URL=true – keep it off on every hosted service (SSRF); the worker sets it to false explicitly.

• BESSER_LLM_SHELL_SANDBOX=off – a development opt-out that runs model-authored commands unconfined.

Sandbox and network¶

• run_command wraps every model-authored command in bubblewrap, which needs an unprivileged user namespace. The worker carries four security_opt entries and no added capability:

  • seccomp=unconfined – Docker’s default seccomp profile blocks the namespace and mount calls bubblewrap makes.

  • apparmor=unconfined – the default AppArmor profile denies mount.

  • systempaths=unconfined – Docker masks parts of /proc, and on stock kernels such as Amazon Linux’s the kernel then refuses the private /proc that bubblewrap mounts for each command.

  • no-new-privileges:true – no process in the container can gain privileges through a setuid binary.

  Without the first three the worker fails closed and refuses every command, as it does on a kernel that forbids unprivileged user namespaces. The trade-off is plain: systempaths=unconfined unmasks /proc for the worker’s own root process. Generated code never runs as that process; it runs inside bubblewrap’s user namespace. Running the worker as a non-root user would narrow the exposure further.

• Each shell session has a network namespace of its own, which a pasta process (the image’s passt package) connects to the internet with no port forwarded either way: concurrent runs can all use the same port, and none can reach another’s server or the worker’s own API on 127.0.0.1. pasta needs the tun device, hence devices: /dev/net/tun on the worker. BESSER_LLM_SHELL_NETWORK is private there, so a worker without the device refuses every command rather than share its network; auto (the code default) falls back to the shared namespace with a warning, and shared keeps it. init: true reaps the processes bubblewrap leaves behind as they exit, which the Python server as PID 1 does not. BESSER_LLM_SHELL_SESSION_IDLE_SECONDS and BESSER_LLM_SHELL_SESSION_MAX are passed through the same environment block.

• A run’s commands share one long-lived sandbox, its shell session, so a server the agent starts in one command is still up for the next. It is torn down when the run ends in any way, when its folder is deleted, and when the worker exits (the session’s control channel is a pipe to the worker process). Two settings bound what sessions hold: BESSER_LLM_SHELL_SESSION_IDLE_SECONDS (900) closes a session that has had no command for that long, and BESSER_LLM_SHELL_SESSION_MAX (10) caps live sessions per worker; past it a command gets a one-off sandbox as before. A session costs a bubblewrap process and a small Python supervisor plus whatever the run left running, so size the worker’s memory for BESSER_LLM_SHELL_SESSION_MAX running apps rather than for one command at a time.

• The validators that execute generated code run in sandboxes of the same kind, each fresh and never the run’s shell session, with the network cut: the import check after each file write, the import smoke check, the startup, create and API probes, and the tsc, cargo check and npm run build checks. When the worker cannot start the sandbox they are skipped and reported as unverified, never run unconfined. Tool results and findings then say only that the shell sandbox is unavailable; look in the worker’s log for the cause.

• Two steps need the network and get it, but are sandboxed all the same, so package install scripts see read-only /usr/local, no other runs and a stripped environment: the Phase 1 npm install of a scaffolded frontend and the pip install --dry-run dependency check. When the sandbox cannot start, the install is skipped with a logged reason, and the frontend build check then reports the frontend’s dependencies as not installed.

• Each run gets its own writable $HOME beside its workspace (<run dir>.sandbox-home), where installs and the npm / cargo caches go; /usr/local and /root are read-only. The telemetry folder and the incident log folder (BESSER_INCIDENT_LOG_DIR, default /app/incidents, which names other runs) are masked from every run. Deleting a run removes its sandbox $HOME along with its workspace.

• cargo check has no network, so only crates that the run already fetched through run_command resolve. Otherwise the check is reported as “dependencies could not be fetched”, a skipped check rather than a compile failure.

• The worker is only on smartgen_network, not besser_network: it cannot reach the backend, the frontend or the modeling agent by name, and still has outbound internet for the LLM endpoints.

• It publishes 127.0.0.1:9001 only, so the host’s reverse proxy must route /besser_api/spec-driven/ to it, except /besser_api/spec-driven/push-to-github and /besser_api/spec-driven/import-github-run, which stay on the backend with the rest of /besser_api/.

• Give /besser_api/transform-agent-model-json (agent personalization) a proxy_read_timeout of about 600 s. It makes one LLM call per selected option in sequence, so several options exceed nginx’s 60 s default and return a 504 while the backend keeps calling the LLM.

Deploying¶

The Deploy WME to EC2 workflow (target: backend or all) builds and pushes both images, then recreates besser-wme-backend and besser-wme-smartgen. Before it builds anything, it reads the host’s compose file and stops, with nothing pushed or restarted, when a service it would deploy is not defined there or when besser-wme-smartgen is not on the smartgen_worker image. After the restart it compares BESSER_BUILD_SHA in both containers with the commit it built, then checks that the API answers 200.

Verification¶

Run these on the host, in the directory of its compose file:

# Both containers run the commit that was deployed
docker compose exec -T besser-wme-backend printenv BESSER_BUILD_SHA
docker compose exec -T besser-wme-smartgen printenv BESSER_BUILD_SHA

# The worker has its toolchains, and bubblewrap can start
docker compose exec -T besser-wme-smartgen sh -c \
  'node --version && tsc -v && cargo --version && kotlinc -version && bwrap --version && ruff --version'
docker compose exec -T besser-wme-smartgen python -c \
  "from besser.spec_driven_agent.execution.sandbox import sandbox_selftest_error as e; print(e() or 'sandbox OK')"
docker compose exec -T besser-wme-smartgen python -c \
  "from besser.spec_driven_agent.execution.sandbox import shell_network_is_private as p; print('private network' if p() else 'SHARED network')"

# Shell tools and toolchain validation are on in the worker ...
curl -s http://127.0.0.1:9001/besser_api/spec-driven/config | python3 -m json.tool | grep -E 'shell_tools_enabled|toolchain_validation_enabled'
# ... and off in the backend
docker compose exec -T besser-wme-backend python -c \
  "import besser.utilities.web_modeling_editor.backend.constants.constants as c; print(c.LLM_ENABLE_SHELL_TOOLS, c.LLM_ENABLE_TOOLCHAIN_VALIDATION)"

# The API answers through the public URL
curl -s -o /dev/null -w '%{http_code}\n' https://<editor-host>/besser_api/

Expected: the same commit twice, the tool versions, sandbox OK, private network, both features true in the worker, False False in the backend, and 200.