← Back to dashboard

Remote Worker (Ansible, Kubernetes & image-promote runners)

Formerly "Ansible." The Settings panel is now Configuration → Remote Worker. The doc path (docs/integrations/ansible.md) is unchanged so existing links resolve.

What is it?

The Remote Worker panel configures the dashboard's three off-host runners. Each runs its work as a one-shot cloud task — a container launched in the target cloud, run once, and destroyed when it exits — and all three share the same per-cloud network settings (cluster, subnet, security group, role, ACR, VPC connector).

Each runner picks its backend per the job's target cloud (see Per-target-cloud backend): an AWS-target job runs on ECS Fargate, an Azure-target job on ACI, a GCP-target job on Cloud Run. The Ansible and Kubernetes runners can also run local (a Docker sibling / in-process); image-promote always runs in its target cloud.

The four backends:

Why a cloud runner? Two independent reasons:

• Private subnets — the target VM (Ansible) or cluster API (Kubernetes) has no route back to the dashboard host. A task launched in the target cloud sits on the right network.
• Corp proxy — a TLS-inspecting corporate egress proxy can't validate the cluster API's private-CA cert during inspection, so it rejects direct kubectl/helm calls (e.g. an HTTP 526 "invalid SSL certificate", or the proxy's own block page). A one-shot cloud task has clean egress and side-steps the proxy.

Read these first: - docs/config-management.md — philosophy, best practices, the security argument for one-shot runners, and where SaaS extends this. - docs/storage-management.md — full reference for the four storage backends (AWS S3, Azure Blob, GCS, Local / UNC) the Ansible runner pulls assets from, and the migrate flow. - runners/promote/README.md — the image-promote runner, which shares this panel's per-cloud infra.

This page is the integration-specific guide: the config-field reference, the two runners, the shared cloud infrastructure and its fallback chains, per-cloud prerequisites, the Ansible playbook how-to, and troubleshooting.

Storage and execution targets are independent (Ansible runner). You can store assets in S3 and run them against on-premises Proxmox hosts, or store them on a corporate UNC share and target EC2 instances — any combination works (with one constraint: cloud runners can't read from a UNC backend; see storage-management.md). The Kubernetes runner has no storage dependency — it streams manifests over stdin.

Per-target-cloud backend

Each runner chooses its backend from the job's target cloud, not a single global switch. The selector offers Local or that cloud's matching task service — there is no cross-cloud option, because the network, identity, and storage a job needs live in its target cloud:

So you can run, say, Kubernetes ops for EKS clusters in-process but AKS clusters on an ACI task (clean egress to *.azmk8s.io), and Ansible against AWS VMs on Fargate but Azure VMs locally — each independently, per runner.

Image-promote is inherently per-target-cloud already: promoting to AWS runs on ECS, to Azure on ACI, to GCP on Cloud Run (no local — a promote always runs in the destination cloud, where it stages the converted disk).

Config keys

A per-cloud key takes precedence; when blank, the runner falls back to the global ansible_runner / k8s_runner (default local) — so existing single-runner configs keep working unchanged. On open, the panel pre-fills the per-cloud selectors from any global value (mapped to its matching cloud) so you see the effective config and can then adjust each cloud.

Config panel field reference

Every field on Configuration → Remote Worker, grouped as the panel groups them. Values are stored in the dashboard's config-service store and can also be set via the matching environment variable (the env name is the config key upper-cased — pydantic BaseSettings, no prefix). Defaults and meanings are taken from web_dashboard/config.py.

Runner backends

The panel shows a runner × cloud grid: for each runner (Ansible, Kubernetes) pick Local or the matching cloud service per target cloud. Each selector writes a per-cloud key; the global keys remain as a fallback (see Per-target-cloud backend).

There is no ansible_runner_local. A Config-Management run against an on-prem Kubernetes cluster (cloud = local) always uses the local runner — it is the only backend with a route to your LAN — so there is no key to set, and a stray one is ignored rather than honored. See Kubernetes-cluster and database targets.

Shared cloud infrastructure — AWS / ECS

These knobs are reused by the Ansible runner, the Kubernetes runner, and the image-promote runner (see Shared cloud infrastructure).

The AWS region comes from the dashboard's AWS config (aws_region, default us-east-1), not a Remote-Worker field.

Shared cloud infrastructure — Azure / ACI

The ACI resource group and location come from the Azure config (azure_resource_group, default vm-cli-rg; azure_location, default centralus) — there are no separate Remote-Worker fields for them. Azure SP credentials (azure_client_id / _secret / _tenant_id / _subscription_id) are inherited from the Azure config.

Shared cloud infrastructure — GCP / Cloud Run

The GCP project comes from gcp_project_id and the region falls back to gcp_region (default us-central1) — both from the GCP config.

Ansible runner details

The ECS Ansible image is ansible_ecs_image (env ANSIBLE_ECS_IMAGE, default chrweav/ansible-winrm:latest). The local runner image is ansible_local_image (env ANSIBLE_LOCAL_IMAGE, same default). The AWS SSH key for EC2 targets comes from ansible_ssh_key_sm_name (env ANSIBLE_SSH_KEY_SM_NAME, default ec2/ssh-keypair) — see Cloud VM SSH keys. The final-fallback username for an unrecognised cloud tag is ansible_default_user (default ec2-user).

Kubernetes runner

Shared cloud infrastructure

The Ansible runner, the Kubernetes runner, and the image-promote runner all reuse the same per-cloud cluster / subnet / SG / role / ACR / VPC settings. Set them once and all three pick them up. Each runner reads the shared ansible_* (and Azure/GCP) keys directly, with the promote runner adding its own optional promote_runner_* overrides on top.

What each runner reads

Fallback chains

• Image-promote → Ansible (AWS): promote_runner_ecs_cluster → ansible_ecs_cluster; promote_runner_ecs_subnet_id → ansible_ecs_subnet_id; promote_runner_ecs_security_group_ids → ansible_ecs_security_group_ids; promote_runner_ecs_execution_role_arn → ansible_ecs_execution_role_arn. (The promote runner additionally needs a task role with S3 write — that one has no Ansible equivalent because the Ansible runner doesn't stage to S3.)
• Kubernetes runner (ECS): reuses ansible_ecs_* (cluster, subnet, SG, execution role, cpu, memory) + aws_region; only the task family (k8s-runner) and image (k8s_runner_image) differ.
• Kubernetes runner (ACI): reuses azure_resource_group / azure_location / ansible_aci_subnet_id / ansible_aci_acr_server/username/password.
• Kubernetes runner (GCP): reuses gcp_project_id / gcp_region (or gcp_ansible_cloud_run_region) / gcp_ansible_vpc_connector.

The takeaway: configure the ECS cluster + subnet + SG + role once (or the ACI subnet + ACR, or the GCP region + VPC connector once), and the Ansible runner, the Kubernetes runner, and image-promote all use it. The promote_runner_* keys exist only for installs that want the promote task on different infra than config-mgmt — most single-tenant installs leave them blank. See runners/promote/README.md for the full promote-runner key list.

Kubernetes runner

k8s_runner_<cloud> (falling back to the global k8s_runner) controls how the dashboard runs cluster-API operations — kubectl apply, kubectl delete, helm repo add/helm upgrade, kubectl get secret — for a cluster, chosen by that cluster's cloud. These back the entitle agent install, the External Secrets Operator (ESO) rollout, and mgmt-plane operations.

When to use a cloud backend

Use ecs / aci / gcp when direct kubectl/helm from the dashboard host fails because of a TLS-inspecting corporate egress proxy. The symptom is a TLS / SSL-certificate error when the proxy inspects traffic to a cluster API server that presents a private-CA cert it can't validate — for example an HTTP 526 ("invalid SSL certificate"), or the proxy's own block page. A one-shot cloud task has clean egress to the cluster API and side-steps the proxy entirely.

(The same private-subnet reasoning as the Ansible runner also applies: a cloud task can reach a cluster API that has no route back to the dashboard host.)

Reachability caveat

The cloud task talks to the cluster's public API endpoint over the bearer token in the prepped kubeconfig. The task still needs that endpoint to be reachable from the cloud-runner network:

• The cluster API must have a public endpoint (or one reachable from the runner's subnet / VPC connector).
• If the cluster restricts the API to authorized CIDRs / IP allow-lists, add the runner's egress (the Fargate task's public IP or NAT range, the ACI subnet, or the Cloud Run VPC-connector egress) to that allow-list, or the task's kubectl calls will time out.

Configuration

Pick the backend per cluster cloud in Configuration → Remote Worker → Kubernetes runner (EKS / AKS / GKE each get their own Local-or-cloud selector) and, if it's a cloud backend, make sure the shared cloud infrastructure for that cloud is set (the k8s runner reuses the Ansible runner's ECS / ACI / Cloud Run network plumbing). Override the image only if you mirror dtzar/helm-kubectl to a private registry — set k8s_runner_image.

Per-cloud prerequisites

Only needed for the cloud backends (ecs / aci / gcp). The local backends need nothing beyond the Docker socket (Ansible) or in-container kubectl/helm (Kubernetes).

AWS (ECS Fargate)

• ECS Fargate cluster the dashboard can run-task against (ansible_ecs_cluster; reuses the BT Gateway cluster by default).
• Task execution role (ansible_ecs_execution_role_arn) with service-role/AmazonECSTaskExecutionRolePolicy — this is what ECS uses to pull the image (from a private ECR) and write CloudWatch logs. It is not the role the container code runs as. Required for private-registry images; can be blank if the image is public.
• Task role vs execution role: the Ansible and Kubernetes runners need only the execution role — neither container makes signed AWS API calls (Ansible SSHes to the VM; the k8s task uses a bearer-token kubeconfig). The image-promote runner additionally needs a task role (the identity the container assumes via the task metadata endpoint) with s3:PutObject on the staging bucket; see runners/promote/README.md.
• Subnet + security group (ansible_ecs_subnet_id, ansible_ecs_security_group_ids) with egress to the target VM / cluster API (and to your image registry on 443). A private subnet is recommended; it needs a NAT route for image pulls.

Azure (ACI)

• Subscription with ACI quota. Register the provider if this subscription hasn't used ACI: az provider register --namespace Microsoft.ContainerInstance.
• Service principal — the dashboard's existing azure_client_id / azure_client_secret / azure_tenant_id / azure_subscription_id. It needs Contributor (or a custom role allowing container-group create/delete) on the resource group ACI runs in (azure_resource_group).
• ACR credentials (ansible_aci_acr_server / _username / _password) only when the runner image lives in a private ACR. ACI uses these as image-registry credentials at container-group create time to pull the image; they are passed as secure values, not stored on the container. Leave blank for the public chrweav/ansible-winrm / dtzar/helm-kubectl images.
• VNet subnet (ansible_aci_subnet_id) when the container group must run inside a private VNet to reach the target.

GCP (Cloud Run Jobs)

• APIs enabled: run.googleapis.com (and compute.googleapis.com / iam.googleapis.com for the surrounding flows).
• Service account with:

• VPC reach when the job must SSH to a private RFC-1918 target — Cloud Run Jobs run in a Google-managed VPC by default and can't reach private addresses without one of the two modes below (otherwise SSH times out and the play fails UNREACHABLE, container exit code 4). Direct VPC egress wins when both are set.

• Direct VPC egress (preferred — no standing infra): set both gcp_run_network (VPC name) and gcp_run_subnetwork (a subnet in the Cloud Run region). The job's NIC lands straight in the subnet — no connector to provision or pay for, and immune to the connector's shared-core zonal stockouts. Egress stays private-ranges-only. Ensure a firewall rule allows tcp:22 from the subnet range to the target VM.

  gcp_run_network=dashboard-sandbox-vpc gcp_run_subnetwork=dashboard-sandbox-vm-subnet

• VPC connector (legacy): create a Serverless VPC Access connector and set gcp_ansible_vpc_connector:

  bash gcloud compute networks vpc-access connectors create ansible-runner \ --region us-central1 --network default --range 10.8.0.0/28

  then set gcp_ansible_vpc_connector=projects/PROJECT_ID/locations/us-central1/connectors/ansible-runner.

Cloud VM SSH keys (Ansible runner)

Cloud VM targets authenticate with an SSH key, not a password. The Ansible runner pulls the private key from the cloud's secret store at run time:

A legacy AWS fallback exists: ansible_ssh_key_secret (env ANSIBLE_SSH_KEY_SECRET, default AWS_KEY) — a Password Safe secret title. Prefer ansible_ssh_key_sm_name.

GCP example — store the key and grant access:

gcloud secrets create ssh-ansible-keypair --replication-policy="automatic"
gcloud secrets versions add ssh-ansible-keypair --data-file=~/.ssh/id_rsa
gcloud secrets add-iam-policy-binding ssh-ansible-keypair \
  --member="serviceAccount:SA_EMAIL" \
  --role="roles/secretmanager.secretAccessor"

Using a Secrets-Management secret in a run

Beyond the SSH key, a run can pull secrets from Secrets Management — a DB-stored secret or an external vault reference (aws_sm://, gcp_sm://, azure_kv://, bt_safe://) — without the operator ever seeing the value. The Use a secret panel on /config-mgmt offers three bindings:

Using a secret requires the secrets:use permission (admins and legacy unrestricted users bypass). The use is audited — kinds + var names only, never the source refs or values — and any resolved value is scrubbed from the job output.

Cloud runners: hardened per provider (and the store requirement)

On the cloud runners the value is not placed in the task's plaintext env or on the command line. Each secret is delivered through the provider's own secret channel, and the container decodes a non-secret manifest into a 0600 vars file before running ansible-playbook -e @file:

Because ECS and Cloud Run reference a store secret rather than carrying its value, a variable/become secret used on those runners must already live in that cloud's store. If it doesn't, the run is rejected up front with an actionable message — move it there via Secrets → migrate, then reference it as aws_sm://<name> / gcp_sm://<name>. ACI has no such requirement. The SSH-key secret always rides the existing SSH_KEY_B64 channel and needs no migration.

Managed-account checkout (BeyondTrust Password Safe)

When Password Safe is enabled (password_safe_enabled), a run can also use a Password Safe managed account as the login identity — instead of referencing a stored secret, the operator picks an account from a live list and the dashboard checks out its credential just-in-time at run time. The operator never sees the value; the checkout is scrubbed from output and audited exactly like the secret path above (and needs the same secrets:use permission).

How to use it: on /config-mgmt, pick Target → On-prem host (IP / hostname) and enter a system registered in Password Safe (a cloud VM's IP works too). The dashboard looks up that host's managed systems + accounts and shows an account picker (each tagged [SSH key] or [password]). Selecting one:

• sets ansible_user to the account name;
• injects its credential as the connection secret — an SSH-key account becomes the connection key; a password account becomes ansible_ssh_pass / ansible_password (Windows targets need ansible_connection: winrm in the play);
• optionally, a second managed account can be picked for the become/sudo password (ansible_become_password).

Across many hosts (bulk runs). The picker above pins system_id and account_id, and both belong to one managed system — so the same reference cannot be reused across a fleet: it would check out a single machine's credential and connect to every host with it. Correct only if the account happens to be domain-linked, wrong for a local account, and nothing would report which. A bulk run therefore sends the account name, and each job resolves it against the host it is actually configuring before checking anything out. Consequences:

• Each host checks out its own credential. Because the resolved reference replaces the submitted one wholesale, that host's own [SSH key] / [password] nature decides the checkout mode — the sample host's flag is never trusted.
• A host with no account by that name fails only its own job, with a message naming the host and the account.
• Matching accepts the {user};{suffix} form that cloud-native onboarding registers (the AWS Systems Manager plugin appends a scope suffix), so picking svc-ansible matches svc-ansible;local. Domain-linked accounts resolve too.
• The account list you choose from is read from one selected target as a sample — it supplies the names, not the ids.

Not available for Kubernetes / database batches. Those run a localhost play with no SSH connection to authenticate, and the run path silently ignores the connection-identity fields. A single run can absorb that quietly; a batch would leave you believing a credential had been applied to every cluster — so managed_account, managed_become, secret_ssh_key_source and secret_become_source are refused outright for a non-VM batch. Named secret_vars are honored there and stay available.

Local and Azure (ACI) runners inject the credential inline — the local runner via a 0600 vars file, ACI via secure_value — so a checked-out managed account works on either out of the box.

ECS and Cloud Run reference a store secret (the task identity fetches it at launch), which a checked-out (ephemeral) credential has none of — so they're rejected unless "Ephemeral cloud secrets" is enabled (Settings → Ansible). When on, the credential is written to that cloud's store as a short-lived, RBAC-locked secret, injected via the provider's channel, then force-deleted after the run — see Ephemeral cloud secrets.

SSH-password targets require sshpass in the runner image (already true for the built-in on-prem SSH path). The lookup and checkout go through ps-cli, authenticated by the configured Password Safe OAuth client (pscli_api_url / pscli_client_id / pscli_client_secret).

In-playbook Password Safe lookup (beyondtrust.secrets_safe)

The managed-account checkout above is out-of-band: the dashboard fetches the credential and injects it. The complementary pattern is an in-playbook lookup — the play fetches its own secrets from Password Safe at runtime via the beyondtrust.secrets_safe Galaxy collection's secrets_safe_lookup plugin (and the beyondtrust.password_safe management modules). Use it for app secrets, API tokens, or DB credentials a task consumes — as opposed to the connection credential, which the checkout path handles.

Ready-to-run starters live in examples/playbooks/password-safe/.

Several shipped samples support this optionally. Rather than only living in the dedicated demos, the plays that consume an app secret each declare an optional …_secret var — set it to a SECRET path (folder/title) and the value is fetched mid-run; leave it blank and the play behaves exactly as before:

Two implementation notes that matter if you adapt the pattern:

• The fetch writes to a private _ps_* fact and is resolved at the use site, never back onto the caller-supplied variable. Ansible extra vars outrank set_fact, so writing back would be silently ignored whenever the value was also supplied via an extra var or a "Use a secret" binding — the play would quietly use the wrong one. When both are set, the Password Safe path wins.
• Connection credentials (ansible_password, become, SSH keys) deliberately stay with the run-form panel, which covers cases a lookup can't (SSH keys, sshpass, ephemeral cloud secrets). tests/test_playbook_ps_lookup.py pins these invariants across the samples.

Auto-injected credentials. The lookup runs on the Ansible controller (the runner container) and reads PASSWORD_SAFE_API_URL / PASSWORD_SAFE_CLIENT_ID / PASSWORD_SAFE_CLIENT_SECRET. When Password Safe is enabled (password_safe_enabled) and the ps-cli OAuth client is configured, the dashboard auto-injects those three env vars into every runner (Local, ECS, ACI, Cloud Run) — reusing the same pscli_api_url / pscli_client_id / pscli_client_secret config as the checkout path, so there's nothing extra to set per run.

• The client secret rides the same per-run env channel each runner already uses for the SSH private key — the ECS runTask override (not the task-def revision history), the Cloud Run job env, an ACI secure_value, or a 0600 --env-file locally — never on a command line, and scrubbed from job output. No ansible_cloud_ephemeral_secrets_enabled gate is required (that gate is only for the checked-out managed-account path).
• Both runner images (chrweav/ansible-winrm, chrweav/ansible-cloud) ship the two collections (via beyondtrust-bips-library), so the lookup resolves on either — rebuild
• push them before relying on it.
• The OAuth client needs the usual API-registration permissions (Secrets → Read, Requests → Create, Credentials → Read, plus scope for the paths the play touches).

Bulk runs (one asset, many targets)

/config-mgmt runs one asset against one target. To apply a playbook across a fleet, select rows on the Inventory page (/inventory) — filter to what you want, tick them, and a run panel appears. Each selected resource becomes its own job, all tagged with a shared batch_id, dispatched through the ordinary run path so every permission check, secret-store validation and runner decision behaves exactly as it does for a single run. Queueing lands you on /jobs?batch_id=… — the batch filtered out of the job list, with a status rollup across all of it.

Every run is claimed from the jobs table by the job runner (the worker service in the compose files), the same way Kubernetes and database runs are. A batch therefore survives a dashboard restart, and its jobs execute concurrently across WORKER_REPLICAS (default 3) rather than one at a time — that number is the ceiling on how many hosts a batch touches simultaneously.

One kind per run. Selecting a VM locks the checkboxes on Kubernetes clusters and databases, and vice versa. The kinds aren't interchangeable at any level: a VM run SSHes to a host, while k8s/database runs are localhost plays reaching out over a kubeconfig or DB login — different request fields, a different runner, and a playbook written for one is meaningless against the other.

Rows that can never be a target are disabled with the reason on hover: virtual desktops (no Ansible target exists behind a seat), Proxmox / Nutanix VMs (their deploy records a node + VMID rather than an address — target them through their hypervisor group instead), and databases whose engine or cloud has no runner. Those reasons are computed server-side by the same rule the endpoint enforces, so the page can't offer a checkbox the API would reject.

Two limits worth knowing:

• 50 targets per batch. Each is a job, so an unbounded "select all" against a large estate would fan out unbounded work.
• Selection problems refuse the whole request; per-target failures don't. A mixed selection or an untargetable row is caught before any job exists. But several checks in the run path turn on the target's cloud, so a mixed-cloud VM batch can be valid for one host and not another — those come back in the response's failed list and are named in the toast, while the rest still run.

Secrets and managed accounts. The inventory panel covers asset / SSH user / extra vars. For a run needing a Secrets-Management secret or a Password Safe managed account, use Continue on the Config Management page →, which carries the selection over and applies the full run form to it — see Managed-account checkout for how an account is matched across many hosts.

Full treatment in docs/config-management.md.

Storage prerequisite (Ansible runner)

The Ansible runner fetches its assets (playbooks, scripts, packages) from a storage backend. At least one backend must be configured and active on /storage before the Remote Worker / Ansible feature flag can be enabled.

The four backends — S3, Azure Blob, GCS, Local Filesystem / UNC — are configured on the dedicated /storage page. Picking the right backend:

Configuration steps, asset upload, migration between backends, and per-backend IAM details all live in docs/storage-management.md. (The Kubernetes runner has no storage dependency.)

Enable in the dashboard

1. Open /storage and configure at least one backend; pick it as active (required for the Ansible runner).
2. Open Settings → Integrations. The Remote Worker toggle, previously greyed out, is now selectable.
3. Click Configure on the Remote Worker row to set the runner backends — pick Local or the matching cloud service per target cloud for each runner — the per-cloud SSH usernames, and, for cloud backends, the shared cloud infrastructure.
4. Toggle Remote Worker on. No restart required.

Per-cloud SSH user (Ansible runner)

Each cloud's stock image ships with a different default username (ec2-user / azureuser / gcp-user), so the panel exposes three fields rather than one:

The pre-fill is non-clobbering: a value the operator types by hand is never overwritten when they switch targets. The submitted ansible_user is whatever the field holds at submit time.

Ansible: local Docker runner (on-premises and cloud targets)

The local runner is automatic: no extra infrastructure is needed beyond the Docker socket already mounted in docker-compose.yml. It handles both on-premises hypervisors and cloud VMs — the asset is always fetched from storage regardless of where the target lives. It is also the only runner that can target on-premises hypervisors and the only one that forwards WinRM ansible_password extra vars.

How the inventory is built

When you click Run, the dashboard calls GET /api/config-mgmt/inventory, which returns a dynamic Ansible JSON inventory built from every on-premises hypervisor integration that is both enabled and has a host configured.

Hypervisors that are not enabled or have no host set are silently omitted — the target picker only shows what is actually reachable. Cloud VMs appear in separate optgroups populated from the AWS / Azure / GCP tab caches.

WinRM and the runner image (pywinrm)

Any Windows / WinRM target — on-prem Hyper-V or a Windows cloud VM (AWS / Azure / GCP) — needs pywinrm in the runner image. The dashboard's default runner image, chrweav/ansible-winrm:latest, is upstream willhallonline/ansible plus pywinrm, so Windows works out of the box on every runner — no image change needed. (Source: runners/ansible-winrm/.)

This matters only if you override the image. Upstream willhallonline/ansible does not bundle pywinrm, so pointing a runner at it (or any image without pywinrm) makes Windows runs fail with "pywinrm is not installed". The image settings, all defaulting to chrweav/ansible-winrm:latest:

Beyond the image, a Windows run needs WinRM enabled and reachable on the target (Enable-PSRemoting -Force on Hyper-V; ports 5985/5986 open to the runner) — and on the cloud runners the credential supplied via Use a secret, since they don't forward plaintext extra vars.

Proxmox SSH note

The local runner authenticates to Proxmox via SSH using proxmox_password (the root@pam password). If you configured Proxmox with API token only (no password), the SSH connection will fail. Either: - Set PROXMOX_PASSWORD in addition to the token, or - Target Proxmox VMs individually by IP rather than using the proxmox group.

ESXi SSH note

SSH is disabled by default on ESXi. Enable it via: Host → Manage → Services → TSM-SSH → Start, or:

vim-cmd hostsvc/enable_ssh

Changing the local Ansible image

ANSIBLE_LOCAL_IMAGE=chrweav/ansible-winrm:latest   # the default

Any image with ansible-playbook on its PATH works. The playbook and inventory are bind-mounted into /ansible/ inside the container. Note: an image without pywinrm (e.g. upstream willhallonline/ansible) can't drive Windows/WinRM targets.

Provisioning assets (.sh / .ps1 / .rpm / .deb)

In addition to Ansible playbooks (.yml), you can upload scripts and packages to the same storage backend. The dashboard auto-generates a wrapper playbook based on the file extension.

Two ways to upload:

• /storage page — file picker + Upload button, goes to the active backend.
• /config-mgmt page — same upload form, plus inline run controls.

Either way, the upload hits POST /api/storage/upload and the file appears in the asset picker on next refresh. You can also write directly to the underlying bucket / share with the cloud's native tools (aws s3 cp, az storage blob upload, gsutil cp) if you'd rather script it.

The Config Mgmt tab shows all asset types in the picker. A colour badge indicates the type (Playbook / Script / PowerShell / RPM / DEB).

Extra vars are forwarded only to playbooks. For scripts and packages the field is accepted but ignored — pass runtime parameters via the script itself or encode them in the filename.

Cloud VM target discovery (Ansible runner)

The Config Mgmt tab reads the instance lists already cached by the AWS, Azure, and GCP tabs — no extra API calls are needed. The target picker shows three optgroups:

If you have not yet navigated to the cloud tab (so the cache is empty), visit it once to populate the list, then return to Config Mgmt.

Ansible playbook structure

On-premises hypervisor playbook

Target the proxmox, vsphere, hyperv, nutanix, or xcpng group (whichever is configured). Or use on_premises to hit all of them.

# harden-proxmox.yml
- hosts: proxmox
  become: yes
  tasks:
    - name: Ensure auditd is running
      service:
        name: auditd
        state: started
        enabled: true

# restart-hyperv-service.yml
- hosts: hyperv
  tasks:
    - name: Restart the dashboard service
      win_service:
        name: DashboardSvc
        state: restarted

Cloud VM playbook (single-host, ad-hoc)

For cloud targets the dashboard passes the IP as -i <host>, to Ansible:

# hardening.yml
- hosts: all
  become: yes
  tasks:
    - name: Ensure sshd is running
      service:
        name: sshd
        state: started
        enabled: true

Provisioning asset examples

Script (install-agent.sh) — upload a .sh file; the dashboard wraps it automatically:

#!/bin/bash
set -euo pipefail
curl -fsSL https://packages.example.com/agent.sh | bash
systemctl enable --now example-agent

RPM package (my-agent-1.0.rpm) — upload the .rpm directly. The dashboard generates:

- hosts: all
  become: yes
  tasks:
    - name: Copy my-agent-1.0.rpm to remote
      ansible.builtin.copy:
        src: /ansible/assets/my-agent-1.0.rpm
        dest: /tmp/my-agent-1.0.rpm
    - name: Install my-agent-1.0.rpm
      ansible.builtin.dnf:
        name: /tmp/my-agent-1.0.rpm
        state: present
        disable_gpg_check: true

Sample playbooks

Ready-to-adapt starters for Linux and Windows cloud VMs live in examples/playbooks/ — patching, SSH hardening, admin-user creation, Docker, node_exporter, nginx (Linux); Windows updates, firewall, Chocolatey, local admin, and IIS (Windows). See examples/playbooks/README.md for how to run each. There are also two cluster-building sets for on-prem hosts — Docker Swarm (examples/playbooks/swarm/): init, join, open ports, stack deploy, status, leave; and k3s (examples/playbooks/k3s/): server-init, join, open ports, kubeconfig, status, uninstall. Both work the same way: because a run targets one host at a time, the cluster is built node-by-node with the join token relayed between runs, so they need the local runner (see that README for the walkthrough and the token-visibility caveat — either token can be routed through Password Safe instead of job output).

The k3s set closes the loop with the section below: k3s-kubeconfig.yml rewrites k3s's loopback API address to the node's real one and prints a registration-ready payload, so the cluster you just built can be registered (cloud = local) and then become a Config-Management target itself.

Linux samples run via the cloud or local runner; Windows (WinRM) samples run via the local runner, which forwards the ansible_password extra var the WinRM connection needs (the cloud runner is SSH-only and doesn't forward extra vars).

Kubernetes-cluster and database targets (localhost runs)

Registered or provisioned Kubernetes clusters and databases are selectable Config-Management targets too, but they don't SSH anywhere — Ansible's kubernetes.core and community.postgresql/mysql/general modules run a hosts: localhost, connection: local play and connect out to the API server (via a kubeconfig) or the DB endpoint (via login vars).

• Dedicated runner image. These runs use chrweav/ansible-cloud (config key ansible_cloud_image) — Debian-based, carrying kubernetes.core, community.postgresql, community.mysql, community.general, the helm/kubectl binaries, and psycopg2/PyMySQL/pymssql. Build/push it from runners/ansible-cloud/ (multi-arch recommended) and, if you don't use public Docker Hub, mirror it to ECR/ACR/Artifact Registry. The winrm VM image is never used for these targets.
• The runner follows line-of-sight to the endpoint. Not a preference — the two cases point in opposite directions:
• Cloud-hosted (aws / azure / gcp). The control plane / DB endpoint is private to its VPC, so the run executes on a transient ECS / ACI / Cloud Run task in-subnet, reusing the same ansible_ecs_subnet_id / ansible_aci_subnet_id / gcp_ansible_vpc_connector network config as the VM cloud runner. The local runner can't reach those RFC1918 endpoints and its egress hits the corporate TLS proxy, so an ansible_runner_<cloud>: local override is rejected. The backend is ansible_runner_<cloud>, defaulting to that cloud's native runner.
• On-prem (cloud = local). A registered resource on your LAN, where an in-cloud task has no route at all: a Kubernetes cluster registered from a kubeconfig — see examples/playbooks/k3s/ for building one — or a database registered against a Password Safe managed account, see Databases → Registering an existing database. Those runs execute in a sibling container on the dashboard host, which is the only thing that can reach them: same ansible_cloud_image, same localhost play, same scrubbing. Two consequences — the dashboard host needs a working docker CLI and network reach to the endpoint (a dashboard deployed in a cloud has neither, and the run is refused with a message saying so), and local filesystem assets work fine, because the "move it to S3 first" rule exists only for the in-cloud runners that can't read this host's disk.

Databases follow the same split. A provisioned database is always cloud-hosted; a registered one may be either — cloud = local for an on-premises database, or the cloud it already lives in. Registered OCI is the one gap: it can be registered, but no runner resolves for oci, so it is refused as a target. - Auto-injected, scrubbed connection material. The kubeconfig is token-prepped server-side (a short-lived bearer token replaces the cloud exec-auth block) and delivered via K8S_AUTH_KUBECONFIG/KUBECONFIG. The DB admin credential comes from one of two places, following the row's source: a provisioned database's is read from its provisioning job plus the encrypted config store, while a registered one has no provisioning job — its Password Safe managed account is checked out just-in-time at launch and never persisted. Either way it arrives as db_login_* extra-vars. Both ride the runner task's ephemeral env and are redacted from job output. An operator can still bind extra Secrets-Management named vars (e.g. a new role's password) via Use a secret; SSH-only options (become password, SSH key, managed-account) don't apply. - Durability. Dispatched by the job worker as an ansible_cloud_run job (it launches a cloud task that can outlive a request worker's recycle).

Starters: examples/playbooks/k8s/ and examples/playbooks/database/. Smoke-test the image directly with docker run … chrweav/ansible-cloud ansible-playbook -i 'localhost,' -c local … against a kind/k3d cluster or a throwaway Postgres/MySQL container.

Troubleshooting

Ansible — local Docker runner

"Target X is not a configured hypervisor" — the hypervisor integration is either disabled or has no host set. Enable it and fill in the host in Settings → Integrations.

No targets appear in the picker — no on-premises hypervisor is both enabled and configured. Check Settings → Integrations and confirm that both the toggle is on and the host field is filled.

"docker: command not found" — the Docker socket is not mounted. Verify docker-compose.yml includes the /var/run/docker.sock bind mount and restart the stack.

SSH authentication failed (Proxmox / vSphere / XCP-ng) — the stored password must work for SSH (not just the management API). For Proxmox, this means PROXMOX_PASSWORD must be set (API-token-only auth is not sufficient for SSH). For ESXi, SSH must be enabled on the host.

Hyper-V: "WinRM connection refused" — WinRM is not enabled. Run Enable-PSRemoting -Force on the Hyper-V host.

"pywinrm is not installed" (any Windows/WinRM target, any runner) — you've overridden the runner image with one that lacks pywinrm (e.g. upstream willhallonline/ansible). The default chrweav/ansible-winrm:latest includes it; either clear the override or point it at an image that has pywinrm. See WinRM and the runner image (pywinrm).

Container starts but can't reach the hypervisor — the Ansible container runs on the same Docker network as the dashboard (compose default bridge). If the hypervisor is on a separate VLAN, ensure the Docker host has a route to it.

Ansible — asset storage

Storage backend configuration, asset-list issues, and per-provider IAM permission errors live in docs/storage-management.md. The items below are Ansible-runner-specific concerns that the storage page doesn't cover.

"No active storage backend" when running — the feature flag got enabled while a backend was active, but it's since been deactivated. Re-pick a backend on /storage and Save.

"Permission denied" on .sh asset at run time — the auto-generated wrapper uses ansible.builtin.script which copies + runs the file with executable: /bin/bash. If the remote rejects it, write a custom .yml playbook with an explicit mode: '0755' copy + a task to invoke it.

.ps1 asset fails with "WinRM connection refused" — the target's inventory hostvars don't have ansible_connection=winrm. Hyper-V hostvars set this automatically. For other hypervisors hosting Windows guests, you'll need a custom playbook that sets vars: explicitly, or extend the relevant services/<hypervisor>_service.py to detect Windows guests.

Cloud VMs not in the target list — the list is read from the in-memory cache populated by the AWS / Azure / GCP tabs. Visit the relevant cloud tab first so the cache is warm, then return to Config Mgmt.

SSH authentication failed on cloud target (AWS) — verify ansible_ssh_key_sm_name is set and the IAM role has secretsmanager:GetSecretValue on that secret.

SSH authentication failed on cloud target (GCP) — verify gcp_ssh_key_secret_name is set and the service account has roles/secretmanager.secretAccessor on the secret. Ensure the public key is in the instance's ~/.ssh/authorized_keys (injected at launch).

Ansible — cloud runners

ECS task fails to start — check CloudWatch logs for the task family ansible-config-mgmt. Common causes: missing execution role, ECR pull error, or subnet routing to the target.

GCP: "Permission denied" creating Cloud Run Job — add roles/run.admin and roles/iam.serviceAccountUser to the service account.

GCP: logs empty after successful job — add roles/logging.viewer:

gcloud projects add-iam-policy-binding PROJECT_ID \
  --member="serviceAccount:SA_EMAIL" \
  --role="roles/logging.viewer"

GCP: Cloud Run job can't reach target host (play fails UNREACHABLE, "ssh: connect to host … port 22: Operation timed out", container exit code 4) — give the runner VPC reach: set gcp_run_network + gcp_run_subnetwork for direct VPC egress (preferred, no standing infra), or gcp_ansible_vpc_connector for a Serverless VPC Access connector — matching the Cloud Run region to the target's region. Also confirm a firewall rule permits tcp:22 from the runner's subnet range to the VM.

Azure: ACI runner UNREACHABLE / ssh: connect to host <ip> port 22: Operation timed out — the ACI container has no route to the target VM's private IP. Set ansible_aci_subnet_id to a VNet-delegated subnet with line-of-sight to the target subnet; when unset it now falls back to the gateway's azure_aci_subnet_id. With no subnet the container group is public and cannot reach private targets. (A working PRA Shell Jump to the same VM confirms the gateway's subnet reaches it — reuse that subnet.)

Kubernetes runner

Direct kubectl/helm fails with HTTP 526 / TLS errors — a corp egress proxy is inspecting TLS to the cluster's private-CA API. Set that cluster's cloud to a cloud backend (k8s_runner_<cloud> = ecs / aci / gcp) so the op runs from a task with clean egress.

Cloud k8s task times out reaching the API — the cluster API isn't reachable from the runner's network. Confirm the cluster has a public endpoint and add the runner's egress IP/CIDR to the cluster's authorized-networks allow-list (see Reachability caveat).

"Kubernetes ECS/ACI/Cloud Run runner is not configured" — the runner couldn't resolve a required shared field. ECS needs ansible_ecs_subnet_id and ansible_ecs_execution_role_arn; ACI needs azure_resource_group; GCP needs gcp_project_id and a region (gcp_region or gcp_ansible_cloud_run_region). Set them on Configuration → Remote Worker / the relevant cloud config.

Image pull fails on the cloud k8s task — the stock dtzar/helm-kubectl:latest is on Docker Hub. Behind a private registry, mirror it and set k8s_runner_image (ECS needs ansible_ecs_execution_role_arn with ECR pull; ACI needs ansible_aci_acr_*).