← Back to dashboard

Entitle Integration

Audience: operator · Profile: both · Read this when: you want access to expire on its own instead of being revoked by someone remembering.

The Entitle integration has four independent tracks that share one Entitle tenant + API token. You can enable any combination:

The previous approval-gate track (which blocked dashboard actions behind an Entitle approval + webhook) has been removed. Secret read/update/delete are now admin-only; cloud deploys run without an approval round-trip.

Resource registration

When enabled, each resource the dashboard provisions is registered as its own entitle_integration via the entitleio/entitle Terraform provider:

Managed SQL Server is gated off. The Entitle SQL Server connector needs sysadmin / CONTROL SERVER, which RDS-standard, Azure SQL Database and Cloud SQL for SQL Server cannot grant. _ENTITLE_VIABLE_SQLSERVER_PROVIDERS in cloud_database_service is empty today, so all three managed flavors are refused up front — the API returns an error before touching the database, and the UI hides the checkbox via the row's entitle_viable flag. Adding a provider to that set flips it back on; the truth table is pinned by tests/test_entitle_viability.py.

Registered databases can't be onboarded either. A database added with Register existing (source="registered") has no provisioning job and no Terraform state, so there is no admin credential to give the connector — its Password Safe managed account is checked out at run time and never stored. _entitle_ineligible_reason refuses those rows first, whatever the engine, with a message naming the missing credential rather than the SQL Server one above.

Registration is done by entitle_registration_service.py, mirroring terraform_pra_service: it generates HCL, runs terraform apply, records the new integration id, and stashes the Terraform state on the provisioning job so decommission can terraform destroy it. The teardown is wired into VM termination and DB decommission, so removing a resource removes its Entitle integration.

Public vs. private — the Entitle Agent

How Entitle reaches the target determines whether an agent is needed:

• Public resource (reachable from Entitle's cloud) → registers directly, no agent.
• Private resource (our private RDS, PRA-only VMs) → Entitle reaches it through the Entitle Agent, which runs only in a Kubernetes cluster (Helm-installed); there is no Docker/ECS deployment (agent docs).

The agent is shared: one per VPC/network serves every private integration in it, referenced by entitle_agent_token_name. So you provision the agent once per environment (not per build).

Except an OT demo cell, which mints its own. A cell registers against an agent running in its own plant — on a DMZ broker deployed beside it — because an agent outside the plant both misrepresents the architecture and, with the cell's Purdue zoning on, cannot reach it at all: the registration succeeds, the grant approves, and the login fails. See Who brokers identity in the plant.

"Only in a Kubernetes cluster" does not have to mean a real one. For an edge or plant-floor host that will not carry a cluster, KubeSolo is a single-node, etcd-free distribution that takes the same chart unmodified — Config Management playbooks install it and the agent on an on-prem host through a remote agent. That page also covers what the chart's defaults get wrong on one node, and why the agent needs port 8080 as well as 443.

On a POV instance the agent is per POV, not per install. A POV's VMs are on their own private network, so the shared one above cannot reach them. The POV page installs a single-node k3s on a Linux guest and Helm-installs the agent there, minting the token in that POV's own Entitle tenant — see the POV instance guide. The keys on this page describe the instance-wide agent used by managed clusters and stay untouched by it. A registration for a private target fails (non-fatally) with a clear message if no agent is configured. The Entitle agent is the management plane (it mints/revokes the ephemeral SSH account or DB role); the PRA tunnel the dashboard already brokers is the separate access path the user connects through.

Network reachability — the agent must have a route to the private target. To "fetch resources" for an SSH-ephemeral target, the agent pod SSHes into the host and enumerates accounts, so it needs an IP path to the target on port 22. The dashboard-managed clusters build their own self-contained VPC; when the sandbox VPC is configured (gcp_network / aws_vpc_id), the cluster module peers back to the sandbox VPC and opens the lab-VM firewall to the cluster's node+pod ranges so the agent can reach private VMs directly (GCP: gcp_gke peering + a *-allow-ssh-from-k8s firewall; AWS: aws_eks VPC peering + VM-SG ingress). Without that path, the Entitle audit log shows "Failed to fetch the resources of <target>" (a connection timeout). GCP caveat: VPC peering is non-transitive, so the agent reaches sandbox VMs but not Cloud SQL private-IP DBs (behind the sandbox↔servicenetworking peering) — DB JIT uses the PRA tunnel, not the agent.

Provisioning the agent is a one-time, per-environment admin step, and it is wired up: POST /api/k8s/clusters/{id}/entitle-agent Helm-installs the agent into a dashboard-managed cluster as a k8s_entitle_agent job, mints its token via the provider, stashes the value in the secrets backend and records entitle_agent_token_name for later private registrations. The token's lifecycle is tied to the agent's: the remove action and the decommission of the hosting cluster destroy the auto-minted token (freeing the name for the next mint) — an operator-supplied entitle_agent_token_ref is never touched. See docs/design/entitle-resource-registration.md.

Kubernetes clusters

Cluster registration is not a build-form checkbox — a cluster is registered on demand, after it exists, from the Kubernetes page:

A public cluster registers directly, with the API server host, a token and the CA cert passed as Terraform variables. A private cluster registers through the agent instead and passes no credentials — the same public/private split as every other resource type.

The public path's token is the minted ServiceAccount's long-lived token Secret, applied and read back in a single runner invocation (sentinel-wrapped, so noisy runner log capture can't corrupt it). If the cluster never populates that Secret, the job falls back to a TokenRequest (kubectl create token) and logs a warning: TokenRequest lifetimes are capped by the cluster (e.g. ~48 h on GKE), so an integration registered that way stops working when the token expires — re-register before then, or install the agent and use In-Cluster access.

The fine-grained tier is the impersonator model. Rather than Entitle minting cluster credentials, POST /api/k8s/clusters/{id}/impersonator grants the Entra group cluster-wide impersonate on users. Entitle then JIT-binds <prefix>:<email> to a role, and the operator runs kubectl --as=<prefix>:<email> over the API tunnel. The prefix is entitle_k8s_user_prefix (default entitle).

Note that impersonation needs the API (TCP) tunnel, not the Web Jump — a browser jump can't carry impersonation headers. Full context in Kubernetes.

Per-build opt-in (VMs and databases)

For VMs and databases — the resources the dashboard builds — registration is opt-in per build. The build forms (AWS deploy / bulk deploy) show a "Register in Entitle for just-in-time SSH access" checkbox; cloud-database provisioning takes a register_in_entitle flag on POST /api/databases. Registration runs only when both the global entitle_registration_enabled capability and the per-build choice are on. Default off everywhere — nothing is registered until an operator and a builder both opt in. Clusters differ: they are registered on demand after the fact (above), so there is no build-time checkbox for them.

Setup

Settings → Integrations → Entitle, or the first-run wizard. Then per-build, check the "Register in Entitle" box.

SSH private key — not a config field. The key Entitle authenticates with is the counterpart of the keypair cloud-init injected into the VM at build time, resolved per-cloud from the dashboard's existing SSH keypair secret (Azure azure_ssh_keypair_secret_name, AWS ec2_ssh_key_secret / ec2/keypairs/<name>, GCP gcp_ssh_key_secret_name) — not a separately-configured Entitle key. entitle_ssh_private_key_ref exists only as an optional global fallback/override. See the design doc.

Application slugs: application.name is a lowercase catalog slug — postgresql is confirmed; mysql / mssql / ssh are best-effort. Confirm against the entitle_applications data source for your tenant and adjust _APP_SLUG in the service if they differ.

Connecting to an SSH ephemeral-account grant

When a grant on a registered VM is approved, Entitle delivers it (in Teams, for example) as a single bash one-liner:

echo -e '<PEM with literal \n>' > user_NNNN.pem && chmod 600 user_NNNN.pem && ssh -i user_NNNN.pem user_NNNN@<private-ip>

Two things about it are easy to miss:

• Every grant mints a new account. The username (karenwalker_8762, then karenwalker_8274 on the next grant) and the key are both new. When a grant ends, Entitle deletes that account (userdel). A key from an earlier message, or an old username, fails with the same Permission denied (publickey) a wrong key gives. Take the key, username and host from the same grant message.
• The host is a private address. Lab VMs sit on private subnets (10.99.2.0/24 for the Azure vm-subnet). Entitle gets to them through its agent (see above), but that is the management path only. Your machine needs its own route, so start a PRA Network Tunnel through the gateway (clouddb-jumpoint on Azure) before you run ssh. Without it, ssh reports Connection timed out. See Network tunnels need an address pool if the tunnel itself won't come up.

On Linux or macOS, paste the one-liner as it is.

From Windows

The one-liner assumes bash and a POSIX filesystem. Neither is true on Windows, and each of the obvious workarounds breaks in its own way:

• WSL, in a /mnt/c/... directory: chmod does nothing on a Windows drive, so the file stays 0777. ssh then prints Permissions 0777 for 'user_NNNN.pem' are too open and ignores the key. Run the one-liner from your WSL home directory (cd ~ first) instead, or use PowerShell.
• PowerShell ISE: ISE cannot run ssh interactively. The host-key confirmation prompt never shows up, so the session appears to hang, and ssh's normal stderr is displayed as a red NativeCommandError. Use a regular PowerShell window or Windows Terminal.

In a regular PowerShell window, paste all of this at once. Put everything between the quotes of echo -e into $key, and the grant's username and host into the ssh line:

$key = '<everything between the quotes of echo -e>'
$path = "$HOME\.ssh\entitle-$(Get-Date -Format yyyyMMdd-HHmmss).pem"
[IO.File]::WriteAllText($path, ($key -replace '\\n', "`n"))
icacls $path /inheritance:r /grant:r "$($env:USERNAME):(R)"
ssh-keygen -lf $path
ssh -i $path <user>@<host>

What each part is for:

• [IO.File]::WriteAllText, not Out-File or >. It writes the key with no byte-order mark and with LF line endings. A BOM or CRLF endings can stop ssh from reading the key.
• A new, timestamped filename on every run. After the icacls line removes inheritance and leaves you read-only access, you can't overwrite the file, and even Remove-Item -Force fails on it. Writing each grant's key to a new file avoids this. Paste the whole block every time, too: a $key left over from an earlier paste still holds the previous grant's key.
• ssh-keygen -lf prints the key's fingerprint. Compare it with the fingerprint of the key the server actually installed (see Troubleshooting).

Machine identity — JIT cloud credentials via Entitle

There's a separate track that covers what credentials get used when the dashboard executes a privileged action against AWS / Azure / GCP — the Cloud-Identity JIT design.

Today the dashboard's three cloud identities (AWS IAM user, Azure Service Principal, GCP Service Account) carry broad standing privilege all the time. The machine-identity track replaces that with per-request, short-TTL elevations issued through the same Entitle tenant — so an EC2 deploy triggers a fresh IAM role grant scoped to that one action, valid for ~15 minutes, audited end-to-end in Entitle, and auto-revoked afterwards. No long-lived keys in the dashboard.

Setup additions (when enabling the machine-identity track)

Further reading

• docs/design/cloud-identity-jit.md — full design, threat model, per-cloud trade-offs.
• docs/runbooks/cloud-identity-jit-phase-1-entitle-submit.md — first end-to-end Entitle submit-and-poll loop; requires a configured Entitle tenant.

User JIT — Entra-group-backed dashboard permissions via Entitle

The User-JIT track governs what permissions the user has at all — granted just-in-time through Entitle instead of statically assigned by an admin.

Today the dashboard's user permissions (e.g. aws:write, images:delete, dashboard-admin) are static — once granted, they persist until an admin revokes them. The Entitle User-JIT track lets users request those permissions through the same Entitle workflow, and the dashboard recognises the time-bound grant on the next login or token refresh (via Entra group membership).

Setup additions (when enabling the User-JIT track)

The Terraform module under terraform/entitle_user_jit/ covers the Entitle side (one application + workflows + resources + policies). The Entra bootstrap is a separate script: python -m web_dashboard.scripts.bootstrap_entra_groups.

Further reading

• docs/design/entitle-user-jit.md — full design, operation matrix, OAuth resolution flow.
• docs/runbooks/entitle-user-jit-phase-2-bootstrap-entitle.md — Entitle virtual-application provisioner.

Workload identity

Instead of the API token, the dashboard's REST calls to Entitle can present the dashboard's own workload identity: set Authentication → Workload identity on this panel, after setting up the shared identity and adding Entitle to its Pathfinder registration's Product Scopes. Resource registration (the entitleio/entitle Terraform provider) still uses the API key or token, so keep one set. Not yet verified live — see BeyondTrust platform identity.

Troubleshooting

Nothing registers in Entitle — confirm both the global Registration enabled capability and the per-build "Register in Entitle" checkbox are on, then check the job log: docker compose logs app | grep -i entitle. Registration is non-fatal — the VM/DB still provisions; the job message records why registration was skipped or failed.

"an Entitle Agent token named X already exists in the tenant" (the raw provider form is Failed to create the Agent Token, status code: 400 … Resource already exists) — the token mint failed, so the k8s_entitle_agent job died before touching the cluster; nothing was installed and Live Output stays empty. It means the tenant already holds a token of that name while this dashboard holds no copy of its value. The value is returned only at creation and cannot be read back, and ensure_agent_token already tries to recover it from entitle_agent_token_tf_state before minting — so reaching this error means there is no stored state either (config migration treats all three keys as runtime handles and drops them: see config_migrate/classify.py _RUNTIME_HANDLES). Teardown now destroys the auto-minted token (the agent remove action, and the decommission of the hosting cluster), so a conflicting token was minted by an earlier dashboard version, a different dashboard/environment, or by hand. Fix by one of: delete that token in Entitle and retry — which breaks any agent still using it; set ENTITLE_AGENT_TOKEN_NAME to an unused name so a fresh token is minted; or set ENTITLE_AGENT_TOKEN_REF to the existing token value. Both keys are env/.env only — entitle_agent_token_name is read-only in the Settings panel and entitle_agent_token_ref is not in it at all. To check whether an agent is in fact installed, the Kubernetes page's per-cluster flag is just entitle_agent_cluster_id == cluster.id; confirm against reality with helm list -n entitle.

"private target requires entitle_agent_token_name" — the resource is private and no Entitle agent is configured. Either provision the agent (Kubernetes) and set entitle_agent_token_name, or register only public resources.

"Failed to fetch the resources of <target>" (SSH ephemeral) — the agent can't reach the private host on port 22. Almost always a network path: the agent's cluster VPC has no route to the target (e.g. a GKE cluster in its own isolated VPC, and the private VM in the sandbox vm-subnet). Fix = ensure the cluster module peered back to the sandbox VPC — set gcp_network (GCP) / aws_vpc_id (AWS) before provisioning the cluster, then re-provision so the peering + lab-VM firewall are created (see Network reachability above). Verify from a pod: nc -vz <target-private-ip> 22. (For the Kubernetes connector — not SSH — the same message instead means the agent SA lacks cluster RBAC; that's handled by the cluster-admin binding setup_entitle_agent applies.)

Permission denied (publickey) or Connection timed out connecting to an SSH grant — work through these in order (details in Connecting to an SSH ephemeral-account grant):

1. Connection timed out: there's no route from your machine to the private address. Start the PRA Network Tunnel through the gateway first.
2. Permissions 0777 ... are too open: you ran the one-liner in WSL from a /mnt/c/... directory. Run it from ~ instead, or use the PowerShell steps.
3. Permission denied (publickey): the key, username or host is from a different grant than the other two. Each grant creates a new account, and when a grant ends its account is deleted. The server's /var/log/auth.log shows Invalid user <name> for an account that no longer exists. When a key really is rejected, the log shows the fingerprint of the key your client offered. Compare that with ssh-keygen -lf run on your .pem and on the account's ~/.ssh/authorized_keys. On Azure you can read the log without logging in: az vm run-command invoke --command-id RunShellScript --scripts "tail -50 /var/log/auth.log".
4. You can't overwrite or delete the .pem: icacls left you read-only access. Write each new grant's key to a new filename.
5. The prompt never appears, or the output is red NativeCommandError: you're in PowerShell ISE. Use a regular PowerShell window or Windows Terminal.

"entitle_owner_id / entitle_workflow_id is not configured" — both are required to create an integration. Fill them in under Settings → Integrations → Entitle.

terraform init can't find the provider ("was not found in any of the search locations") — the entitleio/entitle provider is baked into the image's read-only provider mirror ($TF_PROVIDER_MIRROR_DIR, served via /etc/terraform.tfrc), and the registry is deliberately not a fallback. Rebuild the image if you changed the Dockerfile provider pre-cache step or the provider version constraint.

Wrong application / connection — application.name and the connection_json keys are application-specific. Verify the slug against the entitle_applications data source and the per-application connection schema in the BeyondTrust integration docs linked above.