Skip to main content

Troubleshooting

Login redirects, signs out, or fails intermittently​

  • Confirm the browser-visible OIDC issuer exactly matches the token issuer.
  • Test OIDC discovery and token endpoints from every application node. One node with blocked egress causes intermittent failures behind a healthy Service.
  • For multiple replicas, configure one shared session-encryption key. Sticky ingress sessions are not the primary fix.
  • In standalone, open http://host.docker.internal:8090, not localhost.

Vault login returns 403​

Decode the pod JWT and confirm its sub matches a ServiceAccount bound to the Vault role. Then inspect vault read auth/<mount>/config:

  • token_reviewer_jwt_set=true: verify the reviewer ServiceAccount and TokenReview permission.
  • token_reviewer_jwt_set=false with disable_local_ca_jwt=true: enable vaultKubernetesAuth.appAuthDelegator; the application ServiceAccount needs TokenReview.
  • Confirm projected-versus-legacy token expectations before enabling legacyToken.

Secret changed but the app still uses old credentials​

Check whether the Secret or ExternalSecret actually synchronized. One missing remote property can leave an ExternalSecret at its last good value. Kubernetes secretKeyRef environment variables are loaded at process start, so roll the Deployment after successful synchronization.

Registration or cluster probe fails​

Verify routing from the DonkeyFleet runtime, the Vault path and username/password keys, TLS trust, and least-privilege ONTAP access. Workstation connectivity alone is not sufficient.

Run says skipped_observe_incomplete​

An ONTAP read failed, pagination did not finish, or record counts disagreed. Do not bypass this guard. Fix reachability, credentials, rate limits, timeouts, or the endpoint response.

Approved work does not execute​

Check effective dry-run, minimum volume age, current profile membership, max_actions_per_run, baseline concurrency, bandwidth, and unresolved ONTAP jobs. Approval is authorization, not a guardrail bypass.

Apply is Ready or In flight for too long​

Inspect the checkpoint, stored job UUID, and last error. Ready work waits for admission or a baseline slot. In-flight work must poll the existing ONTAP job. Do not manually repeat create or initialize while job outcome is unresolved.

The next active-provisioning follow-up normally runs after 15 seconds; long baseline monitoring returns to the normal five-minute cadence.

Baseline reports unhealthy​

Initialization can remain uninitialized and unhealthy for hours. Check transfer and job state; do not suspend it solely for not being complete. Overrun alerts use the larger of the physical-size estimate and a ten-minute floor, multiplied by the configured factor.

Managed relationship becomes suspended after a completed baseline​

Upgrade to a build containing Flyway V16 and restart against the preserved PostgreSQL database. That migration repairs relationships affected by the earlier completed-baseline lifecycle defect.

Run says skipped_peer_down or skipped_orphan_delta_guardrail​

Repair SVM peer health outside DonkeyFleet. For the orphan guardrail, investigate source visibility and UUID identity before changing either the absolute or percentage limit.