CHAT_OUTPUT_BEGIN COMMAND_ID=HOMELAB-TERMINAL-DECISION-136 STATUS=OK RC=0 HOST=pve01 MODE=read-only COMPONENT=cluster-audit REFERENCE_REGISTER_CHECK=OK REFERENCE_SHA256=5763f2f8edc75fcf6f3951f8c95896d1cc112ceccb6cf5b64a383f540a4deb8e ERROR_REGISTER_CHECK=OK ERROR_REGISTER_SHA256=08b3a769b8e38eaa1acec915352fb328180d70caa7ff6cd9d1ec0da426262752 COMMAND_SHA256=04f3260b54a22b3434621d0b827c5ac3bfddc7a2d2fa400e26bc081c3e334b15 DUPLICATE_FAILED_COMMAND_BLOCKED=false EXECUTION_STARTED=true CHANGES_MADE=false SANITIZED=yes SECRETS_INCLUDED=no PRIVATE_ADDRESSES_INCLUDED=no RAW_EVIDENCE_SHA256=9e14acc4431585654d55d0459a67961b329a9bbc8ade0297346746333f398625 SANITIZED_OUTPUT_SHA256=9e14acc4431585654d55d0459a67961b329a9bbc8ade0297346746333f398625 OUTPUT_BEGIN JSON_IMMUTABLE_COMMIT=8457efb7a3b0a859f22b21c16f398f156769bacd TXT_IMMUTABLE_COMMIT=8457efb7a3b0a859f22b21c16f398f156769bacd === IMMUTABLE_135 === COMMAND_ID=HOMELAB-TERMINAL-CLOSURE-135 STATUS=FAIL RC=42 EXECUTION_STARTED=true DUPLICATE_FAILED_COMMAND_BLOCKED=false ERROR_REGISTERED=no TERMINAL_RESULT=BLOCKED WORKFLOW_STATUS=REQUIRES_ACTION TOTAL_FINAL_RISKS=21 NEXT_ACTION=RECONCILE_DUAL_MAIL_CLOUD_BACKUPS HOMELAB_FINAL_RESULT=BLOCKED BLOCKER_CLASS=RECONCILE_DUAL_MAIL_CLOUD_BACKUPS === BLOCKER_COUNTS === PROBE_FAILURES=0 QUORUM_RISKS=0 FAILED_UNIT_RISKS=3 STORAGE_STATE_RISKS=0 CAPACITY_RISKS=0 KERNEL_RISKS=0 BACKUP_POLICY_RISKS=6 LOCAL_BACKUP_ARCHIVES=12 TOTAL_FINAL_RISKS=21 === BLOCKER_EVIDENCE === === BLOCKER_EVIDENCE === Cluster information ------------------- Name: homelab Config Version: 3 Transport: knet Secure auth: on Quorum information ------------------ Date: Tue Jul 21 22:48:26 2026 Quorum provider: corosync_votequorum Nodes: 3 Node ID: 0x00000001 Ring ID: 1.9d Quorate: Yes Votequorum information ---------------------- Expected votes: 3 Highest expected: 3 Total votes: 3 Quorum: 2 Flags: Quorate Membership information ---------------------- Nodeid Votes Name 0x00000001 1 [PRIVATE_IP] (local) 0x00000002 1 [PRIVATE_IP] 0x00000003 1 [PRIVATE_IP] homelab-backup-matrix-refresh.service loaded failed failed Generate unified homelab backup matrix state homelab-cluster-admin-pull.service loaded failed failed Pull cluster-admin observer health into pve01 homelab-vm160-remote-backup.service loaded failed failed VM160 full-image backup streamed from pve02 to pve03 INFO: starting new backup job: vzdump 110 111 112 113 130 150 190 --mailnotification failure --prune-backups 'keep-daily=4' --mode snapshot --storage local --compress zstd --quiet 1 STATUS=WARN TS=2026-07-19T05:14:41Z TYPE=overall-health BAD=5 backup-framework.txt=OK drift-check.txt=WARN service-registry.txt=OK dependency-map.txt=OK golden-state.txt=OK cluster-passport.txt=WARN final-readiness.txt=WARN safe-autoheal.txt=OK kuma-monitor-policy.txt=OK backup-sla.txt=OK backup-coverage-matrix.txt=OK extended-appbackup.txt=OK residual-review.txt=OK forum-snuffleupagus.txt=OK incident-journal.txt=OK duty-admin-v2.txt=OK node-loss-readiness.txt=OK node-loss-runbooks.txt=OK power-loss-readiness.txt=OK power-loss-runbook.txt=OK ungated-health-backlog.txt=OK pve03-root-cleanup.txt=OK pve03-staging-retention.txt=OK pve03-staging-retention-proof.txt=OK pve03-staging-retention-cleanup.txt=OK pve03-failed-unit-cleanup.txt=OK remote-git-sops-age-readiness.txt=OK remote-git-sops-age-policy.txt=OK desired-state-remote-target-trace.txt=OK desired-state-remote-target.txt=OK desired-state.txt=OK runbooks.txt=OK alerting.txt=OK secret-exposure.txt=OK capacity-risk.txt=OK vm-local-dumps-retention.txt=OK vm-local-dumps-cloud.txt=OK vm-backup-policy.txt=OK vm170-vm171-full-strategy.txt=OK external-probe-vps.txt=OK netbird-vps-trust.txt=OK netbird-vps-trust-closure.txt=OK external-probe-runner-decision.txt=OK netbird-vps-trust-clean-seal.txt=OK live-tail-audit.txt=OK vm-backup.txt=OK cluster-admin-observer.txt=OK cluster-admin-restricted-probes.txt=WARN cluster-admin-full-observer.txt=WARN cluster-admin-vm-mail-cloud-backup.txt=OK cluster-admin-webpanel-sync.txt=OK cluster-admin-webpanel.txt=OK INFO: starting new backup job: vzdump 110 111 112 113 130 150 190 --prune-backups 'keep-daily=4' --mode snapshot --mailnotification failure --storage local --compress zstd --quiet 1 STATUS=WARN TS=2026-07-20T05:10:25Z TYPE=overall-health BAD=5 backup-framework.txt=OK drift-check.txt=WARN service-registry.txt=OK dependency-map.txt=OK golden-state.txt=OK cluster-passport.txt=WARN final-readiness.txt=WARN safe-autoheal.txt=OK kuma-monitor-policy.txt=OK backup-sla.txt=OK backup-coverage-matrix.txt=OK extended-appbackup.txt=OK residual-review.txt=OK forum-snuffleupagus.txt=OK incident-journal.txt=OK duty-admin-v2.txt=OK node-loss-readiness.txt=OK node-loss-runbooks.txt=OK power-loss-readiness.txt=OK power-loss-runbook.txt=OK ungated-health-backlog.txt=OK pve03-root-cleanup.txt=OK pve03-staging-retention.txt=OK pve03-staging-retention-proof.txt=OK pve03-staging-retention-cleanup.txt=OK pve03-failed-unit-cleanup.txt=OK remote-git-sops-age-readiness.txt=OK remote-git-sops-age-policy.txt=OK desired-state-remote-target-trace.txt=OK desired-state-remote-target.txt=OK desired-state.txt=OK runbooks.txt=OK alerting.txt=OK secret-exposure.txt=OK capacity-risk.txt=OK vm-local-dumps-retention.txt=OK vm-local-dumps-cloud.txt=OK vm-backup-policy.txt=OK vm170-vm171-full-strategy.txt=OK external-probe-vps.txt=OK netbird-vps-trust.txt=OK netbird-vps-trust-closure.txt=OK external-probe-runner-decision.txt=OK netbird-vps-trust-clean-seal.txt=OK live-tail-audit.txt=OK vm-backup.txt=OK cluster-admin-observer.txt=OK cluster-admin-restricted-probes.txt=WARN cluster-admin-full-observer.txt=WARN cluster-admin-vm-mail-cloud-backup.txt=OK cluster-admin-webpanel-sync.txt=OK cluster-admin-webpanel.txt=OK INFO: starting new backup job: vzdump 110 111 112 113 130 150 190 --compress zstd --quiet 1 --mode snapshot --prune-backups 'keep-daily=4' --mailnotification failure --storage local STATUS=WARN TS=2026-07-21T05:01:14Z TYPE=overall-health BAD=7 backup-framework.txt=OK drift-check.txt=WARN service-registry.txt=OK dependency-map.txt=OK golden-state.txt=OK cluster-passport.txt=WARN final-readiness.txt=WARN safe-autoheal.txt=OK kuma-monitor-policy.txt=OK backup-sla.txt=WARN backup-coverage-matrix.txt=OK extended-appbackup.txt=OK residual-review.txt=OK forum-snuffleupagus.txt=OK incident-journal.txt=OK duty-admin-v2.txt=OK node-loss-readiness.txt=OK node-loss-runbooks.txt=OK power-loss-readiness.txt=OK power-loss-runbook.txt=OK ungated-health-backlog.txt=OK pve03-root-cleanup.txt=OK pve03-staging-retention.txt=OK pve03-staging-retention-proof.txt=OK pve03-staging-retention-cleanup.txt=OK pve03-failed-unit-cleanup.txt=OK remote-git-sops-age-readiness.txt=OK remote-git-sops-age-policy.txt=OK desired-state-remote-target-trace.txt=OK desired-state-remote-target.txt=OK desired-state.txt=OK runbooks.txt=OK alerting.txt=OK secret-exposure.txt=OK capacity-risk.txt=OK vm-local-dumps-retention.txt=OK vm-local-dumps-cloud.txt=OK vm-backup-policy.txt=OK vm170-vm171-full-strategy.txt=OK external-probe-vps.txt=OK netbird-vps-trust.txt=OK netbird-vps-trust-closure.txt=OK external-probe-runner-decision.txt=OK netbird-vps-trust-clean-seal.txt=OK live-tail-audit.txt=OK vm-backup.txt=OK cluster-admin-observer.txt=WARN cluster-admin-restricted-probes.txt=WARN cluster-admin-full-observer.txt=WARN cluster-admin-vm-mail-cloud-backup.txt=OK cluster-admin-webpanel-sync.txt=OK cluster-admin-webpanel.txt=OK /var/lib/vz/dump/vzdump-lxc-110-2026_07_21-03_30_02.tar.zst /var/lib/vz/dump/vzdump-lxc-110-2026_07_19-03_45_38.tar.zst /var/lib/vz/dump/vzdump-lxc-112-2026_07_20-03_30_08.tar.zst /var/lib/vz/dump/vzdump-lxc-110-2026_07_18-03_30_00.tar.zst /var/lib/vz/dump/vzdump-qemu-150-2026_07_19-03_30_16.vma.zst /var/lib/vz/dump/vzdump-lxc-112-2026_07_19-03_45_47.tar.zst /var/lib/vz/dump/vzdump-qemu-150-2026_07_21-03_30_16.vma.zst /var/lib/vz/dump/vzdump-qemu-150-2026_07_20-03_30_14.vma.zst /var/lib/vz/dump/vzdump-lxc-110-2026_07_20-03_30_00.tar.zst /var/lib/vz/dump/vzdump-qemu-150-2026_07_18-03_30_13.vma.zst /var/lib/vz/dump/vzdump-lxc-112-2026_07_21-03_30_10.tar.zst /var/lib/vz/dump/vzdump-lxc-112-2026_07_18-03_30_08.tar.zst HOMELAB_TERMINAL_CLOSURE=BLOCKED OUTPUT_END CHAT_OUTPUT_END NEXT_ACTION=RECONCILE_DUAL_MAIL_CLOUD_BACKUPS HOMELAB_TERMINAL_DECISION=COMPLETE OUTPUT_END CHAT_OUTPUT_END