vCenter Certificate Status alarm caused by expired BACKUP_STORE certificates

Problem

The vCenter Server object shows a persistent alarm:

Certificate Status

However, the active vCenter certificates are still valid.

This can happen when old certificates created during a previous certificate replacement remain in the VMware Endpoint Certificate Store (VECS) BACKUP_STORE and later expire.

Typical symptoms

• The vCenter object shows a red Certificate Status alarm
• MACHINE_SSL_CERT is still valid
• Solution User certificates are still valid
• Trusted root certificates are still valid
• The only expired entries are aliases such as:

bkp___MACHINE_CERT
bkp_machine
bkp_vsphere-webclient
bkp_vpxd
bkp_vpxd-extension
bkp_hvc
bkp_wcp

These entries are historical backups and are not the active certificates used by vCenter.

Why this happens

When vCenter certificates are replaced, older Machine SSL and Solution User certificates can be copied into backup VECS stores so that rollback is possible.

Broadcom documents that expired certificates in BACKUP_STORE or BACKUP_STORE_H5C can trigger the generic Certificate Status alarm.

The behaviour is version-dependent. vCenter 7.0 Update 3 and later avoid triggering the alarm for expired BACKUP_STORE entries, although BACKUP_STORE_H5C can still require cleanup.

Step 1 - Check every VECS store

SSH to the VCSA as root, enter the Bash shell if required, and run:

for store in $(/usr/lib/vmware-vmafd/bin/vecs-cli store list | grep -v TRUSTED_ROOT_CRLS); do
  echo "===== STORE: $store ====="
  /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store "$store" --text |
  grep -ie "Alias" -ie "Not After"
done

Review every Not After date.

Do not assume the Machine SSL certificate is the problem simply because the alarm is named Certificate Status.

Step 2 - Distinguish active certificates from backup certificates

A healthy result may show active certificates valid well into the future:

===== STORE: MACHINE_SSL_CERT =====
Alias : __MACHINE_CERT
Not After : Dec 10 10:38:05 2028 GMT

===== STORE: machine =====
Alias : machine
Not After : Dec 10 10:39:06 2028 GMT

while the backup store contains expired historical entries:

===== STORE: BACKUP_STORE =====
Alias : bkp___MACHINE_CERT
Not After : Dec 5 14:53:14 2025 GMT

Alias : bkp_machine
Not After : Dec 5 14:57:40 2025 GMT

If only the backup certificates are expired, do not renew the active Machine SSL or Solution User certificates.

Step 3 - Protect the vCenter before modifying VECS

Before changing certificate stores, follow Broadcom's rollback requirements.

For a standalone vCenter, take a supported backup or snapshot.

For Enhanced Linked Mode, Broadcom requires coordinated powered-off snapshots of all vCenter Servers in the same SSO domain.

Step 4 - Use vCert to clear expired BACKUP_STORE entries

Download the current vCert package from the official Broadcom KB, copy it to the VCSA, verify the checksum published for that release, extract it, and run:

./vCert.py

From the menu select:

3. Manage Certificates
12. Clear expired certificates in BACKUP_STORE in VECS

The tool identifies expired entries and removes them from the backup store.

A successful run returns OK for the affected aliases.

Step 5 - Confirm the backup store is clean

After exiting vCert, run:

/usr/lib/vmware-vmafd/bin/vecs-cli entry list --store BACKUP_STORE --text |
grep -ie "Alias" -ie "Not After"

If the expired entries were removed and no remaining entries match the filter, the command returns no output.

If BACKUP_STORE_H5C exists in the environment, validate that store as well.

Step 6 - Reset the alarm

Once the expired backup entries are removed and the active certificates are confirmed healthy:

vCenter Server
  -> Monitor
  -> Issues and Alarms
  -> Triggered Alarms
  -> Certificate Status
  -> Reset to Green

The alarm may remain triggered after the underlying condition is corrected, so a manual reset can be required.

Operational impact

Clearing expired backup certificates does not replace the active Machine SSL or Solution User certificates.

Broadcom states that expired certificates in BACKUP_STORE have no operational impact by themselves because they are historical backup copies.

A vCenter service restart is not normally required when vCert successfully clears the backup entries and the alarm can be reset.

Manual fallback

If vCert cannot clear the store, Broadcom documents a manual method.

List the entries:

/usr/lib/vmware-vmafd/bin/vecs-cli entry list --store BACKUP_STORE

Delete only the confirmed expired backup alias:

/usr/lib/vmware-vmafd/bin/vecs-cli entry delete   --store BACKUP_STORE   --alias <expired_backup_alias>

Use the manual method only after confirming that the active certificates are valid and rollback protection exists.

Validation checklist

Confirm all of the following:

• Active Machine SSL certificate is valid
• Solution User certificates are valid
• Trusted Roots are valid
• Expired BACKUP_STORE entries are gone
• vCenter services remain healthy
• Certificate Status is reset to green
• The alarm does not return after the next certificate health evaluation

Prevention / best practice

• Check all VECS stores when a generic certificate alarm appears
• Keep a current VCSA backup
• Use vCert for supported certificate health and cleanup workflows
• Do not renew healthy active certificates just because stale backup entries have expired
• Record the vCenter version because BACKUP_STORE alarm behaviour is version-dependent
• Verify the certificate state after every certificate replacement

Quick diagnostic sequence

# List all stores and certificate expiry dates
for store in $(/usr/lib/vmware-vmafd/bin/vecs-cli store list | grep -v TRUSTED_ROOT_CRLS); do
  echo "===== STORE: $store ====="
  /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store "$store" --text |
  grep -ie "Alias" -ie "Not After"
done

# Check BACKUP_STORE after cleanup
/usr/lib/vmware-vmafd/bin/vecs-cli entry list --store BACKUP_STORE --text |
grep -ie "Alias" -ie "Not After"

Related knowledge

• vCenter Server certificate stores and common certificate alarms
• vCenter virtual disks show 0 GB or blank properties due to SSL trust mismatch
• ESXi Host Certificate Status shows Expiration imminent

Sources

See the authoritative Broadcom references in the article metadata.