Fix SCCM Secondary Server Recovery Failed Issue | ConfigMgr

0
Secondary Server Recovery Failed

Let’s check how to fix SCCM secondary server recovery failed issue. I got this issue while installing a hotfix to the configuration manager secondary site server. You can check Show Install Status wizard shows the replication between secondary and primary is not active.

Secondary Server Recovery Failed

The secondary site hotfix installation (recovery failed). And when I checked the console to get more details about the issue. You can follow the steps mentioned below to get more details of the installation:

  • Administration – Site ConfigurationSites.
  • Select the secondary server.
  • Shows Recovery Failed as the state of the secondary server.
  • Right-click and click on Show Install Status.
[Passed]:Waiting for Database Replication Link State to be active.

I waited for many hours and restarted the secondary server. However, the link between secondary and primary link didn’t come up.

SCCM Secondary Server Recovery Failed Issue | Fix | ConfigMgr
SCCM Secondary Server Recovery Failed Issue | Fix | ConfigMgr

Link State is Degraded

Now, let’s check the Replication Link from monitoring workspace.

  • Navigate to \Monitoring\Overview\Database Replication
  • Check the Link State
  • The link-state = Link Degraded
SCCM Secondary Server Recovery Failed Issue | Fix | ConfigMgr
SCCM Secondary Server Recovery Failed Issue | Fix | ConfigMgr

If you go into the details of the replication link, you can see that the two replication groups failed to replicate between secondary and primary servers.

  • Secondary Site Data – Successful
  • Secondary Site Updates – Failed
  • Secondary_Site_Replication_Configuration – Failed
SCCM Secondary Server Recovery Failed Issue | Fix | ConfigMgr
SCCM Secondary Server Recovery Failed Issue | Fix | ConfigMgr

Troubleshooting

Check the following steps to troubleshoot the secondary server recovery issue.

Logs File

Check the logs on the ConfigMgr.log file on the ConfigMgr secondary server that is failed to recover. I could see two errors but I don’t think the registry error 0x80070002 is not a very significant error.

However, the error related to sending state messages to the primary server Failed is significant error 0x00060002.

INFO: Failed to open registry key SOFTWARE\Microsoft\ConfigMgrBootStrap\BootStrap\SecInstallmsg\ (LastError=0x80070002)
INFO: Failed to send message 0x00060002 to parent.
INFO: Removing byte order marker after reading from file (C:\Program Files\Microsoft Configuration Manager\bin\X64\secondarysiteupdatepackage.xml)
Successfully update secondary site update pacakge status from file C:\Program Files\Microsoft Configuration Manager\bin\X64\secondarysiteupdatepackage.xml
INFO: Successfully begin Automatic Updates detection task
Not recovery mode or not top level site. Skip restoring client piloting packages.
~~===================== Completed Configuration Manager Server Setup ===================== 

Database Replication – Save Diagnostic File

Let’s check Diagnostic files from database replication option.

  • Navigate to MonitoringDatabase Replication
  • Right-click on database replication between the secondary and primary server
  • Select Save Diagnostic Files
  • Save the file into CSV format.
SCCM Secondary Server Recovery Failed Issue | Fix | ConfigMgr
SCCM Secondary Server Recovery Failed Issue | Fix | ConfigMgr

Details of Save Diagnostic Files of Secondary server is given below:

Summary
MEM <-> HS0
Parent Site = MEM
Parent Site State = Replication Active
Child Site = HS0
Child Site State = Replication Active
Parent Site to Child Site Global State = Link Degraded
Parent Site to Child Site Global Synchronization Time = 10/4/2020 5:14:11 PM
Child Site to Parent Site Global State = Link Active
Last Synchronization Time = 10/4/2020 5:24:47 PM
Child Site to Parent Site State = Not Applicable
Child Site to Parent Site Synchronization Time = 1/1/1900 12:00:00 AM
Child Site (HS0)Child Site Configuration
State,Monitored Item,Current Configuration,Description
Unknown,"Machine certificate","cn=MEMCMSecondary.memcm.com
Expires: 2120-07-09","Certificate is still valid for MEMCMSecondary.memcm.com."
Unknown,"SQL Server certificate","cn=SSB Transport Security Certificate
Expires: 2040-08-02","Service Broker certificate is still valid for MEMCMSecondary.memcm.com."
Unknown,"SQL Server port","1433","Port 1433 still valid for MEMCMSecondary.memcm.com."
Unknown,"SQL Server service broker port","4022","Service Broker MEMCMSecondary.memcm.com Port 4022 still valid."
Unknown,"Database file location","C:\Program Files\Microsoft SQL Server\MSSQL12.MSSQLSERVER\MSSQL\DATA\CM_HS0.mdf
C:\Program Files\Microsoft SQL Server\MSSQL12.MSSQLSERVER\MSSQL\DATA\CM_HS0_log.ldf","Configuration Manager Database file location is still valid."
Unknown,"Database file disk free space","C:\ 97GB","No alerts configured. Use Site System properties to configure alerts."
Unknown,"Computer account","memcm.com\MEMCMSecondary$","Configuration Manager Site Server Account MEMCMSecondary.memcm.com still valid."
Unknown,"SQL Server role","smsdbrole_MP, smsdbrole_MCS, smsdbrole_DMP, smsdbrole_siteprovider, smsdbrole_siteserver, smsdbrole_AMTSP, smsdbrole_AIUS, smsdbrole_AITool, smsdbrole_extract, smsdbrole_WebPortal, smsdbrole_MPUserSvc, smsdbrole_MPMBAM, smsdbrole_AUDITMBAM, smsdbrole_EnrollSvr, smsdbrole_DViewAccess, smsdbrole_SUP, smsdbrole_CRP, smsdbrole_DWSS, smsdbrole_CRPPfx, smsschm_users, smsdbrole_DmpConnector, smsdbrole_HMSUser","All Configuration Manager SQL Roles still valid."
Unknown,"Firewall ports","1433, 4022","Configuration Manager SQL Server ports 1433, 4022 still active on Firewall exception."
Unknown,"SQL Server secondary replica",,"_103"
Unknown,"SQL Server availability group failover",,"_103"

Replication Link Analyzer

Let’s analyze the SQL based replication link between secondary and primary servers.

  • Navigate to MonitoringDatabase Replication
  • Right-click on database replication between the secondary and primary server
  • Select Replication Link Analyzer
SCCM Secondary Server Recovery Failed Issue | Fix | ConfigMgr
SCCM Secondary Server Recovery Failed Issue | Fix | ConfigMgr

The replication link analyze shows the following message – Inconsistent public keys can be remediated by initiating a public key transfer between sites HSO and MEM.

The actions recommended to fix the issue

  1. Initiate public key transfer for target site MEM on source site HSO
  2. Check to see if the problem is fixed
  3. Skip this rule
SCCM Secondary Server Recovery Failed Issue | Fix | ConfigMgr
SCCM Secondary Server Recovery Failed Issue | Fix | ConfigMgr
  • Successfully submitted a request to initiate the key transfer for target site MEM on source site HSO.
    • Click on Continue
SCCM Secondary Server Recovery Failed Issue | Fix | ConfigMgr
SCCM Secondary Server Recovery Failed Issue | Fix | ConfigMgr
  • Verifying database CM_MEM has valid security scope for Local System Account
  • The replication link analyzer detected Database CM_MEM does not have a valid security scope for the local system account.
    • Please make sure login for the local system account exists and it has a sysadmin role assigned.
Fix SCCM Secondary Server Recovery Failed Issue | ConfigMgr 1
SCCM Secondary Server Recovery Failed Issue | Fix | ConfigMgr
  • Added NT Authority/System to SysAdmin role in the primary server SQL database.
  • Click OK to save.
SCCM Secondary Server Recovery Failed Issue | Fix | ConfigMgr
SCCM Secondary Server Recovery Failed Issue | Fix | ConfigMgr
  • It seems you need to Reset queued messages on SQLMEMCM.memcm.com for site HSO.
  • The replication link analyzer recommends resetting queued replication messages on SQLMEMCM.memcm.com for site HSO.
  • Click on Reset Queued Messages.
SCCM Secondary Server Recovery Failed Issue | Fix | ConfigMgr
SCCM Secondary Server Recovery Failed Issue | Fix | ConfigMgr
  • Successfully reset queued replication messages on primary SQL server for the secondary server.
SCCM Secondary Server Recovery Failed Issue | Fix | ConfigMgr
SCCM Secondary Server Recovery Failed Issue | Fix | ConfigMgr
  • Create file replication route for site MEM on-site HSO
    • File replication route is required for the site to site communication and the file replication route is missing for site MEM on-site HSO
    • Select Create file replication route option
Fix SCCM Secondary Server Recovery Failed Issue | ConfigMgr 2
  • File Replication route exists for site MEM on site HS0.
Fix SCCM Secondary Server Recovery Failed Issue | ConfigMgr 3

Failed Replication

Link failed after replication link analyzer check 🙁

Fix SCCM Secondary Server Recovery Failed Issue | ConfigMgr 4

Fix?

Let’s check what can you do to fix the issue. The only option left for me is to recover the secondary site again. Go through the following steps to confirm the same:

  • Launch SCCM console
  • Navigate to \Administration\Overview\Site Configuration\Sites
  • Click secondary server and click on Recover Secondary Site from the ribbon menu
Secondary Server Recovery
SCCM Secondary Server Recovery Failed Issue | Fix | ConfigMgr
  • Check the ConfigMgrSetup.log file to confirm
    • INFO: Secondary Site is now active
    • INFO: send message 0x00060007 to parent
    • INFO: send message 0x00060008 to parent
    • INFO: verifying content metadata (1 processed)…
    • Completed Configuration Manager Server Setup
Fix SCCM Secondary Server Recovery Failed Issue | ConfigMgr 5
  • Check the console to confirm whether the site is active:
SCCM Secondary Server Recovery Failed Issue | Fix | ConfigMgr
SCCM Secondary Server Recovery Failed Issue | Fix | ConfigMgr
  • SQL query to confirm whether the secondary is installed with updates or not.
    • If the return value is 1, that means is hotfix is installed successfully.
SCCM Secondary Server Recovery Failed Issue | Fix | ConfigMgr
SCCM Secondary Server Recovery Failed Issue | Fix | ConfigMgr

Resources

LEAVE A REPLY

Please enter your comment!
Please enter your name here

This site uses Akismet to reduce spam. Learn how your comment data is processed.