Fix – CMG Client Communication Failure Error 0x87d0027e

2
SCCM CMG Client Communication Failure

Cloud Management Gateway (CMG) is the most talked feature these days as it became a full release feature from SCCM CB 1802 onwards. If you are looking for the deployment steps, CMG Client communication post is not the right post and you can check the detailed tutorial by Anoop on this here.

Content

  1. SCCM CMG – Problem Description
  2. Troubleshooting – SCCM CMG Client Communication Failure
  3. Deep Dive in to Firewall, PKI, etc. – CMG Client Communication Failure
  4. Solution – CMG Client Communication Failure
  5. SCCM CMG –  Is there limitation in Uploading Client Certs?

SCCM CMG – Problem Description

While working on deploying Cloud Management gateway (CMG) for one of the organization, I came across a client communication failure issue. Thought of putting it as a blog so that others can benefit from it and don’t have to go through the same pain we had.

We deployed Cloud Management Gateway (CMG) in Azure, which was successful and then we tested the features on one of the client machines successfully. When moved on with the testing for a larger UAT group, the client communication was failing for majority of the laptops.

Troubleshooting – SCCM CMG Client Communication Failure

We started the troubleshooting via log files. I try to explain how you can troubleshoot SCCM CMG Client communication Failure issues. When you check the locationservices.log, it will be showing INF (Internet Facing) MP failed to communicate and the MP switching happens between the INF MP and on-prem MP very frequently.

log snippet from locationservices.log (Domain name changed for security reason)

[CCMHTTP] ERROR: URL=https://MYCMG.MYDOMAIN.COM/CCM_Proxy_MutualAuth/720575940XXXXXXXX/SMS_MP/.sms_aut?MPLIST2&PR1, Port=443, Options=448, Code=0, Text=CCM_E_BAD_HTTP_STATUS_CODE LocationServices 6/21/2018 2:17:18 PM 11180 (0x2BAC)
Raising event:
instance of CCM_CcmHttp_Status
{
ClientID = "GUID:B704E758-91A8-479B-A629-747A66E4065B";
DateTime = "20180621121718.306000+000";
HostName = "MYCMG.MYDOMAIN.COM";
HRESULT = "0x87d0027e";
ProcessID = 5448;
StatusCode = 401;
ThreadID = 11180;
};
LocationServices 6/21/2018 2:17:18 PM 11180 (0x2BAC)
Successfully queued event on HTTP/HTTPS failure for server 'MYCMG.MYDOMAIN.COM'. LocationServices 6/21/2018 2:17:18 PM 11180 (0x2BAC)

Similiar error can be seen in the CCMMessaging.log

Successfully queued event on HTTP/HTTPS failure for server 'MYCMG.MYDOMAIN.COM'. CcmMessaging 6/21/2018 1:50:15 PM 7348 (0x1CB4)
Post to https://MYCMG.MYDOMAIN.COM/CCM_Proxy_MutualAuth/720575940XXXXXXXX/ccm_system/request failed with 0x87d00231. CcmMessaging 6/21/2018 1:50:15 PM 7348 (0x1CB4)
[CCMHTTP] ERROR: URL=https://MYCMG.MYDOMAIN.COM/CCM_Proxy_MutualAuth/720575940XXXXXXXX/ccm_system/request, Port=443, Options=448, Code=0, Text=CCM_E_BAD_HTTP_STATUS_CODE CcmMessaging 6/21/2018 2:00:26 PM 9468 (0x24FC)
Raising event:
instance of CCM_CcmHttp_Status
{
ClientID = "GUID:B704E758-91A8-479B-A629-747A66E4065B";
DateTime = "20180621120026.920000+000";
HostName = "MYCMG.MYDOMAIN.COM";
HRESULT = "0x87d0027e";
ProcessID = 5448;
StatusCode = 401;
ThreadID = 9468;
};
CcmMessaging 6/21/2018 2:00:26 PM 9468 (0x24FC)

Deep Dive in to Firewall, PKI, etc. – CMG Client Communication Failure

The first thing we checked here is the port 443 connectivity from this test machine to the CMG public IP using portquery UI tool. Port connectivity was fine and it was listening for port 443 without any issue.

After hours of troubleshooting, we identified that the PKI infrastructure is having multiple Issuing CA’s. Below Picture will give an overview of how the PKI infrastructure is.

CMG Client Communication Failure
PKI Infrastructure

In this scenario, the certificates on the server and few devices were issued by Issuing CA 1 and hence we have uploaded the RootCA, Intermediate CA, Issuing CA 1 to Azure while installing CMG. So whichever machines got certificate enrolled from Issuing CA 1 the client communication was a success and for others, the client communication was failing because they were missing the Certificate Chain.

Solution – CMG Client Communication Failure 

So to rectify the problem, we have to upload all the certs in such a way that their certificate chain is not broken.

For Example, In our case here below is the list of certs that should be provided to Azure while installing the CMG

Root CA
Intermediate CA
Issuing CA 1
Issuing CA 2
Issuing CA 3
Issuing CA 4

SCCM CMG –  Is there limitation in Uploading Client Certs?

Note: Currently there is a restriction to upload only 6 (2 root CA and 4 Intermediate CA)certs while deploying a CMG. This may be changing in future releases.

CMG Client Communication Failure

2 COMMENTS

LEAVE A REPLY

Please enter your comment!
Please enter your name here

This site uses Akismet to reduce spam. Learn how your comment data is processed.