Troubleshoot SCCM Fast Channel Push Notification Issues

Let’s discuss how to Troubleshoot SCCM Fast Channel Push Notification Issues. The fast channel notification feature has been in SCCM products since 2012 SP1. SCCM fast channel notification was mainly used to notify clients about vital policies, collect inventories, etc.

For more details about the run PowerShell script option, refer to the SCCM CB Run PowerShell Script Directly from the Collection post.

SCCM CB 1706 introduced the “RUN Script” option through the fast channel push notification. This post will use a video guide to troubleshoot SCCM CB fast channel push notification issues. A video tutorial about SCCM CB fast channel push notification is here.

Let’s understand Fast channel notifications for clients. SCCM Fast channel notification is a “PUSH” method of notifying clients about the new policies. This communication channel for SCCM client fast notification is TCP (port 10123) or HTTP (port 80).

Sign up to get the best of How To Manage Devices straight to your inbox!

• ConfigMgr Scenario Health Checks | SCCM
• Monitor ConfigMgr Client Status from SCCM Console | Online | Offline
• Overview Of Client Online Status In SCCM Console
• SCCM 2303 Upgrade Step By Step Guide | New Features

How to Troubleshoot SCCM CB Fast Channel Notification Issues

In the video, you’ll find comprehensive details on troubleshooting SCCM CB Fast Channel Notification issues. This guide will cover scenarios where Fast Channel Notifications may fail or encounter issues within the SCCM environment.

What is SCCM Fast Channel Push Notification? – Troubleshoot SCCM Fast Channel Push Notification Issues

The SCCM client communicates to its MP every 15 minutes to confirm that it’s still online. When your client does not show as ONLINE in the SCCM console, we may have a problem with the fast notification communication channel.

SCCM Push Vs. Pull

Historically, SCCM uses the PULL method, expecting the client to ask for new policies regularly. But, the fast channel uses the PUSH method. What is BGB in SCCM? BGB = Fast Channel Notification related components. I don’t know whether this notification channel was codenamed “Big Green Button” or not 😉

What are the Components of SCCM CB Fast Channel Notification?

SCCM CB fast channel notification has three components. The notification manager will be located along with site servers (Primary/Secondary). It generates “push messages” for clients, sends notifications to the BGB server (MP), and stores the results. 

The notification manager initiates push notifications from the site server. The log file BGBmgr.log provides more details about the notification manager. Notification files (*.BOS files) are stored in the INBOX/BGB.box folder. The video tutorial here shows the BOS file being created.

As you can see in the following fast channel notification architecture diagram, when the primary server has an MP component, the notification manager and notification server are also on that primary server.

The notification server will be located along with the Management Point (MP) and secondary sites. It will have TCP and HTTP listeners. These will help listen to the notification manager (DB) push notifications and confirm the client’s online status.

The notification manager pushes result files (*.BTS) from clients. BGBServer.log is the file on the MP setup or site server setup directory.

The Notification Agent is a fast-channel notification component at the SCCM client end. It’s part of the SMS agent (CCMEXEC). The agent establishes a persistent connection with its notification server.

This will receive the PUSH messages from MP. CcmNotificationAgent.log is the log file on the SCCM client device. The log can note MP/Notification server communication errors.

What is the architecture flow of SCCM CB Fast channel push notification?

Why is the SCCM CB Client NOT Showing as ONLINE?

The problem is that the SCCM CB client is not showing as ONLINE in the console. Instead, it always stays offline. The problem is ONLY with FAST notification channel communication; normal deployments and policies are working fine.

Troubleshooting of SCCM CB Fast Channel Notification

First, you must ensure all the notification components are installed correctly on the server and client sides. The following log files can confirm this for installation issues and troubleshooting.

Fast Channel Notification – Server-Side Troubleshooting

I checked the log files on my primary and MP (both are on the same server), and BGBServer.log shows a warning all the time: “WARNING: Notification Server (% systemroot%system32dllhost. exe) with TCP port 10123 is NOT allowed by Windows Firewall on all interfaces I.” But I thought it should work with the port 80 HTTP channel. It was not working as expected.

Following are the extracts of troublesome logs on the BGB notification server.  BGBServer.log

• Starting SMS Notification Server…~~

• lt;SMS_NOTIFICATION_SERVER><08-15-2017 01:16:02.005-330> Server GC is OFF~~

• lt;SMS_NOTIFICATION_SERVER><08-15-2017 01:16:02.006-330> Trigger to start TCP listener~~

• lt;SMS_NOTIFICATION_SERVER><08-15-2017 01:16:02.022-330> The HTTP listener is started~~

• lt;SMS_NOTIFICATION_SERVER><08-15-2017 01:16:02.030-330> Listening connections on port 10123. Waiting for clients to connect…~~

• lt;SMS_NOTIFICATION_SERVER><08-15-2017 01:16:02.051-330> STATMSG: ID=9807 SEV=I LEV=M SOURCE=”SMS Server” COMP=”SMS_NOTIFICATION_SERVER” SYS=SCCMTP1.INTUNE.COM SITE=TP1 PID=3280 TID=1968 GMTDATE=Mon Aug 14 19:46:02.059 2017 ISTR0=”SCCMTP1.INTUNE.COM” ISTR1=”10123″ ISTR2=”” ISTR3=”” ISTR4=”” ISTR5=”” ISTR6=”” ISTR7=”” ISTR8=”” ISTR9=”” NUMATTRS=0

• lt;SMS_NOTIFICATION_SERVER><08-15-2017 01:16:02.061-330> WARNING: Notification Server (%systemroot%\system32\dllhost.exe) with TCP port 10123 is NOT allowed by Windows Firewall on all interfaces.~~

• lt;SMS_NOTIFICATION_SERVER><08-15-2017 01:16:02.062-330> Total online clients: 0 (TCP: 0 HTTP: 0)~~

• lt;SMS_NOTIFICATION_SERVER><08-15-2017 01:21:02.039-330> Generated BGB online status FULL report C:\Program Files\Microsoft Configuration Manager\inboxes\bgb.box\Bgb72ul2.BOS (version: 0) at 08/15/2017 01:21:02~~

• lt;SMS_NOTIFICATION_SERVER><08-15-2017 01:21:02.055-330> WARNING: Notification Server (%systemroot%\system32\dllhost.exe) with TCP port 10123 is NOT allowed by Windows Firewall on all interfaces.~~

• lt;SMS_NOTIFICATION_SERVER><08-15-2017 01:21:02.067-330> Wait 300 seconds for notifications…

• lt;SMS_NOTIFICATION_SERVER><08-15-2017 01:21:02.276-330>

Client-Side Troubleshooting Fast Channel Notification Component

The notification agent was running. But, the  CcmNotificationAgent.log log showed loads of errors. One of the errors indicated that there could be a communication issue between the server and the client.

Error 10060 means ==> A connection attempt failed because the connected party did not appropriately respond after a period, or an established connection failed because the connected host could not respond. BGBAgent component log :-

<![LOG[Bgb client agent is starting...]LOG]!><time="01:23:55.212-330" date="08-15-2017" component="BgbAgent" context="" type="1" thread="6372" file="agentendpoint.cpp:238">
<![LOG[BgbController main thread is started with settings: {bgb enable = 1}, {tcp enabled = 1}, {tcp port = 10123} and {http enabled = 1}.]LOG]!><time="01:23:55.259-330" date="08-15-2017" component="BgbAgent" context="" type="1" thread="6372" file="bgbcontroller.cpp:126">
<![LOG[Startup random sleep for 1 seconds.]LOG]!><time="01:23:55.290-330" date="08-15-2017" component="BgbAgent" context="" type="1" thread="5200" file="bgbcontroller.cpp:416">
<![LOG[Critical Battery: [FALSE]]LOG]!><time="01:23:56.306-330" date="08-15-2017" component="BgbAgent" context="" type="1" thread="5200" file="bgbcommon.cpp:60">
<![LOG[Connection Standy: [FALSE]]LOG]!><time="01:23:56.306-330" date="08-15-2017" component="BgbAgent" context="" type="1" thread="5200" file="bgbcommon.cpp:61">
<![LOG[Network allowed to use: [TRUE]]LOG]!><time="01:23:56.306-330" date="08-15-2017" component="BgbAgent" context="" type="1" thread="5200" file="bgbcommon.cpp:62">
<![LOG[Access point is SCCMTP1.INTUNE.COM. (SSLEnabled = 0)]LOG]!><time="01:23:56.415-330" date="08-15-2017" component="BgbAgent" context="" type="1" thread="5200" file="bgbcontroller.cpp:276">
<![LOG[CRL Checking is Enabled.]LOG]!><time="01:23:56.431-330" date="08-15-2017" component="BgbAgent" context="" type="1" thread="5200" file="bgbcontroller.cpp:284">
<![LOG[Both TCP and http are enabled, let's try TCP connection first.]LOG]!><time="01:23:56.431-330" date="08-15-2017" component="BgbAgent" context="" type="1" thread="5200" file="bgbcontroller.cpp:792">
<![LOG[Connecting to server with IP: 20.20.20.22 Port: 10123 
]LOG]!><time="01:23:56.447-330" date="08-15-2017" component="BgbAgent" context="" type="1" thread="5200" file="bgbtcpclient.cpp:699">
<![LOG[Failed to connect to server with IP v4 address with error 10060. Try next IP...
]LOG]!><time="01:24:17.468-330" date="08-15-2017" component="BgbAgent" context="" type="1" thread="5200" file="bgbtcpclient.cpp:703">
<![LOG[Failed to signin bgb client with error = 80004005.]LOG]!><time="01:24:17.468-330" date="08-15-2017" component="BgbAgent" context="" type="3" thread="5200" file="bgbcontroller.cpp:635">
<![LOG[Connecting to server with IP: 20.20.20.22 Port: 10123 
]LOG]!><time="01:25:17.482-330" date="08-15-2017" component="BgbAgent" context="" type="1" thread="5200" file="bgbtcpclient.cpp:699">
<![LOG[Failed to connect to server with IP v4 address with error 10060. Try next IP...
]LOG]!><time="01:25:38.501-330" date="08-15-2017" component="BgbAgent" context="" type="1" thread="5200" file="bgbtcpclient.cpp:703">
<![LOG[Failed to signin bgb client with error = 80004005.]LOG]!><time="01:25:38.501-330" date="08-15-2017" component="BgbAgent" context="" type="3" thread="5200" file="bgbcontroller.cpp:635">
<![LOG[Fallback to HTTP connection.]LOG]!><time="01:25:38.501-330" date="08-15-2017" component="BgbAgent" context="" type="1" thread="5200" file="bgbcontroller.cpp:828">
[CCMHTTP] ERROR: URL=http://SCCMTP1.Intune.com/bgb/handler.ashx?RequestType=Continue, Port=80, Options=224, Code=0, Text=CCM_E_BAD_HTTP_STATUS_CODE
Successfully queued event on HTTP/HTTPS failure for server 'SCCMTP1.Intune.com'.
Failed to post continue request with error code 87d0027e.

Fix for SCCM CB Fast Channel Notification Issues

The Firewall port 10123 was not opened between the SCCM client and the primary BGB server. I ran the following command from the client, “Telnet 10123,” and it didn’t work (the port was not opened).

I checked the software and hardware firewalls on the server side and discovered that Windows Firewall was blocking the port communication 10123.

Disabled the Windows Firewall on the notification server for testing and restarted the client agent services (SMS Agent) on the client machine. This helped to resolve the fast channel notification issue with the SCCM CB environment.

In an ideal world, you should exclude/exempt port 10123/80 from the hardware and software firewall between the fast channel notification server and agent. This will help to resolve the issue.

More details are available in the video tutorial here

Server Side Logs – After Successful Actions on Fast Channel Notification

I finished sending push tasks (PushID: 1 TaskID: 3) to 1 client and generated the BGB online status DELTA report, two critical lines of SCCM CB fast notification channel server log BGBServer.log.

C:\Program Files\Microsoft Configuration Manager\inboxes\bgb.box\Bgb7cbzg.BOS

• Receiving a message from queue timeout.~~

• lt;SMS_NOTIFICATION_SERVER><08-24-2017 12:07:50.411-330> Retrieving push tasks from database…~~

• lt;SMS_NOTIFICATION_SERVER><08-24-2017 12:07:50.412-330> Retrieving online resync flag from database…~~

• lt;SMS_NOTIFICATION_SERVER><08-24-2017 12:11:50.914-330> Total online clients: 1 (TCP: 1 HTTP: 0)~~

• lt;SMS_NOTIFICATION_SERVER><08-24-2017 12:11:50.916-330> Online/Offline clients since last successful report: 1~~

• lt;SMS_NOTIFICATION_SERVER><08-24-2017 12:11:50.919-330> Generated BGB online status DELTA report C:\Program Files\Microsoft Configuration Manager\inboxes\bgb.box\Bgb7cbzg.BOS (version: 18) at 08/24/2017 12:11:50~~

• lt;SMS_NOTIFICATION_SERVER><08-24-2017 12:11:50.940-330> Get one push message from database.~~

• lt;SMS_NOTIFICATION_SERVER><08-24-2017 12:11:52.184-330> Starting to send push task (PushID: 1 TaskID: 3 TaskGUID: 3700E17A-4BDC-45C7-990E-EA26FF92E5BF TaskType: 4 TaskParam: ) to 1 clients with throttling (strategy: 1 param: 42)~~

• lt;SMS_NOTIFICATION_SERVER><08-24-2017 12:11:52.209-330> Finished sending push task (PushID: 1 TaskID: 3) to 1 clients~~

• lt;SMS_NOTIFICATION_SERVER><08-24-2017 12:11:52.214-330> Starting to send push task (PushID: 1 TaskID: 4 TaskGUID: FD80647D-9748-4C96-AFC8-8BC71E00C235 TaskType: 1 TaskParam: ) to 1 clients with throttling (strategy: 1 param: 42)~~

• lt;SMS_NOTIFICATION_SERVER><08-24-2017 12:11:52.216-330> Finished sending push task (PushID: 1 TaskID: 4) to 1 clients~~

• lt;SMS_NOTIFICATION_SERVER><08-24-2017 12:11:52.237-330> Starting to send push task (PushID: 1 TaskID: 5 TaskGUID: 9D2B274B-F6E3-452F-A1C3-C1C166523EC8 TaskType: 1 TaskParam: ) to 1 clients with throttling (strategy: 1 param: 42)~~

• lt;SMS_NOTIFICATION_SERVER><08-24-2017 12:11:52.238-330> Finished sending push task (PushID: 1 TaskID: 5) to 1 clients~~

• lt;SMS_NOTIFICATION_SERVER><08-24-2017 12:11:52.261-330> Starting to send push task (PushID: 1 TaskID: 6 TaskGUID: CFB76FC4-BCC5-4525-AA53-80BCD4393E46 TaskType: 1 TaskParam: ) to 1 clients with throttling (strategy: 1 param: 42)~~

• lt;SMS_NOTIFICATION_SERVER><08-24-2017 12:11:52.262-330> Finished sending push task (PushID: 1 TaskID: 6) to 1 clients~~

• lt;SMS_NOTIFICATION_SERVER><08-24-2017 12:11:52.285-330> Starting to send push task (PushID: 1 TaskID: 7 TaskGUID: 1ED88E07-5E52-44FD-AF07-73769EDD7FA6 TaskType: 1 TaskParam: ) to 1 clients with throttling (strategy: 1 param: 42)~~

• lt;SMS_NOTIFICATION_SERVER><08-24-2017 12:11:52.286-330> Finished sending push task (PushID: 1 TaskID: 7) to 1 clients~~

• lt;SMS_NOTIFICATION_SERVER><08-24-2017 12:11:52.309-330>

Client BGB Agent Fast Notification:- Log files details of successful operations

After the firewall ports had been changed, the client notification agent started working fine. Following are some of the important log file snippets.

<![LOG[Bgb client agent is starting...]LOG]!><time="12:07:25.115-330" date="08-24-2017" component="BgbAgent" context="" type="1" thread="5368" file="agentendpoint.cpp:238">
<![LOG[BgbController main thread is started with settings: {bgb enable = 1}, {tcp enabled = 1}, {tcp port = 10123} and {http enabled = 1}.]LOG]!><time="12:07:25.162-330" date="08-24-2017" component="BgbAgent" context="" type="1" thread="5368" file="bgbcontroller.cpp:126">
<![LOG[Startup random sleep for 23 seconds.]LOG]!><time="12:07:25.178-330" date="08-24-2017" component="BgbAgent" context="" type="1" thread="3864" file="bgbcontroller.cpp:416">
<![LOG[Critical Battery: [FALSE]]LOG]!><time="12:07:48.178-330" date="08-24-2017" component="BgbAgent" context="" type="1" thread="3864" file="bgbcommon.cpp:60">
<![LOG[Connection Standy: [FALSE]]LOG]!><time="12:07:48.178-330" date="08-24-2017" component="BgbAgent" context="" type="1" thread="3864" file="bgbcommon.cpp:61">
<![LOG[Network allowed to use: [TRUE]]LOG]!><time="12:07:48.178-330" date="08-24-2017" component="BgbAgent" context="" type="1" thread="3864" file="bgbcommon.cpp:62">
<![LOG[Access point is SCCMTP1.Intune.com. (SSLEnabled = 0)]LOG]!><time="12:07:48.178-330" date="08-24-2017" component="BgbAgent" context="" type="1" thread="3864" file="bgbcontroller.cpp:276">
<![LOG[CRL Checking is Enabled.]LOG]!><time="12:07:48.178-330" date="08-24-2017" component="BgbAgent" context="" type="1" thread="3864" file="bgbcontroller.cpp:284">
<![LOG[Both TCP and http are enabled, let's try TCP connection first.]LOG]!><time="12:07:48.178-330" date="08-24-2017" component="BgbAgent" context="" type="1" thread="3864" file="bgbcontroller.cpp:792">
<![LOG[Connecting to server with IP: 20.20.20.22 Port: 10123 
]LOG]!><time="12:07:48.194-330" date="08-24-2017" component="BgbAgent" context="" type="1" thread="3864" file="bgbtcpclient.cpp:699">
<![LOG[Handshake was successful
]LOG]!><time="12:07:49.270-330" date="08-24-2017" component="BgbAgent" context="" type="1" thread="3864" file="bgbtcpclient.cpp:495">
<![LOG[Pass verification on server certificate.]LOG]!><time="12:07:49.329-330" date="08-24-2017" component="BgbAgent" context="" type="1" thread="3864" file="bgbtcpclient.cpp:823">
<![LOG[NetworkInfo: IPAddress 20.20.20.23,fe80::b09e:95a3:172a:4212]LOG]!><time="12:07:49.438-330" date="08-24-2017" component="BgbAgent" context="" type="1" thread="3864" file="bgbconnector.cpp:124">
<![LOG[NetworkInfo: IPSubnet 255.0.0.0,64]LOG]!><time="12:07:49.438-330" date="08-24-2017" component="BgbAgent" context="" type="1" thread="3864" file="bgbconnector.cpp:147">
<![LOG[NetworkInfo: AccessMP SCCMTP1.Intune.com]LOG]!><time="12:07:49.471-330" date="08-24-2017" component="BgbAgent" context="" type="1" thread="3864" file="bgbconnector.cpp:155">
<![LOG[NetworkInfo: IsClientOnInternet 0]LOG]!><time="12:07:49.471-330" date="08-24-2017" component="BgbAgent" context="" type="1" thread="3864" file="bgbconnector.cpp:159">
<![LOG[Update the timeout to 900 second(s)]LOG]!><time="12:07:49.471-330" date="08-24-2017" component="BgbAgent" context="" type="1" thread="3864" file="bgbtcpclient.cpp:916">
<![LOG[Receive signin confirmation message from server, client is signed in.]LOG]!><time="12:08:01.062-330" date="08-24-2017" component="BgbAgent" context="" type="1" thread="3864" file="bgbconnector.cpp:221">
<![LOG[Receive task from server with pushid=1, taskid=3, taskguid=3700E17A-4BDC-45C7-990E-EA26FF92E5BF, tasktype=4 and taskParam=]LOG]!><time="12:11:52.227-330" date="08-24-2017" component="BgbAgent" context="" type="1" thread="3864" file="bgbconnector.cpp:312">
<![LOG[Receive task from server with pushid=1, taskid=4, taskguid=FD80647D-9748-4C96-AFC8-8BC71E00C235, tasktype=1 and taskParam=]LOG]!><time="12:11:52.248-330" date="08-24-2017" component="BgbAgent" context="" type="1" thread="3864" file="bgbconnector.cpp:312">
<![LOG[Receive task from server with pushid=1, taskid=5, taskguid=9D2B274B-F6E3-452F-A1C3-C1C166523EC8, tasktype=1 and taskParam=]LOG]!><time="12:11:52.264-330" date="08-24-2017" component="BgbAgent" context="" type="1" thread="3864" file="bgbconnector.cpp:312">
<![LOG[Successfully sent keep-alive message.]LOG]!><time="12:35:51.339-330" date="08-24-2017" component="BgbAgent" context="" type="1" thread="3864" file="bgbconnector.cpp:290">
<![LOG[Successfully sent keep-alive message.]LOG]!><time="12:50:51.356-330" date="08-24-2017" component="BgbAgent" context="" type="1" thread="3864" file="bgbconnector.cpp:290">

References

• Fast Channel Client Notification in SCCM  – Here
• Fast channel notification and MP replica issues – Here
• What’s New With ConfigMgr’s Client Notification Feature – Here

We are on WhatsApp now. To get the latest step-by-step guides, news, and updates, Join our Channel. Click here. HTMD WhatsApp.

Author

Anoop C Nair is Microsoft MVP! He is a Device Management Admin with more than 20 years of experience (calculation done in 2021) in IT. He is a Blogger, Speaker, and Local User Group HTMD Community leader. His main focus is on Device Management technologies like SCCM 2012, Current Branch, and Intune. He writes about ConfigMgr, Windows 11, Windows 10, Azure AD, Microsoft Intune, Windows 365, AVD, etc.