Intune SCEP HTTP Errors Troubleshooting Made Easy With Joy-#5

Welcome to Part 5 of the series Intune PKI Made Easy With Joy.

Today’s post discusses the different Intune SCEP HTTP errors we may encounter while working with SCEP certificate deployments from Intune. It explains how to identify probable causes to help with quicker troubleshooting and ensure lower downtime.

In the context of Intune, the Simple Certificate Enrollment Protocol (SCEP) can be leveraged to authenticate connections for your apps and corporate resources. SCEP relies on the Certification Authority (CA) certificate to securely facilitate the Certificate Signing Request (CSR) message exchange.

The exchange of messages for the Certificate Signing Request (CSR) is secured using the Certification Authority (CA) certificate within SCEP. When your infrastructure supports SCEP, you can deploy certificates to your devices by utilizing Intune SCEP certificate profiles, a specific type of device profile in Intune.

Patch My PC

Disclaimer: The Intune SCEP HTTP Error causes listed in this article are purely based on experience dealing with a production environment and may not match the order in which MS has listed them in its troubleshooting document.

Previous articles of this series Intune PKI Made Easy With Joy

Index
Demystifying Intune SCEP HTTP Errors
504 Gateway Timeout
502 Bad Gateway
This page cannot be displayed/Can’t reach this page
Intune SCEP HTTP Error 500 – Internal Server Error
NDES Service Account permission/rights issue
MSCEP RA Certificates are Expired (or Deleted or Revoked)
CRP in IIS has Windows Authentication set to Enabled
NDES unable to retrieve CRL from CDP
Intune SCEP Http Error 503 – Service Unavailable
Untrusted Intermediate Certificates in NDES server cert store
The End
Intune SCEP HTTP Errors Troubleshooting Made Easy With Joy-#5 – Table 1

Let’s get started.

Demystifying Intune SCEP HTTP Errors

For an internet-facing device to send the SCEP request to NDES, the request must go via a proxy. Azure AD App Proxy (Microsoft recommended) exposes the internal NDES mscep.dll URL in most setups.

Adaptiva

So, let’s begin with the HTTP errors we may likely get due to Azure AD App Proxy.

Intune SCEP HTTP Errors – AAD App Proxy Errors

504 Gateway Timeout

This mainly occurs if the AAD App Proxy connector is not Running or the Server hosting the connector has gone offline.

Intune SCEP HTTP Errors Troubleshooting Made Easy With Joy-#5 - Fig.1
Intune SCEP HTTP Errors Troubleshooting Made Easy With Joy-#5 – Fig.1

The Microsoft AAD Application Proxy Connector service’s startup type is set to Automatic (Delayed Start) by default.

Intune SCEP HTTP Errors Troubleshooting Made Easy With Joy-#5 - Fig.2
Intune SCEP HTTP Errors Troubleshooting Made Easy With Joy-#5 – Fig.2

As such, if the Server recently underwent a reboot cycle, check whether the connector service has started.

502 Bad Gateway

This mainly occurs due to a typo in the internal URL while creating/configuring the App Proxy application in Azure. As a result, the connector fails to resolve the internal URL, resulting in the following error:

Intune SCEP HTTP Errors Troubleshooting Made Easy With Joy-#5 - Fig.3
Intune SCEP HTTP Errors Troubleshooting Made Easy With Joy-#5 – Fig.3

This page cannot be displayed/Can’t reach this page

This occurs due to incorrect App Proxy configuration for publishing the SCEP URL.

If you choose to use your custom verified domain in Azure as the mscep URL domain rather than the default “msappproxy.net” domain, you must create a CNAME record in the DNS provider for the custom domain to map the requests.

By default, Azure App Proxy requires no additional configuration to use the default “-.msappproxy.net” domain. Also, you must provide an SSL cert if you choose to use a custom domain other than the default “-msappproxy.net” domain.

In this case, I went with the default configuration to save myself from extra work, but I still got the error. Generating Diagnostics For the App Proxy application in Azure tells the error source.

Intune SCEP HTTP Errors Troubleshooting Made Easy With Joy-#5 - Fig.4
Intune SCEP HTTP Errors Troubleshooting Made Easy With Joy-#5 – Fig.4

While configuring the App Proxy application, I left the Pre Authentication set to Azure Active Directory.

Intune SCEP HTTP Errors Troubleshooting Made Easy With Joy-#5 - Fig.5
Intune SCEP HTTP Errors Troubleshooting Made Easy With Joy-#5 – Fig.5

For NDES, the pre-authentication setting within the AAD App Proxy application in Azure needs to be set to Passthrough since no actual AAD user will be accessing the application.

Provided that we are good from the proxy side and AAD App Proxy can resolve the external requests to the internal NDES MSCEP URL, the HTTP errors we will encounter depend on the physical infrastructure settings (NDES and PKI).

Intune SCEP HTTP Errors – Configuration Issues

Intune SCEP HTTP Error 500 – Internal Server Error

There can be several causes for this particular error, and you would need to check several things to narrow them down to the actual cause.

NDES Service Account permission/rights issue

The SCEP pool in IIS runs under the NDES service account identity.

Intune SCEP Error - HTTP Error 500 - IIS SCEP Pool runs under NDES Service Account identity - Ensure NDES Service Account password is not expired and account is in unlocked state.
Intune SCEP HTTP Errors Troubleshooting Made Easy With Joy-#5 – Fig.6

You need to check and ensure that the NDES Service Account

  • is not in a Locked state,
  • password is not Expired,
  • is a member of the local IIS_IURS group on the NDES box (ensure no GPO is modifying the membership of the particular group),
  • the IIS_IURS group is assigned the Impersonate a client after authentication user right (by default, the right is present unless modified),
Intune SCEP HTTP Errors Troubleshooting Made Easy With Joy-#5 - Fig.4
Intune SCEP HTTP Errors Troubleshooting Made Easy With Joy-#5 – Fig.7
  • has Read permission to the MSCEP RA certs private key
Intune SCEP HTTP Errors Troubleshooting Made Easy With Joy-#5 - Fig.8
Intune SCEP HTTP Errors Troubleshooting Made Easy With Joy-#5 – Fig.8

If the above conditions are not satisfied, you will have the Intune SCEP HTTP Error 500 – Internal Server Error.

MSCEP RA Certificates are Expired (or Deleted or Revoked)

NDES service startup depends on the MSCEP RA Certificates. [More of the NDES service startup sequence is discussed later in detail in this post]

Intune SCEP HTTP Errors Troubleshooting Made Easy With Joy-#5 - Fig.9
Intune SCEP HTTP Errors Troubleshooting Made Easy With Joy-#5 – Fig.9

Suppose the certificates are expired, deleted, or revoked from the issuing CA for any cause. In that case, the NDES service will fail to start, resulting in the Intune SCEP HTTP Error 500 – Internal Server Error.

CRP in IIS has Windows Authentication set to Enabled

Ensure CertificateRegistrationSvc in IIS [CRP] has Windows Authentication set to Disabled.

Intune SCEP HTTP Errors Troubleshooting Made Easy With Joy-#5 - Fig.10
Intune SCEP HTTP Errors Troubleshooting Made Easy With Joy-#5 – Fig.10

If it is set to Enabled, as shown in the above snapshot, it may result in Intune SCEP HTTP Error 500 – Internal Server Error.

Note! This particular setting may also result in HTTP Error 503 – Service Unavailable.

Just like the/certsrv/mscep app in the SCEP pool, IIS authentication for CRP also needs to have

  • Anonymous Authentication = Enabled
  • Windows Authentication = Disabled
Issuing CA is Offline or Unreachable from NDES

*********One of the most notable causes of Intune SCEP HTTP Error 500 – Internal Server Error.*********

Network configuration changes done by the network/firewall/proxy team may result in the Issuing CA becoming unreachable or unavailable from the NDES box, resulting in the error. Error events can be viewed from the events below.

Event Source: Windows Logs > Application > NetworkDeviceEnrollmentService
Event ID 8 - The Network Device Enrollment Service cannot retrieve information about the certification authority (0x80004005).  Unspecified error
Event ID 2 - The Network Device Enrollment Service cannot be started (0x80004005). Unspecified error

You can use CertUtil Command on the NDES server to verify CA availability. If your Issuing CA is unreachable or unavailable from the NDES box, you should expect an output like the one below.

PS C:\Windows\system32> certutil -config - -ping
WIN-2CQQDB9STE6.joymalya.xyz\joymalya-WIN-2CQQDB9STE6-CA
Connecting to WIN-2CQQDB9STE6.joymalya.xyz\joymalya-WIN-2CQQDB9STE6-CA ...
Server could not be reached: The RPC server is unavailable. 0x800706ba (WIN32: 1722 RPC_S_SERVER_UNAVAILABLE) -- (32ms)
CertUtil : -ping command FAILED: 0x800706ba (WIN32: 1722 RPC_S_SERVER_UNAVAILABLE)
CertUtil : The RPC server is unavailable.

The same can be verified using another tool named pkiview.msc

Intune SCEP HTTP Errors Troubleshooting Made Easy With Joy-#5 - Fig.11
Intune SCEP HTTP Errors Troubleshooting Made Easy With Joy-#5 – Fig.11

Sometimes, the issuing CA server is online for a multi-tier PKI setup. Still, Certificate Services of the Issuing CA fail to start after a server restart [legit causes] because the Root CA CRL has expired.

Consider the classic example below: When I do a standard ping, my CA server returns a response that means it’s online and available, but a CertUtil ping to the same CA server says otherwise.

PS C:\Windows\system32> ping WIN-2CQQDB9STE6.joymalya.xyz
 
Pinging WIN-2CQQDB9STE6.joymalya.xyz [10.0.0.20] with 32 bytes of data:
Reply from 10.0.0.20: bytes=32 time<1ms TTL=128
Reply from 10.0.0.20: bytes=32 time=1ms TTL=128
Reply from 10.0.0.20: bytes=32 time=1ms TTL=128
Reply from 10.0.0.20: bytes=32 time=1ms TTL=128
 
Ping statistics for 10.0.0.20:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum = 1ms, Average = 0ms
 
PS C:\Windows\system32> certutil -ping WIN-2CQQDB9STE6.joymalya.xyz
Connecting to WIN-2CQQDB9STE6.joymalya.xyz ...
Server could not be reached: The RPC server is unavailable. 0x800706ba (WIN32: 1722 RPC_S_SERVER_UNAVAILABLE) -- (32ms)
 
CertUtil: -ping command FAILED: 0x800706ba (WIN32: 1722 RPC_S_SERVER_UNAVAILABLE)
CertUtil: The RPC server is unavailable.

This can happen for multi-tier PKI infra where the Root CA is primarily offline and the Root CA CRL has expired. The Issuing CA went through a reboot due to any cause. After rebooting, the Issuing CA’s certificate services will fail to start, as when it tries to retrieve the Root CA CRL to verify its cert chain, the verification operation fails.

Intune SCEP HTTP Errors Troubleshooting Made Easy With Joy-#5 - Fig.12
Intune SCEP HTTP Errors Troubleshooting Made Easy With Joy-#5 – Fig.12

Fixing the issue might require engagement from both the Network and/or ADCS teams, as it does not generally fall within the Intune scope.

NDES unable to retrieve CRL from CDP

******This is the other most prominent cause of Intune SCEP HTTP Error 500 – Internal Server Error******

This occurs because the NDES box fails to retrieve the Certificate Revocation List (CRL from CDP) required for the NDES service to start itself.

Remember that NDES is implemented as an ISAPI extension in IIS; as such, you will not see NDES as a service when you check in services.msc. Instead, the service runs within the w3wp.exe IIS worker process.

The NDES service must start successfully for the SCEP pool in IIS to run. The NDES service startup checklist is available here.

In a nutshell, the NDES Service Startup Sequence is –

  • Locate RA cert from machine cert store – X509Objects
  • Acquire the Private Keys – CryptAcquireCertificatePrivateKey
  • Build cert chain – CertGetCertificateChainStart
  • Verify Revocation Status – CertVerifyRevocationStart

If all the steps are completed successfully, the NDES service starts, and you get the success events below

Event Source: Windows Logs > Application > NetworkDeviceEnrollmentService
Event ID 47 - The Network Device Enrollment Service loaded the Registration Authority (RA) key exchange certificate with serial number ###### from the "MY" store.
Event ID 48 - The Network Device Enrollment Service loaded the Registration Authority (RA) signature certificate with serial number ###### from the "MY" store.
Event ID 1 - The Network Device Enrollment Service started successfully.

However, the NDES service fails to start if it fails to verify the MSCEP-RA certificates. Error events are as follows.

Event Source: Windows Logs > Application > NetworkDeviceEnrollmentService
Event ID 8 - The Network Device Enrollment Service cannot retrieve information about the certification authority (0x80004005).  Unspecified error
Event ID 2 - The Network Device Enrollment Service cannot be started (0x80004005). Unspecified error

The event above does not necessarily mean that the MSCEP RA certs are expired or not present (deleted); it can also be because of an Expired CRL or the NDES server unable to retrieve the CRL, though the events above do not expressly state such.

To check the actual reason for the failure to load the RA certs, you need to enable CAPI2 logging-in events [Applications and Services logs > Microsoft > Windows > CAPI2], then restart the Cryptographic Services and IIS [PS command iisreset] and check back in the CAPI2 Operational logs.

CAPI 2  Event Trace for the same is shown below

Event ID 90 - X509 Objects - Loads RA cert
Event ID 70 - Acquire Certificate Private Key - Crypt Function CryptAcquireCertificatePrivateKey
Event ID 10 - Build Chain - Crypt Function CertGetCertificateChainStart to start Chain Building
Event ID 40 - Verify Revocation - Crypt Function CertVerifyRevocationStart to start revocation status
Event ID 52 - Retrieve Object from Network - Crypt Function CryptRetrieveObjectByUrlWireStart to retrieve CRL

The error events start here if the CA CRL has expired or the CRL location is offline.

Error Event ID 53 - Retrieve Object from Network - Crypt Function CryptRetrieveObjectByUrlWire which will report error if CRL not retrievable.

Error Event ID 42 - Reject Revocation Information - Crypt Function CertRejectedRevocationInfo. Here you can see that if CRL was actually retrieved, the failure reason might be due to Action CheckTimeValidity [retrieved CRL has expired]

Error Event ID 41 - Verify Revocation - Crypt Function CertVerifyRevocation will give you the outcome of revocation verification. 
Intune SCEP HTTP Errors Troubleshooting Made Easy With Joy-#5 - Fig.13
Intune SCEP HTTP Errors Troubleshooting Made Easy With Joy-#5 – Fig.13

In the NDES server, you can use the CertUtil command CertUtil -url cache CRL to view the locally cached CRL URLs and then use the CertUtil command certutil -URL <LDAP or HTTP URL of CRL> to verify if the NDES server can retrieve the CRL from CDP.

You will see the URL Retrieval Tool GUI window like this: Select CRLs (from CDP) and click on Retrieve. In an ideal scenario, the status will be OK.

Intune SCEP HTTP Errors Troubleshooting Made Easy With Joy-#5 - Fig.14
Intune SCEP HTTP Errors Troubleshooting Made Easy With Joy-#5 – Fig.14

Important! If a proxy server facilitates communication between servers, ensure that proxy settings in all the servers involved are configured for both user and machine/system context.

In the production environment, a firewall may be in between or a proxy facilitating the communications. For a firewall, you need to ensure you have the proper exceptions set to allow connections. For a proxy, you must set proxy settings in the SYSTEM context for NDES to work. The best way to do this is to use the PSEXEC tool.

Intune SCEP HTTP Errors Troubleshooting Made Easy With Joy-#5 - Fig.15
Intune SCEP HTTP Errors Troubleshooting Made Easy With Joy-#5 – Fig.15

Open a CMD using PSEXEC and confirm the CMD process runs as a SYSTEM using the command whoami. Once confirmed, you can use the netsh winhttp show proxy to see if the proxy settings are correctly configured in the SYSTEM context. If not, use the netsh winhttp to set the proxy and the command to set the same.

Fixing issues with retrieving CRLs or expired CRLs should mostly fall under the scope of ADCS and not the scope of Intune in general.

Once the CRL issue is resolved, restart the Cryptographic Services and IIS on your NDES server and wait to check if NDES services started successfully. [Event ID 1 for the successful startup of NDES]

Intune SCEP Http Error 503 – Service Unavailable

This one is pretty straightforward. In most cases, the IIS SCEP pool crashes (stopped stated). The likely causes are

Untrusted Intermediate Certificates in NDES server cert store

This is very common for multi-tier PKI infrastructure. Hence, if you are facing this issue, the first thing you should check on your NDES box is to open PowerShell and run the following command.

Get-Childitem -Path cert:\LocalMachine\root -Recurse | Where-Object {$_.Issuer -ne $_.Subject}

If the above code gives any output, it means there are untrusted certs present, and you can delete them by running the command below.

Get-Childitem -Path cert:\LocalMachine\root -Recurse | Where-Object {$_.Issuer -ne $_.Subject}| Remove-Object

Note! If you frequently encounter this issue, consider creating a scheduled task to run this command automatically at regular intervals to remove the untrusted certificates.

  • IIS SSL binding certificate was renewed, but the NDES Policy module still points to the old SSL certificate thumbprint

Remember, the certificate to bind port 443 of IIS must be selected while installing the Intune NDES certificate connector. This situation arises primarily when the IIS SSL Binding Certificate gets renewed but not updated with the Intune NDES Certificate Connector.

Intune SCEP HTTP Errors Troubleshooting Made Easy With Joy-#5 - Fig.16
Intune SCEP HTTP Errors Troubleshooting Made Easy With Joy-#5 – Fig.16

You must ensure that the thumbprint of the certificate held by the NDES Policy Module must match the thumbprint of the SSL binding certificate used in IIS.

Intune SCEP HTTP Errors Troubleshooting Made Easy With Joy-#5 - Fig.17
Intune SCEP HTTP Errors Troubleshooting Made Easy With Joy-#5 – Fig.17

The Intune NDES Connector service requires access to the URLs in CRL for proper functioning. As such, after a successful NDES service startup, if, for any reason, the CRL URLs become unreachable again from the NDES box, it may result in HTTP Error 503—Service Unavailable.

Notice that this time, the NDES service has started itself. Otherwise, the same scenario during NDES service startup results in HTTP Error 500—Internal Server Error.

The End

In addition to all the Intune SCEP HTTP Errors as listed above, you may also encounter HTTP Error 414 – Request URI Too Long and HTTP Error 413 – Payload Too Large, which are all due to incorrect IIS Request Filtering settings for the /certsrv/mscep Virtual app.

The causes of the errors presented in this article may not be exhaustive. While troubleshooting, you may encounter an entirely new cause, which is quite possible. I would be very interested to learn if you have discovered any new reason for the above error while troubleshooting in your environment.

Well, that was all I had planned for today. I will be back with some other posts shortly. Until then, continue to stay safe.

We are on WhatsApp. To get the latest step-by-step guides and news updates, Join our Channel. Click here –HTMD WhatsApp.

Author

Joymalya Basu Roy is an experienced IT service professional with almost five years of experience working with Microsft Intune. He is currently working as a Senior Consultant – Architect with Atos India. He is an ex-MSFT, where he worked as a Premiere Support Engineer for Microsoft Intune. He was also associated with Wipro and TCS in the early stages of his career. He was awarded the Microsoft MVP award for Enterprise Mobility in 2021. You can find all his latest posts on his blog site, MDM Tech Space, at https://joymalya.com

11 thoughts on “Intune SCEP HTTP Errors Troubleshooting Made Easy With Joy-#5”

  1. Hello Joy,

    I also ran into HTTP 500 error when setting up NDES server, logged a case with MS and didn’t revert back for 2 weeks and then I came across your blog which helped me to look at the CRL location and turns out in our case the CRL location was wrongly published. thanks and keep sharing!!

    Reply
  2. Great extensive post!

    I’m still confused about the relation with a third party on-prem forward proxy and Azure Application (reverse) proxy.

    I cannot get an Internet connection through our forward on-prem proxy when signing in to Azure via the Intune certificate connector.
    However, if I browse to that URL via IE on my NDES server to https://portal.manage.microsoft.com, I get an HTTP 403 ‘forbidden’. This makes sense since we have to authenticate through our on-prem proxy with an account.

    But after authenticating and configuring this proxy in the Intune Connector configuration, I still don’t get access.

    Any help would be appreciated, thanks!

    Reply
  3. NDES Disable Password Requirement.
    I’ve read a few blogs and articles that say;

    “There is no way for Cisco devices to supply the required password to enroll with NDES/MSCEP, so you need to disable the requirement for a password.”

    This is NOT TRUE, however the whole point of issuing certificates via your PKI infrastructure, is that it can scale dramatically. If you are creating passwords and embedding those passwords in all your enrollments, it can get a little unwieldy. So it may be sensible to remove the password requirement.

    1. Windows Key+R > regedit {Enter} > Navigate to;

    HKEY_LOCAL_MACHINE > SOFTWARE > Microsoft > Cryptography > MSCEP > EnforcePassword > EnforcePassword
    To disable change the value to 0 (zero).

    Disable NDES Password Enforce

    Update: 22/10/21: You may also need to recycle the SCEP application pool in IIS (on the Certificate Services Server)

    From IIS Manager > CA > Application Pools, SCEP. > From the right hand panel > Advanced Settings. > Set Load User Profile to ‘True‘ > OK.

    Again in the right panel > Recycle > From IIS Manager > Sites > Default Web Site. > From the right panel, click Restart.

    Reply
  4. Any chance you have an updated screen shot for CRP in IIS has Windows Authentication set to Enabled . It looks like it doesnt create this website anymore with the new connector

    Reply

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.