Windows Autopilot WhiteGlove Provisioning Backend Process- Deep Dive – Post 4

This is the last concluding article of the Autopilot series, which I started. In my previous articles of this series, I have explained the working details behind Windows Autopilot in depth.

If you have not read them yet, I suggest giving them a read. Let’s see what Window Autopilot WhiteGlove is.

Related Posts

Introduction

If you have read them and know a little bit about it, you would agree that Windows Autopilot has minimal IT overhead and gives the end-users a smooth and customized device provisioning experience, but it is usually time-consuming.

Patch My PC

This is generally true in cases where you have a lot of applications deployed, and you want all of them to be installed before the user can become productive on the device.

In such cases, the provisioning phase will spend a lot of time on the ESP screen with the user sitting in front of the device doing nothing.

It’s more like, start the device -> get it connected to network -> sign-in -> sit back or do other works as it provisions

In this fast-paced, cloud-backed IT world, ideally, this would not be considered a seamless experience. Microsoft came up with something to address this – Windows Autopilot WhiteGlove Provisioning.

Autopilot Whiteglove profile - Intune - Window Autopilot WhiteGlove
Window Autopilot WhiteGlove 1

What is the WhiteGlove process, and how is it different from normal Autopilot provisioning?

Introduced with Windows 10 1903 (codename 19H1, finally MS moved to a relatable codename scheme from its previous RS and TH theme), Windows Autopilot gives IT Admin an option to pre-provision the device before handing it over to the end-user.

This is essential to reduce the time the device stays on Enrollment Status Page when a user goes through the entire process. Especially in cases where many policies and applications (large applications like O365 Pro Plus Suite) are set to be tracked in ESP.

Yes, there is an option in ESP setup to select particular apps to be tracked only, but even with this, the end-user is likely to spend a good time of approximately 20-30 mins or more in the ESP before being presented with the Desktop.

NOTE: The time spent in ESP depends not only on the number of applications or policies but also on another important factor that makes it all work – Network and bandwidth. Poor network connection or limited bandwidth will add to the time spent in the ESP stage.

Whiteglove is an effort from Microsoft to aid the above. The advantage of this is:

  • Having the majority of your applications and policies targeted to the Device context, the local IT can be used to pre-provision the devices before handing them over to the end-users.
  • This will help when the user starts the device for the first time. The time spent in the ESP phase is much lesser when compared to everyday experience.

How do Windows Autopilot WhiteGlove works?

As you start the device for the first time (for new devices) or the fresh OS install as Setup enters the oobeSystem pass – it creates the OOBE phase.

windeploy.exe > oobeldr.exe > msoobe.exe > CloudExperienceHostBroker.exe

As the CloudExperienceHost renders the OOBE Region screen (the 1st screen you get), you must press the Windows button on the keyboard five times.

Device OOBE Screen Region - Window Autopilot WhiteGlove
OOBE Region Screen – Need to press Windows key five times – Window Autopilot WhiteGlove 2

The CloudExperienceHost has special hooks registered to identify keystroke patterns to respond to.

Which causes to bring up the CMD console.

Similarly, pressing the Windows key five times in succession causes CloudExperienceHost to leave the normal OOBE flow and bring up the oobeEnterpriseProvisioning UI

Device OOBE Screen EnterprisProvisioning - Window Autopilot WhiteGlove
CloudExperinecHost renders the oobeEnterpriseProvisioning screen with the option for WhiteGolve provisioning – Window Autopilot WhiteGlove. 3

Select the Windows Autopilot provisioning and click on Continue.

What happens when you click on Continue?

The device will reach out to check the Autopilot provisioning status and, if true, will get the profile downloaded.

Window Performance Recorder - WhiteGlove activites  Window Autopilot WhiteGlove
Autopilot Whiteglove – ETL capture to see activity sequence Window Autopilot WhiteGlove 4

The sequence of activity as per the ETW trace is shown above

  • starts oobeEnterpriseProvisioning activity,
  • checks and applies critical updates related to AutopilotWhiteGloveOobeZdp activity
  • performs autopilot component updates related by AutopilotWhiteGloveUpdate
  • if there is custom device naming defined in the autopilot profile, uses the same to take effect via AutopilotWhitGloveReboot

While this is being done, you see the same screen as below that you would get in the normal Autopilot post getting connected to the Network.

Device OOBE Processing ZDP Updates - Window Autopilot WhiteGlove
Windows Autopilot WhiteGlove – OOBE ZDP update check -Window Autopilot WhiteGlove 5

Something went wrong – AUTOPILOTWHITEGLOVEUPDATE

At this stage, you can get the below error.

This is not critical and will not fail. It mostly occurs if you have run a Sysprep or reset the OS before proceeding with the deployment. KB4505903 update contains the fix for this.

However, you can safely click on Skip.

AUTOPILOTWHITEGLOVEUPDATE Error - Something went wrong - Window Autopilot WhiteGlove
Windows Autopilot WhiteGlove – AutoPilotWhiteGoveUpdate failed Window Autopilot WhiteGlove 6

As we skip and continue, the device retrieves the configuration from the downloaded profile and causes CloudExperienceHost to render the AutopilotWhiteGloveLanding screen.

Device OOBE - WhiteGLove Landing Page - Window Autopilot WhiteGlove
Autopilot Whiteglove – Landing Page – Window Autopilot WhiteGlove 7

The screen shows the details retrieved from the profile and a QR code that can be used via a companion app to check the profile settings.

  • Organization – The default domain as specified by the keyword CloudAssignedTenantDomain
  • Deployment profile – Name of the Autopilot profile as defined by the keyword DeploymentProfileName
  • Assigned user – UPN of the user explicitly added in Intune for this device as specified by the keyword CloudAssignedTenantUPN

WARNING! WhiteGlove requires a LAN connection…

WhiteGlove process will not work without an active LAN connection, as once you start the oobeEnterpriseProvisioning flow, there is no oobeWireless screen to choose Network resulting in an error (Red Screen)

Device OOBE - WhiteGlove Error Red Screen - Window Autopilot WhiteGlove
Autopilot Whiteglove Error – Requires LAN connection to work -Window Autopilot WhiteGlove.8

What happens when you click on Provision?

Clicking the Provision button starts the device provisioning phase. As can be seen from the ETW trace, the sequence is below.

WPR WhiteGlove Activities -Event Logs - Window Autopilot WhiteGlove
ETW Trace of WhiteGlove Provisioning – Event Logs – Window Autopilot WhiteGlove 9

This phase of device provisioning is carried out in a fashion similar to Autopilot Self-Deployment modeUSER LESS and is also tracked by Enrollment Status Page, which shows the tasks performed.

NOTE: The tasks as carried out during ESP are synchronous.

ESP Stage 1 - Device preparation - Window Autopilot WhiteGlove
WhiteGlove provisioning phase 1 – ESP Device Preparation -Window Autopilot WhiteGlove 10

ESP Stage 1 Device PreparationSecuring your hardware

This is when the system prepares and takes ownership of TPM – relates to TpmTaskUpdate as shown in the ETW trace.

Events from Autopilot
=====================
Event ID 177 Configuring TPM for attestation. Current attempt 1 of 10 maximum.
Event ID 201 Windows AIK not found by name.
Event ID 151 AutopilotManager started the TPM maintenance task to update TPM attestion
Event ID 186 AutopilotManager TPM configuration already in progress
Event ID 152 AutopilotManager reported TPM maintenance task is complete
Event ID 252 AutopilotManager reported AIK key was located
Event ID 250 AutopilotManager started AIK certificate aquisition task
Event ID 205 Windows AIK certficate request succeeded and new certificate is available
Event ID 168 AutopilotManager reported MSA TPM device identity was updated
Event ID 169 AutopilotManager set TPM identity confirmed

What can go wrong here – TPM Attestation?

Securing your hardware (Failed: 0x800705b4)

Is your device TPM chip 2.0, is enabled and supports device attestation?

The actual error is that the TPM requirement can be seen under ManagementService events.

Event ID 509 AutopilotManager enabled TPM requirement due to WhiteGlove policy value 1
AutopilotManager enabled TPM requirement due to Window Autopilot WhiteGlove policy
WhiteGlove Provisioning – TPM requirement is set – Window Autopilot WhiteGlove.11

WhiteGlove requires a physical device with TPM 2.0 and support for device attestation.

Virtual Machines are not supported, and in such cases, you will get an error.

You can even get an error on your physical devices if the TPM chip supports 2.0 but has not been upgraded to 2.0.

Also, the TPM chip needs to be in a ready state and support device attestation.

Essentially this is the same requirement as for Autopilot Self-Deploy mode, as Autopilot WhiteGlove device provisioning is carried out in the same fashion as Autopilot Self-Deploy mode – USERLESS provisioning, using the device’s TPM 2.0 hardware to authenticate the device into an organization’s Azure AD tenant.

Error event under Autopilot events

Event ID 156 AutopilotManager reported that MSA TPM is not configured for hardware TPM attestation even though profile indicates it is required. Autopilot cannot proceed.
AutopilotManager reported that MSA TPM is not configured for hardware TPM attestation even though profile indicates it is required. Autopilot cannot proceed. Window Autopilot WhiteGlove
WhiteGlove Provisioning – Error due to TPM – Window Autopilot WhiteGlove 12

In the error repro device I used, the TPM chip is 2.0 supported but was not upgraded. You might get other errors, such as TPM not being found as if TPM is disabled.

The below section is based on the WPR activity trace correlated with a network trace from Fiddler. For this purpose, I have done this in a way that covers the general device provisioning backend activity as well rather than only the WhiteGlove.

Fiddler Network Trace Capture - Windows Autopilot WhiteGlove
Fiddler Network trace as captured to see the endpoints – Window Autopilot WhiteGlove 13

ESP Stage 1 Device Preparation – Joining your organization’s Network

During this phase, ESP tracks the AAD join. Relates to PerformDeviceEnrollment, AADDiscovery, JoinDevice The task in the ETW trace.

NOTE: Autopilot WhiteGlove during the device provisioning phase (IT Technician Flow) does not process Hybrid AAD join even if it is the specified join method in the autopilot profile. Another similarity to Self-Deploy mode provisioning, which does not supports Hybrid AAD participation at this point!

Event ID 179 AutopilotManager began device enrollment phase AADDiscover
Event ID 181 AutopilotManager completed device enrollment AADDiscover. HRESULT 0x0
Event ID 179 AutopilotManager began device enrollment phase AADConfigure
Event ID 181 AutopilotManager completed device enrollment AADConfigure. HRESULT 0x0 
Event ID 179 AutopilotManager began device enrollment phase AADEnroll
Event ID 181 AutopilotManager completed device enrollment AADEnroll. HRESULT 0x0 

AAD Join process in a nutshell (Generalized and valid for all scenarios)

As the system gets connected to the Network,

  • It checks the status and gets the profile downloaded based on the result. Autopilot provisioning
  • Using the CloudAssignedOobeConfig bitmap value retrieved from the autopilot profile, it configures OOBE to skip the EULA screen and causes CloudExperienceHost to load the CloudDomainJoinweb apphttps://login.microsoftonline.com/WebApp/CloudDomainJoin/4

The CloudDomainJoin web app is Client Code (HTML) which uses Javascript (Worker API) to call WinRT APIs via the host process to do the device join/registration by utilizing the functions as implemented in dsreg.dll

Export Functions - dsreg.dll - Component behing AAD Join - Window Autopilot WhiteGlove
AAD Join – Behind the Scenes component dsreg.dll – Window Autopilot WhiteGlove 14

If the Autopilot check is returned false, there is no skipping the EULA screen. The rest is the same as follows.

  • CloudDomainJoin web app Sends a GETrequest to discover auth endpoints retrieving the configurations.login.microsoftonline.com/commonOpen-ID
  • With the auth endpoints retrieved, it builds a sign-in request (GETcall) to obtain a ID_Token to.Azure DRS

For consumer OOBE flow, at this point, it renders the default Sign in with Microsoft work or school account cloud sign-in page on the screen. As the user enters UPN and clicks on next…

Default Sign in with Microsoft cloud join sign-in page - Window Autopilot WhiteGlove
Device, not Autopilot provisioned? Default Sign-in with Microsoft screen – Window Autopilot WhiteGlove 15
  • It sends a GET call for realm information. The response to this call tells whether the tenant is managed or federated to reach out to the federation STS endpoint for auth.https://login.microsoftonline.com/common/userrealm/

For Autopilot user-driven scenario, till this is pre-negotiated using the values retrieved from the autopilot profile. Instead of the default Sign-in with Microsoft screen, this is how the user is presented with the custom branded tenant Sign-in page.

Custom branded Azure sign-in page - Window Autopilot WhiteGlove
Device Autopilot provisioned? Custom branded sign-in page – Window Autopilot WhiteGlove. 16

As the user performs sign-in, the rest of the process follows as below

It sends the credentials for auth to https://login.microsoftonline.com/common/login?cxhflow=OOBE &cxhver=1.0 &cxhplatform=Desktop &cxhplatformversion=10.0.18362

Noticed the cash flow value passed as a parameter. “OOBE” refers to the join performed during OOBE. If performed post OOBE from Settings, this would be “MOSET.” This information is useful in the backend to decide on device functionality support – joined from OOBE or post OOBE using Settings.

For managed tenants only, a temporary auth buffer is created to be used by Winlogon post completion of Setup, directly taking the user to the desktop screen, bypassing the Windows sign-in. For Hybrid AAD join, this is not followed.

For Autopilot Self-deployment mode and WhiteGlove, it leverages the device’s TPM 2.0 hardware to authenticate the device into the Azure AD tenant.

  • For successful auth, an.ID_Token is returned to the CloudDomainJoin web app
Auto-Enrollment scope needs to be configured previously as the The ID_Token
as returned contains the below details as claims

1.  mdm_enrollment_url
2.  mdm_tou_url or tou_url 
 
MDM Scope for Auto-Enrollment - Intune - Window Autopilot WhiteGlove
Windows Autopilot – Need to enable MDM scope to the auto-enroll device to MDM – Window Autopilot WhiteGlove.17
  • If you have Terms of Use configured, will navigate to as specified in the claim to render the same on-screen for user acceptance.CloudDomainJoin web apptou_url

The web app starts the process. Worker API calls a WinRT API implemented in dsreg.dll for the same purpose.CloudDomainJoinAzure DRS Discovery

  • It sends a GET call to https://enterpriseregistration.windows.net/joymalya.com/discover?api-version=1.6
The discovery operation response:

"DiscoveryService":{"DiscoveryEndpoint":"https:\/\/enterpriseregistration.windows.net\/joymalya.com\/Discover","ServiceVersion":"1.6"},
"DeviceRegistrationService":{"RegistrationEndpoint":"https:\/\/enterpriseregistration.windows.net\/EnrollmentServer\/DeviceEnrollmentWebService.svc","RegistrationResourceId":"urn:ms-drs:enterpriseregistration.windows.net","ServiceVersion":"1.0"},
"AuthenticationService":{"OAuth2":{"AuthCodeEndpoint":"https:\/\/login.microsoftonline.com\/deaa1de1-ef2c-41e3-9a53-81f52c39d1c3\/oauth2\/authorize","TokenEndpoint":"https:\/\/login.microsoftonline.com\/deaa1de1-ef2c-41e3-9a53-81f52c39d1c3\/oauth2\/token"}},
"IdentityProviderService":{"Federated":false,"PassiveAuthEndpoint":"https:\/\/login.microsoftonline.com\/joymalya.com\/wsfed"},"DeviceJoinService":{"JoinEndpoint":"https:\/\/enterpriseregistration.windows.net\/EnrollmentServer\/device\/","JoinResourceId":"urn:ms-drs:enterpriseregistration.windows.net","ServiceVersion":"1.0"},
"KeyProvisioningService"":{"KeyProvisionEndpoint":"https:\/\/enterpriseregistration.windows.net\/EnrollmentServer\/key\/","KeyProvisionResourceId":"urn:ms-drs:enterpriseregistration.windows.net","ServiceVersion":"1.0"},"WebAuthNService":{"ServiceVersion":"1.0","WebAuthNEndpoint":"https:\/\/enterpriseregistration.windows.net\/webauthn\/deaa1de1-ef2c-41e3-9a53-81f52c39d1c3\/","WebAuthNResourceId":"urn:ms-drs:enterpriseregistration.windows.net"},"DeviceManagementService":{"DeviceManagementEndpoint":"https:\/\/enterpriseregistration.windows.net\/manage\/deaa1de1-ef2c-41e3-9a53-81f52c39d1c3\/","DeviceManagementResourceId":"urn:ms-drs:enterpriseregistration.windows.net","ServiceVersion":"1.0"},"MsaProviderData":{"SiteId":"295958","SiteUrl":"enterpriseregistration.windows.net"},"PrecreateService":{"PrecreateEndpoint":"https:\/\/enterpriseregistration.windows.net\/EnrollmentServer\/device\/precreate\/deaa1de1-ef2c-41e3-9a53-81f52c39d1c3\/","PrecreateResourceId":"urn:ms-drs:enterpriseregistration.windows.net","ServiceVersion":"1.0"},"TenantInfo":{"TenantId":"deaa1de1-ef2c-41e3-9a53-81f52c39d1c3","TenantName":"joymalya.com"},"AzureRbacService":{"RbacPolicyEndpoint":"https:\/\/pas.windows.net"},"BPLService":{"BPLProxyServicePrincipalId":"dda27c27-f274-469f-8005-cce10f270009","BPLResourceId":"urn:ms-drs:enterpriseregistration.windows.net","BPLServiceEndpoint":"https:\/\/enterpriseregistration.windows.net\/aadpasswordpolicy\/deaa1de1-ef2c-41e3-9a53-81f52c39d1c3\/","ServiceVersion":"1.0"},"DeviceJoinResourceService":{"Endpoint":"https:\/\/enterpriseregistration.windows.net\/EnrollmentServer\/device\/resource\/deaa1de1-ef2c-41e3-9a53-81f52c39d1c3\/","ResourceId":"urn:ms-drs:enterpriseregistration.windows.net","ServiceVersion":"1.0"}}

The response as received contains information about, Provisioningng endpoint, etc. This info is locally cached and can be viewed using the command line.Azure DRS URIdsregcmd

The web app starts the process with the discovery data and the ID_Token available. This is a Worker API call to invoke WinRT API implemented in dsreg.dll for the same purpose.

  • The API, as leveraged, generates a key pair to create a device CSR. In addition, a 2nd key pair, called the, is also made, which will be used to protect the SSO tokens (Primary Refresh Token).
  • The ID_Token and the public portion of TPMattestation data is sent as part of the registration/join request. Storage/Transport keyAzure DRS

This is a POST call to the Azure DRS endpoints retrieved during Discovery.

The get join response operation callback was successful. 

Activity Id: 34b7ca40-5367-4625-bd70-36b52d473449 

Server response was: {"Certificate":{"Thumbprint":"A2A8F71D926FBCDA6848B740BCB2ED6A135150D0",
"RawBody":"MIID8jCCAtqgAwIBAgIQCWkLOIfrK6ZBy..."},
"User":{"Upn":"[email protected]"},
"MembershipChanges":[
{"LocalSID":"S-1-5-32-544",
"AddSIDs":["S-1-12-1-****","S-1-12-1-*****","S-1-12-1-****"]}]}

For Autopilot, Azure DRS does not create a new device object but instead looks for the pre-created Azure AD device object (the GUID is provided in the Join request) to enable it and update the object properties.

The Join Request and Response for Autopilot Self-Deploy and WhiteGlove pre-provisioning looks like this.

Event ID 102 under User Device Registration

The initialization of the join request was successful. Inputs:
JoinRequest: 11 (DEVICE_AUTO_DDID)
Domain: prolick.onmicrosoft.com
Event ID 104 under User Device Registration

Server response was: {"Certificate":{"Thumbprint":"4AD998D09187656576C0CA055DA3035AFD09102D",
"RawBody":"MIID9TCCAt2gAwIBAgIQIOZS..."},
"ResponseStatus":
{"message":"Successfully updated pre-created device with id 
d48e9478-7056-4df8-bce7-11a18b8c0027.",
"traceId":"6c0110f3-e6df-4b83-a80d-1781ff5d1525",
"time":"09-20-2019 15:37:40Z"},
"MembershipChanges":[{"LocalSID":"S-1-5-32-544",
"AddSIDs":["S-1-12-1-****","S-1-12-1-*****"]}]}

I noticed that there isn’t any UPN specified here.

Azure DRS Upon receiving the request, creates a device object in Azure AD and sends back a cert to the device, stored in the Personal Store of the LocalMachine account.

The Subject Name of this certificate is the Azure AD device GUID and can be viewed using CMD with the command. dir Cert:\LocalMachine\My\ | where { $_.Issuer -match "CN=MS-Organization-Access" } | fl

Certificate Store - Local Machine - MS-Organization-Access cert - AAD device GUID - Window Autopilot WhiteGlove
MS-Organization-Access Cert – SN = Azure AD Device GUID – Window Autopilot WhiteGlove 18

For a user-driven model, the device also gets the Azure PRT simultaneously. For WhiteGlove pre-provisioning, it is received during the user flow.

This completes the Azure DRS Join process.

Wondering what the MS-Organization-P2P-Access[2018] cert is for? The P2P certificate is pushed down by Azure AD during authentication of the user in the device to support remote desktop connectivity to another Azure AD joined device (peer-to-peer). The target device will authenticate this certificate against Azure AD before the remote connection is established. 

As performed from Settings > Accounts > Access work or school > Connect, the Azure AD registration has the same backend process. The only difference is the implementation at the device end – it registers the device to Azure, not a Join.

Things that can go wrong here…

Joining your organization’s Network (Failed 0x801C03F3)

Have you deleted the Azure AD device object pre-created during DDS registration?

Remember, in my previous post, I hinted at this! Well, this is where you will see the reason. As Azure DRS receives the join request, it searches for the associated Azure AD device object with the ZTDID stamping.

Suppose you have deleted the entry from Azure for Self-Deploy mode and WhiteGlove.

In that case, it results in an error as no match is found against the associated device guide, and you would need to remove the device and re-register it back to DDS for provisioning.

This security mechanism is implemented to stop unknown devices from joining Azure AD under the userless scheme and gain access to resources.

The error you will come across in events are below

Event ID 180 AutopilotManager failed during device enrollment phase AADEnroll. HRESULT=0x801C03F3
AutopilotManager failed during device enrollment phase AADEnroll. HRESULT=0x801C03F3 - Window Autopilot WhiteGlove
WhiteGlove Provisioning – Autopilot failed during AADEnroll – Window Autopilot WhiteGlove 19

This error does not tell much unless you check the User Device Registration events

Event ID 204 

The get join response operation callback failed with exit code: Unknown HResult Error code: 0x801c03f3. 
 Activity Id: 9a8f324f-4515-4c9e-86c1-ad2337de3e4d 
 The server returned HTTP status: 400 
 Server response was: {"Code":"AuthenticationError","Subcode":"DeviceNotFound","Message":"A device with the expected device identifiers was not found","TraceId":"9a8f324f-4515-4c9e-86c1-ad2337de3e4d","Time":"09-16-2019 14:55:45Z"}
A device with the expected device identifiers was not found.
Error code: 0x801c03f3 - Window Autopilot WhiteGlove
WhiteGlove Provisioning – AAD Device Object not found – Window Autopilot WhiteGlove 20

Note: During WhiteGlove pre-provisioning, even though the error comes up in the User Device Regsitration events, no User Device Association is performed. The process is a complete non-user affinity process.

ESP Stage 1 Device preparation – Registering your device for mobile management

This relates to the task in the ETW trace, which enrolls the device to the tenant configured MDM service.MDMEnroll

MDM Enroll process in a nutshell

It starts with the web app Worker API calls the MDM Enrollment API (implemented in mdmregistration.dll) to register the device to the configured MDM service.CloudDomainJoin

Learn more about the Microsoft Mobile Device Enrollment protocol here.

But to proceed, it requires a token to authenticate to the Enrollment Service. This is the task you see in the ETW trace analysis.AccessESTSTicket

NOTE: In the case of user-driven mode, the user has already been authenticated, and the device completed the join process and can establish itself to Azure AD. For self-deploy and whiteglove, the device auth and join are conducted against the device’s TPM secure identity.

, To obtain the web app, get a from the Enrollment Service silently using the cookies as stored during the authentication. MDM Access tokenCloudDomainJoinauth codeAzure DRS

Remember the ID_Token as received during Azure DRS contains the mdm_enrollment_url in claims, which points to https://enrollment.manage.microsoft.com/enrollmentserver/discovery.svc

The corresponding GET call

https://login.microsoftonline.com/common/oauth2/authorize?client_id=29d9ed98-a469-4536-ade2-f981bc1d605e

where the client_idcorresponds to the MDM App as configured in Azure.

MDM configured in Azure - Window Autopilot WhiteGlove
MDM configured in tenant – Window Autopilot WhiteGlove 21

The response type requested for this call is which the worker API will use to obtain a token to the Enrollment Server.GETresponse_type=codeCloudDomainJoinAccess

This is a POST call to

https://login.microsoftonline.com/common/oauth2/token

In response, it receives a token that contains an Access token, ID_token, and a refresh token. Essentially it is done during the Azure DRS. The process only post Discovery and cached, as this is required to reach out to the BearerMDM_TOU_URL

The CloudDomainJoin web app extracts the token from it to pass it to the MDM Enrollment API which is implemented is mdmregistration.dll to proceed with the enrollment.Access

{
"token_type":"Bearer",
"scope":"mdm_delegation",
"expires_in":"3599","ext_expires_in":"3599",
"expires_on":"1568823035","not_before":"1568819135",
"resource":"https://enrollment.manage.microsoft.com/",
"access_token":"eyJ0eXAiOiJKV1QiLC..."
}
Exported Functions - mdmresgitration.dll - component behind MDM Enrollment - Window Autopilot WhiteGlove
Export Functions of mdmregistration.dll – the component behind MDM enrollment – Window Autopilot WhiteGlove 22

The main part involved here is DeviceEnroller.exe which works by implementing the WinRT APIs as provided via exported functions in the mdm dlls like mdmregistration.dll and others. This is a long explanation and deserves an article of its own!

The enrollment process is essentially the same as the Azure Join process. The MDM Enrollment API will cause the device to create a CSR to be sent to the enrollment server and, in return, will get a cert, the Subject Name of which will be the Intune Device GUID.

Microsoft Intune MDM Device cert - Local Machine Cert Store - Window Autopilot WhiteGlove
MDM Device Cert – SN = MDM Device GUID – Window Autopilot WhiteGlove 23

Things that can go wrong here

Registering your device for mobile management (Failed 0x81036501)

Do you have multiple MDM solutions configured in your tenant?

Scenario 1

Under Azure AD > Mobility (MDM and MAM), you have two entries for Intune

  • Microsoft Intune
  • Microsoft Intune Enrollment
Multiple entries for Intune - reason for failure. Autopilot discovery failed to find a valid MDM. HRESULT = 0x81036501 - Window Autopilot WhiteGlove
Microsoft Intune and Microsoft Intune Enrollment under Mobility – Window Autopilot WhiteGlove 24

The Microsoft Intune Enrollment does not aid in enrollment but gets created for CA enforcement during registrations, mainly designed to enforce MFA. It is recommended not to enforce MFA via CA for registration. There are other ways.

Scenario 2

Under Azure AD > Mobility (MDM and MAM), you have separate MDM configured

Multiple unique MDM configured for tenant. Autopilot discovery failed to find a valid MDM. HRESULT = 0x81036501 - Window Autopilot WhiteGlove
Multiple unique MDM configured for tenant – Window Autopilot WhiteGlove 25

For scenario 1, MDM Discovery URL in both, the entries are the same but resolve to two different client_id and, as such, create a conflict failing.

For scenario 2, DeviceDiscovery fails to decide which endpoint to reach for enrollment.

For either of the above, the error events you would find will be

Event ID 302 AutopilotManager device enrollment failed duinrg stage DeviceDiscovery with error 0x81036501

Event ID 180 AutopilotManager failed during device enrollment phase DeviceDiscovery. HRESULT = 0x81036501 

Event ID 115 Autopilot discovery failed to find a valid MDM. Confirm that the AAD tenant is properly provisioned and licensed for exactly one MDM. HRESULT = 0x81036501  
WhiteGlove Provisioning - Failure due to multiple MDM configured. HRESULT = 0x81036501 - Window Autopilot WhiteGlove
WhiteGlove Provisioning – Failure due to multiple MDM configured – Window Autopilot WhiteGlove 26

ESP Stage 1 Device preparation – Preparing your device for mobile management

During this task, ESP waits for the policy providers to complete their registration

  • Intune (the MDM service itself) and
  • Intune Management Extension (from 1903, ESP can track win32 apps as well)

There isn’t a failure point here, but it takes time at this task since it is waiting for Intune to deliver the IME MSI installer package and then waits for IME to initialize and get the policies it would process so that ESP can track the same.

More information hereby Michael Niehaus

ESP Stage 2 Device setup

ESP Stage 2 - Device setup - Window Autopilot WhiteGlove
ESP Stage 2 – Device setup – Window Autopilot WhiteGlove 27

As ESP completes stage 1, it moves to the next stage, Device setup, where it tracks the implementation of the policies as deployed to Device context (and apps with install context as device). The failure points here can be

  • App installation failure identified by the LastHResult of the app log
  • SCEP/PKCS cert failure due to NDES related errors

Provisioning Status – GREEN or RED screen?

If the pre-provisioning is successful, the device presents you with the GREEN screen, and you have the option to RESEAL.

Windows Autopilot WhiteGlove - GREEN Screen showing provisioning succes - Window Autopilot WhiteGlove
2 Windows Autopilot WhiteGlove – GREEN Screen showing provisioning success – Window Autopilot WhiteGlove 28

If you would like to check the events for success, you can do that as well

Windows Autopilot WhiteGlove - Event ID 303 success - Window Autopilot WhiteGlove
2Windows Autopilot WhiteGlove – Event ID 303 success – Window Autopilot WhiteGlove 29

However, any error during provisioning leads to the below RED screen.

Autopilot WhiteGlove - RED Screen - Provisioning Failed - RESET/RETRY
Windows Autopilot WhiteGlove – RED Screen showing provisioning failed – Window Autopilot WhiteGlove 30

As an analogy, WhiteGlove pre-provisioning can be compared to booting the system to Audit mode to prepare the device as in the audit system config pass.

Information: You can boot Windows to Audit mode from OOBE with the key combination CTRL+SHIFT+F3 at the first OOBE screen or with the command line using Sysprep /audit utilizing the SHIFT+F10 key combo.

Thus we can safely say that Autopilot Whiteglove gives us the same benefits as the Audit mode config pass saving us from the manual works, but here comes the contrast.

Audit mode runs using the default Built-in Administrator account, which is again disabled while exiting the pass.

WhiteGlove works in oobeSystem pass. The technician flow is done under the temporary defaultuser0 account.

However, both are reunited again at their end with the same ideology – RESEAL, a Microsoft-Windows-Deployement component from the Windows Unattended Setup reference.

In Audit mode, post completing the device preparation and setup, you would need to run a Sysprep /shutdown /oobe. to exit the Audit mode config pass and set OOBE as the start point for the next boot before the shutdown, which is identical to RESEAL action

Component Name="Microsoft-Windows-Deployment"
Reseal
ForceShutdownNow: true/false
Mode: OOBE/Audit

What happens when you click on Reseal?

As mentioned above, the RESEAL button behaves analogously to sysprep /shutdown /oobe

  • it prepares Windows Setup to start from OOBE on the next boot, and
  • shuts down the computer immediately with no end-user interaction

At this point, the system is ready to be handed over to the end-user.

State of the device – Before and After WhiteGlove Preprovisioning

  • Windows Autopilot WhiteGlove Provisioning Backend Process- Deep Dive - Post 4 1
  • Windows Autopilot WhiteGlove Provisioning Backend Process- Deep Dive - Post 4 2

Azure AD Devices Audit showing DRS and Intune as initiators for Device Update activity as a result of Join and Enrollment

  • Windows Autopilot WhiteGlove Provisioning Backend Process- Deep Dive - Post 4 3
  • Windows Autopilot WhiteGlove Provisioning Backend Process- Deep Dive - Post 4 4

Post provisioning device status – Intune and Azure

  • Windows Autopilot WhiteGlove Provisioning Backend Process- Deep Dive - Post 4 5
  • Windows Autopilot WhiteGlove Provisioning Backend Process- Deep Dive - Post 4 6

As you can see, the device is enrolled and is in the managed state, but compliance shows as. This is due to the fact – till this point, the device is treated as a non-user affinity device.Not Evaluated

Information Alert!

Intune does not evaluates complaince for a userless device, reason being 
the Built-in Device Complaince Policy has check criteria related to user 
affinity which will always be false for a userless deployement.

 Windows Autopilot WhiteGlove Provisioning Backend Process- Deep Dive - Post 4 7
Window Autopilot WhiteGlove

What happens as the end-user starts the device?

As the end-user starts the device, it starts from the OOBE again as per the Reseal action and follows the same flow as a normal autopilot provisioning would

However, in this case, AutopilotManager will check and see that the device is already provisioned, so it will not download the profile again.

  • Setup starts which take the user through the usual Region, Keyboard, and additional Keyboard layout screen. CloudExperienceHost
  • If not connected via LAN, the user is presented with the Network screen to select an active network. As the user clicks and proceeds, it again checks for ZDP updates.
  • During this time, autopilotmanager checks that the device is already provisioned and retrieves the values from the profile to drive the rest of OOBE.
  • It pre-negotiates end-points to present end-user with the custom branded sign-in page.

Till this point, it is the defaultuser0 profile in Session 1 driving the process.

Windows Autopilot Whiteglove - User Flow - Session 1 defaultuser0
Windows Autopilot Whiteglove – User Flow – Session 1 defaultuser0 31

User Device Association events are triggered to associate the UPN with the corresponding device object in Intune and Azure AD as the user performs sign-in.

You can see the User Device Association events in Azure Device Audits below.

Windows Autopilot Whiteglove - UDA events in Azure Audit logs
Windows Autopilot Whiteglove – UDA events in Azure Audit logs 32

As the device gets associated with the user, you would also notice Intune initiating Update device activity on the Azure Device object to update the compliance state.

Windows Autopilot Whiteglove - Intune initiating complaince state update
Windows Autopilot Whiteglove – Intune initiating compliance state update 33

There is a session change happening at this point.

  • Winlogon logs off defaultuser0, and profile cleanup tasks are initiated. (This temporary user account is ready to be discarded at this point)
  • Winlogon performs login against the original user account using the credential buffer created by the CloudDomainJoin web app during the Azure DRS sign-in.

For the Hybrid AAD Join type, there is no cred buffer created that Winlogon can use, and as such user is presented with the Winlogon UI screen requiring a manual sign-in.

The First Sign-In Animation is displayed post which comes to the ESP to complete its last stage – tracking the Account setup.

Windows Autopilot Whiteglove - User flow - ESP Account Setup
Windows Autopilot Whiteglove – User flow – ESP Account Setup 34

As ESP completes tracking Account Setup, the device prompts for Hello enrollment if Hello for Business is set.

Event ID 358 User Device Registration

Windows Hello for Business provisioning will be launched. 
Device is AAD joined ( AADJ or DJ++ ): Yes 
User has logged on with AAD credentials: Yes 
Windows Hello for Business policy is enabled: Yes 
Windows Hello for Business post-logon provisioning is enabled: Yes 
Local computer meets Windows hello for business hardware requirements: Yes 
User is not connected to the machine via Remote Desktop: Yes 
User certificate for on premise auth policy is enabled: No 
Windows Autopilot Whiteglove - User Flow - Hello Enrollment
Windows Autopilot Whiteglove – User Flow – Hello Enrollment 35

Once done, for join type Azure AD only, the user will be automatically logged in and presented with the Desktop.

Windows Autopilot WhiteGlove- User Flow Completed - User presented with Desktop
Windows Autopilot WhiteGlove- User Flow Completed – User presented with Desktop 36

Something Odd That I Noticed?

View Diagnostics in case of failure doesn’t work.

If you get to the RED screen due to an error as I covered above, if you click on the View Diagnostics button, it opens a File Explorer window

Windows Autopilot WhiteGlove - RED Screen - View Diagnostic opens File Explorer window
Windows Autopilot WhiteGlove – RED Screen – View Diagnostic opens File Explorer window 37

, If you have a USB drive attached and choose a folder for log collection and click on Select Folder, it fails to state, “Provisioning information could not be located. Contact the customer IT admin to troubleshoot.”

Windows Autopilot WhiteGlove - RED Screen - View Diagnostic Fails to capture provisioning info
Windows Autopilot WhiteGlove – RED Screen – View Diagnostic Fails to capture provisioning info 38

Well, it’s a bit odd that you would yourself be the IT admin or support at this error stage.

But you still have a fully functional Windows running behind this RED screen…

You have your usual arsenal of Win+R launching Event Viewer, Registry, PowerShell, or Shift+F10 run Admin context CMD.

Windows Autopilot WhiteGlove - Tools to collect logs from OOBE itself
Windows Autopilot WhiteGlove – Tools to collect logs from OOBE itself 39

Yeah, the reference snap is over the GREEN screen, but it’s the same for the RED screen. And did I forget to mention that you have to work Alt+Tab to cycle through the windows?

Windows Autopilot WhiteGlove - Alt+Tab to cycle through tools in OOBE
Windows Autopilot WhiteGlove – Alt+Tab to cycle through tools in OOBE 40

defaultuser0 is still present under C:\Users?

This is something that you would see very rarely. Sometimes, the profile cleanup doesn’t happen as it should due to some unknown reasons.

During the session change that occurs in OOBE before the ESP Account phase comes up, if there is a restart, the system starts with the original user account profile (as provided during Azure sign-in) in Session 1 itself.

Running in, Winlogon initiates the original user account login in another session started by smss.exe – since, for the same boot cycle, a new user login causes a unique user session initiation.defaultuser0session 1session 2

  • Windows Autopilot WhiteGlove Provisioning Backend Process- Deep Dive - Post 4 8
  • Windows Autopilot WhiteGlove Provisioning Backend Process- Deep Dive - Post 4 9

In such a case, when you are presented with the desktop screen, you will still see the account folder if you go.c:\Usersdefaultuser0

Windows Autopilot Whiteglove - defaultuser0 profile still present post provisioning
Windows Autopilot Whiteglove – defaultuser0 profile still present post provisioning 41

This gets removed automatically if you do a restart manually. So the profile cleanup task waits for a restart, but I couldn’t seem to find a task scheduled for the same πŸ™

Ending

This completes the 4 part series of Windows Autopilot, which I started. I hope with each article of this series. I was able to provide you with some valuable information and clarify the underlying working principles of Windows Autopilot.

Well, I might come up with a supplementary post for this series about re-provisioning Autopilot devices, but till then, as I always say before ending – read something every day, learn something every day!

Resources

24 thoughts on “Windows Autopilot WhiteGlove Provisioning Backend Process- Deep Dive – Post 4”

  1. Im getting an error while
    Registering your device for mobile management, Failed: 3, 0x801c03f3

    Already checked TPM version on the laptop which is 2.0
    Can you please advise?

    Reply
    • Ideally the error should be for this event in User Device Registration, which happens if you have unknowingly deleted the Azure AD device object which was precreated as part of DDS registration.

      The get join response operation callback failed with exit code: Unknown HResult Error code: 0x801c03f3.
      Activity Id: #####
      The server returned HTTP status: 400
      Server response was: {“Code”:”AuthenticationError”,”Subcode”:”DeviceNotFound”,”Message”:”A device with the expected device identifiers was not found”,”TraceId”:”######”,”Time”:”01-10-2019 14:04:25Z”}

      Reply
  2. Great Article, I see on your article it says “Need to enable MDM scope to auto-enroll device to MDM”

    is this 100% needed? Currently we restrict enrollement to Intune to a group of users. How would we restrict access if we set this to all?

    Reply
    • You can set MDM scope enabled to particular user groups instead of enabling for all users. Anyways for whiteglove pre-provisioning process it will not cause an issue as it is a userless process. But think when you handover the device to a user who is not part of the MDM scope. Then how will the user device association happen? The ID_Token as returned dueing auth for Azure DRS wont contain the MDM endpoint information, or say if you have MAM scope enabled for that user, instead of device management enrollment endpoint, it would point to the WIP management endpoint. This is a conflict scenario.

      Also this MDM scope applies to Windows devices only (Android and iOS are not affected by this and needs to be restricted in other ways)

      If in your environment, you have restricted Windows enrollment via MDM scope, it is advisable to hand over whiteglove provisoned devices to users who are part of the MDM scope.

      Reply
      • Thanks for the reply. We have found though that with the Mdm scope set to a group of users. White glove does not complete. The device never enrolls in intune. If we set to all it works.

      • We are just trying to provision the devices being giving to the end user. It never finishes the intune enrollment and we have aadenroll errors. If we set to all it enrolls in intune and apps start to deploy.

  3. I get enrolment working with no issues, but had the same trouble on 1909 with the l view diagnostics not working.

    Main issue I’m seeing is that Bitlocker doesn’t seem to encrypt the devices during the white glove process and in a secure environment it means devices being deployed to users that haven’t yet encrypted which doesn’t meet the security requirements.

    Reply
    • Hi James,

      Pre encrypting devices with Bitlocker is an option with Config Manager via Task Sequence but not via Intune.
      For an Intune managed device, Bitlocker encryption gets triggered in the User Phase during ESP when post completing Device Setup and running the FSIA before ESP enters the Account Setup phase.
      This is by design as I know.

      Regards,
      Joy

      Reply
  4. Very useful blog post, thank you for that. For “Device preparation – Preparing your device for mobile management”, you said “There isn’t a failure point here, but you will see it takes time at this task since it is waiting for…” Turns out things can go wrong here as well. I have a Surface 7 that I’m working on, with Windows 10 Pro 1909 build 18363.657, TPM 2.0, wired network. It sits at this step exactly 30 minutes each time, and it briefly flashes the message “Failed: 0x800705b4” before going into the red screen. Retried several times; same outcome. Any suggestions?

    Reply
    • Hi Bodek,

      Thanks for pointing this out. In general, the error code you mentioned β€œFailed: 0x800705b4” under device preparation phase usually relates to TPM, where Windows fails to prepare TPM and take ownership. Since Autopilot Whiteglove phase carries out in similar fashion of Self-deploy mode, it relies on device auth since this is a userless phase and TPM plays an important role. Can you check if you are getting any TPM errors. Also can you check on the same device if it can do a Autopilot Self-Deploy. I think if TPM is causing the issue, you should get the same error. Let me know.

      Regards,
      Joy

      Reply
      • Thanks for the reply. On that device, I ended up resetting Windows and starting from scratch. It worked fine the second time.

  5. Very useful article. I have one problem working with Whiteglove and I hope maybe you have some input.
    We are currently applying autopilot profile which works fine, but then we do the white glove, and we have some apps added in ESP that will block device that is assigned to device. because we want the apps to be installed before the user log on. Most of the time the apps we have added dont get innstalled in the white glove and it shows green screen after just a minute with success, but apps not installed. but if I then do a “fresh restart” from intune on the device and try the whiteglove once more, then the apps get instaled. you have seen this issue before?

    Reply
  6. Nice article!

    I’m having an issue getting started with the whiteglove procedure.

    When booting Windows the first time after installation with an USB, the CloudExperienceHost wizard automatically proceeds to the network step, thus skipping the language dialog.

    So, I’m not able to press the Windows key 5 times…

    Exact same system, same USB works fine.

    Do you know if we can force Windows to proceed to oobeEnterpriseProvisioning?

    Reply
  7. Hi,
    I would like to tailor a windows build and use this within my Autopilot provisioning.
    We use MDT for our builds, how would i incorporate my mdt build with the above?

    thanks

    Reply
  8. Windows 11 does not allow to use of autopilot pre-provisioning.
    Is it a known issue?
    Pressing Windows key 5 times does not take any effect with Windows 11.
    The same device with the same pressing with Windows 10 works as expected.

    Reply
  9. Great article, thanks for the time you put in.
    One question about the Reseal, is the sysprep /shutdown /oobe absolutely the same as the Reseal button?
    So with a device that is running on error and I am confident that everything fits, could I manually run the command to get back out of pre-provisioning?

    Reply
  10. I also cannot pre-provision Windows 11 on a computer by pressing the Windows key 5 times. If I reimage the same computer with Windows 10 and press the Windows key 5 times it works properly, just not with Windows 11. Is anyone else running into this issue?

    Reply
  11. I figured out why pressing the Windows key 5-times in Windows 11 was not working for me. As it turns out, at least in my case, when I reached the “Choose your region” screen I kept pressing the Windows key (many more times than 5) and nothing would happen UNTIL I made sure that I had actually selected the REGION screen by clicking on my region (not next) before trying to click the Windows key 5-times. This worked for me, so I’m guessing that the REGION screen must be in the foreground and selected in Windows 11 before pressing the Windows key 5-times will work.

    Reply
  12. Can you please explain this better?
    “For the Hybrid AAD Join type, there is no cred buffer created that Winlogon can use, and as such user is presented with the Winlogon UI screen requiring a manual sign-in.”

    According to Microsoft Docs, it should still prompt you to sign in using Azure AD Credentials – “On the branded sign-on screen, enter the user’s Azure Active Directory credentials.”

    See here under the “User Flow” Section: https://learn.microsoft.com/en-us/mem/autopilot/pre-provision#scenarios

    Reply
  13. Is there a way we can say the device is pre-provisioned or a regular Autopilot device. Trying to create a dynamic group with that information for segregation.

    Reply

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.