Windows Autopilot WhiteGlove Provisioning Backend Process- Deep Dive – Post 4

4

This is the last concluding article of the Autopilot series which I started. In my previous articles of this series, I have explained in depth the working details behind Windows Autopilot. If you have not read them yet, I would definitely suggest to give them a read. Let’s see what is Window Autopilot WhiteGlove.

Related Posts

Introduction

If you have read them and/or you know a little bit about it, you would agree that though Windows Autopilot comes with minimal IT overhead and gives the end users a smooth and customized device provisioning experience, but it usually is time consuming.

This is generally true in cases where you have a lot of applications deployed and you want all of them to be installed before the user can become productive on the device. In such cases, the provisioning phase will spend a lot of time in the ESP screen with the user sitting in front of the device doing nothing. It’s more like, start the device -> get it connected to network -> sign-in -> sit back or do other works as it provisions

In this fast paced cloud backed IT world, ideally this would not be considered as a seamless experience. Well Microsoft came up with something to address this – Windows Autopilot WhiteGlove Provisioning

Autopilot Whiteglove profile - Intune - Window Autopilot WhiteGlove
Window Autopilot WhiteGlove

What is WhiteGlove process and how it is different from normal Autopilot provisioning?

Introduced with Windows 10 1903 (codename 19H1, finally MS moved to a relatable codename scheme from it’s previous RS and TH theme), Windows Autopilot gives IT Admin an option to pre-provision the device before handing it over to the end user.

This is essentially to reduce the time the device stays in Enrollment Status Page when user goes through the entire process. Especially in cases where there are many policies and applications (large applications like O365 Pro Plus Suite) are set to be tracked in ESP.

Yes there is an option in ESP setup to select particular apps to be tracked only, but even with this, the end user is likely to spend a good time of approximately 20-30 mins or more in the ESP before being presented with the Desktop.

NOTE: The time spent in ESP is not only dependent on the number of applications or polices but another important factor which makes it all work – network and bandwidth. Poor network connection or limited bandwidth will add to the time spent in ESP stage.

Whiteglove is an effort from Microsoft to aid the above. The advantage of this is:

  • Having majority of your applications and policies targeted to Device context, the local IT can be used to pre-provision the devices before handing it over to the end users.
  • This will help when the user starts the device for the first time, the time spent in ESP phase is much lesser when compared to regular experience.

How does Windows Autopilot WhiteGlove works?

As you start the device for the first time (for new devices) or for fresh OS install as Setup enters the oobeSystem pass – starts the OOBE phase.

windeploy.exe > oobeldr.exe > msoobe.exe > CloudExperienceHostBroker.exe

As the CloudExperienceHost renders the OOBE Region screen (the 1st screen that you get), you need to press the Windows button on the keyboard for 5 times.

Device OOBE Screen Region - Window Autopilot WhiteGlove
OOBE Region Screen – Need to press Windows key 5 times – Window Autopilot WhiteGlove

The CloudExperienceHost has special hooks registered to identify keystroke patterns to respond to. Like Shift+F10 which causes to bring up the CMD console. Similarly pressing the Windows key for 5 times in succession causes CloudExperienceHost to leave the normal OOBE flow and bring up the oobeEnterpriseProvisioning UI

Device OOBE Screen EnterprisProvisioning - Window Autopilot WhiteGlove
CloudExperinecHost renders the oobeEnterpriseProvisioning screen with option for WhiteGolve provisioning – Window Autopilot WhiteGlove

Select the Windows Autopilot provisioning and click on Continue.

What happens when you click on Continue?

The device will reach out to check the Autopilot provisioning status and if true will get the profile downloaded.

Window Performance Recorder - WhiteGlove activites  Window Autopilot WhiteGlove
Autopilot Whiteglove – ETL capture to see activity sequence Window Autopilot WhiteGlove

The sequence of activity as per ETW trace as shown above

  • starts oobeEnterpriseProvisioning activity ,
  • checks and applies critical updates related by AutopilotWhiteGloveOobeZdp activity
  • performs autopilot component updates related by AutopilotWhiteGloveUpdate
  • if there is custom device naming defined in autopilot profile, applies the same to take effect via AutopilotWhitGloveReboot

While this is being done, you see the same screen as below that you would get in the normal Autopilot post getting connected to network.

Device OOBE Processing ZDP Updates - Window Autopilot WhiteGlove
Windows Autopilot WhiteGlove – OOBE ZDP update check -Window Autopilot WhiteGlove

Something went worng – AUTOPILOTWHITEGLOVEUPDATE

At this stage you can get the below error. This is not critical and will not result in failure. It mostly occurs if have ran a Sysprep or reset the OS before proceeding with the deployement. KB4505903 update contains the fix for this. However, you can safely click on Skip.

AUTOPILOTWHITEGLOVEUPDATE Error - Something went wrong - Window Autopilot WhiteGlove
Windows Autopilot WhiteGlove – AutoPilotWhiteGoveUpdate failed Window Autopilot WhiteGlove

As we skip and continue, the device retrieves the configuration from the downloaded profile and causes CloudExperienceHost to render the AutopilotWhiteGloveLanding screen.

Device OOBE - WhiteGLove Landing Page - Window Autopilot WhiteGlove
Autopilot Whiteglove – Landing Page – Window Autopilot WhiteGlove

The screen shows the details as retrieved from the profile along with a QR code which can used via a companion app to check the profile settings.

  • Organization – The default domain as specified by the keyword CloudAssignedTenantDomain
  • Deployment profile – Name of the Autopilot profile as specified by keyword DeploymentProfileName
  • Assigned user – UPN of the user explicitly added in Intune for this device as specified by the keyword CloudAssignedTenantUPN

WARNING ! WhiteGlove requires LAN connection…

WhiteGlove process will not work without an active LAN connection, as once you start the oobeEnterpriseProvisioning flow, there is no oobeWireless screen to choose Network resulting in error (Red Screen)

Device OOBE - WhiteGlove Error Red Screen - Window Autopilot WhiteGlove
Autopilot Whiteglove Error – Requires LAN connection to work -Window Autopilot WhiteGlove

What happens when you click on Provision?

Clicking the Provision button starts the device provisioning phase. The sequence as can be seen from the ETW trace is below.

WPR WhiteGlove Activities -Event Logs - Window Autopilot WhiteGlove
ETW Trace of WhiteGlove Provisioning – Event Logs – Window Autopilot WhiteGlove

This phase of device provisioning is carried out in fashion similar to Autopilot Self-Deployment modeUSER LESS and is also tracked by Enrollment Status Page which shows the tasks as being performed.

NOTE: The tasks as carried out during ESP are synchronous in nature.

ESP Stage 1 - Device preparation - Window Autopilot WhiteGlove
WhiteGlove provisioning phase 1 – ESP Device Preparation -Window Autopilot WhiteGlove

ESP Stage 1 Device PreparationSecuring your hardware

This is when system prepares and takes ownership of TPM – relates to TpmTaskUpdate as shown in the ETW trace.

Events from Autopilot
=====================
Event ID 177 Configuring TPM for attestation. Current attempt 1 of 10 maximum.
Event ID 201 Windows AIK not found by name.
Event ID 151 AutopilotManager started the TPM maintenance task to update TPM attestion
Event ID 186 AutopilotManager TPM configuration already in progress
Event ID 152 AutopilotManager reported TPM maintenance task is complete
Event ID 252 AutopilotManager reported AIK key was located
Event ID 250 AutopilotManager started AIK certificate aquisition task
Event ID 205 Windows AIK certficate request succeeded and new certificate is available
Event ID 168 AutopilotManager reported MSA TPM device identity was updated
Event ID 169 AutopilotManager set TPM identity confirmed

What can possibly go wrong here – TPM Attestation?

Securing your hardware (Failed: 0x800705b4)

Is your device TPM chip 2.0, is enabled and supports device attestation?

The reason behind the actual error is TPM requirement can be seen under ManagementService events

Event ID 509 AutopilotManager enabled TPM requirement due to WhiteGlove policy value 1
AutopilotManager enabled TPM requirement due to Window Autopilot WhiteGlove policy
WhiteGlove Provisioning – TPM requirement is set – Window Autopilot WhiteGlove

WhiteGlove requires physical device with TPM 2.0 and support for device attestation.

Virtual Machines are not supported and in such cases, you will get an error. You can even get an error on your physical devices if TPM chip supports 2.0 but has not been upgraded to 2.0. Also the TPM chip needs to be in ready state and support device attestation.

Essentially this is the same requirement as for Autopilot Self-Deploy mode, as Autopilot WhiteGlove device provisioning is carried out in same fashion as Autopilot Self-Deploy mode – USERLESS provisioning, using the device’s TPM 2.0 hardware to authenticate the device into an organization’s Azure AD tenant.

Error event under Autopilot events

Event ID 156 AutopilotManager reported that MSA TPM is not configured for hardware TPM attestation even though profile indicates it is required. Autopilot cannot proceed.
AutopilotManager reported that MSA TPM is not configured for hardware TPM attestation even though profile indicates it is required. Autopilot cannot proceed. Window Autopilot WhiteGlove
WhiteGlove Provisioning – Error due to TPM – Window Autopilot WhiteGlove

The error repro device I used, the TPM chip is 2.0 supported but was not upgraded to. You might get other errors such TPM not found if TPM is in disabled state

The below section is based on the WPR activity trace correalated with a network trace from Fiddler. For the purpose, I have done this in a way, which covers the general device provisioning backend activity as well rather than only the WhiteGlove.

Fiddler Network Trace Capture - Windows Autopilot WhiteGlove
Fiddler Network trace as captured to see the endpoints – Window Autopilot WhiteGlove

ESP Stage 1 Device Preparation – Joining your organization’s network

During this phase, ESP tracks the AAD join. Relates to PerformDeviceEnrollment, AADDiscovery, JoinDevice task in the ETW trace.

NOTE: Autopilot WhiteGlove during the device provisionig phase (IT Technician Flow) does not processes Hybrid AAD join even if it is the specified join method in the autopilot profile. Another similarity to Self-Deploy mode provisioning which does not supports Hybrid AAD join at this point!

Event ID 179 AutopilotManager began device enrollment phase AADDiscover
Event ID 181 AutopilotManager completed device enrollment AADDiscover. HRESULT 0x0
Event ID 179 AutopilotManager began device enrollment phase AADConfigure
Event ID 181 AutopilotManager completed device enrollment AADConfigure. HRESULT 0x0 
Event ID 179 AutopilotManager began device enrollment phase AADEnroll
Event ID 181 AutopilotManager completed device enrollment AADEnroll. HRESULT 0x0 

AAD Join process in a nutshell (Generalized and valid for all scenarios)

As system gets connected to network,

  • It checks Autopilot provisioning status and based on result, gets the profile downloaded.
  • Using the CloudAssignedOobeConfig bitmap value retrieved from the autopilot profile, it configures OOBE to skip the EULA screen and causes CloudExperienceHost to load the CloudDomainJoinwebapphttps://login.microsoftonline.com/WebApp/CloudDomainJoin/4

The CloudDomainJoin webapp is Client Code (HTML) which uses Javascript (Worker API) to call WinRT APIs via the host process to do the device join/registration by utilizing the functions as implemented in dsreg.dll

Export Functions - dsreg.dll - Component behing AAD Join - Window Autopilot WhiteGlove
AAD Join – Behind the Scenes component dsreg.dll – Window Autopilot WhiteGlove

If Autopilot check is returned false, there is no skipping the EULA screen. Rest is same as followed.

  • CloudDomainJoin webapp sends a GETrequest to login.microsoftonline.com/common to discover auth endpoints retrieving the Open-ID configurations.
  • With the auth endpoints retrieved, it builds a sign-in request (GETcall) to obtain an ID_Token to Azure DRS.

For consumer OOBE flow, at this point, it renders the default Sign in with Microsoft work or school account cloud sign-in page on screen. As user enters UPN and clicks on next…

Default Sign in with Microsoft cloud join sign-in page - Window Autopilot WhiteGlove
Device not Autopilot ptovisioned? Default Sign-in with Microsoft screen – Window Autopilot WhiteGlove
  • It sends a GET call to https://login.microsoftonline.com/common/userrealm/ for realm information. The response to this call tells whether the tenant is managed or federated to reach out to federation STS endpoint for auth.

For Autopilot user-driven scenario, till this is pre-negotiated using the values as retrieved from the autopilot profile. This is how, instead of the default Sign-in with Microsoft screen, user is presented with the custom branded tenant Sign-in page.

Custom branded Azure sign-in page - Window Autopilot WhiteGlove
Device Autopilot provisioned? Custom branded sign-in page – Window Autopilot WhiteGlove

As the user performs sign-in, rest of the process follows as below

It sends the credentials for auth to https://login.microsoftonline.com/common/login?cxhflow=OOBE &cxhver=1.0 &cxhplatform=Desktop &cxhplatformversion=10.0.18362

Noticed the cxhflow value passed as parameter. “OOBE” refers the join is performed during OOBE. If performed post OOBE from Settings, this would be “MOSET”. This information is useful in backend to decide device functionality support – joined from OOBE or post OOBE using Settings.

For managed tenants only, a temporary auth buffer is created to be used by Winlogon post completion of setup, directly taking the user to the desktop screen bypassing the Windows sign-in. For Hybrid AAD join, this is not followed.

For Autopilot Self-deployment mode and WhiteGlove, it leverages the device’s TPM 2.0 hardware to authenticate the device into Azure AD tenant.

  • For successful auth, an ID_Token is returned to the CloudDomainJoin webapp.
Auto-Enrollment scope needs to be configured previously as the The ID_Token
as returned contains the below details as claims

1.  mdm_enrollment_url
2.  mdm_tou_url or tou_url 
 
MDM Scope for Auto-Enrollment - Intune - Window Autopilot WhiteGlove
Windows Autopilot – Need to enable MDM scope to auto-enroll device to MDM – Window Autopilot WhiteGlove
  • If you have Terms of Use configured, CloudDomainJoin webapp will navigate to tou_url as specified in the claim to render the same on screen for user acceptance.

The CloudDomainJoin webapp starts the Azure DRS Discovery process. Worker API calls a WinRT API implemented in dsreg.dll for the same purpose.

  • It sends a GET call to https://enterpriseregistration.windows.net/joymalya.com/discover?api-version=1.6
The discovery operation response:

"DiscoveryService":{"DiscoveryEndpoint":"https:\/\/enterpriseregistration.windows.net\/joymalya.com\/Discover","ServiceVersion":"1.6"},
"DeviceRegistrationService":{"RegistrationEndpoint":"https:\/\/enterpriseregistration.windows.net\/EnrollmentServer\/DeviceEnrollmentWebService.svc","RegistrationResourceId":"urn:ms-drs:enterpriseregistration.windows.net","ServiceVersion":"1.0"},
"AuthenticationService":{"OAuth2":{"AuthCodeEndpoint":"https:\/\/login.microsoftonline.com\/deaa1de1-ef2c-41e3-9a53-81f52c39d1c3\/oauth2\/authorize","TokenEndpoint":"https:\/\/login.microsoftonline.com\/deaa1de1-ef2c-41e3-9a53-81f52c39d1c3\/oauth2\/token"}},
"IdentityProviderService":{"Federated":false,"PassiveAuthEndpoint":"https:\/\/login.microsoftonline.com\/joymalya.com\/wsfed"},"DeviceJoinService":{"JoinEndpoint":"https:\/\/enterpriseregistration.windows.net\/EnrollmentServer\/device\/","JoinResourceId":"urn:ms-drs:enterpriseregistration.windows.net","ServiceVersion":"1.0"},
"KeyProvisioningService"":{"KeyProvisionEndpoint":"https:\/\/enterpriseregistration.windows.net\/EnrollmentServer\/key\/","KeyProvisionResourceId":"urn:ms-drs:enterpriseregistration.windows.net","ServiceVersion":"1.0"},"WebAuthNService":{"ServiceVersion":"1.0","WebAuthNEndpoint":"https:\/\/enterpriseregistration.windows.net\/webauthn\/deaa1de1-ef2c-41e3-9a53-81f52c39d1c3\/","WebAuthNResourceId":"urn:ms-drs:enterpriseregistration.windows.net"},"DeviceManagementService":{"DeviceManagementEndpoint":"https:\/\/enterpriseregistration.windows.net\/manage\/deaa1de1-ef2c-41e3-9a53-81f52c39d1c3\/","DeviceManagementResourceId":"urn:ms-drs:enterpriseregistration.windows.net","ServiceVersion":"1.0"},"MsaProviderData":{"SiteId":"295958","SiteUrl":"enterpriseregistration.windows.net"},"PrecreateService":{"PrecreateEndpoint":"https:\/\/enterpriseregistration.windows.net\/EnrollmentServer\/device\/precreate\/deaa1de1-ef2c-41e3-9a53-81f52c39d1c3\/","PrecreateResourceId":"urn:ms-drs:enterpriseregistration.windows.net","ServiceVersion":"1.0"},"TenantInfo":{"TenantId":"deaa1de1-ef2c-41e3-9a53-81f52c39d1c3","TenantName":"joymalya.com"},"AzureRbacService":{"RbacPolicyEndpoint":"https:\/\/pas.windows.net"},"BPLService":{"BPLProxyServicePrincipalId":"dda27c27-f274-469f-8005-cce10f270009","BPLResourceId":"urn:ms-drs:enterpriseregistration.windows.net","BPLServiceEndpoint":"https:\/\/enterpriseregistration.windows.net\/aadpasswordpolicy\/deaa1de1-ef2c-41e3-9a53-81f52c39d1c3\/","ServiceVersion":"1.0"},"DeviceJoinResourceService":{"Endpoint":"https:\/\/enterpriseregistration.windows.net\/EnrollmentServer\/device\/resource\/deaa1de1-ef2c-41e3-9a53-81f52c39d1c3\/","ResourceId":"urn:ms-drs:enterpriseregistration.windows.net","ServiceVersion":"1.0"}}

The response as received contains information about Azure DRS URI, Microsoft Passportprovisioning end-point,etc. This info is locally cached and can be veiwed using the dsregcmd command line.

With the discovery data and the ID_Token available, CloudDomainJoin webapp starts the Azure DRS Join process. This is a Worker API call to invoke WinRT API implemented in dsreg.dll for the same purpose.

  • The API as leveraged generates a key-pair to create a device CSR. In addition a 2nd key-pair, callled the Storage/Transport key, is also created which will be used to protect the SSO tokens (Primary Refresh Token) .
  • The CSR along with the ID_Token and the public portion of Storage/Transport key with TPMattestation data is sent to Azure DRS as part of the regsitration/join request.

This is a POST call to the Azure DRS endpoint as retrieved during Discovery.

The get join response operation callback was successful. 

Activity Id: 34b7ca40-5367-4625-bd70-36b52d473449 

Server response was: {"Certificate":{"Thumbprint":"A2A8F71D926FBCDA6848B740BCB2ED6A135150D0",
"RawBody":"MIID8jCCAtqgAwIBAgIQCWkLOIfrK6ZBy..."},
"User":{"Upn":"[email protected]"},
"MembershipChanges":[
{"LocalSID":"S-1-5-32-544",
"AddSIDs":["S-1-12-1-****","S-1-12-1-*****","S-1-12-1-****"]}]}

For Autopilot, Azure DRS does not creates a new device object but instead, looks for the pre-created Azure AD device object (the GUID is provided in the Join request) to enable it and update the object properties.

The Join Request and Resposne for Autopilot Self-Deploy and WhiteGlove preprovisioning looks like this.

Event ID 102 under User Device Registration

The initialization of the join request was successful. Inputs:
JoinRequest: 11 (DEVICE_AUTO_DDID)
Domain: prolick.onmicrosoft.com
Event ID 104 under User Device Registration

Server response was: {"Certificate":{"Thumbprint":"4AD998D09187656576C0CA055DA3035AFD09102D",
"RawBody":"MIID9TCCAt2gAwIBAgIQIOZS..."},
"ResponseStatus":
{"message":"Successfully updated pre-created device with id 
d48e9478-7056-4df8-bce7-11a18b8c0027.",
"traceId":"6c0110f3-e6df-4b83-a80d-1781ff5d1525",
"time":"09-20-2019 15:37:40Z"},
"MembershipChanges":[{"LocalSID":"S-1-5-32-544",
"AddSIDs":["S-1-12-1-****","S-1-12-1-*****"]}]}

Noticed that there isn’t any UPN specified here.

Azure DRS upon recieving the request creates a device object in Azure AD and sends back a cert to the device which is stored to Personal Store of LocalMachine account.

The Subject Name of this certificate is the Azure AD device GUID and can be viewed using CMD with command dir Cert:\LocalMachine\My\ | where { $_.Issuer -match "CN=MS-Organization-Access" } | fl

Certificate Store - Local Machine - MS-Organization-Access cert - AAD device GUID - Window Autopilot WhiteGlove
MS-Organization-Access Cert – SN = Azure AD Device GUID – Window Autopilot WhiteGlove

For a user driven mode, device also gets the Azure PRT at the same time. For WhiteGlove preprovisioning, it is received during the user flow.

This completes the Azure DRS Join process.

Wondering what the MS-Organization-P2P-Access[2018] cert is for? The P2P certificate is one that is pushed down by Azure AD during authentication of the user in the device, for the purpose of supporting remote desktop connectivity to another Azure AD joined device (peer-to-peer). The target device will authenticate this certificate against Azure AD, before the remote connection is established. 

The Azure AD regsitration as performed from Settings > Accounts > Access work or school > Connect has the same backend process as explained above. The only difference is the implementation at the device end – it regsiters the device to Azure, not a Join.

Things that can go wrong here…

Joining your organization’s network (Failed 0x801C03F3)

Have you deleted the Azure AD device object pre-created during DDS registration?

Remember in my previous post, I hinted on this! Well this is where you will actually see the reason. As Azure DRS receives the join request, it searches for the associated Azure AD device object which has the ZTDID stamping.

If you have deleted the entry from Azure, for Self-Deploy mode and WhiteGlove, it results in error as no match is found against the associated device guid and you would need to remove the device and re-register it back to DDS for provisioning.

This is a security mechanism implemented to stop unknown devices from getting joined to Azure AD under userless scheme and gain access to resources.

The error you will come across in events are below

Event ID 180 AutopilotManager failed during device enrollment phase AADEnroll. HRESULT=0x801C03F3
AutopilotManager failed during device enrollment phase AADEnroll. HRESULT=0x801C03F3 - Window Autopilot WhiteGlove
WhiteGlove Provisioning – Autopilot failed during AADEnroll – Window Autopilot WhiteGlove

This error does not tells much unless you check the User Device Registration events

Event ID 204 

The get join response operation callback failed with exit code: Unknown HResult Error code: 0x801c03f3. 
 Activity Id: 9a8f324f-4515-4c9e-86c1-ad2337de3e4d 
 The server returned HTTP status: 400 
 Server response was: {"Code":"AuthenticationError","Subcode":"DeviceNotFound","Message":"A device with the expected device identifiers was not found","TraceId":"9a8f324f-4515-4c9e-86c1-ad2337de3e4d","Time":"09-16-2019 14:55:45Z"}
A device with the expected device identifiers was not found.
Error code: 0x801c03f3 - Window Autopilot WhiteGlove
WhiteGlove Provisioning – AAD Device Object not found – Window Autopilot WhiteGlove

Note: During WhiteGlove preprovisioning even though above the error comes up in the User Device Regsitration events, there is no User Device Association performed. The process is a complete non-user affinity process.

ESP Stage 1 Device preparation – Registering your device for mobile management

This relates to the MDMEnroll task in the ETW trace which enrolls the device to the tenant configured MDM service.

MDM Enroll process in a nutshell

It starts with the CloudDomainJoin webapp Worker API calling the MDM Enrollment API (implemeted in mdmregistration.dll) to register the device to tenant configured MDM service.

Learn more about the Microsoft Mobile Device Enrollment protocol here.

But to proceed, it requires an Access token to authenticate to the Enrollment Service. This is the ESTSTicket task you see in the ETW trace analysis.

NOTE: In case of user-driven mode, the user has already been authenticated and device completed the join process and is capable of authenticating itself to Azure AD. For self-deploy and whiteglove, the device auth and join is completed against the device TPM secure identity.

To obtain the MDM Access token, the CloudDomainJoin webapp obtains an auth code from the Enrollment Service silently using the cookies as stored during the authentication to Azure DRS.

Remember the ID_Token as recieved during Azure DRS contains the mdm_enrollment_url in claims, which points to https://enrollment.manage.microsoft.com/enrollmentserver/discovery.svc

The corresponding GET call

https://login.microsoftonline.com/common/oauth2/authorize?client_id=29d9ed98-a469-4536-ade2-f981bc1d605e

where the client_idcorresponds to the MDM App as configured in Azure.

MDM configured in Azure - Window Autopilot WhiteGlove
MDM configured in tenant – Window Autopilot WhiteGlove

The response type requested for this GET call is response_type=code which the CloudDomainJoin worker API will use to obtain an Access token to the Enrollment Server.

This is a POST call to

https://login.microsoftonline.com/common/oauth2/token

In response, it receives a Bearer token which contains an Access token, ID_token and a refresh token. Essentially it is done during the Azure DRS process only post Discovery and cached, as this is required to reach out to the MDM_TOU_URL

The CloudDomainJoin webapp extracts the Access token from it, to pass it to the MDM Enrollment API which is implemented is mdmregistration.dll to proceed with the enrollment.

{
"token_type":"Bearer",
"scope":"mdm_delegation",
"expires_in":"3599","ext_expires_in":"3599",
"expires_on":"1568823035","not_before":"1568819135",
"resource":"https://enrollment.manage.microsoft.com/",
"access_token":"eyJ0eXAiOiJKV1QiLC..."
}
Exported Functions - mdmresgitration.dll - component behind MDM Enrollment - Window Autopilot WhiteGlove
Export Fucntions of mdmregistration.dll – the component behind MDM enrollment – Window Autopilot WhiteGlove

The main component involved here is DeviceEnroller.exe which works by implementing the WinRT APIs as provided via exported functions in the mdm dlls like mdmregistration.dll and others. This is a long explanation and deserves an article of it’s own!

The enrollment process is essentially same as the Azure Join process where the MDM Enrollment API will cause the device to create a CSR to be sent to the enrollment server and in return will get a cert, the Subject Name of which will be the Intune Device GUID.

Microsoft Intune MDM Device cert - Local Machine Cert Store - Window Autopilot WhiteGlove
MDM Device Cert – SN = MDM Device GUID – Window Autopilot WhiteGlove

Things that can go wrong here

Registering your device for mobile management (Failed 0x81036501)

Do you have multiple MDM solution configured in your tenant?

Scenario 1

Under Azure AD > Mobility (MDM and MAM) you have two entries for Intune

  • Microsoft Intune
  • Microsoft Intune Enrollment
Multiple entries for Intune - reason for failure. Autopilot discovery failed to find a valid MDM. HRESULT = 0x81036501 - Window Autopilot WhiteGlove
Microsoft Intune and Microsoft Intune Enrollment under Mobility – Window Autopilot WhiteGlove

The Microsoft Intune Enrollment does not aids in enrollment but gets created for CA enforcement during enrollments, mainly created to enforce MFA. It is recomended to not enforce MFA via CA for enrollment. There are other ways.

Scenario 2

Under Azure AD > Mobility (MDM and MAM) you have seperate MDM configured

Multiple unique MDM configured for tenant. Autopilot discovery failed to find a valid MDM. HRESULT = 0x81036501 - Window Autopilot WhiteGlove
Multiple unique MDM configured for tenant – Window Autopilot WhiteGlove

For scenario 1, MDM Discovery URL in both the entries are same but resolves to two different client_id and as such creates a conflict resulting failure.

For scenario 2, DeviceDiscovery fails to decide which endpoint to reach for enrollment.

For either of above, the error events you would find will be

Event ID 302 AutopilotManager device enrollment failed duinrg stage DeviceDiscovery with error 0x81036501

Event ID 180 AutopilotManager failed during device enrollment phase DeviceDiscovery. HRESULT = 0x81036501 

Event ID 115 Autopilot discovery failed to find a valid MDM. Confirm that the AAD tenant is properly provisioned and licensed for exactly one MDM. HRESULT = 0x81036501  
WhiteGlove Provisioning - Failure due to multiple MDM configured. HRESULT = 0x81036501 - Window Autopilot WhiteGlove
WhiteGlove Provisioning – Failure due to multiple MDM configured – Window Autopilot WhiteGlove

ESP Stage 1 Device preparation – Preparing your device for mobile management

During this task, ESP actually waits for the policy providers to complete their registration

  • Intune (the MDM service itself) and
  • Intune Management Extension (from 1903, ESP can track win32 apps as well)

There isn’t a failure point here, but you will see it takes time at this task since it is waiting for Intune to deliver the IME msi installer package and then waits for IME to initialize and get the policies it would process so that ESP can track the same. More information here by Michael Niehaus

ESP Stage 2 Device setup

ESP Stage 2 - Device setup - Window Autopilot WhiteGlove
ESP Stage 2 – Device setup – Window Autopilot WhiteGlove

As ESP completes stage 1, it moves to the next stage Device setup where it tracks the implementation of the policies as deployed to Device context (and apps with install context as device). The failure points here can be

  • App installation failure identified by the LastHResult of the app log
  • SCEP/PKCS cert failure due to NDES related errors

Provisioning Status – GREEN or RED screen?

If the pre-provisioning is success, device presents you with the GREEN screen and you have the option to RESEAL.

Windows Autopilot WhiteGlove - GREEN Screen showing provisioning succes - Window Autopilot WhiteGlove
Windows Autopilot WhiteGlove – GREEN Screen showing provisioning succes – Window Autopilot WhiteGlove

If you would like to check the events for a success, you can do that as well

Windows Autopilot WhiteGlove - Event ID 303 success - Window Autopilot WhiteGlove
Windows Autopilot WhiteGlove – Event ID 303 success – Window Autopilot WhiteGlove

However, any error during provisioning leads to the below RED screen.

Autopilot WhiteGlove - RED Screen - Provisioning Failed - RESET/RETRY
Windows Autopilot WhiteGlove – RED Screen showing provisioning failed – Window Autopilot WhiteGlove

As an analogy, WhiteGlove preprovisioning can be compared to booting the system to Audit mode to prepare the device as in auditSystem config pass.

Information: You can boot Windows to Audit mode from OOBE with the key combination CTRL+SHIFT+F3 at the first OOBE screen or with the command line using Sysprep /audit utilizing the SHIFT+F10 key combo.

Thus we can safely say that Autopilot Whiteglove essentially gives us the same benefits as Audit mode config pass saving us from the manual works, but here comes the contrast

Audit mode runs using the default Built-in Administrator account which is again disabled while exiting the pass.

WhiteGlove works in oobeSystem pass. The technician flow is done under temporary defaultuser0 account.

However, both are reunited again at their end with the same ideology – RESEAL, a Microsoft-Windows-Deployement component from Windows Unattended Setup reference.

In Audit mode, post completing the device preparation and setup, you would need to run a Sysprep /shutdown /oobe to exit the Audit mode config pass and set OOBE as the start point for the next boot before shutdown which is identical to RESEAL action

Component Name="Microsoft-Windows-Deployment"
Reseal
ForceShutdownNow: true/false
Mode: OOBE/Audit

What happens when you click on Reseal?

As mentioned above, the RESEAL button behaves analogous to sysprep /shutdown /oobe

  • it prepares Windows Setup to start from OOBE on the next boot, and
  • shuts down the computer immediately with no end-user interaction

At this point the system is ready to be handed over to the end-user.

State of the device – Before and After WhiteGlove Preprovisioning

  • Windows Autopilot WhiteGlove Provisioning Backend Process- Deep Dive - Post 4 1
  • Windows Autopilot WhiteGlove Provisioning Backend Process- Deep Dive - Post 4 2

Azure AD Devices Audit showing DRS and Intune as intiator for Device Update activity as result of Join and Enrollment

  • Windows Autopilot WhiteGlove Provisioning Backend Process- Deep Dive - Post 4 3
  • Windows Autopilot WhiteGlove Provisioning Backend Process- Deep Dive - Post 4 4

Post provisioning device status – Intune and Azure

  • Windows Autopilot WhiteGlove Provisioning Backend Process- Deep Dive - Post 4 5
  • Windows Autopilot WhiteGlove Provisioning Backend Process- Deep Dive - Post 4 6

As you can see, device is enrolled and is in managed state at this point, but compliance shows as Not Evaluated. This is due to the fact – till this point, the device is treated as a non-user affinity device.

Information Alert!

Intune does not evaluates complaince for a userless device, reason being 
the Built-in Device Complaince Policy has check criteria related to user 
affinity which will always be false for a userless deployement.

 Windows Autopilot WhiteGlove Provisioning Backend Process- Deep Dive - Post 4 7
Window Autopilot WhiteGlove

What happens as the end user starts the device?

As end user starts the device, it starts from the OOBE again as per the Reseal action and follows the same flow as a normal autopilot provisioning would

However, in this case, AutopilotManager will check and see that device is already provisioned so it will not download the profile again.

  • Setup starts CloudExperienceHost which takes user through the usual Region, Keyboard and additional Keyboard layout screen.
  • If not connected via LAN, user is presented with the Network screen to select an active network. As the user connects and proceeds, it again checks for ZDP updates.
  • During this time, autopilotmanager checks that device is already provisioned and retrieves the values from the profile to drive the rest of OOBE.
  • It pre-negotiates end-points to present end user with the custom branded sign-in page.

Till this point, it the defaultuser0 profile in Session 1 driving the process.

Windows Autopilot Whiteglove - User Flow - Session 1 defaultuser0
Windows Autopilot Whiteglove – User Flow – Session 1 defaultuser0

As user performs sign-in, User Device Association events are triggered to associate the UPN with the corresponding device object in Intune and Azure AD.

You can see the User Device Association events in Azure Device Audits as below

Windows Autopilot Whiteglove - UDA events in Azure Audit logs
Windows Autopilot Whiteglove – UDA events in Azure Audit logs

As the device gets associated with user, you would also notice Intune initiating Update device activity on the Azure Device object to update complaince state.

Windows Autopilot Whiteglove - Intune initiating complaince state update
Windows Autopilot Whiteglove – Intune initiating complaince state update

There is a session change happening at this point –

  • Winlogon logs off defaultuser0 and profile cleanup tasks are initiated. (This temporary user account is ready to be discarded at this point)
  • Winlogon performs log-in against the original user account using the credential buffer as created by CloudDomainJoin webapp during the Azure DRS sign-in.

For Hybrid AAD Join type, there is no cred buffer created that Winlogon can use and as such user is presented with the Winlogon UI screen requiring a manual sign-in.

The First Sign-In Animation is displayed post which comes the ESP to complete it’s last stage – tracking the Account setup.

Windows Autopilot Whiteglove - User flow - ESP Account Setup
Windows Autopilot Whiteglove – User flow – ESP Account Setup

As ESP completes tracking Account Setup, if Hello for Business is set, device prompts for Hello enrollment

Event ID 358 User Device Registration

Windows Hello for Business provisioning will be launched. 
Device is AAD joined ( AADJ or DJ++ ): Yes 
User has logged on with AAD credentials: Yes 
Windows Hello for Business policy is enabled: Yes 
Windows Hello for Business post-logon provisioning is enabled: Yes 
Local computer meets Windows hello for business hardware requirements: Yes 
User is not connected to the machine via Remote Desktop: Yes 
User certificate for on premise auth policy is enabled: No 
Windows Autopilot Whiteglove - User Flow - Hello Enrollment
Windows Autopilot Whiteglove – User Flow – Hello Enrollment

Once done, for join type Azure AD only, user will be automatically logged in and presented with the Desktop.

Windows Autopilot WhiteGlove- User Flow Completed - User presented with Desktop
Windows Autopilot WhiteGlove- User Flow Completed – User presented with Desktop

Something Odd That I Noticed?

View Diagnostics in case of failure doesn’t works

If you get to the RED screen due to any error as I covered above, if you click on the View Diagnostics button, it opens a File Explorer window

Windows Autopilot WhiteGlove - RED Screen - View Diagnostic opens File Explorer window
Windows Autopilot WhiteGlove – RED Screen – View Diagnostic opens File Explorer window

If you have a USB drive attached and you choose a folder for log collection and click on Select Folder, it fails stating “Provisioning information could not be located. Contact the customer IT admin to troubleshoot

Windows Autopilot WhiteGlove - RED Screen - View Diagnostic Fails to capture provisioning info
Windows Autopilot WhiteGlove – RED Screen – View Diagnostic Fails to capture provisioning info

Well its a bit odd that you would yourself be the IT admin or support at this error stage.

But you still have a fully functional Windows running behind this RED screen…

As such, you have your usual arsenal of Win+R to launch Event Viewer, Registry or PowerShell, or Shift+F10 to run Admin context CMD.

Windows Autopilot WhiteGlove - Tools to collect logs from OOBE itself
Windows Autopilot WhiteGlove – Tools to collect logs from OOBE itself

Yeah the reference snap is over the GREEN screen but its same for RED screen as well. And did I forgot to mention that you have a working Alt+Tab as well to cycle through the windows.

Windows Autopilot WhiteGlove - Alt+Tab to cycle through tools in OOBE
Windows Autopilot WhiteGlove – Alt+Tab to cycle through tools in OOBE

defaultuser0 is still present under C:\Users?

This is something that you would see very rarely. Sometimes due to some unknown reasons, the profile cleanup doesn’t happen as it should.

During the session change that occurs in OOBE prior to the ESP Account phase coming up, if there is a restart, the system starts with the original user account profile (as provided during Azure sign-in) in Session 1 itself.

But if there is no restart, as in the snap below, you will see that though system logged off defaultuser0 running in session 1, Winlogon initiates the original user account log-in in another session initiated by smss.exe – session 2, since for same boot cycle, new user login causes a new user session initiation.

  • Windows Autopilot WhiteGlove Provisioning Backend Process- Deep Dive - Post 4 8
  • Windows Autopilot WhiteGlove Provisioning Backend Process- Deep Dive - Post 4 9

In such case, when you are presented with the desktop screen, if you go to c:\Users you will still see the defaultuser0 account folder.

Windows Autopilot Whiteglove - defaultuser0 profile still present post provisioning
Windows Autopilot Whiteglove – defaultuser0 profile still present post provisioning

This gets removed automatically if you do a restart manually. So the profile cleanup task waits for a restart, but I couldn’t seem to find a task scheduled for the same 🙁

Ending

This completes the 4 part series of Windows Autopilot which I started. I hope with each article of this series, I was able to provide you with some valuable information and clarifying the underlying working principles of Windows Autopilot.

Well, I might come with a supplementary post for this series about re-provisioning Autopilot devices, but till then, as I always say before ending – read something everyday, learn something everyday!

Resources

4 COMMENTS

  1. Im getting an error while
    Registering your device for mobile management, Failed: 3, 0x801c03f3

    Already checked TPM version on the laptop which is 2.0
    Can you please advise?

    • Ideally the error should be for this event in User Device Registration, which happens if you have unknowingly deleted the Azure AD device object which was precreated as part of DDS registration.

      The get join response operation callback failed with exit code: Unknown HResult Error code: 0x801c03f3.
      Activity Id: #####
      The server returned HTTP status: 400
      Server response was: {“Code”:”AuthenticationError”,”Subcode”:”DeviceNotFound”,”Message”:”A device with the expected device identifiers was not found”,”TraceId”:”######”,”Time”:”01-10-2019 14:04:25Z”}

LEAVE A REPLY

Please enter your comment!
Please enter your name here

This site uses Akismet to reduce spam. Learn how your comment data is processed.