Deep Dive into delivery optimization

You can imagine that when a new Windows patch or a new version of a software is released and has to be installed on every Devices and many PCs start to download the content at the same time from a destination outside the corporate network, the Internet break outs are very busy. Or if the internet connection of a certain location has a low bandwidth the download of a large app takes a long time. But more and more organisations move their applications as well as the update management to the cloud and reduce their on premise infrastructure. To solve this problem microsoft has introduced a very good technology called Delivery Optimization. In this blog we would like to clarify what is DO and take a deeper look at how this works.

What is Delivery Optimization?

In the past, with Microsoft Endpoint Manager Configuration Manager and classic device management, the complete application content was provided by content servers located directly at the site. Thus, the Internet break outs for the content download on the various clients were not burdened. In addition, there are also technologies such as branch and peer cache to spread the load across different clients. By moving the client management to the cloud, the problem arises that the complete content download always has to go through the Internet breakouts. This brings various problems. On the one hand, the download can be very slow at certain locations and on the other hand, much more bandwidth is consumed which leads to costs but also to a high load.

To overcome these problems Microsoft has introduced Delivery Optimization. Delivery Optimisation is a cloud managed peer to peer technology to reduce the bandwidth usage when downloading packages like e.g. WufB or Win32 apps by distributing the packages across multiple devices. It is a intelligent distributed cache that makes it possible to distribute the download of packages to other sources, thus not burdening the corporate internet breakouts and also increasing the download speed.

Delivery Optimization is an intelligent cache where the peers are managed by a web service and are only offered as peers depending on the battery level and the load of the client.

What are the prerequisites?

  • Min Windows Os Version from 1511 (With each later version more features were added)
  • Open port TCP 7680 on clients and local firewalls for requests from other peers
  • Internet connection with access to the delivery optimisation service:

For communication between clients and the Delivery Optimization cloud service:

*.do.dsp.mp.microsoft.com

For Delivery Optimization metadata:

*.dl.delivery.mp.microsoft.com
*.emdl.ws.microsoft.com

For the payloads (optional):

*.download.windowsupdate.com
*.windowsupdate.com

For group peers across multiple NATs (Teredo):

win1910.ipv6.microsoft.com

What content is supported?

More and more content has been added, which is supported by Deliver Optimization. Not all content is supported in older windows versions. the following content is supported (as of oct 2022):

  • Windows Updates [Win1511] (Feature and Quality Updates as well as language packs and driver)
  • Windows Store (for Business) files [Win1511]
  • Windows Defender definition updates [Win1511]
  • Intune Win32 apps [Win1709]
  • Microsoft 365 Apps and updates [Win1709]
  • MS Edge Updates [Win1809]
  • ConfigMan Express updates [Win1709 + ConfigMan 1711]
  • Dynamic Updates [Win1903]
  • MDM Agent [Windows 11]

How does Delivery Optimization work?

In short:

The content to be downloaded is divided into small chunks. The client asks the DO service who within its group has the currently downloaded chunk in its cache. DO Server responds with a list of clients. The client requests the download from this client.

In detail:

  1. A policy is defined in Intune or SCCM to configure the behavior of the peers for Delivery Optimization, e.g., with whom a client is allowed to talk.
  2. The client is searching for updates and contact the Windows Update Service. WU returns the URL from the CDN.
  3. When a Windows client requests content from the CDN, it first receives metadata about the content to be downloaded. The content metadata is a hash file containing the SHA-256 block-level hashes of each piece in the file (typically one piece = 1 MB). This meta data is checked for authenticity if it comes from a trusted source.
  4. The client contacts the Delivery Optimization Service to get a list of clients which keep the required sources available in their cache. If no client has the required sources in the cache, then the download is requested directly from the CDN.
  5. The Client contacts the peers. The peer that responds the fastest will be used to download the sources. When a client retrieves a 1MB chunk of content from another client, it is matched with the hash from the metadata. If it is invalid, the chunk is discarded, and the client is locked and can no longer be used as a source.
  6. When the download is complete Delivery Optimization put all the chunk together and get the final file. After that, the retriever of the file (e.g., Windows Update Service) checks the whole file and matches the signature. The client will keep some chunks in his cache and will report this to the Delivery Optimization Service.

How can I define to which client the content will be provided to?

To define which client can talk to which client there is the so called download mode. Here you can define that every client within a network is a possible peer but also every client on the internet. In my opinion the most sensible method is to choose peering within a private group. Here you can define with the help of a group id which clients are in a group.

The goal should be to make the groups as large as possible to increase the chance that the content is available on another client and to offer many peers but only so large that the clients are close together network-wise to ensure a high speed.

What is the DO group Id and how can I distribute the group Id?

The Group ID is a guid which can be distributed e.g. via a policy, DHCP option or also other possibilities to make a custom grouping of the clients. Only clients with the same group id share their cached content with each other. The distribution via DHCP makes sense especially for AADJ only devices.


To distribute the group ID with DHCP it must be stored in the DHCP option 235. An ID can be created with the PowerShell command on one random windows client.

New-Guid

How you distribute the IDs depends on your network topology but it is recommended to create a GUID for each site and attach it via DHCP to the networks belonging to it. (As already said the larger the group the more efficient DO will be).

How can I configure Delivery optimisation in Intune?

  • Open the MEM Portal
  • Navigate to Devices -> Configuration profiles
  • Click + Create profile
  • Select Windows 10 and later as platform and Templates as Profile type
  • Select Delivery optimization
  • Click Create
  • Enter a Name
  • Click Next
  • Enter your Configuration
  • My recommendation (Can differ from environment to environment and must be monitored and adjusted):
SettingValue
Download modeHttp blended with peering across private group (2)
Restrict Peer SelectionNot configured
Group ID sourceDHCP user option
Bandwidth optimisation typeNot configured
Delay background HTTP download (in seconds)60
Delay foreground HTTP download (in seconds)10
Minimum RAM required for peer caching (in GB)1
Minimum disk size required for peer caching (in GB)32
Minimum content file size for peer caching (in MB)10
Minimum battery level required to upload (in %)40
Modify each drive%SystemDrive% (Not filled out use default)
Maximum cache age (in days)120
Maximum cache size typePercentage
Maximum cache size (in %)20
VPN peer cachingDisabled
Caching server fully qualified domain names (FQDN) or IP addressesEmpty
Delay foreground download Cache Server fall back (in seconds)0
Delay background download Cache Server fall back (in seconds)0
  • Click Next

Where are DO files stored?

C:\Windows\DeliveryOptimization but the drive can be changed withing the DO settings.

Manual Cleanup DO Cached file

Use disk cleanup utility to clean the DO cache:

Delivery Optimization Configs in the Registry

  • HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\DeliveryOptimization\Config
  • HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\DoSvc

You can for example change the start value to 4 to disable DO.

How can I monitor Delivery optimisation?

Especially at the beginning when you introduce delivery optimization but also continuously you should take a close look at the performance and the monitoring to adjust them if necessary or to detect errors. To monitor how well deliver optimization works there are two ways. The first is to look at the performance metrics on the client and the other is to evaluate the telemetry data in log analytics. Let’s take a look at this.

Analyse on the Client

There are a number of Powersehll commands that help to query the status:

  • Get-DeliveryOptimizationPerfSnap
    • This command gives you an overall view on download count, statistic of the data size downloaded from different sources and many other useful informations
  • Get-DeliveryOptimizationPerfSnapThisMonth
    • PerfSnap summary of the Month
  • Get-DeliveryOptimizationStatus
    • Overview of all individual content downloads with many detailed informations like download time
  • Get-DeliveryOptimizationLog (evaluated privileges required)
    • List all DoSVC logs
  • Enable-DeliveryOptimizationVerboseLogs / Disable-DeliveryOptimizationVerboseLogs (>= Win2004)
    • Enable Verbose logging to get more detailed informations (consume lot of log entries)

Windows Options

You can also find a statistic of Delivery Optimizations in Windows Settings under Windows Update.

Navigate to: Settings -> Windows Update -> Advance Options -> Delivery Optimization -> Activity monitor

Analyse via Log analytics

Within Update Compliance there is a Delivery Optimization category that provides you with a baic dashboard based on the telemetry data. This is a good starting point but with the infinite possibilities of KQL and Workbooks you can also create your own dashboards. For this you need to distribute the commercial id to the clients. How to distribute it and how to enable the central collection of telemetry data is explained very well in this blog post.

When does delivery optimization not make sense

As mentioned, it makes sense in delivery optimization to make the groups as large as possible. However, if the groups are smaller than 10 devices then delivery optimization is not a viable option. Also, delivery optimization does not make sense in VPN-only or Wi-Fi networks. In these cases you can use Microsoft Connected Cache.

What is Microsoft Connected cache (MCC)

Microsoft Connected Cache an server component that acts as an dynamic cache that is inside the network and also acts as a peer. MCC caches the content based on the requests from the clients. The Connected Cache feature can be enabled on an SCCM content server or currently running a private preview of a stand alone MCC. This is a container image running on a Linux machine. The advantage of MCC is if not many clients are online at a site then the data is still served from a mcc this leads to even better bandwidth reductions and it can be used in networks where peer-to-peer cannot be used.

How does MCC stand alone works

In the Microsoft Docs is very well explained how the Standalone MCC works.

The following steps describe how MCC is provisioned and used.

  1. The Azure Management Portal is used to create MCC nodes.
  2. The MCC container is deployed and provisioned to a server using the installer provided in the portal.
  3. Client policy is configured in your management solution to point to the IP address or FQDN of the cache server.
  4. Microsoft end-user devices make range requests for content from the MCC node.
  5. An MCC node pulls content from the CDN, seeds its local cache stored on disk, and delivers content to the client.
  6. Subsequent requests from end-user devices for content come from the cache.

If an MCC node is unavailable, the client will pull content from CDN to ensure uninterrupted service for your subscribers.

Other nice blog posts about DO and MCC

2 thoughts on “Deep Dive into delivery optimization

  1. Do you have any experience in working with DO while also using the Windows 10 security baseline in Intune? My current guess is that the “deny all incoming connections” setting of the FW blocks DO from working.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s