During the past few weeks, our customers and partners have reported migration throughput performance issues to us. In this blog, we are providing recommendations to our customers and partners, current notifications to Microsoft partners and customers, an explanation of throttling, and how Quest complies with Microsoft’s guidance and best practices.
- We are recommending to our customers and partners to schedule their OneDrive and SharePoint migrations during nighttime and weekend hours within their region.
- Be aware of potential migration throughput issues If you plan to migrate during evening hours within your region.
- You should expect significant migration throughput issues If you plan to migrate during daytime hours within your region.
- You should expect to see an increase in throttling during migrations. Throttling can appear as HTTP 429 errors (“Too Many Requests”) or HTTP 503 errors (“Server Too Busy”).
Current notifications to Microsoft partners and customers
On or around March 24th, Microsoft posted the following notification publicly:
Many SharePoint Online and OneDrive customers run business-critical applications against the service that run in the background. These include content migration, Data Loss Prevention (DLP), and backup solutions. During these unprecedented times, we are taking steps to ensure that SharePoint Online and OneDrive services remain highly available and reliable for your users who depend on the service more than ever in remote work scenarios.
In support of this objective, we have implemented tighter throttling limits on background apps (migration, DLP and backup solutions) during weekday daytime hours. You should expect that these apps will achieve very limited throughput during these times. However, during evening and weekend hours for the region, the service will be ready to process a significantly higher volume of requests from background apps.
https://docs.microsoft.com/en-us/sharepointmigration/sharepoint-online-and-onedrive-migration-speed
On March 24th, Microsoft published MC207439 in the Microsoft 365 admin center:
Update April 17th: Awareness of Microsoft 365 temporary feature adjustments
SharePoint:
- We are rescheduling specific backend operations to regional evening and weekend business hours. Impacted capabilities include migration, DLP, apps that scan the service, and delays in file management after uploading a new file, video or image.
- Reduced video resolution for playback videos.
- The following blog is now available with some additional information about these temporary changes: Awareness of temporary adjustments in SharePoint Online
Throttling
Customers and partners can experience throttling when running migrations that exceeds resource usage limits. SharePoint Online throttles any further request from the user for a period of time.
What are the throttling limits? Microsoft does not publish what the exact rules are for throttling.
Throttling is normal behavior. Microsoft states that:
Throttling is in place to protect the reliability and availability of the service. Throttling rules cannot be disabled or suspended.
https://docs.microsoft.com/en-us/sharepointmigration/sharepoint-online-and-onedrive-migration-speed
Quest / Metalogix migrations tools implement retry code to retry the migration after a period of time in response to throttling. However, the retries stop after a preset number of times. Thereafter, the customer should resubmit the migration job during an off peak time.
Quest complies with Microsoft requirements for retry code, decorating HTTP traffic, and using app based authentication (OAuth).
How Quest complies with Microsoft guidance and best practices
Quest has complied with Microsoft guidance and best practice by implementing specific product features.
| Guidance | Quest Response |
| Decorate your traffic so we know who you are | Our Quest products decorate traffic with our company name and unique application names. |
| Leverage the Retry-After HTTP header | Our Quest products detect the Retry-After HTTP header and delay retrying the sending of HTTP Requests. |
| Use app-based authentication | Authentication is the process of proving you are who you say you are. Our migration products require that an app is installed in the organization’s Azure app directory. Our desktop Quest migration products require a user to authenticate. The user authenticates to the app. Each desktop Quest migration product uses a unique app ID. Our SaaS-based migration products require a user to provide authentication credentials to our service. The service uses the credentials to authenticate to the app. One unique app ID is used for our SaaS-based migration products. All our app IDs are registered with Microsoft. |
| Use OAuth for authorization | OAuth is about authorization. Authorization is the act of granting an authenticated user permission to do something. Our desktop Quest migration products use delegated permission. Our SaaS-based migration products follow the application permission specification. Note: Essentials for Office 365 supports OAuth for Hyper Migration and Hyper Backup. OAuth does not work with SSO or Multifactor Authentication. |
| Use the Azure migration API | The Quest migration products migrate content to SharePoint Online and OneDrive for Business using the Azure migration API Whenever possible. There are a few exceptions in which CSOM calls must still be used to support content migrations. Our desktop migration products support the use of public and private storage containers. |
| Use migration specific API createMigrationJob for migration traffic to reduce throttling | The createMigrationJob method creates a new migration import job and queues it up for later processing by a separate timer job. The job will consume a well-formed (pre-defined format) import package that is located in the Azure Blob Storage Containers specified in this method. https://docs.microsoft.com/en-us/sharepoint/dev/apis/migration-api-overview All Quest migration products use the createMigrationJob method when migrating content via the Azure migration API. |
| Use Asynchronous Metadata Read (AMR) API for migration traffic to reduce throttling | Also known as, the SharePoint Migration Export API. The goal of the new Migration Asynchronous Metadata Read (AMR) API is to reduce the number of calls, reduce throttling, and improve overall migration performance for [Microsoft] customers. Instead of calling thousands of calls to query information from SPO, the new Migration Asynchronous Metadata Read can return the same amount of data in a single read. https://docs.microsoft.com/en-us/sharepoint/dev/apis/export-amr-api Quest is currently researching AMR for use with features related to incremental migration, structure creation, and permission settings. |
Helpful links
Avoid getting throttled or blocked in SharePoint Online
General migration performance guidance
https://docs.microsoft.com/en-us/sharepointmigration/sharepoint-online-and-onedrive-migration-speed
One solution. Many workloads.
Temporary changes in process schedule for SharePoint Online services
Product links
Click here for more information on the On Demand Migration solutions.
Click here for more information on the Metalogix solutions.
