1. Home
  2. CloudReady Status

CloudReady Status

Exoprise continuously monitors thee operations of its service — using CloudReady sensors from all over the world in addition to other services. If there are any interruptions in service, planned or unplanned, a note will be posted here.

Core ServersPublic SitesAlarmsDependant ServicesService Comms API

Thursday August 20, 2020 1:45 AM

  • Microsoft Teams appears to have recovered at this time and the sensors were able to create Audio Video conferences again. This issue was caused by a prolonged Microsoft Teams outage and manifested through disconnected AV conference sessions.

Wednesday August 19, 2020 5:00 PM

  • Fixes were deployed for the OneDrive sensor to account for newly released and updated OneDrive pages that contained errors on the page and with Microsoft deployment. The sensors will update themselves over the next few hours.

Wednesday August 19, 2020 12:00 PM

  • Microsoft Teams is having outage issues for the past few hours and this is being reflected via the Teams AV sensors. We are continuing to observe the outage details and our sensor executions.

Tuesday August 18, 2020 6:00 PM

  • The Exoprise Teams AV Bot was upgraded to the latest Microsoft Communications SDK codebase. Previously the Teams AV Bot was on a Beta SDK and the Bot was updated to the version 1 GA. We are continuing to deploy the updates to the Bot and investigate any new scaling requirements that may be required from the new SDK. There may be intermittent errors from interaction with the Teams AV Bot.

Tuesday August 19, 2020 3:00 PM

  • Fixes were deployed for the SharePoint sensor to account for newly released and updated SharePoint pages that contained errors on upload pages. OneDrive is still being affected.

Friday July 31st, 2020 12:00 PM

  • Microsoft is actively changing SharePoint and OneDrive modern page and those changes are being deployed across subsets of tenants. This is affecting SharePoint upload pages and OneDrive pages which are broken. The issue has been reported to Microsoft and we are awaiting fixes.

Wednesday July 15th, 2020 1:00 PM

  • A fix was deployed for Okta SSO integration and monitoring.

Tuesday July 14th, 2020 4:00 PM

  • Okta sign in forms have been updated and okta-based Single Sign-on may no longer be working for browser tests. We are investigating the issue. The current suggested workaround is to switch to “managed” (non-Federated) accounts for tenants that are leveraging Okta SSO.

Thursday June 11th, 2020 6:05 PM

  • Servers in the PUB-CANADA region completed their updates and recovered. We will investigate better planning and quicker updates in the future.

Thursday June 11th, 2020 5:10 PM

  • Servers in the PUB-CANADA region are being upgraded and taking longer than expected. We will update this page when the upgrades are complete.

Thursday June 11th, 2020 10:00 AM

  • The Azure US-EAST region VoIP sensors have recovered. We will update this page with the cause when we receive one from Microsoft.

Thursday June 11th, 2020 8:00 AM

  • Microsoft is reporting Azure issues in the US-EAST region that is affect the Azure East VoIP sensors. We will post an update when we get one.

Tuesday June 2nd, 2020 7:30 PM

  • We deployed a fix for the Teams AV sensor and it should be rolling out to any of the sensors over the next few hours. Microsoft changed the internal call initiation interface.

Tuesday June 2nd, 2020 6:30 PM

  • Call creation to the Exoprise AV Bot is failing and we continue to investigate. We hope to have an update by tomorrow morning.

Tuesday June 2nd, 2020 4:30 PM

  • We are investigating issues with our Exoprise AV Bot for the Teams AV Sensor. We hope to have and update or fix within the next two hours.

Thursday May 28th, 2020 4:00 PM

  • Email sensors and autodiscover testing for Office 365 have not reported any failures within the past 24 hours. There may still be cached DNS entries out there for poorly configured DNS servers or providers.

Thursday May 28th, 2020 8:30 AM

  • DNS entries for bad autodiscover servers appear to be subsiding as caches are updated depending on your provider. We will continue to monitor the situation.

Wednesday May 27th, 2020 8:00 AM

  • We are investigating issues with Exchange Online and Exchange Web Services. There are potential issues with Autodiscover and their round-robin DNS. Autodiscover servers are redirecting to broken autodiscover services. These are Microsoft issues and the offending autodiscover servers are not responding and causing timeouts.
  • A potential workaround is to avoid autodiscover and utilize OAuth or hardcode the Exchange Web Services URL: https://outlook.office365.com/EWS/Exchange.asmx

We will update this page when we understand more or if the autodiscover servers, DNS entries are fixed.

Tuesday May 26th, 2020 8:00 AM

  • We are investigating issues with SharePoint sensors and accelerating a possible upgrade. Microsoft’s modern SharePoint interfaces have been updated and are causing incompatibilities with our sensors. Investigations are on-going and some customers may have received upgrades already.
  • A possible workaround is to disable the modern SharePoint interface for the sites that are being monitored.

Wednesday March 18th, 2020 8:00 AM

  • A fix was deployed for this issue. Any Management Clients that were downloaded in the previous 24 hours should be uninstalled and re-installed.

Wednesday March 18th, 2020 6:00 AM

  • We are currently experiencing a problem with new site installations. We are investigating the issue and hope to have more information about a resolution at 8:00 AM.

Tuesday January 14th, 2020 4:30 PM

  • We have fixed the problem with OAuth registrations that require tenant consent. Apparently, additional delegate permissions are now required by Azure AD and this is an undocumented breaking change. We are still waiting to hear from Microsoft why or when this change was implemented.

Monday January 13th, 2020 12:05 PM

  • Microsoft’s Azure AD OAuth registration is currently broken for tenant admin consents and this has been reported to the Microsoft Azure AD team. Currently, Azure AD OAuth registrations with tenant admin consent just loop on successful consent and never post back refresh token or registration information. Existing OAuth registrations are not affected.We are awaiting more information from Microsoft when they fix the issue on their side and will post back any status or timeline information we receive.

December 17th, 2019 10:05 PM

  • The Teams AV Conferencing sensors have recovered. We are continuing to investigate the cause of this outage related to 429 throttling errors. We apologize for the inconvenience that the outage caused.

December 17th, 2019 5:30 PM

  • We are investigating an issue with the new (BETA) Microsoft Teams AV Conferencing sensor which is receiving a 429 throttling error message.

October 2nd, 2019 12:20 PM

  • A fix for the Office 365 portal sensors was deployed. We apologize for the inconvenience that the specific sensor outage caused.

October 2nd, 2019 7:30 AM

  • We have identified the login procedure changes related to Azure AD and are preparing a fix for staging. Once we have the fix complete, we will deploy to staging and then production. We estimate the fix to be deployed later this afternoon.

October 2nd, 2019 5:30 AM

  • We are investigating Office 365 Portal sensor issues for some tenants. We hope to have an update within the 1 to 2 hours.

September 30th, 2019 7:00 PM

  • Engineering updated and re-built indexes on newer tables and collections. We apologize for any inconvenience.

September 30th, 2019 5:30 PM

  • We are investigating some slow responding servers and sensor data ingestion.

May 2nd, 2019 12:28 PM

  • We have rolled back changes that were made to the autoresponder infrastructure and the problem should be corrected. We apologize for any inconvenience this may have caused.

May 2nd, 2019 12:05 PM

  • We are investigating potential problems with our autoresponder infrastructure.  There may be false email delivery problems reported.