We had a temporary connection issue with OAuth connected accounts starting at 14:14 UTC. We apologize for any downtime this may have caused for your workflows involving OAuth connected accounts.
The good news is we were able to recover many of these connections automatically using the refresh token sequence on the connected authentication APIs.
We pushed a change that caused the token refresh process for a handful of OAuth accounts to fail for roughly 2 hours. Certain apps (such as Discord and Zoom) immediately revoke previously-issued refresh tokens when a new refresh token request is made.
Because we do not log these tokens outside of our encrypted database, the refresh tokens we stored became out of sync with those the third party app assigned, and the accounts were no longer able to connect to the target API.
We were able to rollback the change, and then trigger the rotation of refresh tokens using the original configuration.
This replaced the dead access tokens with live refreshed ones, restoring access to workflows. Simultaneously the app’s OAuth sequence was restored, allowing new accounts to be connected.
We are tightening our controls around App updates and adding additional peer review policies to prevent this mistake from happening again in the future.
We are also adding additional monitoring to connected account heartbeats to ensure anomalies are detected sooner and automatically.
We appreciate your patience with this issue.
If you have additional questions, please reach out to us at support@pipedream.com