All Airflow deployments are now operating normally. We continue to monitor the issue but do not expect any further service interruptions.
Posted Mar 10, 2022 - 20:46 UTC
Update
While repairing the system, some Fernet keys (used to encrypt passwords in the Airflow connection configuration and variable configuration) are being restored. During this time, customers may see errors such as "could not create Fernet object: Incorrect padding", as well as variables or connections being marked as Invalid. The keys are currently being restored, and this error should disappear soon.
Posted Mar 10, 2022 - 19:29 UTC
Update
This issue is continuing to affect customers wishing to update their airflow deployments. The Astronomer team have identified the underlying issue and is actively working to deliver a fix.
Posted Mar 10, 2022 - 16:57 UTC
Identified
An issue has been identified where images are successfully built and pushed, but the deployments are not being updated with the new images. This is intermittently affecting both CI/CD builds as well as builds triggered by the astro CLI tool.
Astronomer will be monitoring and manually updating deployments with out of date images, but if you notice an image not being deployed, please contact support.
Posted Mar 10, 2022 - 04:25 UTC
This incident affected: Astro Hybrid (Deployment Management, Cloud Image Repository).