We have implemented a preliminary fix and Airflow clusters are now coming up successfully. We will continue to investigate a permanent fix for the issue.
Customers may experience longer than normal start up times while we work on continued maintenance.
Please reach out to Support if you experience any failures with starting Airflow clusters.
Posted Feb 17, 2023 - 06:56 PST
Monitoring
We would inform that we have implemented a preliminary fix and Airflow clusters are now coming up successfully.
Customers may experience longer than normal start up times while we work on continued maintenance.
Please reach out to Support if you experience any failures with starting Airflow clusters.
Posted Feb 16, 2023 - 20:19 PST
Identified
The library used by airflow relies on a common python package which was recently upgraded in the open-source community and is causing the breaking changes. In the meantime, we are exploring ways to bring a resolution into QDS airflow that serves as a workaround for this upstream issue.
Posted Feb 16, 2023 - 14:15 PST
Update
We are investigating options to resolve the issue. We will continue to post the status.
Posted Feb 16, 2023 - 09:18 PST
Investigating
For customers using Airflow, there is an issue with clusters that are running. All other aspects of Qubole operations are functioning normally.
These Airflow clusters do not automatically terminate due to the nature of their function. Any currently running Airflow clusters are functioning normally. If an Airflow cluster is terminated or restarted for any reason then it will not come back up as the loading of Airflow will fail.
We are currently working to resolve the issue and will post an update in the next 2 hours or earlier if new information is available or service is restored.
Posted Feb 16, 2023 - 06:36 PST
This incident affected: api.qubole.com Environment (AWS) (Cluster Operations).