Qubole DevOps has completed the validation of all affected environments from this incident. As noted in prior updates and applicable across all affected environments: Customers will need to re-process commands which were submitted prior to or during the unavailability window. Qubole DevOps will be marking most commands as Cancelled which were stuck during this the impact timeframe.
We appreciate everyone's patience through this process and are officially closing this incident.
Posted Aug 19, 2019 - 23:06 PDT
Update
Qubole DevOps has brought up the azure.qubole.com environment and verified full functionality. Customers will need to re-process commands which were submitted prior to or during the unavailability window. Qubole DevOps will be marking most commands as Cancelled which were stuck during this the impact timeframe.
Qubole DevOps continues to work to bring up and verify the oraclecloud.qubole.com and will provide another update shortly.
Posted Aug 19, 2019 - 22:55 PDT
Update
Qubole DevOps has brought up the us.qubole.com and in.qubole.com environment and verified full functionality. Customers will need to re-process commands which were submitted prior to or during the unavailability window. Qubole DevOps will be marking most commands as Cancelled which were stuck during this the impact timeframe.
Qubole DevOps continues to work to bring up and verify the other impacted environments azure.qubole.com and oraclecloud.qubole.com and will provide another update shortly.
Posted Aug 19, 2019 - 22:22 PDT
Update
Qubole DevOps has brought up the api.qubole.com and eu-central-1.qubole.com environment and verified full functionality. Customers will need to re-process commands which were submitted prior to or during the unavailability window. Qubole DevOps will be marking most commands as Cancelled which were stuck during this the impact timeframe.
Qubole DevOps continues to work to bring up and verify the other impacted environments.
Posted Aug 19, 2019 - 21:57 PDT
Update
Qubole DevOps is in the process of bringing up the api.qubole.com environment. A few supporting application components are in the process of starting up and full functionality will be restored shortly.
In parallel, Qubole DevOps is bringing up the us.qubole.com environment.
We will address other affected environments once these two environments are fully functional. Again, we appreciate your patience through this process and assure you that we have all of the right resources addressing the incident with the highest urgency.
Posted Aug 19, 2019 - 21:05 PDT
Update
We are continuing to work on a fix for this issue.
Posted Aug 19, 2019 - 20:52 PDT
Identified
Qubole DevOps, Engineering and Support are currently working on the issue with the most urgent priority. Currently site availability, cluster operations and command processing are unavailable. We appreciate your patience while we work to restore service at the earliest possible point.
Posted Aug 19, 2019 - 20:35 PDT
Update
We are continuing to investigate this issue.
Posted Aug 19, 2019 - 20:09 PDT
Investigating
Qubole is investigating a major outage across environments. We will provide regular updates as we have more information.