Significant progress has been made in resolving the Notebook outage. Each environment that serves Notebooks has been refreshed with new nodes and the Notebooks functionality is successfully responding in AWS. AWS customers will now see Notebooks as available and functional while, in parallel, the Qubole team continues its production testing to confirm stability and true resolution to the outage. We continue to work to restore Notebooks on GCP and will be updating the status page regularly
Posted Apr 21, 2021 - 11:08 PDT
Update
Devops has passed some additional hurdles in final testing of the redeployment on QA. The new deployment is undergoing retesting before being rolled to production.
Posted Apr 19, 2021 - 10:43 PDT
Update
Redeployment is still in process, and expected to finish between today (4/15) and tomorrow. This entry will be updated again as redeployment is finished, or as Devops provides further information.
Posted Apr 15, 2021 - 13:32 PDT
Identified
Devops encountered a dependency issue in their attempted fix. They are performing a package rebuild, which will need to be tested tomorrow, Wednesday, before deployment. Devops is attempting to finalize both testing and deployment tomorrow, though deployment may occur on Thursday, 4/15/2021.
Posted Apr 13, 2021 - 19:20 PDT
Investigating
Devops resolved a blocking reference error over the weekend. They continue to investigate specific notebook failures on us.qubole.com. At this time a few notebooks appear to be opening normally, though the vast majority still have an issue. Devops is investigating specific failure instances to determine a root cause that will resolve all ongoing failures.
Posted Apr 12, 2021 - 06:12 PDT
Update
Devops is continuing component deployment to resolve this issue. We will update status again as we have a new ETA, or confirmation of resolution.
Posted Apr 08, 2021 - 19:01 PDT
Update
Devops has confirmed the issue has been resolved for all environments except us.qubole, for which their ETA is still Thursday.
Posted Apr 06, 2021 - 12:42 PDT
Update
Devops has implemented several fixes in the Qubole environment during the last 4 days in order to resolve this ongoing outage. At this point they believe there's one component deployment still pending, where they have sorted out the issues related to deployment and will be performing a rigorous testing.
Devops anticipate to get this fix rolled out by Thursday (4/8). This page will continue to be updated as the implementation is completed.
Posted Apr 06, 2021 - 11:00 PDT
Identified
Devops is implementing several fixes in the Qubole environment today (4.2.2021) and tomorrow (4.3.2021) in order to resolve this ongoing outage. This page will continue to be updated as the implementation is completed.
Posted Apr 02, 2021 - 15:32 PDT
Update
Devops has identified a potential cause for this issue and is testing a fix in an internal environment. If the fix is successful, this ticket will be updated to reflect the plan to implement the change across Qubole environments.
Posted Mar 31, 2021 - 13:18 PDT
Update
Devops is continuing to investigate this issue, with the intent to find an immediate workaround.
Posted Mar 30, 2021 - 11:35 PDT
Investigating
Customers attempting to access notebooks will encounter a "500 Internal Server" error.
Posted Mar 23, 2021 - 07:13 PDT
This incident affected: us.qubole.com Environment (AWS) (Notebooks), in.qubole.com Environment (AWS) (Notebooks), and Notebooks.