As we get ready to ship our next major release R54 on azure.qubole.com, we are excited about several new capabilities that we are delivering including:
- Qubole on Azure continues to be a focus. With this release, in addition to the features above we have added support for the following Azure specific capabilities:
* Rubix local disk cache for improved performance of Presto workloads.
* FastPath optimizations that deliver improved performance of QDS driver for Presto queries. With FastPath, the overall command latency for a Presto query before it begins execution reduces to about 1.5 seconds when the cluster is active.
* Improved cluster start times in under 5 minutes (20% reduction) using Azure ARM templates for cluster deployments.
* Block storage upscaling to allow easy scaling of cluster storage.
- Continue the march towards enterprise readiness with role-based access controls for multiple Qubole artifacts such as commands, and data stores.
- Enhanced data privacy and protection with Apache Ranger integration for Hive workloads to enable granular data access controls at the row and column level.
- Strengthen financial governance suite by quantifying TCO savings from spot nodes for compute workloads that run on Qubole.
- Complementary enhancements to Spark such as Sparklens - a profiling tool to understand the scaling limits of Spark applications, Rubix local disk cache to speed up Spark jobs. We are also offering Spark 2.3.2, Presto 0.208 (Beta) with this release and making Hive 2.1 generally available.
- Like before, data science and data engineering interfaces continue to improve and below are some key highlights:
* Enhanced Presto Notebook Experience with a native interpreter implementation and concurrency support.
* Airflow offering is easier-to-use and enterprise-ready than ever before with Python 3.5 environment and the ability to manage add-on packages, and an easy way to edit and sync DAGs and other files.
- Python 2.7 is now the default in QDS Clusters for many Python libraries are have either dropped or are in the process of dropping support for Python 2.6 completely. E.g.: pip 10, Pillow, Pytest.
- Help customers to troubleshoot and resolve common issues with the platform through our first product troubleshooting guide.
The full release notes can be read here when available: http://docs.qubole.com/en/latest/release-notes
If you have questions, please let us know at: https://qubole.zendesk.com
Oct 31, 09:46 PDT