Release Notes 4.3

The 4.3 release notes were released on 11/06/2023 and describe the following:

Compatibility Matrix

System Requirement

Details

Supported OS

  • CentOS - 7.x

  • RHEL - 7.x / 8.x

Supported Nvidia driver

CUDA version from 10.1 up to 11.4.3

Storage version

49

Driver compatibility

  • JDBC 4.5.8

  • ODBC 4.4.4

  • NodeJS

  • .NET 3.0.2

  • Pysqream 3.2.5

  • Spark

SQream Acceleration Studio

Version 5.6.0

New Features and Enhancements

► A new SQLoader will enable you to load data into SQreamDB from other databases.

► Access control permissions in SQreamDB have been expanded, allowing roles to now grant and revoke access to other roles for the following:

  • VIEWS

  • FOREIGN TABLE

  • COLUMN

  • CATALOG

  • SERVICE

To learn more about how and when you should use this new capability, visit Permissions.

► RocksDB’s metadata scale-up improvements have been implemented.

SQreamDB Studio Updates and Improvements

SQream Studio version 5.6.0 has been released.

Known Issues

  • Percentile is not supported for Window Functions.

  • Performance degradation when using VARCHAR partition key in a Window Functions expression

  • In SQreamDB minor versions 4.3.9 and 4.3.10, granting permissions through the Acceleration Studio might result in an error, even though the permission has been successfully granted.

Version 4.3 resolved Issues

SQ No.

Description

SQ-11108

Slow COPY FROM statements using ORC files

SQ-11804

Slow metadata optimization

SQ-12721

maxConnectionInactivitySeconds flag issue when executing Batch Shell Program ETLs

SQ-12799

Catalog queries may not be terminated

SQ-13112

GRANT query queue issue

SQ-13201

INSERT INTO statement error while copying data from non-clustered table to clustered table

SQ-13210, SQ-13426

Slow query execution time

SQ-13225

LoopJoin performance enhancement supports =, >, <, and <= operators

SQ-13322

Cleanup operation case-sensitivity issue

SQ-13401

The JDBC driver causes the log summery of INSERT statements to fail

SQ-13453

Metadata performance issue

SQ-13460

GRANT ALL ON ALL TABLES statement slow compilation time

SQ-13461

WHERE clause filter issue

SQ-13467

Snapshot issue causes metadata failure

SQ-13529

Pysqream concurrency issue

SQ-13566, SQ-13694

S3 access to bucket failure when using custom endpoint

SQ-13587

Large number of worker connections failure

SQ-13947

Unicode character issue when using Tableau

SQ-14094

Metadata server error stops workers and query queue

SQ-14268

Internal runtime memory issue

SQ-14724

Alias issue when executing DELETE statement

SQ-13387

Simple query slow compilation time due to metadata size

Configuration Adjustments

► You may now configure the object access style and your endpoint URL with Virtual Private Cloud (VPC) when using AWS S3.

Visit Amazon Web Services to learn more about how and when you should use these two new parameters:

  • AwsEndpointOverride

  • AwsObjectAccessStyle

Deprecations

► CentOS Linux 7.x

  • As of June 2024, CentOS Linux 7.x will reach its End of Life and will not be supported by SQreamDB. This announcement provides a one-year advance notice for our users to plan for this change. We recommend users to explore migration or upgrade options to maintain ongoing support and security beyond this date.

  • REHL 8.x is now officially supported.

► INT96

Due to Parquet’s lack of support of the INT96 data type, SQream has decided to deprecate this data type.

► Square Brackets []

The [], which are frequently used to delimit identifiers such as column names, table names, and other database objects, are officially deprecated to facilitate the use of the ARRAY data type. To delimit database object identifiers, use double quotes "".

► VARCHAR

The VARCHAR data type is deprecated to improve the core functionalities of the platform and to align with the constantly evolving ecosystem requirements.

  • Support in the VARCHAR data type ends at September 30th, 2023.

  • VARCHAR is no longer supported for new customers, effective from version 2022.1.3.

  • The TEXT data type is replacing the VARCHAR and NVARCHAR data types.

Upgrading to v4.3

  1. Generate a back-up of the metadata by running the following command:

    $ select backup_metadata('out_path');

    Tip

    SQream recommends storing the generated back-up locally in case needed.

    SQream runs the Garbage Collector and creates a clean backup tarball package.

  2. Shut down all SQream services.

  3. Copy the recently created back-up file.

  4. Replace your current metadata with the metadata you stored in the back-up file.

  5. Navigate to the new SQream package bin folder.

  6. Run the following command:

    $ ./upgrade_storage <levelDB path>

  7. Version 4.3 introduces a service permission feature that enables superusers to grant and revoke role access to services. However, when upgrading from version 4.2 or earlier to version 4.3 or later, this feature initializes access to services and to catalog tables, causing existing roles to lose their access to services, catalog tables and consequently also to the UI (Catalog tables may also be used to determine user access rights and privileges. The UI can integrate with these permissions to control what actions users are allowed to perform in the database.).

There are two methods of granting back access to services:

To grant back access to catalog tables and the UI, you may either grant access to all system roles, using your public role:

GRANT ALL PERMISSIONS ON CATALOG <catalog_name> TO public;

Or individually grant access to selected roles:

GRANT ALL PERMISSIONS ON CATALOG <catalog_name> TO <role_name>;

Note

Upgrading from a major version to another major version requires you to follow the Upgrade Storage step. This is described in Step 7 of the Upgrading SQream Version procedure.