What’s new in 2020.3

SQream DB v2020.3 contains new features, improved performance, and bug fixes.

New features

  • Parquet and ORC files can now be exported to local storage, S3, and HDFS with COPY TO and foreign data wrappers.

  • We’ve introduced new error tolerance features when loading data with foreign data wrappers

  • TEXT is ramping up with new features (previously only available with VARCHARs):

  • SQream Acceleration Studio v5.1

    • New log viewer helps you track and debug what’s going on in SQream DB
    • Dashboard now also available for non-k8s deployments
    • The editor contains a new query concurrency tool for date and numeric ranges

Enhancements

  • Error handling for CSV FDW
  • Enable logging errors - ORC, Parquet, CSV
  • Add limit and offset options to csv_fdw import
  • Enable logging errors to an external file when skipping CSV, Parquet, and ORC errors
  • Option to specify date format to the CSV FDW
  • Support all existing Varchar functions with Text on GPU
  • Support INSERT INTO + ORDER BY optimization for non-clustered tables
  • Performance improvements with I/O

Bug fixes

  • Better error message when passing the max errors limit
  • showFullExceptionInfo is no longer restricted to developer mode
  • StreamAggregateA reduction error when performing aggregation on a null column
  • Insert into query fails with “”Error at Sql phase during Stages “”rewriteSqlQuery”“
  • Cast from varchar to nvarchar doesn’t remove the spaces
  • Internal Runtime Error t1.size() == t2.size() when querying sqream_catalog.delete_predicates
  • spoolMemoryGB and limitQueryMemoryGB show incorrectly in the runtime global section of show_conf
  • Casting empty text to int causes illegal memory access
  • Copy from TEXT field is 1.5x slower than the VARCHAR equivalent
  • TPCDS 10TB - Internal runtime error (std::bad_alloc: out of memory) on 2020.1.0.2
  • Not equal join on not existing TEXT crashes the system
  • Internal runtime time error when using TEXT (tpcds)
  • Copying CSV with a quote in the middle of a field to TEXT field does not produce an error
  • Can’t monitor long network insert loads with SQream
  • Upper & like performance on Nvarchar
  • Insert into from 4 instances get stuck (hanging)
  • An invalid formatted CSV can cause an insufficient memory error on a COPY FROM statement if a quote isn’t closed and the file is much larger than system memory.
  • TEXT columns can’t be used with an outer join together with an inequality check (!= , <>)

Known Issues & Limitations

  • Cast from TEXT to a DATE or DATETIME errors when the TEXT column contains NULL
  • Casting an empty TEXT field to an INT type returns 0 instead of erroring
  • Multiple COUNT( distinct ... ) operations on the TEXT data type are currently unsupported
  • Multiple COUNT( distinct ... ) operations within the same query are limited to “developer mode” due to an instability that was identified. If you rely on this feature, contact your SQream account manager to enable this feature.

Upgrading to v2020.3

Versions are available for IBM POWER9, RedHat (CentOS) 7, Ubuntu 18.04, and other OSs via Docker.

Contact your account manager to get the latest release of SQream DB.