What’s new in 2020.3

Storage version: 30

Compatible with Drivers

  • Nvidia driver: 387.26 (Supported CUDA versions 9.1 - 10.1 )
  • JDBC - all versions starting from JDBC 4.0.0, the latest recommended version JDBC 4.3.0.
  • ODBC 4.0 (Linux) (no new version was released)
  • ODBC 4.0 (Windows) (no new version was released)
  • Native C++ 4.0 (no new version was released)
  • NodeJS - all the versions starting from NodeJS 4.0, the latest recommended version NodeJS 4.2.
  • Pysqream - pysqream 3.1.0 (the latest new version).
  • SQream Studio v5.1

New features

  • Export to S3 and HDFS storage
  • Export to Parquet and ORC with enhanced error tolerance
  • Import with enhanced error tolerance
  • Scalar functions for Text data type:
    • Substring, lower, ltrim, charindex, replace, etc.
    • Binary operators - ||, [not] like, [not] like, etc.
    • Casts from/to text
  • New command-line utility for parsing parameters in SQL files
  • SQream Studio - new version - v5.1
    • In addition to the editor, Log Viewer has been added
    • The editor allows splitting a single query into multiple concurrent executions based on date or numeric range segments
    • Dashboard is also available on non-K8S installation

Enhancements

  • Error handling for CSV FDW
  • Enable logging errors - ORC, Parquet, CSV
  • Add limit and offset options to csv_fdw import
  • Enable logging errors to an external file when skipping CSV, Parquet, and ORC errors
  • Option to specify date format to the CSV FDW
  • Support all existing Varchar functions with Text on GPU
  • Support INSERT INTO + ORDER BY optimization for non-clustered tables
  • Performance improvements with I/O

Bug fixes

  • Better error message when passing the max errors limit
  • showFullExceptionInfo is no longer restricted to developer mode
  • StreamAggregateA reduction error when performing aggregation on a null column
  • Insert into query fails with “”Error at Sql phase during Stages “”rewriteSqlQuery”“
  • Cast from varchar to nvarchar doesn’t remove the spaces
  • Internal Runtime Error t1.size() == t2.size() when querying sqream_catalog.delete_predicates
  • spoolMemoryGB and limitQueryMemoryGB show incorrectly in the runtime global section of show_conf
  • Casting empty text to int causes illegal memory access
  • Copy from TEXT field is 1.5x slower than the VARCHAR equivalent
  • TPCDS 10TB - Internal runtime error (std::bad_alloc: out of memory) on 2020.1.0.2
  • Not equal join on not existing TEXT crashes the system
  • Internal runtime time error when using TEXT (tpcds)
  • Copying CSV with a quote in the middle of a field to TEXT field does not produce an error
  • Can’t monitor long network insert loads with SQream
  • Upper & like performance on Nvarchar
  • Insert into from 4 instances get stuck (hanging)
  • An invalid formatted CSV can cause an insufficient memory error on a COPY FROM statement if a quote isn’t closed and the file is much larger than system memory.
  • TEXT columns can’t be used with an outer join together with an inequality check (!= , <>)

Known issues

  • Cast from TEXT to date/datetime returns an error when the TEXT column contains NULL
  • Cast empty TEXT to int returns 0 instead of returning error
  • Performance degradation - ReadParquet Rechunk (~13%)
  • Performance degradation - ReadORC (~4%)
  • The option to run multiple count distinct operation (runTimeGlobal/distinctAggregatesOnHost) is allowed only in developer mode
  • Multiple count distinct operation is not supported on TEXT data types

Upgrading to v2020.3

Versions are available for IBM POWER9, RedHat (CentOS) 7, Ubuntu 18.04, and other OSs via Docker.

Contact your account manager to get the latest release of SQream DB.