Configuration¶

The Configuration page describes SQream’s method for configuring your instance of SQream and includes the following topics:

Overview
Configuring Your Instance of SQream
Session-Based Configuration
Cluster-Based Configuration
Configuration Flag Types
Configuration Commands
Configuration Roles
Showing All Flags in the Catalog Table

Overview ¶

Modifications that you make to your configurations are persistent based on whether they are made at the session or cluster level. Persistent configurations are modifications made to attributes that are retained after shutting down your system.

Configuring Your Instance of SQream ¶

The Configuring Your Instance of SQream section describes the following:

Configuring SQream Using the SQream Configuration File
Configuring SQream Using a Legacy Configuration File

Configuring SQream Using the SQream Configuration File ¶

You can configure your instance of SQream using the SQream configuration file (config.json). This file displays all the parameters required for configuring your instance of SQream and are read-only. The parameter settings in this file are stored at the metadata level and are applied globally to all workers connected to it.

The following is an example of a worker configuration file:

{
    “cluster”: “/home/test_user/sqream_testing_temp/sqreamdb”,
    “gpu”:  0,
    “licensePath”: “home/test_user/SQream/tests/license.enc”,
    “machineIP”: “127.0.0.1”,
    “metadataServerIp”: “127.0.0.1”,
    “metadataServerPort”: “3105,
    “port”: 5000,
    “useConfigIP”” true,
    “legacyConfigFilePath”: “home/SQream_develop/SqrmRT/utils/json/legacy_congif.json”
}

You can access the SQream configuration file from the legacyConfigFilePath parameter shown above.

Configuring SQream Using a Legacy Configuration File ¶

The legacy configuration file provides access to the read/write flags used in SQream’s previous configuration method. A link to this file is provided in the legacyConfigFilePath parameter in the worker configuration file.

The following is an example of the legacy configuration file:

{
   “developerMode”: true,
   “reextentUse”: false,
   “useClientLog”: true,
   “useMetadataServer”” false
}

Session-Based Configuration ¶

Session-based configurations are not persistent and are deleted when your session ends. This method enables you to modify all required configurations while avoiding conflicts between flag attributes modified on different devices at different points in time.

The SET flag_name command is used to modify flag attributes. Any modifications you make with the SET flag_name command apply only to the open session, and are not saved when the session ends.

For more information, see the following:

Using SQream SQL - modifying flag attributes from the CLI.
SQream Acceleration Studio - modifying flag attributes from Studio.

Cluster-Based Configuration ¶

SQream uses cluster-based configuration, enabling you to centralize configurations for all workers on the cluster. Only flags set to the regular or cluster flag type have access to cluster-based configuration. Configurations made on the cluster level are persistent.

For more information, see the following:

Using SQream SQL - modifying flag attributes from the CLI.
SQream Acceleration Studio - modifying flag attributes from Studio.

For more information on flag-based access to cluster-based configuration, see Configuration Flag Types below.

Configuration Flag Types ¶

The flag type attribute can be set for each flag and determines its write access as follows:

Regular: session-based read/write flags that can be stored in the metadata file
Cluster: global cluster-based read/write flags that can be stored in the metadata file
Worker: single worker-based read-only flags that can be stored in the worker configuration file

The flag type determines which files can be accessed and which commands or commands sets users can run.

The following table describes the file or command modification rights for each flag type:

Flag Type	Legacy Configuration File	ALTER SYSTEM SET	Worker Config File
Regular	Can modify	Can modify	Cannot modify
Cluster	Cannot modify	Can modify	Cannot modify
Worker	Cannot modify	Cannot modify	Can modify

Configuration Commands ¶

The configuration commands are associated with particular flag types based on permissions.

The following table describes the commands or command sets that can be run based on their flag type:

Flag Type	Command	Description	Example
Regular	SET <flag_name>	Used for modifying flag attributes.	SET developerMode=true
Cluster	ALTER SYSTEM SET <flag-name>	Used to storing or modifying flag attributes in the metadata file.	ALTER SYSTEM SET <heartbeatInterval=12;>
Cluster	ALTER SYSTEM RESET <flag-name \| ALL>	Used to remove a flag or all flag attributes from the metadata file.	ALTER SYSTEM RESET <heartbeatInterval ALTER SYSTEM RESET ALL>
Regular, Cluster, Worker	SHOW <flag-name> \| ALL	Used to print the value of a specified value or all flag values.	SHOW <heartbeatInterval>
Regular, Cluster, Worker	SHOW ALL LIKE	Used as a wildcard character for flag names.	SHOW <heartbeat*>
Regular, Cluster, Worker	show_conf UF	Used to print all flags with the following attributes: Flag name Default value Is developer mode (Boolean) Flag category Flag type	rechunkThreshold,90,true,RND,regular
Regular, Cluster, Worker	show_conf_extended UF	Used to print all information output by the show_conf UF command, in addition to description, usage, data type, default value and range.	spoolMemoryGB,15,false,generic,regular,Amount of memory (GB) the server can use for spooling,”Statement that perform “”group by””, “”order by”” or “”join”” operation(s) on large set of data will run much faster if given enough spool memory, otherwise disk spooling will be used resulting in performance hit.”,uint,,0-5000
Regular, Cluster, Worker	show_md_flag UF	Used to show a specific flag/all flags stored in the metadata file.	Example 1: * master=> ALTER SYSTEM SET heartbeatTimeout=111; Example 2: * master=> select show_md_flag(‘all’); heartbeatTimeout,111 Example 3: * master=> select show_md_flag(‘heartbeatTimeout’); heartbeatTimeout,111

Configuration Roles ¶

SQream divides flags into the following roles, each with their own set of permissions:

Generic – Flags that can be modified by standard users on a session basis.
Admin – Flags that can be modified by administrators on a session and cluster basis using the ALTER SYSTEM SET command.

The following table describes the Generic and Admin configuration flags:

Showing All Flags in the Catalog Table ¶

SQream uses the sqream_catalog.parameters catalog table for showing all flags, providing the scope (default, cluster and session), description, default value and actual value.

The following is the correct syntax for a catalog table query:

SELECT * FROM sqream_catalog.settings

The following is an example of a catalog table query:

externalTableBlobEstimate, 100, 100, default,
varcharEncoding, ascii, ascii, default, Changes the expected encoding for Varchar columns
useCrcForTextJoinKeys, true, true, default,
hiveStyleImplicitStringCasts, false, false, default,