Installing SQream Using Binary Packages¶

Overview¶

The binary package SQream installation method is based on a file package provided by SQream. The files in this package must be installed on your machine and modified as required.

The Installing SQream Using Binary Packages page provides recommendations for production deployments of SQream and describes the following:

Installing SQream
Upgrading Your SQream Version

Installing SQream ¶

This procedure describes how to install SQream using Binary packages and must be done on all servers.

To install SQream using Binary packages:

Copy the SQream package to the /home/sqream directory for the current version:
```
$ tar -xf sqream-db-v<2020.2>.tar.gz
```
Append the version number to the name of the SQream folder. The version number in the following example is v2020.2:
```
$ mv sqream sqream-db-v<2020.2>
```
Move the new version of the SQream folder to the /usr/local/ directory:
```
$ sudo mv sqream-db-v<2020.2> /usr/local/
```

Change the ownership of the folder to sqream folder:

$ sudo chown -R sqream:sqream  /usr/local/sqream-db-v<2020.2>

Navigate to the /usr/local/ directory and create a symbolic link to SQream:
```
$ cd /usr/local
$ sudo ln -s sqream-db-v<2020.2> sqream
```
Verify that the symbolic link that you created points to the folder that you created:
```
$ ls -l
```
Verify that the symbolic link that you created points to the folder that you created:
```
$ sqream -> sqream-db-v<2020.2>
```
Create the SQream configuration file destination folders and set their ownership to sqream:
```
$ sudo mkdir /etc/sqream
$ sudo chown -R sqream:sqream /etc/sqream
```
Create the SQream service log destination folders and set their ownership to sqream:
```
$ sudo mkdir /var/log/sqream
$ sudo chown -R sqream:sqream /var/log/sqream
```
Navigate to the /usr/local/ directory and copy the SQream configuration files from them:

$ cd /usr/local/sqream/etc/
$ cp * /etc/sqream

The configuration files are service configuration files, and the JSON files are SQream configuration files, for a total of four files. The number of SQream configuration files and JSON files must be identical.

Note

Verify that the JSON files have been configured correctly and that all required flags have been set to the correct values.

Create a cluster.

You can create a cluster either on your machine or in storage.

The following is an example of the message displayed when the cluster is created:

/home/sqream/bin/SqreamStorage -C -r /home/sqream/sqream_cluster
log4cxx: No appender could be found for logger (file_utils).
log4cxx: Please initialize the log4cxx system properly.
Setting cluster version to: 38

For more information on creating clusters, see SqreamStorage.

As in the example below, update your sqream<#>_config.json file:
1. Copy and paste the path to the directory where yous cluster is located.
2. Update all parameters in the current configuration files.

Tip

For more information about setting the limitQueryMemoryGB flag, see the recommended spool settings.

Note

A unique instanceID must be used in each JSON file. In the example above, the instanceID sqream_2 is changed to sqream_3.

Note

A separate sqream<#>_config.json file exists for each worker on your system. The json file corresponding to each worker must be updated. The name of each current configuration file includes the number of the worker. For example, the current configuration file for Worker 8 is sqream8_config.json.

Note the following:

The value of the metadataServerIp parameter must point to the IP that the metadata is running on.

The value of the machineIP parameter must point to the IP of your local machine.

The values are the same on servers running metadataserver, and different on other server nodes.

Insert your license file path into the licensePath parameter in your current configuration file (see above).

The following is an example of a license file path:

6C5B5CEF705FD72C21CA991E6A00A7410589DED028366FF
9441AC6FA8C5FB8FDF2BD22BF328261FCD84455941634EC
38FDB361DC0B2DE81A5A4120C1AE58D0B4EECCE2AD9D97C
047C54838413F50ACC78F74603407AC864B81D23745F841
35CFFF60561886546CF3A8F9A4E0049BB099210CB43FB33
2DDB3B91E9EB9774B39065E7DBB6E1982E8DACCD732AC6B
8532A15BD2C8049A1975C470C3A091DFFE382E4F52A0C5E
24C8F16490DC2D192EB9AED2CE7F2B76F513FCEA6C5B648
E370DDABA9FC29AEB77AC620FDFBC71663FAE18CCE64A2D
B7E69750F74288AD474930CDF3F38023766AE371DC3B639
3A6547620F

Update all parameters in the previous sqream<#>_config_legacy.json configuration files for each worker, as shown in the following example:

{
    "diskSpaceMinFreePercent": 10,
    "enableLogDebug": false,
    "insertCompressors": 8,
    "insertParsers": 8,
    "isUnavailableNode": false,
    "logBlackList": "webui",
    "logDebugLevel": 6,
    "nodeInfoLoggingSec": 60,
    "useClientLog": true,
    "useMetadataServer": true,
    "spoolMemoryGB": 5,
    "waitForClientSeconds": 18000
}

Tip

For more information about setting the limitQueryMemoryGB flag, see the recommended spool settings.

Optional - To run additional SQream services, copy the required sqream<#>_config_legacy.json configuration files and create additional JSON files:
```
$ cp sqream2_config.json sqream3_config.json
$ vim sqream3_config.json
```

Note

A unique instanceID must be used in each JSON file. In the example above, the instanceID sqream_2 is changed to sqream_3.

Optional - If you created additional services in the previous step, verify that you have also created their additional configuration files:

For example, each sqream<#>_config.json configuration file must have a corresponding the path to its corresponding legacy configuration file is sqream<#>_config_legacy.json.
For each SQream service configuration file, do the following:
1. Change the SERVICE_NAME=sqream2 value to SERVICE_NAME=sqream3.
2. Change LOGFILE=/var/log/sqream/sqream2.log to LOGFILE=/var/log/sqream/sqream3.log.

Note

If you are running SQream on more than one server, you must configure the serverpicker and metadatserver services to start on only one of the servers. If metadataserver is running on the first server, the metadataServerIP value in the second server’s /etc/sqream/sqream1_config.json file must point to the IP of the server on which the metadataserver service is running.

Set up servicepicker:
1. Do the following:
```
$ vim /etc/sqream/server_picker.conf
```
2. Change the IP 127.0.0.1 to the IP of the server that the metadataserver service is running on.
3. Change the CLUSTER to the value of the cluster path.

Set up your service files:

$ cd /usr/local/sqream/service/
$ cp sqream2.service sqream3.service
$ vim sqream3.service

Increment each EnvironmentFile=/etc/sqream/sqream2-service.conf configuration file for each SQream service file, as shown below:
```
$ EnvironmentFile=/etc/sqream/sqream<3>-service.conf
```

Copy and register your service files into systemd:

$ sudo cp metadataserver.service /usr/lib/systemd/system/
$ sudo cp serverpicker.service /usr/lib/systemd/system/
$ sudo cp sqream*.service /usr/lib/systemd/system/

Verify that the above have been copied successfully:

$ ls -l /usr/lib/systemd/system/sqream*
$ ls -l /usr/lib/systemd/system/metadataserver.service
$ ls -l /usr/lib/systemd/system/serverpicker.service
$ sudo systemctl daemon-reload

Copy the license into the /etc/license directory:
```
$ cp license.enc /etc/sqream/
```

If you have an HDFS environment, see Configuring an HDFS Environment for the User sqream.

For more information about installing Studio on a stand-alone server, see Installing Studio on a Stand-Alone Server.

Upgrading Your SQream Version ¶

Upgrading your SQream version requires stopping all running services while you manually upgrade SQream.

To upgrade your version of SQream:

Stop all actively running SQream services.

Note

All SQream services must remain stopped while the upgrade is in process. Ensuring that SQream services remain stopped depends on the tool being used.

For an example of stopping actively running SQream services, see Launching SQream with Monit.

Verify that SQream has stopped listening on ports 500X, 510X, and 310X:
```
$ sudo netstat -nltp
```

Replace the old version, such as sqream-db-v2020.2, with the new version, such as sqream-db-v2021.1:

$ cd /home/sqream
$ mkdir tempfolder
$ mv sqream-db-v2021.1.tar.gz tempfolder/
$ tar -xf sqream-db-v2021.1.tar.gz
$ sudo mv sqream /usr/local/sqream-db-v2021.1
$ cd /usr/local
$ sudo chown -R sqream:sqream sqream-db-v2021.1

Remove the symbolic link:
```
$ sudo rm sqream
```
Create a new symbolic link named “sqream” pointing to the new version:
```
$ sudo ln -s sqream-db-v2021.1 sqream
```
Verify that the symbolic SQream link points to the real folder:
```
$ ls -l
```
The following is an example of the correct output:
```
$ sqream -> sqream-db-v2021.1
```

Optional - (for major versions) Upgrade your version of SQream storage cluster, as shown in the following example:

$ cat /etc/sqream/sqream1_config.json |grep cluster
$ ./upgrade_storage <cluster path>

The following is an example of the correct output:

    get_leveldb_version path{<cluster path>}
    current storage version 23
upgrade_v24
upgrade_storage to 24
    upgrade_storage to 24 - Done
    upgrade_v25
    upgrade_storage to 25
    upgrade_storage to 25 - Done
    upgrade_v26
    upgrade_storage to 26
    upgrade_storage to 26 - Done
    validate_leveldb
    ...
upgrade_v37
    upgrade_storage to 37
    upgrade_storage to 37 - Done
    validate_leveldb
storage has been upgraded successfully to version 37

Verify that the latest version has been installed:

$ ./sqream sql --username sqream --password sqream --host localhost --databasename master -c "SELECT SHOW_VERSION();"

The following is an example of the correct output:

v2021.1
1 row
time: 0.050603s

For more information, see the upgrade_storage command line program.

For more information about installing Studio on a stand-alone server, see Installing Studio on a Stand-Alone Server.

Installing SQream Using Binary Packages¶

Overview¶

Installing SQream¶

Upgrading Your SQream Version¶

Installing SQream ¶

Upgrading Your SQream Version ¶