Logo

Contents:

  • Getting Started
    • Preparing Your Machine to Install SQreamDB
    • Installing SQreamDB
    • Executing Statements in SQreamDB
    • Performing Basic SQream Operations
      • Running the SQream SQL Client
      • Creating Your First Table
      • Listing Tables
      • Inserting Rows
      • Running Queries
      • Deleting Rows
      • Saving Query Results to a CSV or PSV File
    • Hardware Guide
    • Staging and Development Hardware Guide
  • Installation Guides
    • Installing and Launching SQreamDB
      • Pre-Installation Configuration
      • Installing SQream Using Binary Packages
      • Installing Monit
      • Launching SQream with Monit
    • Installing SQream Studio
      • Installing Studio on a Stand-Alone Server
      • Installing an NGINX Proxy Over a Secure Connection
    • Upgrade Guides
      • Version Upgrade
      • Upgrade-Related Configuration Changes
  • SQreamDB on AWS
  • Operational Guides
    • Access Control
      • Overview
      • Password Policy
      • Managing Roles
      • Permissions
      • Departmental Example
    • Creating or Cloning Storage Clusters
    • Working with External Data
    • Foreign Tables
    • Health Monitoring
    • Deleting Data
    • Logging
    • Query Split
    • Monitoring Query Performance
    • Security
    • Saved Queries
    • Optimization and Best Practices
    • Oracle Migration Guide
  • Configuration Guides
    • Configuring SQream
      • Cluster and Session
      • Workers
      • Modification Methods
      • Parameter Values
      • Showing All Flags in the Catalog Table
    • LDAP
    • Single Sign-On
  • Architecture
    • Internals and Architecture
    • Filesystem and Usage
    • Sizing
  • Acceleration Studio
    • Getting Started with SQream Acceleration Studio
    • Executing Statements and Running Queries from the Editor
    • Viewing Logs
    • Creating, Assigning, and Managing Roles and Permissions
    • Configuring Your Instance of SQreams
  • Connecting to SQreamDB
    • Client Platforms
      • Denodo Platform
      • Informatica Cloud Services
      • MicroStrategy
      • Pentaho Data Integration
      • PHP
      • BI Desktop
      • R
      • SAP BusinessObjects
      • SAS Viya
      • Semarchy
      • SQL Workbench
      • Tableau
      • Talend
      • TIBCO Spotfire
    • Client Drivers
      • SQreamNET
      • Dataiku
      • JDBC
      • Node.JS
      • ODBC
        • Install and Configure ODBC on Windows
        • Install and configure ODBC on Linux
      • Python (pysqream)
      • Spark
      • Trino
  • Data Ingestion Sources
    • Overview
    • Avro
    • CSV
    • Parquet
    • ORC
    • JSON
    • SQLoader As a Service
      • Preparing Oracle for Data Migration
  • External Storage Platforms
    • Azure Blob Storage
    • Google Cloud Platform
    • HDFS Environment
    • Amazon Web Services
  • Feature Guides
    • Query Healer
    • Compression
    • Python User-Defined Functions
    • Workload Manager
    • Concurrency and Locks
    • Data Encryption
      • Overview
      • Encryption Methods
      • Data Types
      • Syntax
      • Examples
      • Limitations
      • Permissions
  • References
    • SQL Statements and Syntax
      • SQL Syntax Features
      • SQL Statements
      • SQL Functions
        • Built-In Scalar Functions
        • User-Defined Functions
        • Aggregate Functions
        • Window Functions
    • Catalog Reference
      • Catalog Tables
      • Additional Tables
      • Examples
    • Command line programs
      • metadata_server
      • sqreamd
      • Multi-Platform Sqream SQL
      • SqreamDB Console
      • Server Picker
      • SqreamStorage
      • sqream sql
      • upgrade_storage
    • SQL Feature Checklist
  • Data Types
    • Supported Data Types
      • Primitive Data Types
      • Array
    • Casts and Conversions
    • Supported Casts
      • Numeric
      • Boolean
      • Integer
      • Floating Point
      • String
      • Date
      • DateTime2
  • Release Notes
    • SQDB release policy
    • 4.x Release Notes
      • Release Notes 4.3
      • Release Notes 4.5
      • Release Notes 4.8
      • Release Notes 4.9
      • Release Notes 4.10
      • Release Notes 4.11
  • Troubleshooting
    • Remedying Slow Queries
    • Resolving Common Issues
    • Identifying Configuration Issues
    • Lock Related Issues
    • Log Related Issues
    • Core Dumping Related Issues
    • Retrieving Execution Plan Output Using SQreamDB Studio
    • Gathering Information for SQream Support
  • Glossary
SQreamDB
  • Troubleshooting
  • Log Related Issues

Log Related Issues

The Log Related Issues page describes how to resolve the following common issues:

Loading Logs with Foreign Tables

Assuming logs are stored at /home/rhendricks/sqream_storage/logs/, a database administrator can access the logs using the Foreign Tables concept through SQream DB.

CREATE FOREIGN TABLE logs
(
  start_marker      TEXT(4),
  row_id            BIGINT,
  timestamp         DATETIME,
  message_level     TEXT,
  thread_id         TEXT,
  worker_hostname   TEXT,
  worker_port       INT,
  connection_id     INT,
  database_name     TEXT,
  user_name         TEXT,
  statement_id      INT,
  service_name      TEXT,
  message_type_id   INT,
  message           TEXT,
  end_message       TEXT(5)
)
WRAPPER csv_fdw
OPTIONS
  (
     LOCATION = '/home/rhendricks/sqream_storage/logs/**/sqream*.log',
     DELIMITER = '|',
     CONTINUE_ON_ERROR = true
  )
;

For more information, see Loading Logs with Foreign Tables.

Counting Message Types

t=> SELECT message_type_id, COUNT(*) FROM logs GROUP BY 1;
message_type_id | count
----------------+----------
              0 |         9
              1 |      5578
              4 |      2319
             10 |      2788
             20 |       549
             30 |       411
             31 |      1720
             32 |      1720
            100 |      2592
            101 |      2598
            110 |      2571
            200 |        11
            500 |       136
           1000 |        19
           1003 |        19
           1004 |        19
           1010 |         5

Finding Fatal Errors

t=> SELECT message FROM logs WHERE message_type_id=1010;
Internal Runtime Error,open cluster metadata database:IO error: lock /home/rhendricks/sqream_storage/rocksdb/LOCK: Resource temporarily unavailable
Internal Runtime Error,open cluster metadata database:IO error: lock /home/rhendricks/sqream_storage/rocksdb/LOCK: Resource temporarily unavailable
Mismatch in storage version, upgrade is needed,Storage version: 25, Server version is: 26
Mismatch in storage version, upgrade is needed,Storage version: 25, Server version is: 26
Internal Runtime Error,open cluster metadata database:IO error: lock /home/rhendricks/sqream_storage/LOCK: Resource temporarily unavailable

Countng Error Events Within a Certain Timeframe

t=> SELECT message_type_id,
.          COUNT(*)
.   FROM logs
.   WHERE message_type_id IN (1010,500)
.   AND timestamp BETWEEN '2019-12-20' AND '2020-01-01'
.   GROUP BY 1;
message_type_id | count
----------------+------
            500 |    18
           1010 |     3

Tracing Errors to Find Offending Statements

If we know an error occured, but don’t know which statement caused it, we can find it using the connection ID and statement ID.

t=> SELECT connection_id, statement_id, message
.     FROM logs
.     WHERE message_level = 'ERROR'
.     AND timestamp BETWEEN '2020-01-01' AND '2020-01-06';
connection_id | statement_id | message
--------------+--------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------
           79 |           67 | Column type mismatch, expected UByte, got INT64 on column Number, file name: /home/sqream/nba.parquet

Use the connection_id and statement_id to narrow down the results.

t=>   SELECT database_name, message FROM logs
.       WHERE connection_id=79 AND statement_id=67 AND message_type_id=1;
database_name | message
--------------+--------------------------
master        | Query before parsing
master        | SELECT * FROM nba_parquet
Previous Next

© Copyright 2025 SQreamDB.