6.8 Release Notes

The primary improvements in the MemSQL 6.8 release are enhancements to HDFS pipelines security and improved ad hoc (first run) query performance.

See the descriptions and changelog below for more information on these new features, as well as the other new features in the 6.8 release.

Data Loading

  • Advanced HDFS Pipelines that import data from HDFS using Kerberos and wire encryption.
  • Performance improvements to Pipelines extractors
  • Added TRAILING NULLCOLS and NULLS DEFINED BY clauses to LOAD DATA and CREATE PIPELINE

Query Optimization, Compilation, and Statistics

  • Can speed up ad-hoc (first query) run times by having queries interpreted and then dynamically compiled during first query execution. This behavior was available as experimental setting in MemSQL 6.7, but is production-ready and enabled by default in 6.8. See the interpret_first documentation for more details.
  • Performance improvements for some outer, anti-, and semi-joins
  • Performance improvements to ROLLUP and CUBE

Resource Governor Improvements

  • LOAD DATA operations run in the resource pool for the current connection where the load operation is running
  • Stored procedures run in the resource pool of the current connection from where they are called
  • Query optimization always runs in the resource pool of the current connection

Functional Extensions

  • A subquery does not require an alias, assuming that removing the alias doesn’t create ambiguity
  • Increased the precedence of || as concat (under sql_mode = PIPES_AS_CONCAT) to be compatible with MySQL and Postgres
  • Added SPLIT() and JSON_TO_ARRAY()
  • Enabled TO_DATE() to support format strings with time-related format options (HH, SS, etc.)
  • Enabled TO_DATE() to support the DD format option
  • Enabled TO_TIMESTAMP() to support YY and FF format options
  • Enabled TO_TIMESTAMP(), TO_DATE(), and TO_CHAR() to support the D format option
  • Enabled TO_TIMESTAMP() and TO_DATE() to support using different punctuation as separator
  • Enabled TO_TIMESTAMP() and TO_DATE() to raise an error instead of returning NULL for certain error cases
  • Enabled TO_CHAR() to support AM and PM
  • Modified how TO_TIMESTAMP() parses 12 when using the HH format option

Maintenance Release Changelog

2019-07-22 Version 6.8.4

  • Removed some unneeded memory allocations that were being done before running parallel queries. This addresses the high CPU usage that sometimes occurred when running these queries.
  • Now, clear the DNS cache on secondary cluster nodes when nodes are modified on the primary cluster.
  • Fixed an issue where ALTER PIPELINE would go into an infinite loop.
  • Fixed a rarely occurring issue where resource pools were incorrectly tracking memory usage.
  • Fixed an issue where the use of AVX2 and SSE2 by LOAD DATA was disabled.
  • Now, if there are too many threads waiting for child threads to be scheduled, switch to non-parallel execution if possible. Fail the query otherwise. Previously, there was a thirty second timeout for launching child threads. This is too long of a wait for some workloads.
  • Now, remove MemSQL’s dependency on the libnsl.so.1 library to make it easier to deploy MemSQL on flavors/versions of Linux such as RHEL 8 that don’t have libnsl.so.1 installed by default.
  • Fixed an issue where the constant folding optimization was disabled in edge cases. The fix speeds up queries that were affected by the issue.
  • Now, address a deadlock that is possible during REBALANCE PARTITIONS if the child aggregator runs out of connection threads.
  • Fixed a crash that was caused by a sync issue when files were written to the plancache during code generation.
  • Now, disable correlated subselects when using ROLLUP() and CUBE(), as it is currently unsupported. Also, fixed an issue where an optimization incorrectly reordered a GROUP BY and join when ROLLUP() or CUBE() was being used.
  • Added new configuration settings to Kafka pipelines to support explicit Kerberos credential caches (sasl.kerberos.ccache) and disabling of kinit (sasl.kerberos.disable.kinit).
  • Now, a pipeline waits for the reference database to be in sync before the pipeline compiles.

2019-06-24 Version 6.8.3

  • Fixed an issue when interpreter_mode is set to interpret_first, where incorrect code was generated. This issue could cause a crash or an infinite loop.
  • Fixed an issue where memsqlctl wouldn’t show license details.

2019-06-10 Version 6.8.2

  • Fixed an issue that was causing high system CPU usage. This issue occurred when running context switch heavy workloads. These workloads contain many fast executing queries that run against multiple connections.
  • Now, a leaf node fails over to its pair if, for sixty seconds, the leaf has all of its threads running queries (as specified by max_connection_threads) and is not maintaining a minimum throughput of five queries per second.
  • Now, a leaf node fails over to its pair if the leaf’s disk usage falls below the value of minimal_disk_space.
  • Fixed an issue where LAST_INSERT_ID(<expression>) was returning an incorrect value.
  • Now, security management commands such as CREATE GROUP, CREATE ROLE and CREATE USER can be run inside of stored procedures.
  • Now, improve the performance of SELECT ... INTO S3.
  • Now, the value of the SIZE field in information_schema.MV_BACKUP_HISTORY is stored in megabytes instead of bytes.
  • Fixed an issue where a user authenticated with Pluggable Authentication Module (PAM) could not run ANALYZE TABLE.
  • Fixed an issue where the bootstrap aggregator would not work on machines where the only configured address is the loopback address.

2019-05-14 Version 6.8.1

  • Initial GA release of MemSQL 6.8
Was this article useful?