5.8 Release Notes
MemSQL 5.8 brings support for SAML 2.0 authentication, security audit logging capabilities, manageability enhancements, and other improvements.
A number of security enhancements have been added in MemSQL 5.8.
SAML 2.0 Authentication
Users can now authenticate to MemSQL by using SAML 2.0 third-party authentication. When properly configured, a MemSQL cluster can now receive and validate SAML 2.0 assertions about a user during authentication with the database. For more information, see the SAML Authentication documentation.
MemSQL now provides audit logging capability that records all database activity and writes log files to a external location. Numerous logging levels are provided to allow system administrators to adjust both the verbosity and types of queries that can be logged. For more information, see the Audit Logging documentation.
As of the time of this publication, audit logging feature is made available and licensed only as part of the MemSQL Advanced Security Option. Before using or implementing this functionality, please consult with your enterprise’s licensing administrator to confirm that your enterprise has purchased the necessary Advanced Security Option license from MemSQL.
MemSQL Ops Agent Authentication
MemSQL Ops now supports inter-agent authentication using unique API tokens. When the primary agent generates a new token, agents on other hosts can follow the primary agent by specifying its unique API token. For more information, see the following topics in the documentation:
Workload Profiling and Management Views
MemSQL 5.8 introduces a new set of management views to help diagnose performance problems. These views allow you to determine where queries and system tasks are spending time, what resources are being consumed, and where bottlenecks may exist in the cluster. For more information, see the following topics in the documentation:
MemSQL Ops Upgrade to 6.0
When you upgrade MemSQL Ops to version 6.0 using:
sudo memsql-ops agent-upgrade --file-path memsql-ops-6.0.6.tar.gz
you will see a warning like:
Additionally, upgrading MemSQL may require manual intervention on rollback (read more at http://docs.memsql.com/docs/58-release-notes). It is highly recommended that you back up all your data before continuing. Would you like to proceed? [y/N]
The manual intervention on rollback mentioned in the message is no longer needed.
We recommend that you do a backup before upgrading. Then enter
y and proceed.
MemSQL Streamliner has been deprecated and will be removed in MemSQL 6.0. For current Streamliner users, we recommend migrating to MemSQL Pipelines instead. MemSQL Pipelines provides increased stability, improved ingest performance, and exactly-once semantics. For more information about Pipelines, see the MemSQL Pipelines documentation.
What Do I Do With My Old Streamliner Spark Cluster?
To upgrade to MemSQL 6.0, you must first remove each of your Streamliner pipelines, and then uninstall the Spark Cluster co-located with your MemSQL cluster. Once each of the pipelines has been removed, simply run
memsql-ops spark-uninstall to uninstall the Spark Cluster.
Can I Still Use Spark With MemSQL?
Yes, MemSQL supports Spark integration via the MemSQL Spark Connector. The connector allows you to leverage your existing Spark clusters to write data directly to MemSQL via a performant and easy-to-use API. For more information on the MemSQL Spark Connector, please see the MemSQL Spark 2.0 Connector GitHub page.
How Do I Get Started With Pipelines?
For more information on Pipelines, please see:
- Fixed an issue where recovering reference databases could be reattached to the cluster either automatically or explicitly via
- Fixed an issue with
SHOW LOAD ERRORSwhere CSV parse errors would not be reported if
FIELDS ENCLOSED BYwas specified when executing
LOAD DATA SKIP ERRORS.
- Fixed an issue with S3 Pipelines where a pipeline would become unresponsive if a corrupt zip file was loaded.
- Due to the new Workload Profiling functionality,
distributed_plancache_summaryhave been removed.
Maintenance Release Changelog
The changelog below contains MemSQL improvements and bug fixes introduced in maintenance or revision releases. For a similar list for MemSQL Ops, see MemSQL Ops Releases.
2018-05-21 Version 5.8.13
- Fixed an issue where invalid
GROUP BYqueries were not correctly locked out.
2018-03-13 Version 5.8.12
- Fixed the issue where on DR cluster failover, the
AUTO_INCREMENTvalues were not getting synchronized on the aggregators. Since aggregator IDs can collide across clusters, this caused
AUTO_INCREMENTcollisions on DR failover.
2018-03-05 Version 5.8.11
- Fixed an issue where it was possible for child aggregators to generate “Leaf failed while executing this query” errors if they failed to connect to a leaf a number of times during certain operations. This error would persist until the child aggregator was restarted.
- Fix a crash caused by
SHOWcommands with long
2018-01-16 Version 5.8.10
- Reduced occurrences of the error “Partition’s table metadata are out of sync for table.”
- Include CPU feature flags in the persisted plan cache lookup for plans stored on disk to prevent the plan from being used on processors without the proper features (avx2, etc.). This protects against the underlying hardware MemSQL runs on changing after a node restart, which can happen in some cloud environments. If this happens query plans will now recompile.
- Add fsync calls to all persisted plan cache files to better guard against process termination in the middle of writing query plans out to disk.
2017-11-28 Version 5.8.9
- Fixed an issue where queries could incorrectly fail due to the max table memory limit being reached.
2017-11-01 Version 5.8.8
- Fixed an issue with synchronously replicated databases where snapshots could sometimes cause a substantial replication slowdown. Asynchronous databases were not affected.
2017-08-25 Version 5.8.7
- Changed default permissions for
SELECT INTO OUTFILEto 666, so that any file created by
INTO OUTFILEis writable by all users on the server host.
2017-08-15 Version 5.8.6
- Performance has been improved for the
ANALYZEcommand by reducing its associated memory overhead.
- Columnstore tables now use mmap less frequently to avoid concurrency issues in the Linux virtual memory subsystem.
- Fixed an issue with workload profiling where it would incorrectly disable tracking extended statistics under rare circumstances.
2017-07-19 Version 5.8.5
- The following built-in functions have been added for working with vectors with 32-bit floating-point numbers:
- MemSQL Pipelines now supports extracting files from a Linux filesystem directory. For more information, see Filesystem Pipelines
- Fixed an issue where reference tables could be dropped under rare circumstances.
- Fixed a possible crash when querying
2017-06-07 Version 5.8.4
- Memory overhead per database has been reduced by up to 30%, depending on whether or not the cluster’s
transaction_buffersystem variable has been set to a low value. For more information, see Configuring Durability.
- Performance has been improved for the
CAST()function when executed on a
- Improved support for
POLYGONtypes with a large number of vertices.
2017-05-04 Version 5.8.2
- Kerberos Authentication now supports regular expressions when granting access to a Kerberos identity. For more information, see Granting a User Kerberos Authentication Permissions.
- Memory overhead of empty tables has been reduced by as much as 40%.
- Memory usage metrics exposed in management views is now more accurate.