Configuring MemSQL for Non-Uniform Memory Access (MemSQL Ops)
When running MemSQL on machines with multiple Non-Uniform Memory Access (NUMA) nodes, you should configure MemSQL for NUMA with
numactl for optimal performance. If you do not configure MemSQL this way, performance will be greatly degraded due to expensive cross-NUMA-node memory access. Configuring for NUMA should be done as part of the installation process; however, you can reconfigure your deployment later, if necessary.
To simplify the configuration process, MemSQL Ops can automatically detect if a host machine has multiple NUMA nodes and then configure MemSQL with
numactl to bind individual MemSQL nodes to NUMA nodes.
Configuring a new cluster for NUMA
Before binding NUMA nodes to specific MemSQL leaf nodes, you must install
On each host, install the
numactlpackage. For example, for Debian-based OSes:
sudo apt-get install numactl
For Redhat/CentOS, run the following:
sudo yum install numactl
Check the number of NUMA nodes your machines by running
numactl --hardware. For example:
numactl --hardware available: 2 nodes (0-1)
The output shows that there are 2 NUMA nodes on this machine, numbered 0 and 1.
Configuring for NUMA through the Ops web interface
If you are managing your cluster using MemSQL Ops, install Ops agents on host machines through the Ops web interface. After the Ops agent is installed on each host machine, you can install MemSQL nodes through the UI. The suggested leaf node count should align to the NUMA node count on each machine. Make sure to install one MemSQL leaf node per NUMA node.
MemSQL Ops will then configure the MemSQL nodes appropriately for NUMA on each machine during deployment.
Configuring for NUMA through the Ops CLI
On each host that will contain MemSQL leaf nodes, create one leaf per NUMA node by running
memsql-ops memsql-deployuntil you have the same number of leaf nodes as NUMA nodes. You will need to choose a different
--portfor each MemSQL node on the same host.
For example, on a host machine with two NUMA nodes, you would deploy two leaf nodes.
memsql-ops memsql-deploy -r leaf --port 3307 memsql-ops memsql-deploy -r leaf --port 3308
memsql-ops memsql-optimizeand confirm the prompt. This will configure
numactlto bind MemSQL nodes to NUMA nodes, and configure MemSQL memory settings. If you have fewer leaf nodes on a machine than the number of NUMA nodes, the
memsql-optimizecommand will display a message like:
You should deploy 1 more leaf on this machine in order to maximize performance.
See MEMSQL-OPTIMIZE for more information.
Restart the cluster for the configuration changes to take effect.
memsql-ops memsql-restart --all
Reconfiguring an existing cluster for NUMA
Follow the steps in the previous section to deploy and optimize your leaf nodes.
Rebalance all MemSQL databases by running REBALANCE PARTITIONS on each database:
REBALANCE PARTITIONS ON <database>