Memory Errors

ERROR 2350: Leaf or aggregator node could not be added because you are using the MemSQL free license which has a total memory limit of 128.00 GB

Issue

The total combined RAM allocated to the nodes in your cluster is larger than the 128 GB limit imposed by the free license. You will encounter this error when trying to add a new node to the cluster, which can happen during initial deployment, or when expanding your cluster.

Solutions

There are a few potential solutions depending on what your needs are:

  • If you need a cluster with more than 128 GB of RAM, you will need an Enterprise license. Sign up for a free 30-day trial and create an Enterprise License trial key before deploying a larger cluster.

  • Redeploy the cluster on a smaller set of machines that will keep you under the 128 GB limit.

  • Deploy your cluster manually using these steps, but with the important following change:

Before you assign a new node a role (memsql-admin add-leaf or memsql-admin add-aggregator), change the maximum amount of memory that will be allocated to that node. This can be done by running memsql-admin optimize --memory-percentage <value> and setting value to some percentage that will keep the total combined RAM size of the cluster lower than 128 GB. For example, the default percentage of RAM that a MemSQL node will reserve for itself is 90% of the physical RAM for that host machine. If that host machine has 32 GB of physical RAM, the default amount of RAM allocated for the node will be 28 GB (rounded down). If you have six host machines with 28 GB of available memory, the total available memory for your cluster would be 6 machines * 28 GB/machine = 168 GB.

To keep it under the 128 GB limit, you would run memsql-admin optimize --memory-percentage 65 after running memsql-admin create-node. The equation for this is 128 GB / 6 machines = 21 GB RAM per machine (rounded down) and 21 GB RAM / 32 GB RAM per machine = 65% of physical RAM can be allocated to the node per machine (rounded down). When created, this will give you a six-node cluster (assuming one node per host machine) with 126 GB of total available RAM. Note: In general, you should round down in your equation so you do not hit the memory limit when performing your deployment operation.

If you have NUMA-capable CPUs on your host machine, and wish to run more than one node per host machine, then your --memory-percent value would have to be reduced accordingly.

Memory allocation is calculated as follows:

  • One node, default settings: Use 90% of physical memory
  • Two nodes, default settings: Use (90% of physical memory) / 2 per node
  • One node, 50% memory specified: Use 50% of physical memory
  • Two nodes, 50% memory specified: Use (50% of physical memory) / 2 per node

ERROR 1712: Not enough memory available to complete the current request. The request was not processed.

For potential causes and solutions, see Identifying and Reducing Memory Usage.

ERROR: 1720 - “Memory usage by MemSQL for tables (XXXXX MB) has reached the value of ‘maximum_table_memory’ global variable (YYYYY MB). This query cannot be executed.”

For potential causes and solutions, see Identifying and Reducing Memory Usage.

ERROR: “Nonfatal buffer manager memory allocation failure. The maximum_memory parameter (XXXXX MB) has been reached.

For potential causes and solutions, see Identifying and Reducing Memory Usage.

Was this article useful?