Create the Object Definition Files min read


The following are definition files that will be used by the MemSQL Operator to create your cluster. Create new definition files and copy and paste the contents of each code block into those files.

Info

The deployment.yaml and memsql-cluster.yaml files have placeholders that must be updated before they can be used.

deployment.yaml

Create a deployment definition file using the template below.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: memsql-operator
spec:
  replicas: 1
  selector:
    matchLabels:
      name: memsql-operator
  template:
    metadata:
      labels:
        name: memsql-operator
    spec:
      serviceAccountName: memsql-operator
      containers:
        - name: memsql-operator
          image: "OPERATOR_IMAGE"
          imagePullPolicy: Always
          args: [
            # Cause the operator to merge rather than replace annotations on services
            "--merge-service-annotations",
            # Allow the process inside the container to have read/write access to the `/var/lib/memsql` volume.
            "--fs-group-id", "5555"
          ]
          env:
            - name: WATCH_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: OPERATOR_NAME
              value: "memsql-operator"

You must edit this file and replace OPERATOR_IMAGE with the either the local memsql/operator Docker image you pulled down (such as "memsql-operator"), or add in an imagePullSecrets section under the spec section and reference a Kubernetes Secret that you can create via kubectl apply.

Refer to the Kubernetes documentation for more information on imagePullPolicy and creating Secrets.

rbac.yaml

Copy the following to create a ServiceAccount definition file that will be used by the MemSQL Operator.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: memsql-operator
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: memsql-operator
rules:
- apiGroups:
  - ""
  resources:
  - pods
  - services
  - endpoints
  - persistentvolumeclaims
  - events
  - configmaps
  - secrets
  verbs:
  - '*'
- apiGroups:
  - policy
  resources:
  - poddisruptionbudgets
  verbs:
  - '*'
- apiGroups:
  - batch
  resources:
  - cronjobs
  verbs:
  - '*'
- apiGroups:
  - ""
  resources:
  - namespaces
  verbs:
  - get
- apiGroups:
  - apps
  - extensions
  resources:
  - deployments
  - daemonsets
  - replicasets
  - statefulsets
  verbs:
  - '*'
- apiGroups:
  - memsql.com
  resources:
  - '*'
  verbs:
  - '*'
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: memsql-operator
subjects:
- kind: ServiceAccount
  name: memsql-operator
roleRef:
  kind: Role
  name: memsql-operator
  apiGroup: rbac.authorization.k8s.io

memsql-cluster-crd.yaml

Create a CustomResourceDefinition file to define the MemSQLCluster resource type.

apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: memsqlclusters.memsql.com
spec:
  group: memsql.com
  names:
    kind: MemsqlCluster
    listKind: MemsqlClusterList
    plural: memsqlclusters
    singular: memsqlcluster
    shortNames:
      - memsql
  scope: Namespaced
  version: v1alpha1
  subresources:
    status: {}
  additionalPrinterColumns:
  - name: Aggregators
    type: integer
    description: Number of MemSQL Aggregators
    JSONPath: .spec.aggregatorSpec.count
  - name: Leaves
    type: integer
    description: Number of MemSQL Leaves (per availability group)
    JSONPath: .spec.leafSpec.count
  - name: Redundancy Level
    type: integer
    description: Redundancy level of MemSQL Cluster
    JSONPath: .spec.redundancyLevel
  - name: Age
    type: date
    JSONPath: .metadata.creationTimestamp

memsql-cluster.yaml

Create a MemSQLCluster definition file to specify the configuration settings for your MemSQL cluster.

apiVersion: memsql.com/v1alpha1
kind: MemsqlCluster
metadata:
  name: memsql-cluster
spec:
  license: LICENSE_KEY
  adminHashedPassword: "HASHED_PASSWORD"
  nodeImage:
    repository: memsql/node
    tag: 6.8.9-24b9cbd386

  redundancyLevel: 1

  serviceSpec:
    objectMetaOverrides:
      labels:
        custom: label
      annotations:
        custom: annotations

  aggregatorSpec:
    count: 3
    height: 0.5
    storageGB: 256
    storageClass: standard

    objectMetaOverrides:
      annotations:
        optional: annotation
      labels:
        optional: label

  leafSpec:
    count: 1
    height: 1
    storageGB: 1024
    storageClass: standard

    objectMetaOverrides:
      annotations:
        optional: annotation
      labels:
        optional: label

You must edit the following placeholders in this file to properly set up your cluster:

  • Change the name value to the cluster name of your choice.

  • Specify your license key and a hashed version of a secure password for the admin database user on the cluster. The admin user is the default user you can use when logging into your cluster. The account is created by the Operator during cluster deployment and has an explicit set of grants (defined at the end of this step) to reduce the scope of this user.

    Note that, as of MemSQL 7.1.4, license checks are now cgroup-aware and respect container resource boundaries for containerized deployments. While this does not change how license checks are performed, nor does it change how capacity is allocated, it does change how the resources allocated to the container are checked.

    To include the license key and hashed password, you have the following two options:

    • Replace LICENSE_KEY with your license key from the MemSQL Customer Portal and change HASHED_PASSWORD to a hashed version of a secure password for the admin database user on the cluster. The following python script shows how to create a hashed password:

      from hashlib import sha1
      print("*" + sha1(sha1('secretpass').digest()).hexdigest().upper())
      
    • Use Kubernetes secrets to pass in the license key and hashed passwords. You will need one or two secrets (they can be separate or the same secret) with the keys set to the correct values. Note: The password still needs to be hashed inside the secret. It cannot be a bare password.

      licenseSecret:
        name: "my-secret"
        key: "license"
      adminHashedPasswordSecret:
        name: "my-secret"
        key: "password"
      
  • Under nodeImage, tag specifies the version of memsql/node that will be deployed in your cluster. This value aligns with the version number of the MemSQL database engine that is running in the container (e.g. 6.8.9-24b9cbd386 contains the 6.8.9 version of MemSQL).

    You can use different versions of the engine by going to the Docker Hub page for memsql/node and selecting a different tag. Because of recent updates to the memsql/node container, you should select a tag that is 6.8.9-24b9cbd386 or newer. Also, running different versions of the MemSQL engine in one cluster is not supported, so you must ensure that each node in your Kubernetes cluster has the same tag value.

  • Change redundancyLevel to 2 if you want to enable high availability. It is highly recommended you set this value to 2 for production deployments. Note: You must have an even number of leaf nodes to enable high availability. Refer to Managing High Availability for more information.

  • The objectMetaOverrides sections are optional. By including these sections, you can override the metadata annotations and labels at either the node or service layer (objectMetaOverrides).

  • Change count to alter the number of aggregator or leaf nodes in your cluster.

  • The height value specifies the vCPU and RAM size of an aggregator or leaf node where a height of 1 equals 8 vCPU cores and 32 GB of RAM. The smallest value you can set is 0.5 (4 vCPU cores, 16 GB of RAM).

  • The storageGB value corresponds to the amount of storage each aggregator or leaf should request for their persistent data volume.

  • The storageClass value specifies which storage class to use for the PersistedVolume in the Kubernetes cluster. You should change this value to align with the default (or custom) storage class available to your cluster.

  • For advanced users, you can also declare a globalVariables section. This is an optional section that allows you to specify values for MemSQL engine variables.

    • Prior to Operator 1.2.0, the supported engine variables are:
      • default_partitions_per_leaf
      • columnstore_segment_rows
      • columnstore_flush_bytes
      • columnstore_window_size
      • transaction_buffer
      • snapshot_trigger_size
      • minimal_disk_space
      • pipelines_max_concurrent
      • auditlog_level
    • As of Operator 1.2.0, nearly all engine variables are supported except for:
      • redundancy_level
      • sync_permissions
      • local_file_system_access_restricted
    • As of Operator 1.2.1, nearly all engine variables are supported except for:
      • redundancy_level
      • sync_permissions
    • Refer to List of Engine Variables for more information.
    globalVariables:
      transaction_buffer: "8m"
      default_partitions_per_leaf: "4"
    
    Info

    If it is not overridden in the globalVariables section, the Operator will set default_partitions_per_leaf to a value equal to the height multiplied by the vCPU cores per unit.

  • Users may declare an envVariables section. This is an optional section that allows environment variables to be specified. The currently supported environment variables include:

    • MALLOC_ARENA_MAX
    envVariables:
      MALLOC_ARENA_MAX: 4
    

Once you have finished creating your definition files, you can deploy your cluster.

To learn more about a configuration option below while remaining on this page, right-click on the configuration option and open it in a separate tab.

The admin user has the following database permissions:

  • USAGE
  • SELECT
  • INSERT
  • UPDATE
  • DELETE
  • CREATE
  • DROP
  • PROCESS
  • INDEX
  • ALTER
  • SHOW METADATA
  • CREATE TEMPORARY TABLES
  • LOCK TABLES
  • CREATE VIEW
  • ALTER VIEW
  • DROP VIEW
  • SHOW VIEW
  • DROP DATABASE
  • CREATE DATABASE
  • CREATE ROUTINE
  • ALTER ROUTINE
  • EXECUTE
  • CREATE PIPELINE
  • DROP PIPELINE
  • START PIPELINE
  • ALTER PIPELINE
  • SHOW PIPELINE