You are viewing an older version of this section. View current production version.

Maps

By default, all data fetched from the specified source is replicated to MemSQL. A map can be used to precisely defines how the source data is replicated.

Use --map <mapper_file> to define the desired mapping. If this parameter is not specified, a best-effort mapping is attempted in order to maintain the source data structure.

Example:

./bin/replicate snapshot conf/conn/oracle.yaml conf/conn/memsql.yaml --map custom_mapping.yaml

Replicate attempts to maintain the source data structure by using the same names for all replicated objects (catalogs, schemas, tables, columns). However, in cases where the source uses schemas but the destination uses catalogs, a different mapping may be required to reconcile different multi-tenancy approaches.

A mapper file contains a set of rules that define how the data fetched from the source, as defined by the associated filter, is replicated to MemSQL, where each rule applies to a single destination catalog.

A list of source catalogs/schemas can be defined for each destination catalog, where the contents of each source catalog/schema will be mapped into the destination catalog.

By using the table-store key, it’s possible to specify that the destination tables are created as either row-store (by specifying ROW) or column-store (by specifying COLUMN). This rule not only applies to those tables that are either explicitly or implicitly specified in the mapper file, but to all tables that are being replicated. Use --dst-schemas to override this option for individual tables. Note: This has no effect for tables where --append-existing or --truncate-existing is used. If the source database also supports row-store and column-store tables, DEFAULT can be specified to preserve the existing source definition for each table.

File Format

table-store: [ROW || COLUMN]
rules:
  # Name of the destination database
  <dst_database_name>:    
    # Names of the source schemas
    source:
    - <src_schema_name>
    - <src_schema_name>
  # Name of the destination database
  <dst_database_name>:
    # Names of the source schemas
    source:
    - <src_schema_name>
    - <src_schema_name>

Example

Oracle to MemSQL

table-store: COLUMN
rules:
  test-org:
    source:
    - [REPLICATE]
  io:
    source:
    - [SOE]

All tables from Oracle’s REPLICATE schema are replicated to MemSQL’s test-org database, while all tables from Oracle’s SOE schema are replicated to MemSQL’s io database. All tables on MemSQL are created as column-store.