Join the MemSQL Community Today
Get expert advice, develop skills, and connect with others.

MemSQL Replicate Overview

MemSQL is a next generation RDBMS that handles both transactional and analytical workloads, and offers a more cost-effective way to provide unequaled query ability at scale.

To best take advantage of this flexibility and scalability, data in existing databases can now be replicated into MemSQL, allowing data-driven applications to work with minimal changes to either code or data. This capability is especially important when the requirements of the original application have grown beyond the ability to scale the current database, or the database has simply become too costly to scale.

MemSQL Replicate is a data ingestion tool for replicating data from a source type into MemSQL, and provides a number of features and functions, including both filters and maps, to tailor how the data is replicated.

Supported Options

Replicate provides support for:

  • Oracle Enterprise Edition (EE) 11g and 12c including Compression and Partitioning and Real Application Clusters (RAC)
  • MemSQL v6.0 and later
  • A single destination type of MEMSQL
  • SQL 2017, 2016, and 2014 standards compliance
  • Customizable filters and maps to tailor the replicated data
  • A command-line interface for installation, configuration, and use

System Requirements

Replicate is currently officially supported on Linux kernel version 2.6.32 or higher.

Replicate requires Java 8. To check the version installed on your machine, run java -version at the command line. Replicate may also run with Java 9 installed, but it has not been tested with this version.

If the Java version is earlier than Java 8, or if Java is not installed, download the Java SE Runtime Environment 8 from www.java.com. Install the Java SE Runtime Environment 8 via the installation instructions provided by Oracle.

After installing Java SE Runtime Environment 8, check the installed Java version by running java -version on the command line to verify the installation was successful.

Since Replicate is a Java-based, it may be possible, though not guaranteed, to work on other operating systems that can run Java SE Runtime Environment 8. Support for other operating systems is planned for future releases.

Oracle Requirements

Use an appropriate JDBC driver for connecting to an Oracle database. As Oracle’s JDBC drivers are backward-compatible, the 18c (18.3) driver is required, even when working with earlier Oracle database versions. The Java 8-compatible ojdbc8 driver JAR file must be placed in the replicate/lib directory.

Configure Oracle

Configure Oracle and a specific Oracle user (oracle_user) to be used with Replicate.

Oracle 11g

  1. Connect to an Oracle user with sysdba privileges.

  2. Enable automatic archival.

    SHUTDOWN IMMEDIATE
    STARTUP MOUNT
    ALTER DATABASE ARCHIVELOG
    ALTER DATABASE OPEN
    
  3. Verify the database log mode by ARCHIVE LOG LIST.

  4. Set LogMiner permissions.

    ALTER DATABASE ADD SUPPLEMENTAL LOG DATA;
    ALTER DATABASE ADD SUPPLEMENTAL LOG DATA(primary key) COLUMNS;
    ALTER DATABASE FORCE LOGGING;
    ALTER SYSTEM SWITCH LOGFILE;
    
  5. Grant permission for parallel query.

    GRANT SELECT ON gv_$session TO oracle_user;
    
  6. Grant permission to obtain metadata.

    GRANT SELECT ON DBA_SEGMENTS TO oracle_user;
    
  7. Grant permission to analyze any table, cluster, or index in the database.

    GRANT ANALYZE ANY TO oracle_user;
    
  8. Grant permission to retrieve current SCN.

    GRANT SELECT ON gv_$database TO oracle_user;
    
  9. Grant permission to retrieve transaction start SCN.

    GRANT SELECT ON gv_$transaction TO oracle_user;
    
  10. Enable Flashback query.

    GRANT FLASHBACK ANY TABLE TO oracle_user;
    
  11. Grant permissions for change data capture (CDC).

    GRANT LOGMINING TO oracle_user;
    GRANT EXECUTE_CATALOG_ROLE TO oracle_user;
    GRANT SELECT ON v_$logmnr_contents TO oracle_user;
    

Oracle 12c

  1. Connect to an Oracle user with sysdba privileges.

  2. Execute ALTER SYSTEM SET common_user_prefix=’’ SCOPE=spfile;.

  3. Enable automatic archival.

    SHUTDOWN IMMEDIATE
    STARTUP MOUNT
    ALTER DATABASE ARCHIVELOG
    ALTER DATABASE OPEN
    
  4. Verify the database log mode by ARCHIVE LOG LIST.

  5. Set LogMiner permissions.

    ALTER DATABASE ADD SUPPLEMENTAL LOG DATA;
    ALTER DATABASE ADD SUPPLEMENTAL LOG DATA(primary key) COLUMNS;
    ALTER DATABASE FORCE LOGGING;
    ALTER SYSTEM SWITCH LOGFILE;
    GRANT LOGMINING TO oracle_user;
    GRANT SELECT ON v_$logmnr_contents TO oracle_user;
    
  6. Grant permission for parallel query.

    GRANT SELECT ON gv_$session TO oracle_user;
    
  7. Grant permission to obtain metadata.

    GRANT SELECT ON DBA_SEGMENTS TO oracle_user;
    
  8. Grant permission to analyze any table, cluster, or index in the database.

    GRANT ANALYZE ANY TO oracle_user;
    
  9. Grant permission to retrieve current SCN.

    GRANT SELECT ON gv_$database TO oracle_user;
    
  10. Grant permission to retrieve transaction start SCN.

    GRANT SELECT ON gv_$transaction TO oracle_user;
    
  11. Enable Flashback query.

    GRANT FLASHBACK ANY TABLE TO oracle_user;
    
  12. Grant permissions for change data capture (CDC).

    GRANT LOGMINING TO oracle_user;
    GRANT EXECUTE_CATALOG_ROLE TO oracle_user;
    GRANT SELECT ON v_$logmnr_contents TO oracle_user;
    

MemSQL Requirements

A valid MemSQL user that can read data from the source database.