This tutorial shows how to run OpenLogReplicator on Docker with Oracle Database 11g running on Docker. All steps are done on a Linux 64 machine. It is assumed that you have already installed Docker.
1. Oracle Database Docker image
Download Oracle binaries database binaries: Oracle Database 11gR2 Express Edition for Linux x64 and save them as oracle-xe-11.2.0-1.0.x86_64.rpm.zip.
Next, create Docker images with Database binaries. For that, you can download the Dockerfiles from Oracle github.
> git clone https://github.com/oracle/docker-images
> cd docker-images/OracleDatabase/SingleInstance/dockerfiles
# put your downloaded binaries to folder 11.2.0.2
> ls 11.2.0.2/oracle-xe-11.2.0-1.0.x86_64.rpm.zip
oracle-xe-11.2.0-1.0.x86_64.rpm.zip
> ./buildContainerImage.sh -v 11.2.0.2 -x
...
Successfully built 1bd064abd286
Successfully tagged oracle/database:11.2.0.2-xe
Oracle Database container image for 'xe' version 11.2.0.2 is ready to be extended:
--> oracle/database:11.2.0.2-xe
Build completed in 141 seconds.
> docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
oracle/database 11.2.0.2-xe 1bd064abd286 15 seconds ago 1.15GB
oraclelinux 7-slim 0a28ba78f4c9 3 weeks ago 132MBAs a result we have image to build Oracle database in a container. Step 1 is completed
2. OpenLogReplicator image setup
2 base images are available: for Ubuntu 20.04 and for CentOS 8. Let’s build Ubuntu image, because the image file is smaller. To build the image you need to download Oracle Instant Client libraries. You need 2 files: instantclient-basic-linux.x64-19.11.0.0.0dbru.zip and instantclient-sdk-linux.x64-19.11.0.0.0dbru.zip
> git clone https://github.com/bersler/OpenLogReplicator-docker
> cd OpenLogReplicator-docker
#copy the downloaded file here:
> ls *.zip
instantclient-basic-linux.x64-19.11.0.0.0dbru.zip instantclient-sdk-linux.x64-19.11.0.0.0dbru.zip
> docker build -t bersler/openlogreplicator:ubuntu-20.04 -f Dockerfile-ubuntu-20.04 .
...
+ su - user1 -c export LD_LIBRARY_PATH=/opt/instantclient_19_11:/opt/librdkafka/lib; /opt/OpenLogReplicator/OpenLogReplicator --version
2021-07-13 21:49:04 [INFO] OpenLogReplicator v.0.9.14-beta (C) 2018-2021 by Adam Leszczynski (aleszczynski@bersler.com), see LICENSE file for licensing information
...
Successfully built 7c013cc1ecb5
Successfully tagged bersler/openlogreplicator:ubuntu-20.04
> docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
oracle/database 11.2.0.2-xe 1bd064abd286 About a minute ago 1.15GB
bersler/openlogreplicator ubuntu-20.04 7c013cc1ecb5 12 minutes ago 460MB
ubuntu 20.04 9873176a8ff5 3 weeks ago 72.7MB
oraclelinux 7-slim 0a28ba78f4c9 3 weeks ago 132MBAs a result we have image to build OpenLogReplicator container. Step 2 is completed
3. Oracle Database 11 XE Docker container
Prepare configuration files for Oracle database:
> mkdir checkpoint
> mkdir fra
> mkdir oradata
> mkdir scripts
> mkdir scripts-db
> mkdir scripts-db/setup
> mkdir scripts-db/startupPlease mind that the docker image for Oracle database and OpenLogReplicator internally run as user with UID:1000/GID:1000. So, if you are running from a different user give write permissions to the created folders/files or change the owner:
> sudo chown -R 1000:1000 *And create scripts-db/startup/1.config.sql with the following content:
alter system set db_create_file_dest='/u01/app/oracle/oradata' scope = both;
alter system set db_recovery_file_dest='/u01/app/oracle/fast_recovery_area' scope = both;
alter system set db_recovery_file_dest_size='50g' scope = both;
SHUTDOWN IMMEDIATE;
STARTUP MOUNT;
ALTER DATABASE ARCHIVELOG;
ALTER DATABASE OPEN;
ALTER DATABASE ADD SUPPLEMENTAL LOG DATA;
ALTER SYSTEM SWITCH LOGFILE;
CREATE TABLESPACE TBLS1 DATAFILE SIZE 100M AUTOEXTEND ON NEXT 100M;
CREATE BIGFILE TABLESPACE TBLS2 DATAFILE SIZE 100M AUTOEXTEND ON NEXT 100M;
CREATE USER USR1 IDENTIFIED BY USR1PWD DEFAULT TABLESPACE "TBLS1" TEMPORARY TABLESPACE "TEMP";
ALTER USER USR1 QUOTA UNLIMITED ON TBLS1;
GRANT "CONNECT" TO USR1;
GRANT "RESOURCE" TO USR1;
GRANT SELECT, FLASHBACK ON SYS.CCOL$ TO USR1;
GRANT SELECT, FLASHBACK ON SYS.CDEF$ TO USR1;
GRANT SELECT, FLASHBACK ON SYS.COL$ TO USR1;
GRANT SELECT, FLASHBACK ON SYS.DEFERRED_STG$ TO USR1;
GRANT SELECT, FLASHBACK ON SYS.ECOL$ TO USR1;
GRANT SELECT, FLASHBACK ON SYS.OBJ$ TO USR1;
GRANT SELECT, FLASHBACK ON SYS.SEG$ TO USR1;
GRANT SELECT, FLASHBACK ON SYS.TAB$ TO USR1;
GRANT SELECT, FLASHBACK ON SYS.TABCOMPART$ TO USR1;
GRANT SELECT, FLASHBACK ON SYS.TABPART$ TO USR1;
GRANT SELECT, FLASHBACK ON SYS.TABSUBPART$ TO USR1;
GRANT SELECT, FLASHBACK ON SYS.USER$ TO USR1;
GRANT SELECT ON SYS.V_$ARCHIVED_LOG TO USR1;
GRANT SELECT ON SYS.V_$DATABASE TO USR1;
GRANT SELECT ON SYS.V_$DATABASE_INCARNATION TO USR1;
GRANT SELECT ON SYS.V_$LOG TO USR1;
GRANT SELECT ON SYS.V_$LOGFILE TO USR1;
GRANT SELECT ON SYS.V_$PARAMETER TO USR1;
GRANT SELECT ON SYS.V_$STANDBY_LOG TO USR1;
GRANT SELECT ON SYS.V_$TRANSPORTABLE_PLATFORM TO USR1;
CREATE USER USR2 IDENTIFIED BY USR1PWD DEFAULT TABLESPACE "TBLS2" TEMPORARY TABLESPACE "TEMP";
ALTER USER USR2 QUOTA UNLIMITED ON TBLS2;
GRANT "CONNECT" TO USR2;
GRANT "RESOURCE" TO USR2;Now it is time to create the database:
> docker run -d --name ORA1 -d --shm-size="1G" -p 4000:1521 \
-e ORACLE_SID=XE \
-e ORACLE_CHARACTERSET=AL32UTF8 \
-e ORACLE_PWD=Welcome1 \
-v `pwd`/fra:/u01/app/oracle/fast_recovery_area \
-v `pwd`/oradata:/u01/app/oracle/oradata \
-v `pwd`/scripts-db:/u01/app/oracle/scripts \
oracle/database:11.2.0.2-xeThe database should be up.
> docker logs -f ORA1
...
#########################
DATABASE IS READY TO USE!
#########################
...
Completed: CREATE BIGFILE TABLESPACE TBLS2 DATAFILE SIZE 100M AUTOEXTEND ON NEXT 100MMost of the time consuming hard work is done. Step 3 is completed.
4. OpenLogReplicator Docker Container
Prepare file scripts/OpenLogReplicator.json with the following content
{
"version": "0.9.14-beta",
"trace": 3,
"trace2": 256,
"sources": [
{
"alias": "S1",
"name": "DB1",
"reader": {
"type": "online",
"user": "USR1",
"password": "USR1PWD",
"server": "//192.168.1.10:4000/XE"
},
"format": {
"type": "json"
},
"checkpoint": {
"interval-mb": 100,
"output-checkpoint": 1
},
"debug": {
"owner": "USR1",
"table": "OPENLOGREPLICATOR",
"flush-buffer": 1
},
"flags": 64,
"memory-min-mb": 32,
"memory-max-mb": 1024,
"tables": [
{"owner": "USR1", "table": ".*"},
{"owner": "USR2", "table": ".*"}]
}
],
"targets": [
{
"alias": "T1",
"source": "S1",
"writer": {
"type": "file"
}
}
]
}Please replace the local network IP: 192.168.1.10 with yours. Please mind that you can’t use localhost here.
Let’s run OpenLogReplicator:
> docker run -d --name OLR1 \
-v `pwd`/fra:/u01/app/oracle/fast_recovery_area \
-v `pwd`/oradata:/u01/app/oracle/oradata \
-v `pwd`/scripts:/opt/OpenLogReplicator/scripts \
-v `pwd`/checkpoint:/opt/OpenLogReplicator/checkpoint \
bersler/openlogreplicator:ubuntu-20.04
d8365722f9abafb7b20c636db9c671492174a3d665a20daa716ad98cc830d0d3
> docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
d8365722f9ab bersler/openlogreplicator:ubuntu-20.04 "/bin/sh -c ./OpenLo…" 5 seconds ago Up 4 seconds OLR1
1e18ea369ab1 oracle/database:11.2.0.2-xe "/bin/sh -c 'exec $O…" 21 minutes ago Up 21 minutes (healthy) 0.0.0.0:4000->1521/tcp, :::4000->1521/tcp ORA1
> docker logs -f OLR1
2021-07-13 22:26:26 [INFO] OpenLogReplicator v.0.9.14-beta (C) 2018-2021 by Adam Leszczynski (aleszczynski@bersler.com), see LICENSE file for licensing information
2021-07-13 22:26:26 [INFO] adding source: S1
2021-07-13 22:26:26 [INFO] will shutdown after committed DML in USR1.OPENLOGREPLICATOR
2021-07-13 22:26:26 [INFO] adding target: T1
2021-07-13 22:26:26 [INFO] connecting to Oracle instance of DB1 to //192.168.10.18:4000/XE
2021-07-13 22:26:26 [INFO] writer is starting: stdout
2021-07-13 22:26:26 [INFO] version: Oracle Database 11g Express Edition Release 11.2.0.2.0 - 64bit Production, context: XE, resetlogs: 1077833148, activation: 2958962042, con_id: 0, con_name:
2021-07-13 22:26:26 [INFO] loading character mapping for AL32UTF8
2021-07-13 22:26:26 [INFO] loading character mapping for AL16UTF16
2021-07-13 22:26:26 [INFO] Oracle Analyzer for DB1 in online mode is starting (flags: 64) from scn: 0
2021-07-13 22:26:26 [INFO] starting sequence not found - starting with new batch with seq: 5
2021-07-13 22:26:26 [INFO] reading dictionaries for scn: 357748
2021-07-13 22:26:26 [INFO] - found: SYS.CCOL$ (dataobj: 29, obj: 32, cols: 11), part of cluster
2021-07-13 22:26:26 [INFO] - found: SYS.CDEF$ (dataobj: 29, obj: 31, cols: 21), part of cluster
2021-07-13 22:26:26 [INFO] - found: SYS.COL$ (dataobj: 2, obj: 21, cols: 24), part of cluster
2021-07-13 22:26:27 [INFO] - found: SYS.DEFERRED_STG$ (dataobj: 132, obj: 132, cols: 22)
2021-07-13 22:26:27 [INFO] - found: SYS.ECOL$ (dataobj: 123, obj: 123, cols: 3)
2021-07-13 22:26:27 [INFO] - found: SYS.OBJ$ (dataobj: 18, obj: 18, cols: 21)
2021-07-13 22:26:27 [INFO] - found: SYS.SEG$ (dataobj: 8, obj: 14, cols: 20), part of cluster
2021-07-13 22:26:27 [INFO] - found: SYS.TAB$ (dataobj: 2, obj: 4, cols: 37), part of cluster
2021-07-13 22:26:27 [INFO] - found: SYS.TABPART$ (dataobj: 568, obj: 568, cols: 26)
2021-07-13 22:26:27 [INFO] - found: SYS.TABCOMPART$ (dataobj: 590, obj: 590, cols: 35)
2021-07-13 22:26:27 [INFO] - found: SYS.TABSUBPART$ (dataobj: 580, obj: 580, cols: 26)
2021-07-13 22:26:27 [INFO] - found: SYS.USER$ (dataobj: 10, obj: 22, cols: 25), part of cluster
2021-07-13 22:26:27 [INFO] last confirmed scn: 357748, starting sequence: 5, offset: 0
2021-07-13 22:26:27 [INFO] found redo log version: 0x0b200000
2021-07-13 22:26:27 [INFO] processing redo log: group: 6 scn: 356069 to 0 seq: 5 path: /u01/app/oracle/oradata/XE/redo06.log offset: 1024
{"scn":356069,"tm":1626213996000,"payload":[{"op":"chkpt","seq":5,"offset":1536}]}
All looks good. Keep the window open and open another one with Oracle SQL client:
> docker exec -ti ORA1 sqlplus sys/Welcome1@XE as sysdba
SQL*Plus: Release 11.2.0.2.0 Production on Tue Jul 13 22:28:04 2021
Copyright (c) 1982, 2011, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Express Edition Release 11.2.0.2.0 - 64bit Production
SQL> create table USR1.TEST1(a varchar2(5), b numeric(5));
Table created.
SQL> insert into USR1.TEST1 values('test', 101);
1 row created.
SQL> commit;
Commit complete.In the window with log OpenLogReplicator you should see the following log messages:
2021-07-13 22:30:36 [INFO] updated schema: USR1.TEST1 (dataobj: 20038, obj: 20038, cols: 2), primary key missing
{"scn":357888,"tm":1626215447000,"xid":"2.29.386","payload":[{"op":"begin"}]}
{"xid":"2.29.386","payload":[{"op":"c","schema":{"owner":"USR1","table":"TEST1"},"after":{"A":"test","B":101}}]}
{"xid":"2.29.386","payload":[{"op":"commit"}]}This concludes the last step and the first tutorial.