Database In Memory

Saturday, March 5, 2016

CRS-8503 [] [] [] [] [] [] [] [] [] [] [] []

2016-03-05 20:02:45.720 [ORAROOTAGENT(26842)]CRS-8500: Oracle Clusterware ORAROOTAGENT process is starting with operating system process ID 26842

2016-03-05 20:02:53.572 [CLSECHO(27164)]CRS-10001: 05-Mar-16 20:02 AFD-9204: false

2016-03-05 20:03:45.967 [ORAROOTAGENT(26842)]CRS-5818: Aborted command 'check' for resource 'ora.storage'. Details at (:CRSAGF00113:) {0:21:2} in /u01/app/oracle/diag/crs/tnc2/crs/trace/ohasd_orarootagent_root.trc.

2016-03-05 20:04:46.133 [ORAROOTAGENT(28430)]CRS-8500: Oracle Clusterware ORAROOTAGENT process is starting with operating system process ID 28430

2016-03-05 20:05:01.921 [MDNSD(4300)]CRS-5602: mDNS service stopping by request.

2016-03-05 20:05:05.559 [MDNSD(4300)]CRS-8504: Oracle Clusterware MDNSD process with operating system process ID 4300 is exiting

2016-03-05 20:05:10.100 [OHASD(3131)]CRS-10301: Unable to contact Oracle high availability service on peer nodes for cluster wide commands. Details at (:CCL00001:) in /u01/app/oracle/diag/crs/tnc2/crs/trace/ohasd.trc.

2016-03-05 20:05:46.196 [ORAROOTAGENT(28430)]CRS-5818: Aborted command 'check' for resource 'ora.storage'. Details at (:CRSAGF00113:) {0:23:2} in /u01/app/oracle/diag/crs/tnc2/crs/trace/ohasd_orarootagent_root.trc.

2016-03-05 20:06:46.354 [ORAROOTAGENT(28952)]CRS-8500: Oracle Clusterware ORAROOTAGENT process is starting with operating system process ID 28952

2016-03-05 20:07:46.412 [ORAROOTAGENT(28952)]CRS-5818: Aborted command 'check' for resource 'ora.storage'. Details at (:CRSAGF00113:) {0:25:2} in /u01/app/oracle/diag/crs/tnc2/crs/trace/ohasd_orarootagent_root.trc.

2016-03-05 20:08:46.588 [ORAROOTAGENT(29487)]CRS-8500: Oracle Clusterware ORAROOTAGENT process is starting with operating system process ID 29487

2016-03-05 20:09:46.644 [ORAROOTAGENT(29487)]CRS-5818: Aborted command 'check' for resource 'ora.storage'. Details at (:CRSAGF00113:) {0:27:2} in /u01/app/oracle/diag/crs/tnc2/crs/trace/ohasd_orarootagent_root.trc.

2016-03-05 20:10:46.819 [ORAROOTAGENT(30584)]CRS-8500: Oracle Clusterware ORAROOTAGENT process is starting with operating system process ID 30584

2016-03-05 20:10:46.857 [OCSSD(21147)]CRS-1656: The CSS daemon is terminating due to a fatal error; Details at (:CSSSC00012:) in /u01/app/oracle/diag/crs/tnc2/crs/trace/ocssd.trc

2016-03-05 20:10:46.857 [OCSSD(21147)]CRS-1603: CSSD on node tnc2 shutdown by user.

2016-03-05 20:10:46.856 [CSSDAGENT(21135)]CRS-5818: Aborted command 'start' for resource 'ora.cssd'. Details at (:CRSAGF00113:) {0:15:27} in /u01/app/oracle/diag/crs/tnc2/crs/trace/ohasd_cssdagent_root.trc.

2016-03-05 20:10:47.001 [ORAAGENT(4287)]CRS-5818: Aborted command 'clean' for resource 'ora.asm'. Details at (:CRSAGF00113:) {0:9:6} in /u01/app/oracle/diag/crs/tnc2/crs/trace/ohasd_oraagent_oracle.trc.

2016-03-05 20:10:51.009 [ORAAGENT(4287)]CRS-5011: Check of resource "ora.asm" failed: details at "(:CLSN00006:)" in "/u01/app/oracle/diag/crs/tnc2/crs/trace/ohasd_oraagent_oracle.trc"

2016-03-05 20:10:51.013 [OHASD(3131)]CRS-2757: Command 'Clean' timed out waiting for response from the resource 'ora.asm'. Details at (:CRSPE00163:) {0:9:6} in /u01/app/oracle/diag/crs/tnc2/crs/trace/ohasd.trc.

Sat Mar 05 20:10:51 2016

Errors in file /u01/app/oracle/diag/crs/tnc2/crs/trace/ocssd.trc (incident=193):

CRS-8503 [] [] [] [] [] [] [] [] [] [] [] []

Incident details in: /u01/app/oracle/diag/crs/tnc2/crs/incident/incdir_193/ocssd_i193.trc

2016-03-05 20:10:52.062 [OCSSD(21147)]CRS-8503: Oracle Clusterware OCSSD process with operating system process ID 21147 experienced fatal signal or exception code 6

Sweep [inc][193]: completed

2016-03-05 20:10:57.452 [OHASD(3131)]CRS-2757: Command 'Start' timed out waiting for response from the resource 'ora.cssd'. Details at (:CRSPE00163:) {0:15:27} in /u01/app/oracle/diag/crs/tnc2/crs/trace/ohasd.trc.

2016-03-05 20:10:57.541 [CSSDAGENT(30935)]CRS-8500: Oracle Clusterware CSSDAGENT process is starting with operating system process ID 30935

2016-03-05 20:10:57.696 [OHASD(3131)]CRS-2878: Failed to restart resource 'ora.asm'

2016-03-05 20:10:57.711 [OHASD(3131)]CRS-2878: Failed to restart resource 'ora.cssd'

2016-03-05 20:10:57.721 [GPNPD(4339)]CRS-2329: GPNPD on node tnc2 shut down.

2016-03-05 20:10:57.735 [GPNPD(4339)]CRS-8504: Oracle Clusterware GPNPD process with operating system process ID 4339 is exiting

2016-03-05 20:10:58.862 [GIPCD(4408)]CRS-8504: Oracle Clusterware GIPCD process with operating system process ID 4408 is exiting

2016-03-05 20:23:00.782 [CLSECHO(9793)]CRS-10001: 05-Mar-16 20:23 AFD-9204: false

2016-03-05 21:32:19.942 [CLSECHO(11011)]CRS-10001: 05-Mar-16 21:32 AFD-9204: false

$ tail -100f /u01/app/oracle/diag/crs/tnc2/crs/trace/ocssd.trc

016-03-05 20:10:50.299585 : CSSD:1096018240: clssscWaitOnEventValue: after CmInfo State val 3, eval 1 waited 1010 with cvtimewait status 4294967186

2016-03-05 20:10:50.401486 : CSSD:1121986880: clssnmvDHBValidateNCopy: node 1, tnc1, has a disk HB, but no network HB, DHB has rcfg 352673332, wrtcnt, 1011537, LATS 2188456, lastSeqNo 1011534, uniqueness 1457225306, timestamp 1457226649/2189006

2016-03-05 20:10:50.401595 : CSSD:1118832960: clssnmvDHBValidateNCopy: node 1, tnc1, has a disk HB, but no network HB, DHB has rcfg 352673332, wrtcnt, 1011538, LATS 2188456, lastSeqNo 1011535, uniqueness 1457225306, timestamp 1457226649/2189396

2016-03-05 20:10:50.416981 : CSSD:1085483328: clssnmvDHBValidateNCopy: node 1, tnc1, has a disk HB, but no network HB, DHB has rcfg 352673332, wrtcnt, 1011539, LATS 2188466, lastSeqNo 1011536, uniqueness 1457225306, timestamp 1457226649/2189446

2016-03-05 20:10:51.303616 : CSSD:1096018240: clssscWaitOnEventValue: after CmInfo State val 3, eval 1 waited 1000 with cvtimewait status 4294967186

2016-03-05 20:10:51.351707 :GIPCHTHR:1114102080: gipchaWorkerWork: workerThread heart beat, time interval since last heartBeat 30130loopCount 32

2016-03-05 20:10:51.404980 : CSSD:1121986880: clssnmvDHBValidateNCopy: node 1, tnc1, has a disk HB, but no network HB, DHB has rcfg 352673332, wrtcnt, 1011540, LATS 2189456, lastSeqNo 1011537, uniqueness 1457225306, timestamp 1457226650/2190006

2016-03-05 20:10:51.405087 : CSSD:1118832960: clssnmvDHBValidateNCopy: node 1, tnc1, has a disk HB, but no network HB, DHB has rcfg 352673332, wrtcnt, 1011541, LATS 2189456, lastSeqNo 1011538, uniqueness 1457225306, timestamp 1457226650/2190406

2016-03-05 20:10:51.420767 : CSSD:1085483328: clssnmvDHBValidateNCopy: node 1, tnc1, has a disk HB, but no network HB, DHB has rcfg 352673332, wrtcnt, 1011542, LATS 2189476, lastSeqNo 1011539, uniqueness 1457225306, timestamp 1457226650/2190446

Trace file /u01/app/oracle/diag/crs/tnc2/crs/trace/ocssd.trc

DDE: Flood control is not active

Incident 193 created, dump file: /u01/app/oracle/diag/crs/tnc2/crs/incident/incdir_193/ocssd_i193.trc

CRS-8503 [] [] [] [] [] [] [] [] [] [] [] []

CLSB:1106217280: Oracle Clusterware infrastructure error in OCSSD (OS PID 21147): Fatal signal 6 has occurred in program ocssd thread 1106217280; nested signal count is 1

Action:
Initially Analyze the alert log files and trace files

Start investigating on Failure Node and Living node in a cluster, Possible reasons for node eviction as follows
1) Check alert log file on Node 1, Some times If there is problem with Interconnect, Node 1 kicks out Node 2 from the cluster.
2) Blocking port can Make Node 2 Evicts from Cluster
Stop and Start CRS on the Failure Node will resolve the issue

--Nikhil Tatineni--
--12c RAC Cluster--

Friday, March 4, 2016

Oracle GoldenGate: Creating Virtual IP address (VIP) to manage high availability

Step1:

Update /etc/hosts file with new Goldengate VIP

vi /etc/hosts

#Goldengate

192.168.1.23 goldengate-vip gg-vip

##Update /etc/hosts Completed##

Step2:

Create new vip with root user using "appvipcfg"

/u01/app/12.1.0.2/grid/bin/appvipcfg create -network=1 -ip=192.168.1.23 -vipname=goldengate-vip -user=root

#-----

[root@tnc1 ~]# crsctl stat res -p |grep -ie .network -ie subnet |grep -ie name -ie subnet

REGISTRATION_INVITED_SUBNETS=

NAME=ora.net1.network

USR_ORA_SUBNET=192.168.1.0

Step3:

Set permissions for oracle on new goldengate VIP

[root@tnc1 ~]# /u01/app/12.1.0.2/grid/bin/crsctl setperm resource goldengate-vip -u user:oracle:r-x

#------

checking status of resources

[root@tnc1 ~]# crsctl status resource -t

--------------------------------------------------------------------------------

Name Target State Server State details

--------------------------------------------------------------------------------

Local Resources

--------------------------------------------------------------------------------

ora.FRA.dg

ONLINE ONLINE tnc1 STABLE

ONLINE ONLINE tnc2 STABLE

ora.GRID.dg

ONLINE ONLINE tnc1 STABLE

ONLINE ONLINE tnc2 STABLE

ora.LISTENER.lsnr

ONLINE ONLINE tnc1 STABLE

ONLINE OFFLINE tnc2 STABLE

ora.NEVOT.dg

ONLINE ONLINE tnc1 STABLE

ONLINE ONLINE tnc2 STABLE

ora.OCNEW.dg

ONLINE ONLINE tnc1 STABLE

ONLINE ONLINE tnc2 STABLE

ora.SDXH.dg

ONLINE ONLINE tnc1 STABLE

ONLINE ONLINE tnc2 STABLE

ora.asm

ONLINE ONLINE tnc1 Started,STABLE

ONLINE ONLINE tnc2 Started,STABLE

ora.net1.network

ONLINE ONLINE tnc1 STABLE

ONLINE OFFLINE tnc2 STABLE

ora.ons

ONLINE ONLINE tnc1 STABLE

ONLINE OFFLINE tnc2 STABLE

--------------------------------------------------------------------------------

Cluster Resources

--------------------------------------------------------------------------------

goldengate-vip

1 OFFLINE OFFLINE STABLE

ora.LISTENER_SCAN1.lsnr

Step4:

Start new Goldengate VIP with Oracle User:

[oracle@tnc1 ~]$ crsctl start resource goldengate-vip

CRS-2672: Attempting to start 'goldengate-vip' on 'tnc2'

CRS-2676: Start of 'goldengate-vip' on 'tnc2' succeeded

[oracle@tnc1 ~]$ crsctl status resource goldengate-vip

NAME=goldengate-vip

TYPE=app.appvipx.type

TARGET=ONLINE

STATE=ONLINE on tnc2

Step5:

ping new Goldengate VIP

[oracle@tnc1 ~]$ ping -c4 goldengate-vip

PING goldengate-vip (10.10.10.191) 56(84) bytes of data.

64 bytes from goldengate-vip (10.10.10.191): icmp_seq=1 ttl=64 time=2.43 ms

64 bytes from goldengate-vip (10.10.10.191): icmp_seq=2 ttl=64 time=0.328 ms

64 bytes from goldengate-vip (10.10.10.191): icmp_seq=3 ttl=64 time=0.458 ms

64 bytes from goldengate-vip (10.10.10.191): icmp_seq=4 ttl=64 time=0.476 ms

--- goldengate-vip ping statistics ---

4 packets transmitted, 4 received, 0% packet loss, time 3010ms

rtt min/avg/max/mdev = 0.328/0.923/2.431/0.872 ms

New VIP is configured for Goldengate for Automatic Failover

--Nikhil Tatineni--

--12c GoldenGate--

ORA-00265:instance recovery required,

Scenario: Converting database from no archive log to archive log mode

SQL> alter database archivelog;

alter database archivelog

ERROR at line 1:

ORA-00265:instance recovery required, cannot set ARCHIVELOG mode

Performed Instance Recovery as requested and converted database to archivelog Mode

SQL> alter database open;

Database altered.

SQL> shut immediate;

Database closed.

Database dismounted.

ORACLE instance shut down.

SQL> alter database mount;

alter database mount

ERROR at line 1:

ORA-01034: ORACLE not available

Process ID: 0

Session ID: 0 Serial number: 0

SQL> exit

Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production

With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,

Advanced Analytics and Real Application Testing options

[oracle@tnc1 ~]$

[oracle@tnc1 ~]$ db

[oracle@tnc1 ~]$ sql

SQL*Plus: Release 12.1.0.2.0 Production on Thu Mar 3 01:43:52 2016

Connected to an idle instance.

SQL> startup mount;

ORACLE instance started.

Total System Global Area 549453824 bytes

Fixed Size 2926616 bytes

Variable Size 293603304 bytes

Database Buffers 247463936 bytes

Redo Buffers 5459968 bytes

Database mounted.

SQL> alter database archivelog;

Database altered.

---Nikhil Tatineni--

---12C Database---

GoldenGate ERROR OGG-01224

Found All ogg process are abended on Goldengate Home

started the manager and again it is abended

View report file of the mgr

GGSCI (tnc1.localdomain) 4> info all

Program Status Group Lag at Chkpt Time Since Chkpt

MANAGER STOPPED

EXTRACT ABENDED ECBN 00:00:09 00:32:52

REPLICAT ABENDED RCBN 00:00:00 00:50:36

GGSCI (tnc1.localdomain) 19> view report mgr

***********************************************************************

Oracle GoldenGate Manager for Oracle

Version 12.2.0.1.1 OGGCORE_12.2.0.1.0_PLATFORMS_151211.1401

Linux, x64, 64bit (optimized), Oracle 12c on Dec 11 2015 16:10:04

2016-03-04 10:43:08 WARNING OGG-01877 Missing explicit accessrule for server collector.

2016-03-04 10:43:08 INFO OGG-00960 Access granted (rule #5).

Source Context :

SourceModule : [mgr.main]

SourceID : [/scratch/aime/adestore/views/aime_adc4150408/oggcore/OpenSys/src/app/mgr/mgr.c]

SourceFunction : [init_functions]

SourceLine : [1600]

ThreadBacktrace : [12] elements

: [/u02/goldengate/12.1/libgglog.so(CMessageContext::AddThreadContext()+0x1b) [0x2b54d81560

bb]]

: [/u02/goldengate/12.1/libgglog.so(CMessageFactory::CreateMessage(CSourceContext*, unsigne

d int, ...)+0x134) [0x2b54d8150254]]

: [/u02/goldengate/12.1/libgglog.so(_MSG_ERR_TCP_GENERIC(CSourceContext*, char const*, CMes

sageFactory::MessageDisposition)+0x30) [0x2b54d81327b7]]

: [./mgr(init_functions(int, char**)+0xa51) [0x5dd101]]

: [./mgr(main_loop(int, char**)+0x33) [0x5dfb63]]

: [./mgr [0x5e0043]]

: [./mgr(ggs::gglib::MultiThreading::MainThread::ExecMain()+0x66) [0x53a386]]

: [./mgr(ggs::gglib::MultiThreading::Thread::RunThread(ggs::gglib::MultiThreading::Thread::

ThreadArgs*)+0x147) [0x53e757]]

: [./mgr(ggs::gglib::MultiThreading::MainThread::Run(int, char**)+0x12d) [0x53ed6d]]

: [./mgr(main+0x3b) [0x5e06cb]]

: [/lib64/libc.so.6(__libc_start_main+0xf4) [0x3aeee1d994]]

: [./mgr(__gxx_personality_v0+0x389) [0x4d0e89]]

2016-03-04 10:43:08 ERROR OGG-01224 Address already in use.

2016-03-04 10:43:08 ERROR OGG-01668 PROCESS ABENDING.

Manager is abended with ERROR OGG-01224

Analysis:

some other process is holding the manager and its port number

[root@tnc1 ~]# ps -ef | grep mgr

root 12049 11901 0 10:45 pts/2 00:00:00 grep mgr

oracle 25378 1 0 09:57 ? 00:00:01 ./mgr PARAMFILE /u02/goldengate/12.1/dirprm/mgr.prm REPORTFILE /u02/goldengate/12.1/dirrpt/MGR.rpt PROCESSID MGR USESUBDIRS

[root@tnc1 ~]# netstat -nap | grep 7809

tcp 0 0 0.0.0.0:7809 0.0.0.0:* LISTEN 25378/./mgr

Resolution:

Killed the manager and restarted abended Extract process

[root@tnc1 ~]# kill -9 25378

[root@tnc1 ~]# ps -ef | grep mgr

root 12102 11901 0 10:45 pts/2 00:00:00 grep mgr

GGSCI (tnc1.localdomain) 5> start mgr !

Manager started.

GGSCI (tnc1.localdomain) 6> info mgr

Manager is running (IP port tnc1.localdomain.7809, Process ID 13523).

--Nikhil Tatineni--

--12c GoldenGate--

GoldenGate ERROR OGG-06220 Classic Extract does not support multitenant container

GoldenGate Replicat Abended with "ERROR OGG-06220"

Database Version:

Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production

PL/SQL Release 12.1.0.2.0 - Production

CORE 12.1.0.2.0 Production

TNS for Linux: Version 12.1.0.2.0 - Production

NLSRTL Version 12.1.0.2.0 - Production

Database Language and Character Set:

NLS_LANGUAGE = "AMERICAN"

NLS_TERRITORY = "AMERICA"

NLS_CHARACTERSET = "WE8MSWIN1252"

Source Context :

SourceModule : [er.redo.ora]

SourceID : [/scratch/aime/adestore/views/aime_adc4150408/oggcore/OpenSys/src/app/er/redo/oracle/redoora.c]

SourceFunction : [ce_validate_config]

SourceLine : [5896]

ThreadBacktrace : [12] elements

: [/u02/goldengate/12.1/libgglog.so(CMessageContext::AddThreadContext()+0x1b) [0x2abd153e40bb]]

: [/u02/goldengate/12.1/libgglog.so(CMessageFactory::CreateMessage(CSourceContext*, unsigned int, ...)+0x134) [0x2abd153de254]]

: [/u02/goldengate/12.1/libgglog.so(_MSG_ERR_CLASSIC_NOT_SUPPORT_CDB(CSourceContext*, CMessageFactory::MessageDisposition)+0x29) [0x2abd153a94e8]]

: [/u02/goldengate/12.1/extract(REDO_validate_config(bool, unsigned int*, bool&)+0x1339) [0x7798e9]]

: [/u02/goldengate/12.1/extract(ggs::er::OraTranLogDataSource::setup()+0x6c) [0x76c20c]]

: [/u02/goldengate/12.1/extract [0x603b76]]

: [/u02/goldengate/12.1/extract(ggs::gglib::MultiThreading::MainThread::ExecMain()+0x60) [0x6e05e0]]

: [/u02/goldengate/12.1/extract(ggs::gglib::MultiThreading::Thread::RunThread(ggs::gglib::MultiThreading::Thread::ThreadArgs*)+0x14d) [0x6e17cd]]

: [/u02/goldengate/12.1/extract(ggs::gglib::MultiThreading::MainThread::Run(int, char**)+0xb1) [0x6e18b1]]

: [/u02/goldengate/12.1/extract(main+0x3b) [0x60500b]]

: [/lib64/libc.so.6(__libc_start_main+0xf4) [0x3aeee1d994]]

: [/u02/goldengate/12.1/extract [0x573069]]

2016-03-03 14:15:49 ERROR OGG-06220 Classic Extract does not support multitenant container databases.

2016-03-03 14:15:49 ERROR OGG-01668 PROCESS ABENDING.

Findings:

Classic Extracts are not supported on Multitenant database

Resolution:

Delete Classic Extract and Configure Integrated Extract

--Nikhil Tatineni--

--12c GoldenGate---

ERROR OGG-01028

http://oracle-in-memory.blogspot.com/2015/02/error-ogg-01028.html

ERROR OGG-01232

http://oracle-in-memory.blogspot.com/2015/02/error-ogg-01232_1.html

ERROR 0GG-01031

http://oracle-in-memory.blogspot.com/2015/01/error-0gg-01031.html

ERROR OGG-00870

http://oracle-in-memory.blogspot.com/2016/03/goldengate-error-ogg-00870.html

ERROR OGG-00446 Unable to lockfile

http://oracle-in-memory.blogspot.com/2016/03/error-ogg-00446-unable-to-lock-file.html

ERROR OGG-10144

http://oracle-in-memory.blogspot.com/2016/03/goldengate-error-ogg-10144.html

ERROR OGG-01224

http://oracle-in-memory.blogspot.com/2016/03/goldengate-error-ogg-01224.html

GoldenGate ERROR OGG-10144

GoldenGate Extract in Build-stage Abended with errors

SourceModule : [ggparam.util]

SourceID : [/scratch/aime/adestore/views/aime_adc4150408/oggcore/OpenSys/src/gglib/ggparam/paramutil.cpp]

SourceFunction : [generate_msg]

SourceLine : [895]

ThreadBacktrace : [14] elements

: [/u02/goldengate/12.1/libgglog.so(CMessageContext::AddThreadContext()+0x1b) [0x2b3be3a620bb]]

: [/u02/goldengate/12.1/libgglog.so(CMessageFactory::CreateMessage(CSourceContext*, unsigned int, ...)+0x134) [0x2b3be3a5c254]]

: [/u02/goldengate/12.1/libgglog.so(_MSG_ERR_GGPARAM_NO_MATCH(CSourceContext*, char const*, unsigned int, char const*, CMessageFactory::M

essageDisposition)+0x41) [0x2b3be3a2056a]]

: [/u02/goldengate/12.1/libggparam.so(ggs::ggparam::error::generate_msg(unsigned long)+0xfaf) [0x2b3be2d52e27]]

: [/u02/goldengate/12.1/libggparam.so(ggs::ggparam::error::issue(bool)+0x90) [0x2b3be2d50bc0]]

: [/u02/goldengate/12.1/libggparam.so(ggs::ggparam::dict::read(ggs::ggparam::config, char const*, bool, bool, bool)+0x643) [0x2b3be2d49ce

3]]

: [/u02/goldengate/12.1/extract(get_infile_params(time_elt_def*, time_elt_def*, char**, ggs::gglib::ggdatasource::DataSourceParams&, ggs:

:Heartbeat::MapGeneratorParams&)+0x1ff) [0x5d089f]]

: [/u02/goldengate/12.1/extract [0x6038ea]]

: [/u02/goldengate/12.1/extract(ggs::gglib::MultiThreading::MainThread::ExecMain()+0x60) [0x6e05e0]]

: [/u02/goldengate/12.1/extract(ggs::gglib::MultiThreading::Thread::RunThread(ggs::gglib::MultiThreading::Thread::ThreadArgs*)+0x14d) [0x

6e17cd]]

: [/u02/goldengate/12.1/extract(ggs::gglib::MultiThreading::MainThread::Run(int, char**)+0xb1) [0x6e18b1]]

: [/u02/goldengate/12.1/extract(main+0x3b) [0x60500b]]

: [/lib64/libc.so.6(__libc_start_main+0xf4) [0x3aeee1d994]]

: [/u02/goldengate/12.1/extract [0x573069]]

2016-03-03 15:16:18 ERROR OGG-10144 (ecbn.prm) line 6: Parameter [ASMPASSWORD] is not valid for this configuration.

Analysis:

When we configure Oracle GoldenGate on 12c Multitenant database. Classic capture is not loaded or supported on 12c Multitenant database. Here extracts runs integrated mode, extracts collects LCR from Log Mining server.

Extract Parameter file with Tranlogoptions

extract ecbn

SETENV (ORACLE_HOME = "/u01/app/oracle/product/12.1.0.2/db_1/")

SETENV (ORACLE_SID = “P1TNCD”)

TRANLOGOPTIONS ASMUSER ggsasm@ASM, ASMPASSWORD oracle

USERID c##gg@P1TNCD password oracle

DISCARDFILE /u02/goldengate/12.1/dirdsc/ecbn.dsc,PURGE, MEGABYTES 100

SOURCECATALOG CBN

TABLE police.persons;

Extract Parameter file After:

extract ecbn

SETENV (ORACLE_HOME = "/u01/app/oracle/product/12.1.0.2/db_1/")

SETENV (ORACLE_SID = “P1TNCD”)

USERID c##gg@P1TNCD password oracle

DISCARDFILE /u02/goldengate/12.1/dirdsc/ecbn.dsc,PURGE, MEGABYTES 100

SOURCECATALOG CBN

TABLE police.persons;

Resolution:

Removed TRANLOGOPTIONS ASMUSER param file from extract and restart it

GGSCI (tnc1.localdomain) 50> info all

Program Status Group Lag at Chkpt Time Since Chkpt

MANAGER RUNNING

EXTRACT STARTING ECBN 00:00:08 00:00:46

--Nikhil Tatineni--

--12c GoldenGate--

GoldenGate ERROR OGG-00870

Today My Integrated Replicat abended with error ERROR OGG-00870

Source Context :

SourceModule : [iroci]

SourceID : [/scratch/aime/adestore/views/aime_adc4150408/oggcore/OpenSys/src/gglib/ggdbora/iroci.cpp]

SourceFunction : [IROCI_init_context]

SourceLine : [991]

ThreadBacktrace : [11] elements

: [/u02/goldengate/12.1/libgglog.so(CMessageContext::AddThreadContext()+0x1b) [0x2b7de91740bb]]

: [/u02/goldengate/12.1/libgglog.so(CMessageFactory::CreateMessage(CSourceContext*, unsigned int, ...)+0x134) [0x2b7de916e254]]

: [/u02/goldengate/12.1/libgglog.so(_MSG_ERR_DB_SQL_ERROR(CSourceContext*, int, char const*, CMessageFactory::MessageDisposition)+0x3a) [0x2b7de9155cc7]]

: [/u02/goldengate/12.1/replicat(IROCI_init_context(OCISESS_context_def*, int*, int*, unsigned int, char const*)+0x1ae5) [0x75e0a5]]

: [/u02/goldengate/12.1/replicat [0x63193d]]

: [/u02/goldengate/12.1/replicat(ggs::gglib::MultiThreading::MainThread::ExecMain()+0x60) [0x6e9e10]]

: [/u02/goldengate/12.1/replicat(ggs::gglib::MultiThreading::Thread::RunThread(ggs::gglib::MultiThreading::Thread::ThreadArgs*)+0x14d) [0x6ead9d]]

: [/u02/goldengate/12.1/replicat(ggs::gglib::MultiThreading::MainThread::Run(int, char**)+0xb1) [0x6eae81]]

: [/u02/goldengate/12.1/replicat(main+0x3b) [0x634a5b]]

: [/lib64/libc.so.6(__libc_start_main+0xf4) [0x3aeee1d994]]

: [/u02/goldengate/12.1/replicat [0x56b119]]

2016-03-04 00:47:08 ERROR OGG-00870 Database error 942 (ORA-00942: table or view does not exist SQL select apply# from sys."_DBA_APPLY" where apply_name = :1).

2016-03-04 00:47:08 ERROR OGG-01668 PROCESS ABENDING.

Actions:

Connected to the CDB$ROOT database and alter the session to Pluggable database "CBN" and granted data dictionary privileges to goldengate user "c##gg"

[oracle@tnc2 12.1]$ sqlplus / as sysdba

SQL*Plus: Release 12.1.0.2.0 Production on Fri Mar 4 00:48:39 2016

Connected to:

Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production

With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,

Advanced Analytics and Real Application Testing options

SQL> alter session set container=cbn;

Session altered.

SQL> grant select on sys.props$ to c##gg;

Grant succeeded.

SQL> grant select any dictionary to c##gg;

Grant succeeded.

After Replicat came-up automatically

GGSCI (tnc1.localdomain) 21> info all

Program Status Group Lag at Chkpt Time Since Chkpt

MANAGER RUNNING

EXTRACT RUNNING ECBN 00:00:07 00:00:09

REPLICAT RUNNING RCBN 00:00:00 00:00:23

--Nikhil Tatineni--

--12c GoldenGate

Database In Memory

Saturday, March 5, 2016

CRS-8503 [] [] [] [] [] [] [] [] [] [] [] []

Friday, March 4, 2016

Oracle GoldenGate: Creating Virtual IP address (VIP) to manage high availability

ORA-00265:instance recovery required,

GoldenGate ERROR OGG-01224

GoldenGate ERROR OGG-06220 Classic Extract does not support multitenant container

GoldenGate ERROR OGG-10144

GoldenGate ERROR OGG-00870

Querys to monitor RAC

Followers