Sunday, September 11, 2016

12c RAC > Clusterware logs

Recently upgraded my RAC cluster from 11.2.0.4 to 12.1.0.2
Clusterware logs are centralized in  RAC 12c 

>> In 11gR2 RAC Cluster, we have different directories to investigate or troubleshoot issues in cluster <<

/u01/app/11.2.0.4/grid/log/tnc2

[oracle@tnc2 tnc2]$ ll
total 316
drwxr-xr-x 3 root   root       4096 Jul 13 12:38 acfs
drwxr-x--- 3 oracle oinstall   4096 Aug 20 12:54 acfslog
drwxr-x--- 2 oracle oinstall   4096 Jul 13 12:37 acfsrepl
drwxr-x--- 2 root   oinstall   4096 Jul 13 12:37 acfsreplroot
drwxr-xr-x 2 root   oinstall   4096 Jul 13 12:37 acfssec
drwxr-x--- 2 oracle oinstall   4096 Jul 13 12:37 admin
drwxrwxr-t 4 root   oinstall   4096 Jul 13 12:37 agent
-rw-rw-r-- 1 oracle oinstall 221280 Sep  2 17:56 alerttnc2.log
drwxrwxrwt 2 oracle oinstall   4096 Sep  2 17:55 client
drwxr-x--- 2 root   oinstall   4096 Jul 13 12:41 crflogd
drwxr-x--- 2 root   oinstall   4096 Jul 13 12:41 crfmond
drwxr-x--- 2 root   oinstall   4096 Aug 26 13:47 crsd
drwxr-x--- 2 oracle oinstall   4096 Aug 20 12:51 cssd
drwxr-x--- 2 root   oinstall   4096 Aug 20 15:25 ctssd
drwxr-x--- 4 oracle oinstall   4096 Jul 13 12:37 cvu
drwxr-x--- 2 oracle oinstall   4096 Jul 13 12:37 diskmon
drwxr-x--- 2 oracle oinstall   4096 Jul 13 12:41 evmd
drwxr-x--- 2 oracle oinstall   4096 Sep  2 13:58 gipcd
drwxr-x--- 2 root   oinstall   4096 Jul 13 12:37 gnsd
drwxr-x--- 2 oracle oinstall   4096 Sep  2 14:32 gpnpd
drwxr-x--- 2 oracle oinstall   4096 Jul 13 12:38 mdnsd
drwxr-x--- 2 root   oinstall   4096 Aug 25 15:28 ohasd
drwxrwxr-t 5 oracle oinstall   4096 Aug 24 15:51 racg
drwxr-x--- 2 oracle oinstall   4096 Jul 13 12:37 srvm

>> But in 12c RAC cluster, Oracle centralized logs & Location as follows << 
>> It is easy to troubleshoot Issues in Clusterware Now << 

[oracle@tnc2 trace]$ pwd
/u01/app/oracle/diag/crs/tnc2/crs/trace

/u01/app/oracle/diag/crs/<Node Name>/crs/trace

> Using Alert log file, we can find Root cause in New 12c RAC Cluster 

[oracle@tnc2 trace]$ ls -ltr alert.log 
-rw-rw---- 1 oracle oinstall 124737 Sep 11 20:39 alert.log

<< << << << Use ADRCI to get ADR location :) 

[oracle@tnc2 grid]$ adrci

ADRCI: Release 12.1.0.2.0 - Production on Sun Sep 11 21:04:24 2016
Copyright (c) 1982, 2014, Oracle and/or its affiliates.  All rights reserved.

ADR base = "/u01/app/oracle"
adrci> show homes
ADR Homes: 
diag/tnslsnr/tnc2/listener_scan1
diag/tnslsnr/tnc2/mgmtlsnr
diag/tnslsnr/tnc2/listener
diag/rdbms/mipd09/mipd092
diag/rdbms/_mgmtdb/-MGMTDB
diag/rdbms/db12c/db12c_2
diag/asm/+asm/+ASM2
diag/crs/tnc2/crs

>> >> >>
--Nikhil Tatineni--
--12c RAC--


Querys to monitor RAC

following few  Query's will help to find out culprits-  Query to check long running transaction from last 8 hours  Col Sid Fo...