Sunday, January 29, 2017

ORA-24324, ORA-24323, ORA-01090

scenario #
trying to startup / shutdown database encountered with following error #
ORA-24324: service handle not initialized
ORA-24323: value not allowed
ORA-01090: shutdown in progress - connection is not permitted

[oracle@star1 dbs]$ sqlplus /nolog
SQL*Plus: Release 11.2.0.3.0 Production on Fri Jan 27 18:36:01 2017
Copyright (c) 1982, 2011, Oracle.  All rights reserved.
SQL> conn / as sysdba
Connected to an idle instance.
SQL> shut immediate;
ORA-24324: service handle not initialized
ORA-24323: value not allowed
ORA-01090: shutdown in progress - connection is not permitted
SQL> exit
Disconnected

[oracle@star1 dbs]$ ps -ef | grep pmon
oracle    1337   667  0 18:36 pts/0    00:00:00 grep pmon

ORA-24324 During Startup or Shutdown (Doc ID 794293.1)
checked for any oracle server process i.e is up and running on server # and kill them ..
I followed document from Metalink to resolve this issue in my environment 

kill them if any oracle server process are running # ###
oracle@star1(star) ~> ps -ef | grep ora_ | grep $ORACLE_SID
oracle    1072     1  0 18:26 ?        00:00:00 ora_diag_star

oracle@star1(star) ~> ps -ef | grep ora_ | grep $ORACLE_SID
oracle    1072     1  0 18:26 ?        00:00:00 ora_diag_star

oracle@star1(star) ~> kill -9 1072
oracle@star1(star) ~> ps -ef | grep ora_ | grep $ORACLE_SID

A) Check for shared memory and semaphores 

oracle@star1(star) ~> ipcs -mt
------ Shared Memory Attach/Detach/Change Times --------
shmid      owner      attached             detached             changed         
0          root       Nov 18 21:39:27      Nov 18 21:40:23      Nov 18 21:39:19 
425985     oracle     Jan 27 18:44:16      Jan 27 18:47:09      Jan 27 18:26:51 
458754     oracle     Jan 27 18:44:16      Jan 27 18:47:09      Jan 27 18:26:51 
491523     oracle     Jan 27 18:44:16      Jan 27 18:47:09      Jan 27 18:26:51 

oracle@star1(star) ~> ipcrm -m 425985
oracle@star1(star) ~> ipcrm -m 458754
oracle@star1(star) ~> ipcrm -m 491523
oracle@star1(star) ~> ipcs -mt

------ Shared Memory Attach/Detach/Change Times --------
shmid      owner      attached             detached             changed         
0          root       Nov 18 21:39:27      Nov 18 21:40:23      Nov 18 21:39:19

removed shared memory and started database successfully
kindly go through Metalink "Doc ID 794293.1" to resolve this issue

--Nikhil Tatineni--



ARCH: Error 19809

### Scenario #
trying to startup database,
encountered with "ORA-03113: end-of-file on communication channel"
Investigated root cause from alert log file 
# Instance failed with "ARCH: Error 19809" 

inv
SQL> startup;
ORACLE instance started.

Total System Global Area 3206836224 bytes
Fixed Size                  2232640 bytes
Variable Size            1996492480 bytes
Database Buffers         1191182336 bytes
Redo Buffers               16928768 bytes
Database mounted.
ORA-03113: end-of-file on communication channel
Process ID: 793
Session ID: 122 Serial number: 3

###

Message from  database Alert log file #

ARC2: Becoming the 'no SRL' ARCH
ARC1: Becoming the heartbeat ARCH
Errors in file /u01/app/oracle/diag/rdbms/star/star/trace/star_arc1_827.trc:
ORA-19815: WARNING: db_recovery_file_dest_size of 21474836480 bytes is 100.00% used, and has 0 remaining bytes available.
**********************************************************
ARCH: Error 19809 Creating archive log file to '/u01/app/oracle/flash_recovery_area/star/archivelog/2017_01_27/o1_mf_1_83467_%u_.arc'
ARC1: Error 19809 Creating archive log file to '/u01/app/oracle/flash_recovery_area/star/archivelog/2017_01_27/o1_mf_1_83468_%u_.arc'
ARCH: Archival stopped, error occurred. Will continue retrying
ORACLE Instance star - Archival Error
ORA-16038: log 1 sequence# 83468 cannot be archived
ORA-19809: limit exceeded for recovery files
ORA-00312: online log 1 thread 1: '/u01/oradata/star/redo01.log'
Errors in file /u01/app/oracle/diag/rdbms/star/star/trace/star_ora_793.trc:
ORA-16038: log 2 sequence# 83467 cannot be archived
ORA-19809: limit exceeded for recovery files
ORA-00312: online log 2 thread 1: '/u01/oradata/star/redo02.log'
USER (ospid: 793): terminating the instance due to error 16038
System state dump requested by (instance=1, osid=793), summary=[abnormal instance termination].
System State dumped to trace file /u01/app/oracle/diag/rdbms/star/star/trace/star_diag_758.trc
Dumping diagnostic data in directory=[cdmp_20170127181404], requested by (instance=1, osid=793), summary=[abnormal instance termination].
Instance terminated by USER, pid = 793


######
Resolution #
Edit pfile # and increase "db_recovery_file_dest_size" 
startup database with pfile and 
create spfile from pfile 
work on purging archive logs using RMAN and freeup space in archive log directory 
##
#Oracle
Nikhil Tatineni

Querys to monitor RAC

following few  Query's will help to find out culprits-  Query to check long running transaction from last 8 hours  Col Sid Fo...