Category Archives: OPatch

Check patch level of Oracle RDBMS from OS

Most software only have a single patch process to update the software, however relational databases are a bit different.  You have both a binary set of files for the software and the internal configuration and objects within the database itself.   Thus, it is necessary to validate data both patch levels at the OS level and database level.   Mismatch patches at this level can cause significant errors during operations. 

Check Oracle Patch level at OS

To see the binary level patching of the Oracle software packages,  you have to use the opatch utility to see the internal entries in oraInventory directory.  Use the following opatch command to view your oraInventory. 



/code
opatch lsinventory

sun1{orcl}-~> opatch lsinventory
Oracle Interim Patch Installer version 12.2.0.1.23
Copyright (c) 2021, Oracle Corporation.  All rights reserved.


Oracle Home       : /oracle/product/12.2.0/db_home1
Central Inventory : /oracle/oraInventory
   from           : /oracle/product/12.2.0/db_home1/oraInst.loc
OPatch version    : 12.2.0.1.23
OUI version       : 12.2.0.1.4
Log file location : /oracle/product/12.2.0/db_home1/cfgtoollogs/opatch/opatch2021-08-18_08-31-56AM_1.log

Lsinventory Output file location : /oracle/product/12.2.0/db_home1/cfgtoollogs/opatch/lsinv/lsinventory2021-08-18_08-31-56AM.txt
--------------------------------------------------------------------------------
Local Machine Information::
Hostname: sun1.testdomain.com
ARU platform id: 23
ARU platform description:: Solaris Operating System (SPARC 64-bit)


Installed Top-level Products (1):

Oracle Database 12c                                                  12.2.0.1.0
There are 1 products installed in this Oracle Home.


Interim patches (3) :

Patch  32473172     : applied on Fri Jun 04 12:55:51 EDT 2021
Unique Patch ID:  24158925
Patch description:  "OJVM RELEASE UPDATE 12.2.0.1.210420 (32473172)"
   Created on 1 Apr 2021, 09:12:15 hrs UTC
   Bugs fixed:
     25811105, 25890046, 26023042, 26570134, 27000702, 27461740, 27952586
     27642235, 27539876, 28502098, 28915870, 29254623, 29774362, 30160625
     30534662, 30855101, 31306261, 26716835, 31668872, 29224710, 28394726
     24817447, 32165759, 31997805, 26451793, 32523206, 29415774

Patch  32692908     : applied on Fri Jun 04 12:50:09 EDT 2021
Unique Patch ID:  24229706
   Created on 13 May 2021, 11:26:43 hrs PST8PDT
   Bugs fixed:
     29839481, 31193511
   This patch overlays patches:
     32507738
   This patch needs patches:
     32507738
   as prerequisites

Patch  32507738     : applied on Fri Jun 04 12:46:30 EDT 2021
Unique Patch ID:  24169281
Patch description:  "Database Apr 2021 Release Update : 12.2.0.1.210420 (32507738)"
   Created on 9 Apr 2021, 05:33:05 hrs UTC
   Bugs fixed:
     7391838, 8480838, 8932139, 8975044, 12763598, 13554903, 14221306
     14690846, 15931756, 16002385, 16438495, 16727454, 16942578, 17027695
     17533661, 17947871, 17958365, 18308268, 18521691, 18594510, 18774543
     18878420, 18986501, 19072655, 19211433, 19285025, 19327292, 19526548
     19614243, 19647894

Oracle Patch Level in Database





Most software only have a single patch process to update the software, however relational databases are a bit different.  You have both a binary set of files for the software and the internal configuration and objects within the database itself.   Thus, it is necessary to validate data both patch levels at the OS level and database level.   Mismatch patches at this level can cause significant errors during operations. 


To see the patch level of the internal database libraries you have to query the data dictionary view “dba_registry_sqlpatch”.    Connect to one of the databases you have in your current server using sqlplus and execute the below query.


column action_time format a28
column action format a8
column version format a8
column comments format a30
column status format a10
set line 200 pages 500
select patch_id,  version, status, Action,Action_time from dba_registry_sqlpatch order by action_time desc;




orcl (SYS)> column action_time format a28
column action format a8
column version format a8
column comments format a30
column status format a10
set line 200 pages 500
select patch_id,  version, status, Action,Action_time from dba_registry_sqlpatch order by action_time desc;orcl (SYS)> orcl (SYS)> orcl (SYS)> orcl (SYS)> orcl (SYS)> orcl (SYS)>

  PATCH_ID VERSION  STATUS     ACTION   ACTION_TIME
---------- -------- ---------- -------- ----------------------------
  32228578 12.2.0.1 SUCCESS    APPLY    19-MAY-21 02.53.44.053902 PM
  32119931 12.2.0.1 SUCCESS    APPLY    19-MAY-21 02.53.43.930720 PM
  31668898 12.2.0.1 SUCCESS    ROLLBACK 19-MAY-21 02.53.43.919599 PM
  31741641 12.2.0.1 SUCCESS    APPLY    01-DEC-20 08.50.09.424579 PM


Oracle 12c – Apply and review patches

In Oracle 12c, Patching is performed to correct discovered bugs and close security vulnerabilities.   Oracle provides four types of Patches:

Interim Patches – single/collection of bug fixes or one-offs.

Diagnostic Patches – Diagnose or verify a fix.

Patch Set Updates (PSU) – Collection of high impact, low risk, proven fixes for products and components

Critical Patch Updates (CPU) – Collection of patches for security vulnerabilities.

Patching of the RDBMS can be done with two methods: Manually or EM Cloud Control Patch Wizard.   Additionally, the patching processes have added:  Online Patching and Queryable Inventories.   This article will give a brief overview of these Features.

 

Manual Patching

Step in manually patching are:

  1. Logon to oracle support https://support.oracle.com
  2. Click Patches & Updates tab.
  3. Enter the patch number you are looking for on Patches % Updates page.
  4. Click download on patch for your OS
  5. Click download Patch Metadata.
  6. Review the documentation for specific installation instructions.
  7. Use the OPatch utility to install all patches.

 

 

EM Cloud Control Patch Wizard

The EM Cloud Control Patch Wizard has the following benefits over Manual Patching:

  1. Integrate Patch work with Oracle support and provides single view to determine patch needs.
  2. End to End workflow of patching, resolving patch conflicts.
  3. Easy determination of Patch needs for single system.
  4. Patch plans can be saved as templates.
  5. Out of place patching is possible for support of RAC and GRID infrastructure.
  6. EM Cloud allows you create a patch plan to apply patches to one or more targets. A patch can be added to plan only if it’s for same release and OS.

 

EM Cloud can apply patches with two methods:

  1. Online Mode – EM connects to Oracle Support, searches for necessary patches, resolves any conflicts and adds them to patch plan.
  2. Offline Mode – Used when EM cannot connect to Oracle Support. EM searches for patches manually uploaded to software library and adds them to patch plan.

 

 

 

 

Online Patches

Before 11g all patches require relink of RDBMS libraries and a shutdown of system.  Now a large number of Patches can be done online, they contain .so files and do not require relink of RDBMS.

Benefit of Online Patches:

  1. No downtime.
  2. persistent across shutdown
  3. Allow rolling patches in RAC
  4. Fast install.

Downside of Online Patches:

  1. Require more memory
  2. Online patches not available for all OS
  3. Not all patches are hot patches.

 

 

 

Queryable Patch Inventory

Queryable Patch inventory is new 12c, implemented with DBMS_QOPATCH package and provides view of patches installed just as OPATCH lsinventory command.    DBMS_QOPATCH queries the OUI Oracle Universal Installer inventory to gather this metadata.

It provides the following:

  1. Query patches installed from SQL prompt.
  2. Creates reports and provides validation across many installs.
  3. Checks patches on RAC from single node.

 

DBMS_QOPATCH components:

GET_OPTACH_BUGS – Provides list of bug fixes per patch number.

GET_OPATCH_COUNT – Provides total number of patches installed.

GET_OPATCH_DATA – Provides top level patch info – number, creation date.

GET_OPATCH_FILES – Provides list of files updated by patch number.

GET_OPATCH_INSTALL_INFO – Provides info such as ORACLE_HOME, patch, inventory_location

GET_OPATCH_LIST – List of Patches installed.

GET_OPATCH_LSINVENTORY – Whole patch inventory

GET_OPATCH_OLAYS –Overlay for specific patch

GET_OPATCH_PREQS – Provides prerequisites for a patch

GET_OPATCH_XSLT – Provides style sheet for opatch XML inventory

GET_PENDING_ACTIVITY – Provides SQL patches applied to single inventory from binaries.

GET_SQLPATCH_STATUS – Patch status from SQL patch registry.

IS_PATCH_INSTALLED – Patches applied info from XML inventory

PATCH_CONFLICT_DETETION – Patch in conflict for give file.

SET_CURRENT_OPINST – inventory information for specific node in RAC.

 

Larry Catt

OCP

Oracle 12c – Apply and review patches

In Oracle 12c, Patching is performed to correct discovered bugs and close security vulnerabilities.   Oracle provides four types of Patches:

Interim Patches – single/collection of bug fixes or one-offs.

Diagnostic Patches – Diagnose or verify a fix.

Patch Set Updates (PSU) – Collection of high impact, low risk, proven fixes for products and components

Critical Patch Updates (CPU) – Collection of patches for security vulnerabilities.

Patching of the RDBMS can be done with two methods: Manually or EM Cloud Control Patch Wizard.   Additionally, the patching processes have added:  Online Patching and Queryable Inventories.   This article will give a brief overview of these Features.

Manual Patching

Step in manually patching are:

  1. Logon to oracle support https://support.oracle.com
  2. Click Patches & Updates tab.
  3. Enter the patch number you are looking for on Patches % Updates page.
  4. Click download on patch for your OS
  5. Click download Patch Metadata.
  6. Review the documentation for specific installation instructions.
  7. Use the OPatch utility to install all patches.

EM Cloud Control Patch Wizard

The EM Cloud Control Patch Wizard has the following benefits over Manual Patching:

  1. Integrate Patch work with Oracle support and provides single view to determine patch needs.
  2. End to End workflow of patching, resolving patch conflicts.
  3. Easy determination of Patch needs for single system.
  4. Patch plans can be saved as templates.
  5. Out of place patching is possible for support of RAC and GRID infrastructure.
  6. EM Cloud allows you create a patch plan to apply patches to one or more targets. A patch can be added to plan only if it’s for same release and OS.

EM Cloud can apply patches with two methods:

  1. Online Mode – EM connects to Oracle Support, searches for necessary patches, resolves any conflicts and adds them to patch plan.
  2. Offline Mode – Used when EM cannot connect to Oracle Support. EM searches for patches manually uploaded to software library and adds them to patch plan.

Online Patches

Before 11g all patches require relink of RDBMS libraries and a shutdown of system.  Now a large number of Patches can be done online, they contain .so files and do not require relink of RDBMS.

Benefit of Online Patches:

  1. No downtime.
  2. persistent across shutdown
  3. Allow rolling patches in RAC
  4. Fast install.

Downside of Online Patches:

  1. Require more memory
  2. Online patches not available for all OS
  3. Not all patches are hot patches.

Queryable Patch Inventory

Queryable Patch inventory is new 12c, implemented with DBMS_QOPATCH package and provides view of patches installed just as OPATCH lsinventory command.    DBMS_QOPATCH queries the OUI Oracle Universal Installer inventory to gather this metadata.

It provides the following:

  1. Query patches installed from SQL prompt.
  2. Creates reports and provides validation across many installs.
  3. Checks patches on RAC from single node.

 

DBMS_QOPATCH components:

GET_OPTACH_BUGS – Provides list of bug fixes per patch number.

GET_OPATCH_COUNT – Provides total number of patches installed.

GET_OPATCH_DATA – Provides top level patch info – number, creation date.

GET_OPATCH_FILES – Provides list of files updated by patch number.

GET_OPATCH_INSTALL_INFO – Provides info such as ORACLE_HOME, patch, inventory_location

GET_OPATCH_LIST – List of Patches installed.

GET_OPATCH_LSINVENTORY – Whole patch inventory

GET_OPATCH_OLAYS –Overlay for specific patch

GET_OPATCH_PREQS – Provides prerequisites for a patch

GET_OPATCH_XSLT – Provides style sheet for opatch XML inventory

GET_PENDING_ACTIVITY – Provides SQL patches applied to single inventory from binaries.

GET_SQLPATCH_STATUS – Patch status from SQL patch registry.

IS_PATCH_INSTALLED – Patches applied info from XML inventory

PATCH_CONFLICT_DETETION – Patch in conflict for give file.

SET_CURRENT_OPINST – inventory information for specific node in RAC.

 

Larry Catt

OCP

How to debug Oracle opatch utility

The following procedure shows how to debug the Oracle opatch utility when you receive an error.

1. Execution of opatch utility results in the following error.

[oracle@mylinux1 OPatch]$ opatch lsinventory
Invoking OPatch 11.1.0.6.6

Oracle Interim Patch Installer version 11.1.0.6.6
Copyright (c) 2009, Oracle Corporation. All rights reserved.

Oracle Home : /u01/app/oracle/product/11.2.0/dbhome_1
Central Inventory : /u01/app/oraInventory
from : /etc/oraInst.loc
OPatch version : 11.1.0.6.6
OUI version : 11.2.0.1.0
OUI location : /u01/app/oracle/product/11.2.0/dbhome_1/oui
Log file location : /u01/app/oracle/product/11.2.0/dbhome_1/cfgtoollogs/opatch/opatch2011-04-02_19-17-51PM.log

Patch history file: /u01/app/oracle/product/11.2.0/dbhome_1/cfgtoollogs/opatch/opatch_history.txt
Inventory load failed… OPatch cannot load inventory for the given Oracle Home.
LsInventorySession failed: Unable to create patchObject

OPatch failed with error code 73
[oracle@mylinux1 OPatch]$

2. Set the environmental variable OPATCH_DEBUG equal to true.

[oracle@mylinux1 OPatch]$ export OPATCH_DEBUG=true
[oracle@mylinux1 OPatch]$

3. Re-execute your opatch utility and every action performed by opatch will be displayed in your buffer. Use the messages to determine the error.

This completes use of debug with the opatch utility.

Larry J. Catt, OCP
oracle@allcompute.com
www.allcompute.com

Oracle error OUI-10058

The OUI (Oracle Universal Installer) relies on an Oracle inventory to recognize the Oracle software which already exists on the system. Multiple files are read during the OUI startup, the first of which is /etc/oraInst.loc. The oraInst.loc file points to the location of the Oracle inventor directory, which could be anywhere. In order to get an inventory of the current system, OUI must have access to this file. The error OUI-10058 is an indication that OUI does not have access to this file. This article covers the typical error and resolution to this problem

Typical error:

OUI-10058 The OUI Inventory on this system does not exist or is not writable.
1. The /opt/oracle/oraInventory/orainstall…..
or
2. You are not a member of the specified installer group.


Solution: If you executed all previous root.sh script as instructed, the most like cause is permission.

1. Logon to the system as root.
2. Change directory to /etc
3. Check the file permissions with the command ll

[root@mylinux /]# ll *ora*

4. If the file permission is not set to 644, change the permissions with the command chmod

[root@mylinux /]# chmod 644 /etc/oraInst.loc.

5. This completes the resolution of OUI-10058, due to file permissions of /etc/oraInst.loc.

Larry J. Catt, OCP 9i, 10g
oracle@allcompute.com
www.allcompute.com

Enable ADMIN Tracing on Oracle Listener:

Enable ADMIN Tracing on Oracle Listener:

Most Oracle database connects are made through the use of an Oracle process called the LISTENER which monitors a certain machine and port for connection requests to one or more database instances. There are times when a DBA or SA will want to monitor connects being made through the listener for various reasons. This article covers the setup of ADMINISTRATIVE level tracing of you Oracle Listener and will work on UNIX, Linux, and Windows based machines.

1. Logon to your Oracle database server as the Oracle software owner.

2. Open up the LSNRCTL utility.

mylinux:>lsnrctl

LSNRCTL for LINUX: Version 10.2.0.4.0 – Production on 3-Jul-2010 15:17:05

Copyright (c) 1991, 2007, Oracle. All rights reserved.

Welcome to LSNRCTL, type “help” for information.

LSNRCTL>

3. If you are not using the default name of the Oracle listener (LISTENER) then define the Oracle listener name with the following command: set current_listener

LSNRCTL> set current_listener listener_orcl
Current Listener is listener_orcl
LSNRCTL>

4. If you have established a password for your oracle listener set it in the LSNRCTL session with the following command: set password

LSNRCTL> set password oracle
The command completed successfully
LSNRCTL>

5. Turn on admin level tracing by specifying 10 or admin as the level with the following command: set trc_level

LSNRCTL> set trc_level 10
Connecting to
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.0.110)(PORT=1521)))
listener_orcl parameter “trc_level” set to admin
The command completed successfully

or

LSNRCTL> set trc_level admin
Connecting to
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.0.110)(PORT=1521)))
listener_orcl parameter “trc_level” set to admin
The command completed successfully
LSNRCTL>

6. Define the name of the trace file that will be produced by the listener process upon establishment of user connection with the command below: set trc_file

LSNRCTL> set trc_file admin_access_trace
Connecting to
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.0.110)(PORT=1521)))
listener_orcl parameter “trc_file” set to admin_access_trace.trc
The command completed successfully
LSNRCTL>

NOTE: You do not have to set a trace directory by default the trace files are placed under: $ORACLE_HOME/network/trace

7. Save the changes made to the listener.ora file with the following command: save_config

LSNRCTL> save_config
Connecting to
(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.0.110)(PORT=1521)))
Saved listener_orcl configuration parameters.
Listener Parameter File /opt/app/oracle/10.2.0/network/admin/listener.ora
Old Parameter File /opt/app/oracle/10.2.0/network/admin/listener.bak
The command completed successfully
LSNRCTL>

8. Exit out of LSNRCTL utility, change directories to ORACLE_HOME/network/trace, list the files present and you can see that the trace has already begun.

LSNRCTL> exit
mylinux:> ls -lrt
total 1008
-rw-r—– 1 oracle dba 408755 Jul 3 15:25 listener_orcl.trc
-rw-r—– 1 oracle dba 43347 Jul 3 15:40 user_access_trace.trc
-rw-r—– 1 oracle dba 34150 Jul 3 15:42 admin_access_trace.trc

9. Logon to the RDBMS as sysdba and exit.

mylinux:> sqlplus ‘system/devlmgr as sysdba’

SQL*Plus: Release 10.2.0.4.0 – Production on Sat Jul 3 15:44:02 2010

Copyright (c) 1982, 2007, Oracle. All Rights Reserved.

Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 – 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL> exit
Disconnected from Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 –
64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
mylinux:>

10. Perform another listing of the files under the directory $ORACLE_HOME/network/admin and you will see that the file admin_access_trace.trc has grown by recording the connection
from user in step 9.

mylinux:> ls -lrt
total 1136
-rw-r—– 1 oracle dba 408755 Jul 3 15:25 listener_orcl.trc
-rw-r—– 1 oracle dba 43347 Jul 3 15:40 user_access_trace.trc
-rw-r—– 1 oracle dba 119638 Jul 3 15:47 admin_access_trace.trc
mylinux:>

11 This information can be used to show where connections in you database are coming from and other statistical information about the network connection being established with administrative options.

Larry J. Catt, OCP 91, 10g
oracle@allcompute.com
www.allcompute.com

OPatch Oracle database still running: Code 73

Oracle’s OPatch utility is used for applying interim CPU patches to maintain the security of your RDBMS. In this article we will cover the error which would be received when you attempt to apply an Oracle patch to a currently executing Oracle database.

1. Logon to your Oracle database server as the Oracle software owner.

2. Change directories to the location of your CPU patch and unzip the file.

unzip p9119226_10204_.zip

3. Change directories into your unzipped patch directory which is named with the CPU number you are applying.

cd 9119226
OPatch apply

conflicts against Oracle Home…
OPatch continues with these patches: 7155248 7155249 7155250 7155251
7155252 7155253 7155254 7197583 7375611 7375613 7375617 7609057 7609058
8309592 8309632 8309642 8568395 8568397 8568398 8568402 8568404 8568405
8836667 8836671 8836675 8836677 8836678 8836681 8836683 8836684 8836686
9119226 9173244 9173248 9173253

Do you want to proceed? [y|n]
y
User Responded with: Y

Running prerequisite checks…
Prerequisite check “CheckActiveFilesAndExecutables” failed.
The details are:

Following executables are active :
/u01/prism/oracle/product/10.2.0/db_1/bin/oracle
UtilSession failed: Prerequisite check “CheckActiveFilesAndExecutables” failed.

OPatch failed with error code 73
mylinux :>

4. As seen above the OPatch utility errored out with code 73 and references the Prerequisite check “CheckActiveFilesAndExecutables”. This error normally occurs when the Oracle database is still up and running, so we check it below with the command ps:

mylinux :> ps -ef|grep smon
cimsora 26981 1 0 02:12 ? 00:00:00 ora_smon_prism
cimsora 4326 11581 0 02:34 pts/1 00:00:00 grep smon

5. As seen above the database is in fact up, so logon as sysdba and shut it down.

mylinux :> sqlplus ‘/ as sysdba’

SQL*Plus: Release 10.2.0.4.0 – Production on Sun Feb 2802:38:36 2010

Copyright (c) 1982, 2007, Oracle. All Rights Reserved.

Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 – Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL> shutdown immediate
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> exit

6. Once the database is shutdown we can re-execute our OPatch install.

Larry J Catt, OCP9i, 10g
oracle@allcompute.com
www.allcompute.com

Wrong OPatch utility for CPU being applied: Code 14

Oracle’s OPatch utility is used for applying interim CPU patches to maintain the security of your RDBMS. In this article we will cover the error which would be received when you use OPatch options that are not supported by your currently installed OPatch utility.

1. Logon to your Oracle database server as the Oracle software owner.

2. Change directories to the location of your CPU patch and unzip the file.

unzip p9119226_10204_.zip

3. Change directories into your unzipped patch directory which is named with the CPU number you are applying.

cd 9119226
OPatch napply -skip_subset -skip_duplicate

4. You receive the following error or similar error during installation.

ORCL :> OPatch napply -skip_subset -skip_duplicate
Invoking OPatch 10.2.0.3.0

Oracle interim Patch Installer version 10.2.0.3.0
Copyright (c) 2005, Oracle Corporation. All rights reserved..

Syntax Error… Unrecognized Command or Option: failed to parse arguments
“unknown option ‘-skip_subset'”
Please use the option ‘OPatch -help’ to get correct syntax

OPatch failed with error code 14
ORCL :>

5. This error is being generated because OPatch 10.2.0.3.0 did not support the napply option or its tags. The solution is to download the latest Oracle Patch 6880880, containing the OPatch utility, and unzip it in your ORACLE_HOME directory.

Larry J Catt, OCP 9i, 10g
oracle@allcompute.com
www.allcompute.com