Huaiyao's Notes

Fix issues when applying psu7 patch

Steps to apply PSU7 for 11.2.0.3.0.

Check before applying patch

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
+ASM:/nas/home/oracle/software/patches>opatch lsinv
Oracle Interim Patch Installer version 11.2.0.3.16
Copyright (c) 2017, Oracle Corporation. All rights reserved.
Oracle Home : /opt/oracle/grid/11gR203
Central Inventory : /opt/oracle/oraInventory
from : /opt/oracle/grid/11gR203/oraInst.loc
OPatch version : 11.2.0.3.16
OUI version : 11.2.0.3.0
Log file location : /opt/oracle/grid/11gR203/cfgtoollogs/opatch/opatch2017-06-22_05-06-15AM_1.log
Lsinventory Output file location : /opt/oracle/grid/11gR203/cfgtoollogs/opatch/lsinv/lsinventory2017-06-22_05-06-15AM.txt
--------------------------------------------------------------------------------
Local Machine Information::
Hostname: rjin00dbs001
ARU platform id: 23
ARU platform description:: Solaris Operating System (SPARC 64-bit)
Installed Top-level Products (1):
Oracle Grid Infrastructure 11.2.0.3.0
There are 1 products installed in this Oracle Home.
There are no Interim patches installed in this Oracle Home.
--------------------------------------------------------------------------------
OPatch succeeded.

Create ocm.rsp

1
2
3
4
5
6
7
8
9
10
+ASM:/export/home/oracle>$ORACLE_HOME/OPatch/ocm/bin/emocmrsp -no_banner -output /export/home/oracle/rjin/ocm.rsp
Provide your email address to be informed of security issues, install and
initiate Oracle Configuration Manager. Easier for you if you use your My
Oracle Support Email address/User Name.
Visit http://www.oracle.com/support/policies.html for details.
Email address/User Name:
You have not provided an email address for notification of security issues.
Do you wish to remain uninformed of security issues ([Y]es, [N]o) [N]: Y
The OCM configuration response file (/export/home/oracle/rjin/ocm.rsp) was successfully created.

Apply patch with root

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
[root@rjin00dbs001 /]# /opt/oracle/grid/11gR203/OPatch/opatch auto /nas/home/oracle/software/patches -ocmrf /export/home/oracle/rjin/ocm.rsp
Executing /opt/oracle/grid/11gR203/perl/bin/perl /opt/oracle/grid/11gR203/OPatch/crs/patch11203.pl -patchdir /nas/home/oracle/software -patchn patches -ocmrf /export/home/oracle/rjin/ocm.rsp -paramfile /opt/oracle/grid/11gR203/crs/install/crsconfig_params
This is the main log file: /opt/oracle/grid/11gR203/cfgtoollogs/opatchauto2017-06-22_05-11-35.log
This file will show your detected configuration and all the steps that opatchauto attempted to do on your system:
/opt/oracle/grid/11gR203/cfgtoollogs/opatchauto2017-06-22_05-11-35.report.log
2017-06-22 05:11:35: Starting Oracle Restart Patch Setup
Using configuration parameter file: /opt/oracle/grid/11gR203/crs/install/crsconfig_params
Stopping CRS...
Stopped CRS successfully
patch /nas/home/oracle/software/patches/16619898 apply successful for home /opt/oracle/grid/11gR203
patch /nas/home/oracle/software/patches/16619892 apply failed for home /opt/oracle/grid/11gR203
Starting CRS...
CRS-4123: Oracle High Availability Services has been started.
opatch auto succeeded.

Got issue

– two issues here:

  1. Patch 16619892 apply failed.
  2. Patch not apply to rdbms home.

Check log

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
2017-06-22 05:14:28: status of apply patch is 18688
2017-06-22 05:14:28: The apply patch output is Oracle Interim Patch Installer version 11.2.0.3.16
Copyright (c) 2017, Oracle Corporation. All rights reserved.
Oracle Home : /opt/oracle/grid/11gR203
Central Inventory : /opt/oracle/oraInventory
from : /opt/oracle/grid/11gR203/oraInst.loc
OPatch version : 11.2.0.3.16
OUI version : 11.2.0.3.0
Log file location : /opt/oracle/grid/11gR203/cfgtoollogs/opatch/opatch2017-06-22_05-14-22AM_1.log
Verifying environment and performing prerequisite checks...
Prerequisite check "CheckActiveFilesAndExecutables" failed.
The details are:
Following executables are active :
/opt/oracle/grid/11gR203/bin/tnslsnr
Prerequisite check "CheckActiveFilesAndExecutables" failed.
The details are:
Following executables are active :
/opt/oracle/grid/11gR203/lib/libclntsh.so.11.1
UtilSession failed: Prerequisite check "CheckActiveFilesAndExecutables" failed.Prerequisite check "CheckActiveFilesAndExecutables" failed.

Error information is clear.

Fix issue 1

Register a dummy db in cluster

1
2
3
+ASM:/export/home/oracle>srvctl add database -d TEST -o /opt/oracle/database/11gR203
+ASM:/export/home/oracle>srvctl status database -d TEST
Database is not running.

Fix issue 2

1
2
3
4
5
6
7
8
9
10
11
12
13
+ASM:/export/home/oracle>ps -ef|grep tns
oracle 36303 1 0 18:54:10 ? 0:02 /opt/oracle/grid/11gR203/bin/tnslsnr LISTENER -inherit
oracle 42541 40006 0 05:46:51 pts/3 0:00 grep tns
+ASM:/export/home/oracle>lsnrctl stop
LSNRCTL for Solaris: Version 11.2.0.3.0 - Production on 22-JUN-2017 05:46:54
Copyright (c) 1991, 2011, Oracle. All rights reserved.
Connecting to (ADDRESS=(PROTOCOL=tcp)(HOST=)(PORT=1521))
The command completed successfully
+ASM:/export/home/oracle>ps -ef|grep tns
oracle 42544 40006 0 05:47:13 pts/3 0:00 grep tns

Patch again

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
[root@rjin00dbs001 /]# /opt/oracle/grid/11gR203/OPatch/opatch auto /nas/home/oracle/software/patches -ocmrf /export/home/oracle/rjin/ocm.rsp
Executing /opt/oracle/grid/11gR203/perl/bin/perl /opt/oracle/grid/11gR203/OPatch/crs/patch11203.pl -patchdir /nas/home/oracle/software -patchn patches -ocmrf /export/home/oracle/rjin/ocm.rsp -paramfile /opt/oracle/grid/11gR203/crs/install/crsconfig_params
This is the main log file: /opt/oracle/grid/11gR203/cfgtoollogs/opatchauto2017-06-22_05-47-50.log
This file will show your detected configuration and all the steps that opatchauto attempted to do on your system:
/opt/oracle/grid/11gR203/cfgtoollogs/opatchauto2017-06-22_05-47-50.report.log
2017-06-22 05:47:50: Starting Oracle Restart Patch Setup
Using configuration parameter file: /opt/oracle/grid/11gR203/crs/install/crsconfig_params
Stopping RAC /opt/oracle/database/11gR203 ...
Stopped RAC /opt/oracle/database/11gR203 successfully
patch /nas/home/oracle/software/patches/16619898/custom/server/16619898 apply successful for home /opt/oracle/database/11gR203
patch /nas/home/oracle/software/patches/16619892 apply successful for home /opt/oracle/database/11gR203
Stopping CRS...
Stopped CRS successfully
patch /nas/home/oracle/software/patches/16619898 apply successful for home /opt/oracle/grid/11gR203
patch /nas/home/oracle/software/patches/16619892 apply successful for home /opt/oracle/grid/11gR203
Starting CRS...
CRS-4123: Oracle High Availability Services has been started.
Starting RAC /opt/oracle/database/11gR203 ...
Started RAC /opt/oracle/database/11gR203 successfully
opatch auto succeeded.

Check patches

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
+ASM:/export/home/oracle>opatch lsinv -oh /opt/oracle/database/11gR203
Oracle Interim Patch Installer version 11.2.0.3.16
Copyright (c) 2017, Oracle Corporation. All rights reserved.
Oracle Home : /opt/oracle/database/11gR203
Central Inventory : /opt/oracle/oraInventory
from : /opt/oracle/database/11gR203/oraInst.loc
OPatch version : 11.2.0.3.16
OUI version : 11.2.0.3.0
Log file location : /opt/oracle/database/11gR203/cfgtoollogs/opatch/opatch2017-06-22_06-02-30AM_1.log
Lsinventory Output file location : /opt/oracle/database/11gR203/cfgtoollogs/opatch/lsinv/lsinventory2017-06-22_06-02-30AM.txt
--------------------------------------------------------------------------------
Local Machine Information::
Hostname: slcs00dbs001
ARU platform id: 23
ARU platform description:: Solaris Operating System (SPARC 64-bit)
Installed Top-level Products (1):
Oracle Database 11g 11.2.0.3.0
There are 1 products installed in this Oracle Home.
Interim patches (2) :
Patch 16619892 : applied on Thu Jun 22 05:52:30 GMT 2017
Unique Patch ID: 16375425
Patch description: "Database Patch Set Update : 11.2.0.3.7 (16619892)"
Created on 28 Jun 2013, 05:46:54 hrs PST8PDT
Sub-patch 16056266; "Database Patch Set Update : 11.2.0.3.6 (16056266)"
Sub-patch 14727310; "Database Patch Set Update : 11.2.0.3.5 (14727310)"
Sub-patch 14275605; "Database Patch Set Update : 11.2.0.3.4 (14275605)"
Sub-patch 13923374; "Database Patch Set Update : 11.2.0.3.3 (13923374)"
Sub-patch 13696216; "Database Patch Set Update : 11.2.0.3.2 (13696216)"
Sub-patch 13343438; "Database Patch Set Update : 11.2.0.3.1 (13343438)"
Bugs fixed:
13554409, 13484963, 13476583, 10263668, 16279211, 12879027, 12748240
13430938, 13566938, 11840910, 13070939, 12976376, 12845115, 14546638
13399435, 13419660, 13448206, 13559697, 14644185, 11715084, 14273397
13616375, 6690853, 13035804, 13812031, 13718279, 13332439, 7509451
12395918, 13015379, 12925089, 13492735, 13804294, 13527323, 14138130
16279401, 14226599, 14263073, 13649031, 16694777, 13632809, 14668670
13358781, 12620823, 14076523, 13612575, 13528551, 15853081, 13914613
13464002, 13911821, 14459552, 12772404, 13427062, 9095696, 14275605
12731940, 16231699, 13370330, 13572659, 13063120, 13923374, 13810393
13903046, 13544396, 14258925, 13454210, 12950644, 12912137, 12780983
12919564, 16382448, 14035825, 13790109, 12959852, 13724193, 12718090
14262913, 16368108, 13596521, 12658411, 12662040, 13705338, 14546575
16299830, 14040433, 13699124, 14127231, 11708510, 13499128, 14546673
14207317, 13786142, 12938841, 13857111, 12983611, 12917230, 13685544
16382353, 13807411, 13493847, 13696216, 16014985, 14398795, 14472647
12971775, 14727310, 12974860, 12535346, 9873405, 13936424, 14110275
16314470, 13657605, 9761357, 13848402, 13524899, 13397104, 13916709
9706792, 16294378, 13593999, 11071989, 15869211, 12612118, 13059165
13502183, 16314466, 16314467, 16314468, 16314469, 12880299, 14176879
13550185, 12345082, 16056266, 14062792, 13579992, 14062793, 13534412
14062794, 14062795, 14052474, 14062796, 14207163, 14062797, 13354082
12780098, 13923995, 12940620, 14053457, 9858539, 9703627, 15862024
13958038, 14220725, 15862022, 13496884, 15862023, 15862021, 15862020
8547978, 13719081, 13384182, 13732226, 13362079, 14063280, 14571027
14063281, 12791981, 16619892, 15841373, 15862019, 14205448, 16794240
13466801, 15862016, 16794242, 15862017, 16794241, 15862018, 16794244
16794243, 14664355, 13981051, 13326736, 13011409, 13385346, 13366202
13041324, 14841409, 13787482, 13257247, 12797420, 16794239, 13001379
16794238, 13645875, 12594032, 12588744, 10242202, 13467683, 12998795
13680405, 14480676, 14480675, 14480674, 13584130, 14512189, 10350832
14589750, 12794305, 14409183, 12583611, 13340388, 12857027, 12646784
16175381, 13420224, 13814739, 12923168, 13035360, 14023636, 12617123
13582702, 13038684, 14613900, 13791364, 12797765, 12847466, 15905421
13250244, 16306019, 12964067, 11063191, 11877623, 13860201, 12755116
13489024, 12913474, 13910420, 13338048, 13855490, 16742095, 13377816
12401111, 13624984, 12678920, 13907462, 12656535, 14548763, 12693626
13036331, 13645917, 13099577, 13742464, 12873183, 13737746, 13457582
13103913, 16530565, 14467061, 12849688, 13080778, 13026410, 13913630
13343438, 12861463, 13098318, 13632717, 12582664, 16344871, 13503598
12905058, 13742436, 13742437, 14841558, 13561951, 13742438, 13742433
12585543, 13742434, 9659614, 12829021, 13742435, 13773133, 14841812
12894807, 13588248, 13561750, 10133521, 14263036, 14390252, 13772618
12764337, 16703112, 12834027, 14301592, 14469008, 14038787, 12960925
14088346
Patch 16619898 : applied on Thu Jun 22 05:48:57 GMT 2017
Unique Patch ID: 16398290
Patch description: "Grid Infrastructure Patch Set Update : 11.2.0.3.7 (16742216)"
Created on 1 Jul 2013, 01:30:49 hrs PST8PDT
Bugs fixed:
16619898, 16315641, 15876003, 14275572, 13919095, 13696251, 13348650
12659561, 14305980, 14277586, 13987807, 14625969, 13825231, 12794268
13000491, 13498267, 15934834, 11675721, 14082976, 12771830, 14515980
14085018, 13943175, 14102704, 14171552, 12594616, 13879428, 12897902
12726222, 12829429, 13079948, 13090686, 12995950, 13251796, 13582411
12990582, 13857364, 15856610, 13082238, 12947871, 13256955, 13037709
14535011, 12878750, 14048512, 11772838, 13058611, 13001955, 13440962
13727853, 13425727, 12885323, 12870400, 14212634, 14407395, 13332363
13430626, 13811209, 12709476, 14168708, 14096821, 14626717, 13460353
13694885, 12857064, 12899169, 13111013, 12558569, 13323698, 10260842
13085732, 16555186, 10317921, 16077216, 13869978, 12914824, 13789135
12730342, 12950823, 13355963, 13531373, 14268365, 13776758, 12720728
13620816, 13023609, 16578706, 13024624, 13039908, 15911134, 13036424
13938166, 13011520, 13569812, 12758736, 13001901, 13077654, 13430715
13550689, 13806545, 13634583, 14271305, 12538907, 13947200, 12996428
13066371, 13483672, 12897651, 13540563, 12896850, 13241779, 12728585
12876314, 12925041, 12650672, 12398492, 12848480, 13652088, 16307750
12917897, 12975811, 13653178, 13371153, 14800989, 10114953, 14001941
11836951, 14179376, 12965049, 14773530, 12765467, 12950415, 15998768
13339443, 13965075, 16210540, 14307855, 12784559, 14242977, 13955385
12704789, 13745317, 13074261, 12971251, 13993634, 13523527, 13719731
13396284, 12639013, 12867511, 12959140, 14748254, 13912274, 12829917
12349553, 12849377, 12934171, 13843080, 14496536, 13899736, 13924431
12680491, 13334158, 10418841, 12832204, 13838047, 13002015, 14639430
15920201, 12791719
--------------------------------------------------------------------------------
OPatch succeeded.