11gR2/Aix – Dedicated Server Proceses Have Large Usla Heap Segment Compared To Older Versions

Friday, 20.07.2012 – Dejan

Kojeg li neugodnog iznena─Ĺenja nakon upgradea sa 9i/10g na 11gR2 !? Opet Oracle bug – ne tako kriti─Źan, ali i ovaj zadaje glavobolju!

Kako/Zašto?

Jedan informacioni sistem, koji je radio na Oracle verziji 10g, upgradean je na 11gR2.

Naime, nakon upgradea, odmah je primije─çeno ve─çe zauze─çe RAM memorije. Po┼íto je znalo biti i oko 1000 spojenih korisnika istovremeno, dolazilo je do znatnih usporenja i pada performansi. Zajedno sa network i system administratorima sam morao na─çi uzrok ove pojave – dr┼ż, ne daj, kopaj, tra┼żi uzrok, usporedjuj sa sistemskim izvje┼ítajima od ranije – sve dok se ne sjetih ovog buga, na kojeg sam davno naletio tra┼że─çi informacije o relinkanju Oracle binariesa. Tad sam samo letimi─Źno preletio preko njega, ne ─Źitaju─çi detaljno, a ─Źak sam i stavio ovaj MOS Note ID (“11gR2/Aix – Dedicated Server Proceses Have Large Usla Heap Segment Compared To Older Versions [ID 1260095.1]“) u Favorites, planiraju─çi ga kasnije detaljno prou─Źiti, no zaboravih na to. Ipak, u podsvijesti mi je ostala natuknica na sta se ovaj “bug” odnosio, pa sam ga br┼że-bolje “prostudirao”. ­čÖé

Naime, o ─Źemu se radi !?

Radi se o bugu “PROCESSES CONSUMING ADDITIONAL MEMORY DUE TO ‘USLA HEAP‘”.

Od verzije 11gR2, Oracle server proces┬á (oracle binary) je linkan sa dodatnim link opcijama: –bexpful i -brtllib

Opcije -bexpfull i -brtllib su dodane, kako bi se omogu─çila “hot patching” funkcionalnost. U prethodnim verzijama 9i, 10g i 11gR1 ove opcije nisu kori┼ítene.

Rješenje?

IBM i Oracle preporu─Źuju upgrade AIX OS-a na verziju 6.1 TL07 ili 7.1 TL01, te potom instalaciju Oracle Patcha Patch:13443029.
Patch 10190759 je za verzije starije od AIX 6.1 TL07 i 7.1 TL-01. U na┼íem slu─Źaju se radi o AIX 6.1 TL06.

Kako izgleda u praksi?

Kao ┼íto je opisano u MOS Note ID 1260095.1, zauze─çe “USLA heap” je otprilike ovoliko:

Oracle Release -> (work USLA heap times 4k pages size)
11.2.0.1.0 -> 7M bytes
11.1.0.7.0 -> 60KB
10.2.0.4.0 -> 420KB

Ovako mo┼żete provjeriti, koliko “USLA heap” memorije jedan oracle proces stvarno zauzima:

– Oracle 10.2.0.4:

oracle$ ps -ef|grep oracleISPRO1

oracle 63701024 1 0 07:34:56 – 0:00 oracleISPRO1 (LOCAL=NO)

oracle$ echo `expr $(svmon -P 63701024 | grep “work USLA heap” | awk ‘{print $7}’) \* 4` kBytes
436 kBytes

– Oracle 11.1.0.7:

oracle$ ps -ef|grep oracleRSKPROD

oracle 39649488 1 0 Jun 19 – 15:50 oracleRSKPROD (LOCAL=NO)

oracle$ echo `expr $(svmon -P 39649488 |grep “work USLA heap” | awk ‘{print $7}’) \* 4` kBytes
76 kBytes

– Oracle 11.2.0.1:

oracle$ ps -ef|grep oracleCALCDB

oracle 57475154 1 0 08:23:11 – 0:02 oracleCALCDB (LOCAL=NO)

oracle$ echo `expr $(svmon -P 57475154 | grep “work USLA heap” | awk ‘{print $7}’) \* 4` kBytes
7220 kBytes

– Oracle 11.2.0.3 i 11.2.0.3.2 bez instaliranog patcha:

oracle$ ps -ef|grep oracleRMCPROD

ora11g 44105964 1 0 11:24:30 – 0:00 oracleRMCPROD (LOCAL=NO)

oracle$ echo `expr $(svmon -P 44105964 | grep “work USLA heap” | awk ‘{print $7}’) \* 4` kBytes
7484 kBytes

– Oracle 11.2.0.3 sa instaliranim patchom (10190759 za verzije starije od AIX 6.1 TL07 i 7.1 TL-01;┬á 13443029 za verzije od AIX 6.1 TL07 i 7.1 TL-01 pa nadalje):

oracle$ opatch lsinventory

Installed Top-level Products (1):

Oracle Database 11g 11.2.0.3.0
There are 1 products installed in this Oracle Home.

Interim patches (1) :

Patch 10190759 : applied on Wed Jun 13 13:28:21 CEST 2012
Unique Patch ID: 14249255
Created on 8 Dec 2011, 15:26:38 hrs UTC
Bugs fixed:
10190759

oracle$ ps -ef|grep oracleRMCPROD

oracle 36569230 1 0 14:01:09 – 0:25 oracleRMCPROD (LOCAL=NO)

oracle$ echo `expr $(svmon -P 36569230| grep “work USLA heap” | awk ‘{print $7}’) \* 4` kBytes
80 kBytes

Mi smo morali odlu─Źiti kako i ┼íta dalje – da li se vratiti na 10g, isprobati onaj workaround opisan na My Oracle Support websajtu, instalirati ovaj interim patch ili pre─çi na 11.2.0.3 sa najnovijim patchsetom. Testirali smo sa 11.2.0.3, ali bug postoji i u toj verziji, iako Oracle Support tvrdi da je bug rje┼íen ve─ç u verziji 11.2.0.2.99, tako da nam to ne donosi ni┼íta. Stoga je odluka pala na instaliranje patcha.

Edit 23.07.2012.:
Otvorio sam Service Request kod Oracle Supporta i pitao ih u vezi ovog problema – njihov odgovor je:

“Hi Dejan,

The issue you reported was investigated in:
Bug 13443029 – AIX: Excess “work USLA heap” process memory use in 11.2 on AIX (Doc ID 13443029.8)
fixed in 11.2.0.4.

Please apply Patch 13443029 – MEMORY LEAK FIX FOR ONLINE PATCHING / HOT PATCHING FOR RDBMS 11.2.0.3 AIX PPC64 on top of 11.2.0.3. This patch can be safely applied on top of 11.2.0.3.2 (your release) too. It is available for download at:

https://updates.oracle.com/download/13443029.html

Please read carefully the README file before applying the patch.

For reference please also read:
11gR2/Aix – Dedicated Server Proceses Have Large Usla Heap Segment Compared To Older Versions (Doc ID 1260095.1)
Things to Consider Before Upgrading to 11.2.0.3 to Avoid Poor Performance or Wrong Results (Doc ID 1392633.1) “

Post a Comment