Recently I have installed a new Virtual Machine in VM Ware Fusion 7 Pro under Oracle Enterprise Linux 6.6 for running Oracle 12c Grid Infrastructure and database (18.104.22.168). Mac OS X is a 10.9.5 (Mavericks). As usual for my tests environment, I deploy the oracle-rdbms-server-12cR1-preinstall rpm package to be sure everything will be correctly configured on my system. During the installation process, there was a first error during link edition of rman (“error invoking target irman ioracle of makefile /u01/app/oracle/product/12.1.0/grid/rdbms/lib/ins_rdbms.mk”). This problem can be fixed by copying the libjavavm12.a library located in $ORACLE_HOME/javavm/jdk/jdk7/lib to $ORACLE_HOME/lib directory, and then retry the link edition by clicking on “Retry” button. So, if you don’t use Grid Infrastructure here or if you install only the database kernel … you won’t have any problem. Maybe, you will encountered the problems detailed above if you create a mutitenant (or single tenant) database (because it needs to run catcon.pl script … but we will see why later). In my case, I was deploying Grid Infrastructure and at the end of the installation process, you have to run root.sh script to configure the CRS stack. At this step, the root.sh script fails with an error on roothas.pl script and a cute “Segmentation fault (core dumped)” error:
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
/u01/app/oracle/product/12.1.0/grid/crs/config/rootconfig.sh: line 131: 4562 Segmentation fault (core dumped) $ROOTSCRIPT $ROOTSCRIPT_ARGS
The command '/u01/app/oracle/product/12.1.0/grid/perl/bin/perl -I/u01/app/oracle/product/12.1.0/grid/perl/lib -I/u01/app/oracle/product/12.1.0/grid/crs/install /u01/app/oracle/product/12.1.0/grid/crs/install/roothas.pl ' execution failed
After analyzing the error, I get the cause of my problem which is located on the perl binary delivered with Oracle. If I run a simple “perl -v”, I got the error.
[oracle@oel6 bin]$ pwd
[oracle@oel6 bin]$ ./perl -v
Segmentation fault (core dumped)
If we go deeper, gdb shows us an error located on PerlIO functions and specifically on PerlIO_default_layers function:
[oracle@oel6 bin]$ gdb perl
GNU gdb (GDB) Red Hat Enterprise Linux (7.2-75.el6)
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
Reading symbols from /u01/app/oracle/product/12.1.0/grid/perl/bin/perl...(no debugging symbols found)...done.
Starting program: /u01/app/oracle/product/12.1.0/grid/perl/bin/perl
warning: no loadable sections found in added symbol-file system-supplied DSO at 0x7ffff7ffa000
[Thread debugging using libthread_db enabled]
Program received signal SIGSEGV, Segmentation fault.
0x0000000000531399 in PerlIO_default_layers ()
Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.149.el6_6.7.x86_64 libgcc-4.4.7-11.el6.x86_64 nss-softokn-freebl-3.14.3-22.el6_6.x86_64
#0 0x0000000000531399 in PerlIO_default_layers ()
#1 0x000000000053070e in PerlIO_resolve_layers ()
#2 0x000000000053050f in PerlIO_openn ()
#3 0x0000000000530206 in PerlIO_fdopen ()
#4 0x0000000000530198 in PerlIO_stdstreams ()
#5 0x0000000000530150 in Perl_PerlIO_stdin ()
#6 0x0000000000435f09 in S_parse_body ()
#7 0x00000000004343bb in perl_parse ()
#8 0x000000000041cf13 in main ()
IMHO, it’s a specific problem on WMWare fusion because on a VM hosted on Virtualbox there’s no problem, and more :
- If I copy the working perl binary from the virtualbox VM to the VMWare VM … execution failed on VMWare
- If I copy the failing perl binary from the VMWare VM to the virtualbox VM … it works fine
At the time I write this post, I have asked to a friend of mine if he can reproduced the problem on a VM hosted on an ESX server, I will update this post as soon as I will get his results. I made a lot of different tests (Downgrading VMWare fusion to Fusion 6 Pro, Installing OEL 6.4, OEL 7.0, Redhat 6.5 and 7, creating my vmdk’s on another disk, changing scsi driver etc … always the same problem). I read on many blogs, people have faced the same problem and have symlinked the failing perl to the system perl binary which runs fine. Doing this can produce different issues because of the INC directory which is not the same, you can encountered some errors due to version compatibility between modules in $OH/perl/lib directory and the perl binary. Indeed Oracle 12cR1 uses a perl v5.14.1 and system perl binary is 5.10 on OEL/RHEL 6, and 5.16 on OEL/RHEL7. I workarounded the problem by recreating the perl binary from sources, but you need to be careful of the $OH/perl/lib directory, because it contains all the perl modules needed by Oracle. See above, the different steps to compile perl binary and replace it in $ORACLE_HOME:
rm -rf perl
curl -O http://www.cpan.org/src/5.0/perl-5.14.1.tar.gz
tar -xvzf perl-5.14.1.tar.gz
mv perl/ perl.OLD
./Configure -des -Dprefix=$ORACLE_HOME/perl -Doptimize=-O3 -Dusethreads -Duseithreads -Duserelocatableinc ; make clean ; make ; make install
rm -rf lib/ man/
cp -r ../perl.OLD/lib/ .
cp -r ../perl.OLD/man/ .
cp ../perl.OLD/bin/dbilogstrip bin/
cp ../perl.OLD/bin/dbiprof bin/
cp ../perl.OLD/bin/dbiproxy bin/
cp ../perl.OLD/bin/ora_explain bin/
Once these steps are done and you have a running perl binary, you can safely run your root.sh script, create multitenant databases, and all the stuff that needs $ORACLE_HOME/bin/perl.
UPDATE: I have just downloaded the last release of Oracle Virtualbox (5.0) on my Macbook and the same problems occurs. They can be resolved by using the same method described in this post.
UPDATE2 : I would like to thank Deiby Gomez for his trick. Indeed, in my original post I forgot to keep some perl binaries that could be useful 😉 : dbilogstrip, dbiprof, dbiproxy and ora_explain. The script above has been modified.
UPDATE3: Danny Bryant from Enkitec has worked closely with Oracle VM and virtual box team and they discovered the origin of the bug. This one should be fixed in a next release. But, if you use Virtualbox 5.0, Danny published a workaround, you can read it following this link: http://dbaontap.com/2016/01/13/vbox5/
Pingback: Virtual Box 5 Broke my 12c RAC | Stojan's Oracle Blog
Hi, thanks for this post, it ended several days of frustration trying to install 22.214.171.124 on Linux 6 in Virtualbox 5.
Pingback: Oracle DB12c on OL7 with Virtual Box 5 installation issues - My Experiences with Oracle Technology
Pingback: Can’t Compile GI 126.96.36.199 and Segmentation Fault » Peasland Database Blog
It was the same under Mac OS X 10.10.5 Yosemite+Parallels+Oracle Linux 6 and 7+Oracle 188.8.131.52
Thank you for your post. Very good and helpful.
VMware Fusion 8
Mac OS X 10.10.5 Yosemite
Oracle Linux 6.6
Hello, I encountered this issue yesterday with VirtualBox 5.0.8 on OSX Yosemite with guest OS Oracle Linux 6u7. In my case, shutting down the VM, changing the paravirtualization interface from “Default” to “Legacy” resolved the issue. I’m not sure if this is a VirtualBox bug or an Oracle DB embedded perl bug.
Got to say thanks. Recently upgraded to a Mac and this post was so helpful when applying PSU under VirtualBox.
Pingback: ins_rdbms.mk and Segmentation fault errors with Oracle 12c - Oracle - Oracle - Toad World
thanks very much
thanks , it is very helpful .
Can be fixed in VMWare Fusion by adding this to your VMX:
cpuid.4.4.eax = “00000000000000000000000000000000”
cpuid.4.4.ebx = “00000000000000000000000000000000”
cpuid.4.4.ecx = “00000000000000000000000000000000”
cpuid.4.4.edx = “00000000000000000000000000000000”
How do you do the last code in a windows environment?
Thank you so much for this code that saved me a lot of time !
But I have to say something, it’s incredible :
I prepared this environment in a VM that failed each time, and I applied your patch and it worked fine.
Today I do it again, exactly the same VM ( a copy ) on another computer ( same architecture, a few différences ) and it’s fine !!!! No need to patch
I did just a difference, I changed the network from Bridged to NAT using VMware
I am really bored about Oracle products and those wastes of time….
If it’s the same VM which have been copied after you patched it .. IMO it’s normal.
Because, the problem is located on the perl binary delivered during the installation process. If you copied the VM after you patched it, you copied the patch and the workaround as well …
Pingback: Installation d’Oracle E-Business Suite 12.2.5 sous deux noeuds Oracle Linux 7.2: installing the datatiers! | b2train.com
Pingback: Vagrant for you RAC test environment | Future Veterans
Pingback: Installation d’Oracle E-Business Suite 12.2.5 sous deux noeuds Oracle Linux 7.2: installing the datatiers! |
Thank you so much. It helped a lot. Just did blindly and worked like charm..
Pingback: Install Cloud Control 13.2 on Mac and Virtual Box | hias222
Pingback: Perl and new Intel CPUs with 4 level cache – Linux & Unix & Database
Encountered the same 12c R2 while creating DB using DBCA and the above sol is not going to help
Any work around is greatly appriciated…..
Pingback: VBox 5.0.10 crash issues with our Hands-On-Lab | Upgrade your Database - NOW!
Pingback: VBox 5.0.10/12 issues with PERL and Seg Faults – UPDATE | Upgrade your Database - NOW!