Debugging core dump from Apache segfault

chrismcb · **Posted:** Mon Aug 27, 2012 4:46 am

Hi,

I've been getting segfaults from apache and have core dumps to analyse.

They end with:

Code:

Core was generated by `/usr/sbin/httpd'.
Program terminated with signal 11, Segmentation fault.
#0  0x000078a901078976 in execute () from /etc/httpd/modules/libphp5.so

From searching online, I should include PHP for the search, however it shows:

Code:

Reading symbols from /usr/bin/php...(no debugging symbols found)...done.

My installed php-related versions are:

Code:

# rpm -qa | grep 'php'
psa-appvault-phpmoney-1.3-8204320080409011649
psa-appvault-phpdig-1.85-8203120080409011645
php-5.2.17-1.el5.art
php-mcrypt-5.2.17-1.el5.art
asl-php-process-5.4.6-4.el5.art
psa-appvault-phpbugtracker-1.19-8203820080416050605
php-common-5.2.17-1.el5.art
php-mysql-5.2.17-1.el5.art
psa-php5-configurator-1.6.1-cos5.build1013120126.11
php-gd-5.2.17-1.el5.art
asl-php-pdo-5.4.6-4.el5.art
psa-appvault-phpsurveyor-0.98-8204320080409011723
psa-appvault-phpbook-1.50-8203220080409011638
psa-appvault-phpwebsite-0.10.2-8203420080409011738
psa-appvault-phpwiki-1.3.11-8204320080409011808
php-ioncube-loader-4.0.5-1.el5.art
php-pdo-5.2.17-1.el5.art
php-imap-5.2.17-1.el5.art
asl-php-common-5.4.6-4.el5.art
asl-php-gd-5.4.6-4.el5.art
psa-appvault-phpmyvisites-2.3-8202820080409011701
php-mbstring-5.2.17-1.el5.art
asl-php-5.4.6-4.el5.art
psa-appvault-phpads-2.0.8-8203520080409011611
psa-appvault-phpmyfamily-1.4.1-8203420080409011655
php-pear-1.7.2-2.el5.art
php-devel-5.2.17-1.el5.art
asl-php-mysql-5.4.6-4.el5.art
psa-appvault-phprojekt-5.2-8200820080409011713
roadsend-php-libs-2.9.8-9.el5.art
php-cli-5.2.17-1.el5.art
php-xml-5.2.17-1.el5.art
asl-php-cli-5.4.6-4.el5.art

Trying to install debuginfo for php results in:

Code:

# debuginfo-install php
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * 4PLESK: plesk.ro.4psa.com
 * 4PSA: download1.ro.4psa.com
 * atomic: www7.atomicorp.com
 * base: mirror.sov.uk.goscomb.net
 * extras: mirror.sov.uk.goscomb.net
 * updates: mirror.sov.uk.goscomb.net
4PLESK                                                                                                                                                                                                                |  951 B     00:02
4PSA                                                                                                                                                                                                                  | 1.1 kB     00:00
asl-3.0                                                                                                                                                                                                               | 1.9 kB     00:00
atomic                                                                                                                                                                                                                | 1.9 kB     00:00
base                                                                                                                                                                                                                  | 1.1 kB     00:00
extras                                                                                                                                                                                                                | 2.1 kB     00:00
mod-pagespeed                                                                                                                                                                                                         |  951 B     00:00
serverdensity                                                                                                                                                                                                         |  951 B     00:00
updates                                                                                                                                                                                                               | 1.9 kB     00:00
Checking for new repos for mirrors
Could not find debuginfo for main pkg: php-5.2.17-1.el5.art.x86_64
Could not find debuginfo pkg for dependency package bzip2-libs-1.0.3-6.el5_5.x86_64
Could not find debuginfo pkg for dependency package glibc-2.5-81.el5_8.4.x86_64
Could not find debuginfo pkg for dependency package glibc-2.5-81.el5_8.4.x86_64
Could not find debuginfo pkg for dependency package glibc-2.5-81.el5_8.4.x86_64
Could not find debuginfo pkg for dependency package glibc-2.5-81.el5_8.4.x86_64
Could not find debuginfo pkg for dependency package glibc-2.5-81.el5_8.4.x86_64
Could not find debuginfo pkg for dependency package e2fsprogs-libs-1.39-34.el5_8.1.x86_64
Could not find debuginfo pkg for dependency package glibc-2.5-81.el5_8.4.x86_64
Could not find debuginfo pkg for dependency package glibc-2.5-81.el5_8.4.x86_64
Could not find debuginfo pkg for dependency package openssl-0.9.8e-22.el5_8.4.x86_64
Could not find debuginfo pkg for dependency package glibc-2.5-81.el5_8.4.x86_64
Could not find debuginfo pkg for dependency package glibc-2.5-81.el5_8.4.x86_64
Could not find debuginfo pkg for dependency package gmp-4.1.4-10.el5.x86_64
Could not find debuginfo pkg for dependency package krb5-libs-1.6.1-70.el5.x86_64
Could not find debuginfo pkg for dependency package krb5-libs-1.6.1-70.el5.x86_64
Could not find debuginfo pkg for dependency package krb5-libs-1.6.1-70.el5.x86_64
Could not find debuginfo pkg for dependency package glibc-2.5-81.el5_8.4.x86_64
Could not find debuginfo pkg for dependency package glibc-2.5-81.el5_8.4.x86_64
Could not find debuginfo pkg for dependency package glibc-2.5-81.el5_8.4.x86_64
Could not find debuginfo pkg for dependency package glibc-2.5-81.el5_8.4.x86_64
Could not find debuginfo pkg for dependency package glibc-2.5-81.el5_8.4.x86_64
Could not find debuginfo pkg for dependency package openssl-0.9.8e-22.el5_8.4.x86_64
Could not find debuginfo pkg for dependency package libxml2-2.6.26-2.1.15.el5_8.2.x86_64
Could not find debuginfo pkg for dependency package zlib-1.2.3-4.el5.x86_64
No debuginfo packages available to install

Anyone able to offer any advice on getting to the bottom of the trace to figure out which PHP script is causing the segfaults?

Thanks

faris · **Joined:** Thu Dec 09, 2004 11:19 am **Posts:** 1846

Um...looks like you are running php 5.2.x. I would hazard a guess that that's your problem. No updates, no support, lots of bugs and no fixes for them.

I know a few people have been stuck on php 5.2 in the past for various reasons, but generally they can be overcome, so that's the direction I'd be looking right now if at all possible.

chrismcb · **Posted:** Mon Aug 27, 2012 4:22 pm

Yeah, I know it's a bit dated, but there's a few sites on the platform which are not compatible with 5.3+.

As they are client sites, it's not feasible to redevelop code to upgrade them free of charge.

If only I could keep the sites in question as 5.2 and the rest up to date!

Am I right in saying all I need is php-debuginfo for 5.2.17-1.el5.art?

If not, can you point me in the right direction?

faris · **Joined:** Thu Dec 09, 2004 11:19 am **Posts:** 1846

There doesn't seem to be one in the repo (at all - not even for 5.3.x).

But what I was getting at was that even if you find a debuginfo rpm and trace the problem, there's not a lot that can be done if you are stuck on 5.2.

chrismcb · **Posted:** Tue Aug 28, 2012 3:46 am

Thanks Faris, but I'm guessing it's a module or a script that's causing the issue.

With the PHP debugging info, it should be able to pinpoint exactly what is causing the error for me to take further action.

chrismcb · **Posted:** Tue Aug 28, 2012 4:10 am

faris wrote:

There doesn't seem to be one in the repo (at all - not even for 5.3.x).

Question then is, is there a different way to get debuginfo for the Atomic repository for any build?
http://www6.atomicorp.com/channels/atom ... 6_64/RPMS/

I have tried using an alternate build I found online (php-debuginfo-5.2.17-1.el5.vmbleed.x86_64.rpm), but gdb complains with messages like:

Code:

Reading symbols from /etc/httpd/modules/libphp5.so...
warning: the debug information found in "/usr/lib/debug/usr/lib64/httpd/modules/libphp5.so.debug" does not match "/etc/httpd/modules/libphp5.so" (CRC mismatch).

Which was expected anyway.

scott · **Posted:** Tue Aug 28, 2012 8:27 am

you'll need to change the debug nil line in the spec file if I recall correctly.

chrismcb · **Posted:** Tue Aug 28, 2012 8:29 am

scott wrote:

you'll need to change the debug nil line in the spec file if I recall correctly.

Sorry, can you point me in the right direction with this?

chrismcb · **Posted:** Tue Oct 02, 2012 9:05 am

Reviving this old thread as I have upgraded to the latest version of PHP:

Code:

php-5.3.17-10.el5.art

But, as faris mentioned, there is no debug packages in the Atomicorp repository for PHP.

Can these be added? or is there another way I can pinpoint the cause of these intermittent segfaults?

Thanks

Debugging core dump from Apache segfault

Who is online