I recently had the pleasure of logging into an AIX 5.3 machine for the first time ever, to debug a corefile.
Firstly, having the mysqld binary and core is not enough, unless you have an identical machine on which to study the corefile. Library mismatches can be a problem.. IBM was kind enough to provide the snapcore utility to solve this easily.
Snapcore will gather all the libraries and create a single archive contain libs, binary, core.
So we now have a file called something like: snapcore_555060.pax
On our dev box, extract the pax archive:
gunzip snapcore_555060.pax.Z
pax -r -f snapcore_555060.pax
On your dev AIX box, make sure you have DBX installed!!
bash-3.00# lslpp -l | grep bos.adt.debugbos.adt.debug 5.3.8.0 COMMITTED Base Application Development
Now, we are ready to debug a core. But we have to instruct DBX to read the libraries
that we got from the pax archive, instead of the default libraries on this system.
dbx -F -p /usr=/home/sbester/core/usr ./mysqld ./core
And since this isn't a DBX tutorial, I'll stop here, but you'll use normal DBX commands to print a stack trace, move up/down frames, print variables, list the source code.