RSS

Troubleshooting sysctl: cannot stat /proc/sys/kernel/core_name_format: No such file or directory Set core dump file name pattern core_pattern

Troubleshooting setting kernel.core_name_format="core.%e.%p.%s.%t" in /etc/sysctl.conf caused error: sysctl: cannot stat /proc/sys/kernel/core_name_format: No such file or directory

What is core dump

The default action of certain signals is to cause a process to terminate and produce a core dump file, a disk file containing an image of the process’s memory at the time of termination. This image can be used in a debugger (e.g., gdb) to inspect the state of the program at the time that it terminated.

In some Linux, the default core dump file name is core. If there are multiple core dump happen, the later core will override the previous one. To keep the all the core dump files for easier core dumps debugging, different core dump should have different name.

core dump is disabled by default, to enable core dump, use ulimit -c

$ ulimit -c unlimited

Then you will see:

$ ulimit -c
unlimited

$ ulimit -a | grep core
core file size          (blocks, -c) unlimited

Symptom

There is suggestion to use kernel.core_name_format control coredump file name, but got following error:

$ sudo sysctl -w kernel.core_name_format="core.%e.%p.%s.%t"
sysctl: cannot stat /proc/sys/kernel/core_name_format: No such file or directory

Troubleshooting

The kernel.core_name_format is not exist in kernel parameters, the correct one is kernel.core_pattern. e.g.

$ sudo sysctl -w kernel.core_pattern="core.%e.%p.%s.%t"
kernel.core_pattern = core.%e.%p.%s.%t

By default, a core dump file is named core, but the /proc/sys/kernel/core_pattern file (since Linux 2.6 and 2.4.21) can be set to define a template that is used to name core dump files. The template can contain % specifiers which are substituted by the following values when a core file is created:

  • %% a single % character
  • %c core file size soft resource limit of crashing process (since Linux 2.6.24)
  • %d dump mode—same as value returned by prctl(2) PR_GET_DUMPABLE (since Linux 3.7)
  • %e executable filename (without path prefix)
  • %E pathname of executable, with slashes ('/') replaced by exclamation marks ('!') (since Linux 3.0).
  • %g (numeric) real GID of dumped process
  • %h hostname (same as nodename returned by uname(2))
  • %i TID of thread that triggered core dump, as seen in the PID namespace in which the thread resides (since Linux 3.18)
  • %I TID of thread that triggered core dump, as seen in the initial PID namespace (since Linux 3.18)
  • %p PID of dumped process, as seen in the PID namespace in which the process resides
  • %P PID of dumped process, as seen in the initial PID namespace (since Linux 3.12)
  • %s number of signal causing dump
  • %t time of dump, expressed as seconds since the Epoch, 1970-01-01 00:00:00 +0000 (UTC)
  • %u (numeric) real UID of dumped process

Debugging core file

The most gdb command used for a core dump file debugging is bt (or backtrace) to show the crash stack. f <n> (or frame <n>) to switch to function call, info locals and info args also helpfule.

Example of gdb session for a core dump debugging:

$ gdb a.out core
...
...
Type "apropos word" to search for commands related to "word"...
Reading symbols from a.out...done.
[New LWP 17239]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".
btCore was generated by `./a.out'.
Program terminated with signal SIGABRT, Aborted.
#0  __GI_raise ([email protected]=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50	../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0  __GI_raise ([email protected]=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x76e06230 in __GI_abort () at abort.c:79
#2  0x00015308 in bar (x=0x1eb6fa8) at foo.c:196
#3  0x00014834 in foo (x=0x1eb6fa8, y=0x1ba9c90) at foo.c:310
#4  0x00014660 in main (argc=1, argv=0x7ea78684) at foo.c:272
(gdb) f 3
#3  0x00014834 in foo (x=0x1eb6fa8, y=0x1ba9c90) at proxyd/proxyd.c:310
310	        ret = bar(x);
(gdb) info args
x = 0x1eb6fa8
y = 0x1ba9c90
(gdb) info locals
ret = 32354640
...

References