久闻systemtap工具的大名,之前也断断续续的看过它的介绍,并且利用CentOS发行版自带的systemtap相关命令试用过几次,感觉还算不错。本文将介绍一下systemtap的安装,以及对应用程序的追踪示例,不过,在此之前,先描述一下systemtap的背景知识以及相关资料。
在Solaris系统上,有一个大名鼎鼎的动态跟踪工具DTRACE,这一个相当棒的工具,曾荣获《华尔街杂志》2006技术创新大奖中的金奖,而在Linux上却没有对应的工具,当然,那是以前,因为我们已经知道,现在Linux上有了systemtap。
和ZFS文件系统一样,DTrace一直都因版权问题而无法移植到Linux上,但Oracle(SUN公司被Oracle收购)在2012年2月宣布发布DTrace for Linux beta版,即将Solaris操作系统的动态跟踪工具移植到他们的Unbreakable Enterprise Kernel(2.6.39)内,也就是说Linux人员终于也可以使用DTrace了,不过DTrace不是本文的主角,所以还是来看systemtap,而关于systemtap与DTrace的比较,请看这里。
一般的Linux发行版,比如Fedora、OpenSuse、CentOS等,已经包含有systemtap的完整支持了,看看在我的这台机器上,试用实例如下:
[root@localhost ~]# uname -a Linux localhost.localdomain 2.6.32-71.el6.x86_64 #1 SMP Fri May 20 03:51:51 BST 2011 x86_64 x86_64 x86_64 GNU/Linux [root@localhost ~]# cat /etc/issue CentOS Linux release 6.0 (Final) Kernel r on an m [root@localhost ~]# mkdir -p /home/work/systemtap/ [root@localhost ~]# cd !$ cd /home/work/systemtap/ [root@localhost systemtap]# vi lsprob.stp [root@localhost systemtap]# cat lsprob.stp probe process( "/usr/local/nginx/sbin/nginx" ).function( "*" ) { printf ( "%s(%s)n" , probefunc(), $$parms) } [root@localhost systemtap]# /usr/local/nginx/sbin/nginx [root@localhost systemtap]# stap -v lsprob.stp Pass 1: parsed user script and 71 library script(s) using 87816virt/24540res/2428shr kb, in 150usr/80sys/366real ms. Pass 2: analyzed script: 1370 probe(s), 260 function(s), 1 embed(s), 0 global(s) using 119184virt/53660res/5280shr kb, in 210usr/140sys/443real ms. Pass 3: translated to C into "/tmp/stap6xNnxj/stap_bf1d7c57b21463aadca831f0f58503b1_586504.c" using 119152virt/53960res/5544shr kb, in 80usr/970sys/2227real ms. Pass 4, preamble: (re)building SystemTap's version of uprobes. Pass 4: compiled C into "stap_bf1d7c57b21463aadca831f0f58503b1_586504.ko" in 10620usr/11870sys/27102real ms. Pass 5: starting run. ngx_time_update() ngx_gmtime(t=0x51015b28 tp=0x7fff7c70ff80) ngx_sprintf(buf=0x68761a fmt=0x460c58) ngx_vslprintf(buf=0x68761a last=0xffffffffffffffff fmt=0x460c58 args=0x7fff7c70fe60) ... ngx_destroy_pool(pool=0xef4a10) ngx_pool_cleanup_file(data=0xef53f8) ngx_event_add_timer(timer=0xfde8 ev=0xf08600) ngx_rbtree_insert(tree=0x689340 node=0xf08628) ngx_handle_read_event(rev=0xf08600 flags=0x0) ngx_pfree(pool=0xefe970 p=0xef4110) ngx_pfree(pool=0xefe970 p=0xef4600) ngx_reusable_connection(c=0x7fa37adcc180 reusable=0x1) ngx_http_run_posted_requests(c=0x7fa37adcc180) ngx_event_expire_timers() ngx_rbtree_min(sentinel=? node=0xf08628) ngx_event_process_posted(cycle=0xeea070 posted=0x689358) ngx_http_keepalive_handler(rev=0xf08600) ngx_palloc(pool=0xefe970 size=0x400) ngx_palloc_large(pool=0xefe970 size=0x400) ngx_alloc(size=0x400 log =0xefe9d0) ngx_unix_recv(c=0x7fa37adcc180 buf=0xef4110 size=0x400) ngx_http_close_connection(c=0x7fa37adcc180) ngx_close_connection(c=0x7fa37adcc180) ngx_event_del_timer(ev=0xf08600) ngx_rbtree_delete(tree=0x689340 node=0xf08628) ngx_epoll_del_connection(c=0x7fa37adcc180 flags=0x1) ngx_reusable_connection(c=0x7fa37adcc180 reusable=0x0) ngx_free_connection(c=0x7fa37adcc180) ngx_destroy_pool(pool=0xefe970) ngx_process_events_and_timers(cycle=0xeea070) ngx_event_find_timer() ngx_epoll_process_events(cycle=0xeea070 timer=0xffffffffffffffff flags=0x1) ^CPass 5: run completed in 20usr/150sys/28527real ms. [root@localhost systemtap]# |
另开一终端,发起一个nginx的HTTP请求:
[root@localhost ~]# curl 127.0.0.1 <html> <head> <title>Welcome to nginx!</title> </head> <body bgcolor= "white" text= "black" > <center><h1>Welcome to nginx!</h1></center> </body> </html> |
从上面内容可以看到,在CentOS 6.0上,利用系统自动的systemtap,可完整的捕获到Nginx的执行过程。下面介绍systemtap的源码安装方式,并且内核也换为新的香草内核。
从ftp://sources.redhat.com/pub/systemtap/releases/下载到当前(2013-01-26)最新的systemtap源码包,注意我们这里要用systemtap来追踪应用程序,因此先看看systemtap源码包里的README文件的utrace部分:
Building a kernel.org kernel:
– Consider applying the utrace kernel patches, if you wish to probe
user-space applications. http://sourceware.org/systemtap/wiki/utrace
Or if your kernel is near 3.5, apply the uprobes and related patches
(see NEWS). Or if your kernel is >= 3.5, enjoy the built-in uprobes.– Build the kernel using your normal procedures. Enable
CONFIG_DEBUG_INFO, CONFIG_KPROBES, CONFIG_RELAY, CONFIG_DEBUG_FS,
CONFIG_MODULES, CONFIG_MODULE_UNLOAD, CONFIG_UTRACE if able
– % make modules_install install headers_install
– Boot into the kernel.
从上面英文可以看到,如果要用systemtap追踪应用程序,那么有三种情况:
1,如果内核版本比较旧,那么需要下载对应的utrace补丁,比如2.6.37。
2,如果内核版本比较新,但仍低于3.5,那么需要打3个系列固定补丁,具体请看systemtap的NEWS文件。
3,如果内核版本大于等于3.5,那好,直接支持。
为了偷懒,我下载了3.7.0版本的内核代码,执行编译:
[root@localhost ~]# cd /usr/src/ [root@localhost src]# tar xjf linux-3.7.tar.bz2 [root@localhost src]# cd linux-3.7/ [root@localhost src]# make menuconfig |
保证选上这个:
Kernel hacking ---> [*] Tracers ---> [*] Enable uprobes-based dynamic events |
我在内核编译菜单里找了半天,也没有找到内核选项CONFIG_UTRACE,其实这个选项在新版本内核里已经没有了,所以不用管它。在make前,检查一下.config文件,确保如下几个选项存在:
[root@localhost src]# cat .config | grep CONFIG_DEBUG_INFO [root@localhost src]# cat .config | grep CONFIG_KPROBES [root@localhost src]# cat .config | grep CONFIG_RELAY [root@localhost src]# cat .config | grep CONFIG_DEBUG_FS [root@localhost src]# cat .config | grep CONFIG_MODULES [root@localhost src]# cat .config | grep CONFIG_MODULE_UNLOAD [root@localhost src]# make [root@localhost src]# make modules [root@localhost src]# make modules_install install headers_install |
编译重启OK,一切顺利。
试试系统自带的systemtap:
[root@localhost ~]# cd /home/work/systemtap/
[root@localhost systemtap]# cat lsprob.stp
probe process(
"/usr/local/nginx/sbin/nginx"
).function(
"*"
)
{
printf
(
"%s(%s)n"
, probefunc(), $$parms)
}
[root@localhost systemtap]# /usr/local/nginx/sbin/nginx
[root@localhost systemtap]# uname -a
Linux localhost.localdomain 3.7.0 #1 SMP Wed Jan 9 04:46:12 CST 2013 x86_64 x86_64 x86_64 GNU/Linux
[root@localhost systemtap]# cat /etc/issue
CentOS Linux release 6.0 (Final)
Kernel r on an m
[root@localhost systemtap]# stap -v lsprob.stp
Pass 1: parsed user script and 71 library script(s)
using
87952virt/24532res/2424shr kb, in 200usr/110sys/310real ms.
semantic error: process probes not available without kernel CONFIG_UTRACE
while
resolving probe point process(
"/usr/local/nginx/sbin/nginx"
).function(
"*"
)
Pass 2: analyzed script: 0 probe(s), 0 function(s), 0 embed(s), 0 global(s)
using
88612virt/25324res/2560shr kb, in 10usr/0sys/8real ms.
Pass 2: analysis failed. Try again with another
'--vp 01'
option.
[root@localhost systemtap]#
提示出错,这很正常,因为系统自带的systemtap版本比较低,还会依赖并判断CONFIG_UTRACE选项是否存在:
[root@localhost systemtap]# stap -V SystemTap translator/driver (version 1.2/0.148 non-git sources) Copyright (C) 2005-2010 Red Hat, Inc. and others This is free software; see the source for copying conditions. |
用新版本的systemtap试试,安装方法也很简单:
[root@localhost systemtap]# ls lsprob.stp systemtap-2.0.tar.gz [root@localhost systemtap]# tar xzf systemtap-2.0.tar.gz [root@localhost systemtap]# cd systemtap-2.0 [root@localhost systemtap-2.0]# ./configure --prefix=/usr/local/systemtap/ [root@localhost systemtap-2.0]# make; make install; [root@localhost systemtap-2.0]# cd .. [root@localhost systemtap]# /usr/local/systemtap/bin/stap -V Systemtap translator/driver (version 2.0/0.148, non-git sources) Copyright (C) 2005-2012 Red Hat, Inc. and others This is free software; see the source for copying conditions. enabled features: LIBRPM LIBSQLITE3 NSS BOOST_SHARED_PTR TR1_UNORDERED_MAP NLS [root@localhost systemtap]# /usr/local/systemtap/bin/stap -v lsprob.stp Pass 1: parsed user script and 90 library script(s) using 186812virt/21572res/2744shr/19532data kb, in 190usr/50sys/244real ms. Pass 2: analyzed script: 1370 probe(s), 272 function(s), 3 embed(s), 0 global(s) using 338640virt/37872res/6068shr/32224data kb, in 1690usr/2180sys/3630real ms. Pass 3: translated to C into "/tmp/stapKIcR2o/stap_b06a01dc8793131b49ab0ed87f010c1c_610081_src.c" using 339056virt/38520res/6072shr/33036data kb, in 240usr/280sys/520real ms. In file included from /usr/local/systemtap/share/systemtap/runtime/linux/task_finder.c:17, from /usr/local/systemtap/share/systemtap/runtime/linux/runtime.h:169, from /usr/local/systemtap/share/systemtap/runtime/runtime.h:17, from /tmp/stapKIcR2o/stap_b06a01dc8793131b49ab0ed87f010c1c_610081_src.c:21: /usr/local/systemtap/share/systemtap/runtime/linux/task_finder2.c: In function ‘__stp_get_mm_path’: /usr/local/systemtap/share/systemtap/runtime/linux/task_finder2.c:441: error: ‘VM_EXECUTABLE’ undeclared (first use in this function) /usr/local/systemtap/share/systemtap/runtime/linux/task_finder2.c:441: error: (Each undeclared identifier is reported only once /usr/local/systemtap/share/systemtap/runtime/linux/task_finder2.c:441: error: for each function it appears in.) In file included from /tmp/stapKIcR2o/stap_b06a01dc8793131b49ab0ed87f010c1c_610081_src.c:88294: /usr/local/systemtap/share/systemtap/runtime/linux/uprobes-inode.c: In function ‘stapiu_change_plus’: /usr/local/systemtap/share/systemtap/runtime/linux/uprobes-inode.c:410: error: ‘VM_EXECUTABLE’ undeclared (first use in this function) /usr/local/systemtap/share/systemtap/runtime/linux/uprobes-inode.c: In function ‘stapiu_get_task_inode’: /usr/local/systemtap/share/systemtap/runtime/linux/uprobes-inode.c:512: error: ‘VM_EXECUTABLE’ undeclared (first use in this function) make[1]: *** [/tmp/stapKIcR2o/stap_b06a01dc8793131b49ab0ed87f010c1c_610081_src.o] Error 1 make: *** [_module_/tmp/stapKIcR2o] Error 2 WARNING: kbuild exited with status: 2 Pass 4: compiled C into "stap_b06a01dc8793131b49ab0ed87f010c1c_610081.ko" in 8080usr/4360sys/12363real ms. Pass 4: compilation failed. Try again with another '--vp 0001' option. [root@localhost systemtap]#
错误更多了?从这里了解到是因为我们用的内核版本太新了,需要换用更新的systemtap:systemtap-20121215.tar.bz2,解压安装:
[root@localhost systemtap]# ls lsprob.stp systemtap-2.0 systemtap-20121215.tar.bz2 systemtap-2.0.tar.gz [root@localhost systemtap]# tar xjf systemtap-20121215.tar.bz2 [root@localhost systemtap]# cd systemtap-20121215.tar.bz2 -bash: cd: systemtap-20121215.tar.bz2: Not a directory [root@localhost systemtap]# cd src/ [root@localhost src]# ./configure --prefix=/usr/local/systemtap/ [root@localhost src]# make; make install
再试试:
[root@localhost systemtap]# /usr/local/systemtap/bin/stap -V Systemtap translator/driver (version 2.1/0.148, non-git sources) Copyright (C) 2005-2012 Red Hat, Inc. and others This is free software; see the source for copying conditions. enabled features: LIBRPM LIBSQLITE3 NSS BOOST_SHARED_PTR TR1_UNORDERED_MAP NLS [root@localhost systemtap]# /usr/local/systemtap/bin/stap -v lsprob.stp Pass 1: parsed user script and 90 library script(s) using 189748virt/24468res/2736shr/22460data kb, in 170usr/40sys/209real ms. Pass 2: analyzed script: 1370 probe(s), 265 function(s), 3 embed(s), 0 global(s) using 341640virt/40840res/6092shr/35216data kb, in 1540usr/2000sys/3328real ms. Pass 3: translated to C into "/tmp/stap0lA0uk/stap_31d31fafe19083e2b9ad121ef41879bd_607455_src.c" using 342052virt/41460res/6072shr/36024data kb, in 240usr/400sys/655real ms. Pass 4: compiled C into "stap_31d31fafe19083e2b9ad121ef41879bd_607455.ko" in 24160usr/6630sys/30387real ms. Pass 5: starting run. ngx_time_update() ngx_gmtime(t=0x50ecb971 tp=0x7fff6851cb80) ngx_sprintf(buf=0x68761a fmt=0x460c58) ... ngx_event_expire_timers() ngx_event_expire_timers(sentinel=? node=0x1f41628) ngx_event_process_posted(cycle=0x1f23070 posted=0x689358) ngx_http_keepalive_handler(rev=0x1f41600) ngx_palloc(pool=0x1f37970 size=0x400) ngx_palloc_large(pool=0x1f37970 size=0x400) ngx_alloc(size=0x400 log=0x1f379d0) ngx_unix_recv(c=0x7fdd38af2180 buf=0x1f2d110 size=0x400) ngx_handle_read_event(rev=0x1f41600 flags=0x0) ngx_process_events_and_timers(cycle=0x1f23070) ngx_event_find_timer() ngx_event_find_timer(sentinel=0x688c00 node=0x1f41628) ngx_epoll_process_events(cycle=0x1f23070 timer=0xfde8 flags=0x1) ngx_time_update() ngx_http_keepalive_handler(rev=0x1f41600) ngx_unix_recv(c=0x7fdd38af2180 buf=0x1f2d110 size=0x400) ngx_http_close_connection(c=0x7fdd38af2180) ngx_close_connection(c=0x7fdd38af2180) ngx_close_connection(ev=0x1f41600) ngx_rbtree_delete(tree=0x689340 node=0x1f41628) ngx_epoll_del_connection(c=0x7fdd38af2180 flags=0x1) ngx_reusable_connection(c=0x7fdd38af2180 reusable=0x0) ngx_free_connection(c=0x7fdd38af2180) ngx_destroy_pool(pool=0x1f37970) ngx_event_expire_timers() ngx_process_events_and_timers(cycle=0x1f23070) ngx_event_find_timer() ngx_epoll_process_events(cycle=0x1f23070 timer=0xffffffffffffffff flags=0x1) ^CPass 5: run completed in 50usr/170sys/108234real ms. [root@localhost systemtap]#
终于正常了。
另外,如果提示这样的错误:
configure: error: missing elfutils development headers/libraries (install elfutils-devel, libebl-dev, libdw-dev and/or libebl-devel)
试试到https://fedorahosted.org/releases/e/l/elfutils/下载对应的elfutils源码包,然后以如下方式配置:
[root@localhost stap]# ./configure –prefix=/home/stap/install/ –with-elfutils=/home/elfutils-x.xxx
注意,一定不要尝试安装elfutils,否则可能将导致系统环境变乱。
参考资料:
甲骨文发布DTrace for Linux beta版
甲骨文发布其Linux内核更新 动态跟踪框架Dtrace备受瞩目
https://lists.linux-foundation.org/pipermail/ksummit-2008-discuss/2008-June/000192.html
http://dtrace.org/blogs/ahl/2011/10/05/dtrace-for-linux-2/
http://redmonk.com/sogrady/2008/07/01/dtrace-vs-systemtap-redux/
转载请注明:爱开源 » systemtap初试用