win32asm / kedr Goto Github PK

There are situations where LeakCheck may output many false positives (see 
http://code.google.com/p/kedr/wiki/Known_Issues) which are currently too 
difficult to avoid.

Perhaps the means to suppress some kinds of errors could be useful here. The 
idea is similar to what Valgrind already uses, esp. for glibc, GTK+ and so on: 
http://valgrind.org/docs/manual/manual-core.html#manual-core.suppress

I guess, we don't need exactly those complex Valdrind-style suppressions, 
something that would allow identify the events to be supressed by their type 
and call stack would probably do.

Something like this:

ALLOC
  func1
  *
  func2
  func3

That would mean, "suppress reporting of memory leaks with the call stack that 
contains the following sequence of items: func1 called from somewhere (*), 
called from func2, which was in turn called from func3".

The call stack does not have to be fully defined, it a part of it matches, that 
would do.

Another example:

FREE
  func_bar

That would mean, "suppress reporting of unallocated frees if such events 
happened in func_bar(), including the functions called from it, etc."

The suppression information could be passed to to LeakCheck via debugfs, for 
example.

What steps will reproduce the problem?
1. Download latest kedr-0.4.2 and untar it
2. Create a new folder, kedr-build and cd kedr-build
3. cmake ../kedr-0.4.2 and build succeed
4. make 
5. Throw error message in in above make step


What is the expected output? What do you see instead?

Expected: Make should pass
Actual: 

In function ‘functions_map_add’:
/home/soibam/kedr_build/core/kedr_base.c:849:63: error: macro 
"hlist_for_each_entry" passed 4 arguments, but takes just 3
  hlist_for_each_entry(map_elem, node_tmp, &map->heads[i], list)
                                                               ^
/home/soibam/kedr_build/core/kedr_base.c:849:2: error: 
‘hlist_for_each_entry’ undeclared (first use in this function)
  hlist_for_each_entry(map_elem, node_tmp, &map->heads[i], list)
  ^
/home/soibam/kedr_build/core/kedr_base.c:849:2: note: each undeclared 
identifier is reported only once for each function it appears in
/home/soibam/kedr_build/core/kedr_base.c:850:2: error: expected ‘;’ before 
‘{’ token
  {
  ^
/home/soibam/kedr_build/core/kedr_base.c:844:21: warning: unused variable 
‘node_tmp’ [-Wunused-variable]
  struct hlist_node* node_tmp;
                     ^
/home/soibam/kedr_build/core/kedr_base.c: In function 
‘functions_map_remove’:
/home/soibam/kedr_build/core/kedr_base.c:876:63: error: macro 
"hlist_for_each_entry" passed 4 arguments, but takes just 3
  hlist_for_each_entry(map_elem, node_tmp, &map->heads[i], list)
                                                               ^
/home/soibam/kedr_build/core/kedr_base.c:876:2: error: 
‘hlist_for_each_entry’ undeclared (first use in this function)
  hlist_for_each_entry(map_elem, node_tmp, &map->heads[i], list)
  ^
/home/soibam/kedr_build/core/kedr_base.c:877:2: error: expected ‘;’ before 
‘{’ token
  {
  ^
/home/soibam/kedr_build/core/kedr_base.c:872:29: warning: unused variable 
‘map_elem’ [-Wunused-variable]
  struct functions_map_elem* map_elem;
                             ^
/home/soibam/kedr_build/core/kedr_base.c:871:21: warning: unused variable 
‘node_tmp’ [-Wunused-variable]
  struct hlist_node* node_tmp;
                     ^
/home/soibam/kedr_build/core/kedr_base.c: In function 
‘function_counters_table_get’:
/home/soibam/kedr_build/core/kedr_base.c:996:61: error: macro 
"hlist_for_each_entry" passed 4 arguments, but takes just 3
  hlist_for_each_entry(elem, node_tmp, &table->heads[i], list)
                                                             ^
/home/soibam/kedr_build/core/kedr_base.c:996:2: error: 
‘hlist_for_each_entry’ undeclared (first use in this function)
  hlist_for_each_entry(elem, node_tmp, &table->heads[i], list)
  ^
/home/soibam/kedr_build/core/kedr_base.c:997:2: error: expected ‘;’ before 
‘{’ token
  {
  ^
/home/soibam/kedr_build/core/kedr_base.c:991:21: warning: unused variable 
‘node_tmp’ [-Wunused-variable]
  struct hlist_node* node_tmp;



What version of the product are you using? On what operating system?

Product: kedr-0.4.1

OS : Ubuntu 13.04 with kernel version 3.11.0-19-generic


Please provide any additional information below.

LeakCheck should handle kfree_rcu() (appeared in version 3.0 of the kernel). 
For the present, it does not and therefore gives false positives.

N.B. kfree_rcu() is actually a macro, so we need to track whatever it unwraps 
to at the binary level.

Reported-by: Brice Goglin

Hello!

I think it would be nice to have an option to flush
statistics at any time.

There are at least two reasons:
 - Certain drivers cannot be unloaded without breaking something. There may be a way to do that, though, but it's certainly tricky, if any. Who would want to unload a video card driver during normal usage, anyway?
 - Sometimes it's useful to isolate leaks that occur during certain time span. 

The patch is pretty hackish, and, I guess, it does require additional work. But 
it works, and, if I understand the code correctly, shouldn't cause any leaks or 
crashes.

I don't have a more recent system handy to test, but on my CentOS 6.3 
2.6.32-358.23.2.el6.x86_64 system I observed the following behaviour with the 
leak checker:

'Unallocated frees' were reported on memory allocated by kzalloc_node() which 
had matching calls to kfree().  Likewise the 'Allocations' were reported as 
being much lower.  Changing all allocations to kzalloc() instead resolved all 
counts.

-Brett Ciphery

What steps will reproduce the problem?
1. mkdir build
2. cd build
3. cmake ..
4. make

What version of the product are you using? On what operating system?
Archlinux testing/linux

Please provide any additional information below.

Scanning dependencies of target kedr
[  2%] Generating arch/x86/lib/inat-tables.h
[  2%] Generating kedr_module.c
[  2%] Generating kedr_base.c
[  2%] Generating kedr_instrumentor.c
[  4%] Generating kedr_functions_support.c
[  4%] Generating kedr_target_detector.c
[  4%] Generating arch/x86/lib/inat.c
[  4%] Generating arch/x86/lib/insn.c
[  7%] Generating kedr.ko, Module.symvers
/tmp/kedr-0.4.1/build/core/kedr_base.c: In function ‘functions_map_add’:
/tmp/kedr-0.4.1/build/core/kedr_base.c:849:63: error: macro 
"hlist_for_each_entry" passed 4 arguments, but takes just 3
  hlist_for_each_entry(map_elem, node_tmp, &map->heads[i], list)
                                                               ^
/tmp/kedr-0.4.1/build/core/kedr_base.c:849:2: error: ‘hlist_for_each_entry’ 
undeclared (first use in this function)
  hlist_for_each_entry(map_elem, node_tmp, &map->heads[i], list)
  ^
/tmp/kedr-0.4.1/build/core/kedr_base.c:849:2: note: each undeclared identifier 
is reported only once for each function it appears in
/tmp/kedr-0.4.1/build/core/kedr_base.c:850:2: error: expected ‘;’ before 
‘{’ token
  {
  ^
/tmp/kedr-0.4.1/build/core/kedr_base.c:844:21: warning: unused variable 
‘node_tmp’ [-Wunused-variable]
  struct hlist_node* node_tmp;
                     ^
/tmp/kedr-0.4.1/build/core/kedr_base.c: In function ‘functions_map_remove’:
/tmp/kedr-0.4.1/build/core/kedr_base.c:876:63: error: macro 
"hlist_for_each_entry" passed 4 arguments, but takes just 3
  hlist_for_each_entry(map_elem, node_tmp, &map->heads[i], list)
                                                               ^
/tmp/kedr-0.4.1/build/core/kedr_base.c:876:2: error: ‘hlist_for_each_entry’ 
undeclared (first use in this function)
  hlist_for_each_entry(map_elem, node_tmp, &map->heads[i], list)
  ^
/tmp/kedr-0.4.1/build/core/kedr_base.c:877:2: error: expected ‘;’ before 
‘{’ token
  {
  ^
/tmp/kedr-0.4.1/build/core/kedr_base.c:872:29: warning: unused variable 
‘map_elem’ [-Wunused-variable]
  struct functions_map_elem* map_elem;
                             ^
/tmp/kedr-0.4.1/build/core/kedr_base.c:871:21: warning: unused variable 
‘node_tmp’ [-Wunused-variable]
  struct hlist_node* node_tmp;
                     ^
/tmp/kedr-0.4.1/build/core/kedr_base.c: In function 
‘function_counters_table_get’:
/tmp/kedr-0.4.1/build/core/kedr_base.c:996:61: error: macro 
"hlist_for_each_entry" passed 4 arguments, but takes just 3
  hlist_for_each_entry(elem, node_tmp, &table->heads[i], list)
                                                             ^
/tmp/kedr-0.4.1/build/core/kedr_base.c:996:2: error: ‘hlist_for_each_entry’ 
undeclared (first use in this function)
  hlist_for_each_entry(elem, node_tmp, &table->heads[i], list)
  ^
/tmp/kedr-0.4.1/build/core/kedr_base.c:997:2: error: expected ‘;’ before 
‘{’ token
  {
  ^
/tmp/kedr-0.4.1/build/core/kedr_base.c:991:21: warning: unused variable 
‘node_tmp’ [-Wunused-variable]
  struct hlist_node* node_tmp;
                     ^
make[4]: *** [/tmp/kedr-0.4.1/build/core/kedr_base.o] Error 1
make[3]: *** [_module_/tmp/kedr-0.4.1/build/core] Error 2
make[2]: *** [core/kedr.ko] Error 2
make[1]: *** [core/CMakeFiles/kedr.dir/all] Error 2
make: *** [all] Error 2

Hello,

When could you rollout a new release of kedr as the version 0.4.1 is no more 
compilable ?

Thanks in advance.

kasprintf() and kvasprintf() declared in <linux/kernel.h> and defined in 
lib/kasprintf.c allocate memory with kmalloc(). These functions are exported 
and are used by a couple of kernel modules.

It should be useful to intercept the calls to these functions, at least in 
LeakCheck.

Currently, the only way to work with several kernels on one machine is to have 
different KEDR installations, one per kernel.
If other application(e.g, testsuite) requires KEDR installed, this application 
should share same politics: one application's installation per kernel. This is 
needed even if application itself doesn't have kernel-space part.
Having several installations of one application for same purpose is very 
inconvenient.

Support for dkms functionality or similar from KEDR would be very useful.

Another "nice-to-have" feature. Not that it's crucial in any way, but it'll 
make life easier in some circumstances.

KEDR's inability to analyze several modules at once is a known problem.

There should be an open issue about it :)

Just an example where it's important: consider DRM-based GPU driver. When it's 
loaded, a whole bunch of other modules is also loaded.

i915, cfbfillrect, cfbimgbit, i2c_algo_bit, cfbcopyarea, drm_kms_helper, drm, 
intel_agp, intel_gtt, agpgart

When buildin Kedr from repository on kernel 3.4.0-rc2, compilation fails with:

/home/winnukem/kedr/build/core/arch/x86/lib/inat.c:32:13: error: conflicting 
types for ‘inat_get_escape_attribute’
/usr/src/packages/BUILD/kernel-3.4.0_rc2_gcov/arch/x86/include/asm/inat.h:101:20
: note: previous declaration of ‘inat_get_escape_attribute’ was here
/home/winnukem/kedr/build/core/arch/x86/lib/inat.c:55:13: error: conflicting 
types for ‘inat_get_group_attribute’
/usr/src/packages/BUILD/kernel-3.4.0_rc2_gcov/arch/x86/include/asm/inat.h:104:20
: note: previous declaration of ‘inat_get_group_attribute’ was here
make[4]: *** [/home/winnukem/kedr/build/core/arch/x86/lib/inat.o] Error 1
make[3]: *** [_module_/home/winnukem/kedr/build/core] Error 2
make[2]: *** [core/kedr.ko] Error 2
make[1]: *** [core/CMakeFiles/kedr.dir/all] Error 2
make: *** [all] Error 2

After changing the inat_get_escape_attribute and inat_get_group_attribute 
declarations to

insn_attr_t inat_get_escape_attribute(insn_byte_t opcode, int last_pfx, 
insn_attr_t esc_attr)
insn_attr_t inat_get_group_attribute(insn_byte_t modrm, int last_pfx, 
insn_attr_t grp_attr),

compilation still fails with

/home/winnukem/kedr/build/core/arch/x86/lib/insn.c: In function 
‘insn_get_opcode’:
/home/winnukem/kedr/build/core/arch/x86/lib/insn.c:204:3: error: implicit 
declaration of function ‘insn_last_prefix’ 
[-Werror=implicit-function-declaration]
cc1: some warnings being treated as errors
make[4]: *** [/home/winnukem/kedr/build/core/arch/x86/lib/insn.o] Error 1
make[3]: *** [_module_/home/winnukem/kedr/build/core] Error 2
make[2]: *** [core/kedr.ko] Error 2
make[1]: *** [core/CMakeFiles/kedr.dir/all] Error 2
make: *** [all] Error 2

After changing insn_get_opcode calls to insn_get_opcode_id, compilation fails 
like in http://code.google.com/p/kedr/issues/detail?id=5

When building KEDR 0.3 on the Linux kernel 3.3.0, compilation fails with the 
following error:

------------------
[ 63%] Generating kedr_cm_schedule.ko, Module.symvers
Generating kedr_cm_vmm.ko, Module.symvers
Generating kedr_cm_cmm.ko, Module.symvers
[ 63%] Generating kedr_leak_check.ko, Module.symvers
/opt/kedr/payloads_callm/waitqueue/functions_support.c: In function 
'kedr_intermediate_func___init_waitqueue_head':
/opt/kedr/payloads_callm/waitqueue/functions_support.c:96: warning: passing 
argument 2 of '__init_waitqueue_head' from incompatible pointer type
include/linux/wait.h:80: note: expected 'const char *' but argument is of type 
'struct lock_class_key *'
/opt/kedr/payloads_callm/waitqueue/functions_support.c:96: error: too few 
arguments to function '__init_waitqueue_head'
make[4]: *** [/opt/kedr/payloads_callm/waitqueue/functions_support.o] Error 1
make[3]: *** [_module_/opt/kedr/payloads_callm/waitqueue] Error 2
make[2]: *** [payloads_callm/waitqueue/kedr_cm_waitqueue.ko] Error 2
make[1]: *** [payloads_callm/waitqueue/CMakeFiles/kedr_cm_waitqueue.dir/all] 
Error 2
make[1]: *** Waiting for unfinished jobs....
------------------

It seems that the signature of __init_waitqueue_head changed in the kernel 
3.3.0:

void __init_waitqueue_head(wait_queue_head_t *q, const char *name, struct 
lock_class_key *key)

In the kernel 3.2.x and earlier, it has no 'name' argument:
void __init_waitqueue_head(wait_queue_head_t *q, struct lock_class_key *key)

The problem seems to affect call tracing facilities only. If they are not used, 
one can disable them when configuring KEDR, as a temporary workaround:

cmake -DKEDR_STANDARD_CALLM_PAYLOADS=off \
    <other_options_if_needed> <path_to_kedr_source_tree>

Kernel's structures file_system_type and file_operations use 'struct module 
*owner' field for module, which owns functions set as callback in these 
structures.
KEDR structures 'kedr_payload' and 'kedr_functions_support' use 'mod' field for 
the same purpose.

I suggest to rename 'mod' to 'owner' in KEDR structures, as it reflects field 
purpose clearly and follows kernel conventions.

This renaming would require corresponded changes in KEDR-dependent projects, 
but it shouldn't be a big problem.

What steps will reproduce the problem?
1. "make" failed in Kernel 3.11.10 x86_64

What is the expected output? What do you see instead?

./kedr_build/fault_indicators/kmalloc/indicator.c:78:2: error: implicit 
declaration of function /random32'

What version of the product are you using? On what operating system?
KEDR 0.4.1
Kernel 3.11.10 x86_64 (Fedora 20)

Please provide any additional information below.

KEDR itself is only a platform for building instruments for analyze kernel 
modules.

Several instruments are already created and included into distributed package - 
KEDR Leak Check, KEDR Fault Simulation and KEDR Call Monitoring.

But documentation refers only to "facilities" provided by the KEDR when refers 
to fault simulation or call monitoring.
Because of this, users sometimes (incorrectly) say, that "KEDR platform allows 
fault simulation and allows to setup scenarios for it.".

It would be better to explicitly distinguish KEDR (as platform) and instruments 
built on it in the documentation.

Current documentation already uses term "LeakCheck" when refers to detecting 
memory leaks functionality of KEDR. Description of other functionalities may be 
given in similar way.

System becomes completely unresponsible few moments after the test  
leak_check.kfree_rcu.01 begins.

Reproduced on OpenSUSE 13.1, Linux kernel 3.11.25 and 3.18.1(vanilla).
On previous distro-specific kernel (3.11.7 or near) the test has passed 
successfully. Same for many others distro/kernel pairs.

Playing with tests/leak_check/kfree_rcu_module/test_module.c shows that 
problems remains when code for 'foo' allocation/deallocation is removed, but 
disappears when code for 'bar' is removed instead.
Without Leak Check given module is loaded/unloaded without errors.

Hello,

I'd like KEDR to track sk_buff alloc/free. I wrote some basic code to track 
__alloc_skb, skb_clone and kfree_skb. It's not perfect because it only tracks 
struct sk_buff allocation/free, not the data buffer it points to, but that 
doesn't matter for me here.

Now here's my problem:

The sk_buff is allocated by me on the send side and given to the network. It's 
basically free'd by the network driver after send (outside of the module I am 
debugging). On the receive side, the incoming sk_buff is allocated by the 
driver before giving it to me, and I take care of freeing it.

I worked around the send side problem by declaring that dev_queue_xmit() takes 
care of freeing my sk_buff. The freeing doesn't occur in dev_queue_xmit() for 
real, but that's where my module last sees its outgoing sk_buff, so that's fine.

On the receive side, it's harder. If the Linux receive model was my module 
calling a core/driver function to get the next incoming sk_buff, I would just 
tell KEDR that this function allocates the sk_buff. But the Linux model is the 
opposite: the core/driver calls a callback in my module with the incoming skb 
as a parameter.

So I basically need a way to tell KEDR "the skb passed to my module here has 
been allocated earlier, now please track it". How can I do that?

Brice Goglin

PS: Yeah, I'll share my code if you're interested

Inside Linux kernel all '-' characters in module name are internally replaced 
with '_'.
KEDR should take this into account when detect target module loading.

E.g., when running with

# kedr start ideapad-laptop

KEDR should detect loading of module ideapad-laptop.

Now it detects loading of this module only if running with

# kedr start ideapad_laptop

So KEDR user should be aware of special processing of '-' character in module 
name. That is inconvenient.

Instrumentation facilities do not work for kernel 3.16-rc2 because the code of 
a kernel module now becomes read-only earlier (if 
CONFIG_DEBUG_SET_MODULE_RONX=y).

That is, earlier than the notifier is called and the instrumentation starts.

Here is the commit: 
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=49822
23e51e8ea9d09bb33c8323b5ec1877b2b51

We need to either do the instrumentation earlier (see how Ftrace got around 
this) or remove RO before and restore it after instrumentation if needed.

put_page() is sometimes used to deallocate memmory pages that were allocated 
with alloc_page() or the like. 

Among other things, page_cache_release(), which is often used by filesystem 
modules, expands to put_page() too.

It may make sense to investigate the typical usage scenarios of put_page() in 
the kernel, intercept the calls to this function and process it as a 
deallocation routine in LeakCheck.

There are functions in the kernel, which may not fail in some conditions.
Examples:
- mempool_alloc never fails if called in process context,
- bio_add_page always succeed, if single page is added to the empty bio.

Some drivers use such functions in fail-never scenarios, and intentionally do 
not check errors. Fault simulation should not break such modules.

It seems, that allowing to specify raw code for replacement function would be 
good way for support fault simulation payloads with conditional points.
This also helps in case when 'normal' function flaw may not be expressed in 
original function call(e.g., when reading function's arguments modify them).

We have found out several kernel functions that are used (at least) in JFS and 
simulating faults in these functions could make KEDR Fault Simulation better.

Here is the (still growing) list of such functions:

kmem_cache_create
kthread_create_on_node
mempool_create
mempool_alloc (Maybe no one expects this function to fail, but still...)
__dquot_alloc_space
filemap_write_and_wait_range
read_cache_page
write_one_page
submit_bio (A really dangerous function. Brings to crash at least in JFS)
d_alloc_root (This function is removed in Linux v3.4 and is replaced by 
d_make_root)