xdp-project / xdp-tools Goto Github PK

Utilities and example programs for use with XDP

License: Other

C 79.16% Makefile 2.03% Shell 6.23% Emacs Lisp 0.27% Python 1.29% Roff 11.01%

xdp-tools's Introduction

The XDP collaboration project

This git repository is a work-in-progress workspace for collaborating on the XDP project. It provides an easy and git-diff trackable way to share notes, benchmarks and work-in-progress items.

Emacs setup and tips

Since the notes in this repository are kept in Emacs org-mode, spending a bit of time getting familiar with the setup and workflow is probably worthwhile.

Project management with org-mode

For project management in org-mode, we are heavily influenced by the setup described by Bernt Hansen in this document. There is a setup file in org-setup.el which contains setup code to configure Emacs to support the project management workflow described by Bernt. We have a few modifications to the workflow:

We don’t use time clocking on tasks
We’ve added a notion of task (or project) ownership, to filter tasks owned by others. Set the OWNER property on a task or project to claim ownership, which will filter the task from other’s agenda.

The project tasks and TODO entries are managed in the top-level xdp-project.org file and in .org files under directory areas/, which in turn links to other .org files with notes etc in the different sub-directories.

Interaction with Github issues and pull requests

There are several people working on this project, who are comfortable with different workflows. We encourage the use of the org-mode project management style described above, and to keep notes and work-in-progress stuff in the repository. However, the Github management features are also valuable, especially for the things that require direct coordination. As such, we encourage the following guidelines for using Github issues and pull requests in this repository:

Issues:

Open issues for tasks that
- require discussion between several people
- you wish to assign to others
Keep issues small and concrete

Pull requests:

Open pull requests when:
- You want explicit feedback on something
- You are changing things in parts of the repository that are primarily edited by someone else (to prevent stomping on other’s work).
But feel free to commit work-in-progress notes or text directly to master

Other worthwhile Emacs tricks

To support using git and Github in Emacs use the magit package with the forge extension. Magit is a full-featured interface for git itself, and Forge adds interaction with Github issues and pull requests directly to the Magit interface. There’s also the magit-stgit package to add support for the ‘stacked git’ package which can be useful for managing upstream kernel patch series.

If using helm for Emacs, the helm-org-rifle package allows quick searching in all open org files.

xdp-tools's People

Contributors

Stargazers

Watchers

Forkers

aspsk netcodergit chaudron vadorovsky igallar98 stevelorenz shoracek bluedotproject gaoxingwang rocker9527 falcondb polynomialdivision nysan jgseong sanholy chenhengqi anusorn pr0gr4m kalimuthu-velappan 0x1bdat arikachen zf1575192187 davemarchevsky tstellar tmtiwari qitaoxu leiqi96 liviozanol hftzone kp0zhiqian rst0git kckeiks doytsujin frankfanslc meox eightzero sebymiano gizahnl krisnova fwxiong rnshah9 msherif1234 ajmmm y7n05h opensource-thg walking-machine dhiver bluca regit menmazqj fmaurer-rh hennessytj webclinic017 maryamtahhan donaldh tjcw orangice1997 jimmy899 fichtl junka engardens pengwu-wp dmitris shhu00 noa33881 dongdong-wang dankamongmen tpapastergiou3 omarerc starrysec martzki igsilya libtraceteam sikyoon7 worldofjarcraft hhoffstaette martinkennelly tzssangglass stintel cclinuxer jontrossbach anlaneg toliu lorenzobianconi beebdev wenlxie bahorn shunghsiyu yangboyd singnic-isinglab enhaut marcos44pl zhongjiechen97 laitianli georgejguo jamestiotio yangzhou1997 sachintiptur vincentmli antonyantony

xdp-tools's Issues

xdp-dump filter support

Allow filtering like tcpdump (maybe just a subset to start with).

Add selftest run to CI

It would be good to actually run the selftests from the CI. Now that the 5.7 kernel has been released it should actually be possible to do, but the travis script will need to spin up a VM to run the tests. We can probably borrow some of the setup from the libbpf github repo.

Handle makefile rebuild when updating libbpf submodule

Issue: After running git submodule update the xdp-tools Makefile system does not detect that it needs to rebuild libbpf.

The full sequence of commands below

First make command compile everything, and warns me that I should run git submodule update:

$ make

** WARNING **: git submodule SHA-1 out-of-sync
 consider running: git submodule update


lib

  util
    CC       params.o
    CC       logging.o
    CC       util.o
    CC       stats.o

  libxdp
    CC       staticobjs/libxdp.o
    LINK     libxdp.a
    CC       sharedobjs/libxdp.o
    LINK     libxdp.so.0.0.3
    M4       xdp-dispatcher.c
    CLANG    xdp-dispatcher.o
    LLC      xdp-dispatcher.o

xdp-filter
[... cut ...]

xdp-loader
make[2]: Nothing to be done for 'all'.
    CC       xdp-loader
    LINK     xdp-dispatcher.o

xdp-dump
make[2]: Nothing to be done for 'all'.
    CC       xdpdump
    CLANG    xdpdump_bpf.o
    LLC      xdpdump_bpf.o
    LINK     xdp-dispatcher.o

Next I run git submodule update which finds a new libbpf version:

$ git submodule update
Submodule path 'lib/libbpf': checked out '3b239425426e4fa1c204ea3c708d36ec3f509702'

Then I run make to get everything compile against the new libbpf, but that does not happen:

$ make

lib

  util
make[2]: Nothing to be done for 'all'.

  libxdp
make[2]: Nothing to be done for 'all'.

xdp-filter
make[2]: Nothing to be done for 'all'.

xdp-loader
make[2]: Nothing to be done for 'all'.

xdp-dump
make[2]: Nothing to be done for 'all'.

AF_XDP: Provide synchronisation between processes using XSKs on the same interface

With the addition of the AF_XDP code we will be attaching XDP programs on behalf of the socket owner to redirect the packets. We should make sure this is synchronised so multiple processes can attach sockets to the same interface without stepping on each others' toes.

Since we already have a synchronisation point when building the dispatcher, this could likely be reused.

See also this upstream discussion about doing this in libbpf using bpf_link (which is an approach that will not work with multi-prog dispatchers today, sadly):
https://lore.kernel.org/bpf/[email protected]/T/#t

xdp-loader - Couldn't attach XDP program

Hello,
I seem to be having a problem loading xdp programs using xdp-loader.
When attempting to load the basic01-xdp-pass XDP program from the xdp-tutorial examples, I get the following error's

xdp-loader load -v --mode skb lo basic01-xdp-pass/xdp_pass_kern.o

Current rlimit is infinity or 0. Not raising
Loading 1 files on interface 'lo'.
XDP program 0: Run prio: 50. Chain call actions: XDP_PASS
libxdp: Couldn't open BPF file
Couldn't attach XDP program on iface 'lo': Kernel verifier blocks program loading(-4007)

Notable errors from -vv

libbpf: added 60 insn from .text to prog xdp/dispatcher
libbpf: load bpf program failed: Invalid argument
libbpf: -- BEGIN DUMP LOG ---
libbpf:
Validating prog0() func#1...
btf_vmlinux is malformed
Arg#0 type PTR in prog0() is not supported yet.
processed 0 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0

libbpf: -- END LOG --
libbpf: failed to load program 'xdp/dispatcher'
libbpf: failed to load object '/usr/lib64/bpf/xdp-dispatcher.o'
libxdp: Couldn't open BPF file
Couldn't attach XDP program on iface 'lo': Kernel verifier blocks program loading(-4007)

Testing Environment

Virtualbox VM (Memory: 2G)
CentOS Linux release 8.3.2011
Kernel: 5.10.2-1.el8.elrepo.x86_64
xdp-tools-1.0.1-1.el8.x86_64 (centos appstream repository)

Compiling the tool from source produces the same errors.
Loading xdp program's using iproute works fine.

Thanks in advance for the help.

Not capture traffic of both sides in case of container interface

I have a cluster setup, I am using docker image of hping3 and place its pod (with IP 10.32.0.2) on one Node-1 (container), Secondly, I am using Nginx server image and place its pod (with IP 10.46.0.1) on another Node-2 (container).

Through first `Node-1 pod I am flooding packets to the container of second Node (means Nginx server node).

On Node-2 (means Nginx server node), in its container Interface vethwe-datapath I have loaded an XDP program by

xdp-filter load --mode skb vethwe-datapath , As you can see,

The problem is that it captures ONLY ONE side traffic, As you can see

with sudo ./xdpdump -i vethwe-datapath -w - | tee ab.pcap | tcpdump -r - -n

but without loading an XDP-program It captures both sides ingoing outgoing as you can see

with sudo ./xdpdump -i vethwe-datapath -w - | tee ab.pcap | tcpdump -r - -n

Why why why? Without loading it works fine and give me traffic of both directions but with loading It captures only one side of traffic? I want to capture both sides by loading an XDP-program

Thank you,

Automate man page export

Tool man pages are exported from README.org in each directory, but this is currently done automatically. We should create a Makefile rule to automate this.

Attach another XDP program to an already-attached list of programs?

Dear XDP developers,

Thanks so much for your work. May I ask about the possibility to try attaching/appending another XDP program to an already-attached list of programs or replacing an attached XDP program?
I am currently developing a XDP-based traffic monitoring algorithm (as a PhD student 😉 ). I wrote the XDP program but face the problem when I want to integrate it with OVS's AF_XDP datapath and DPDK's AF_XDP PMD. Both of them use the static int xsk_setup_xdp_prog(struct xsk_socket *xsk) function (provided by libbpf) to setup the AF_XDP socket and there is a "dummy" built-in XDP program be loaded inside this function (by calling xsk_load_xdp_prog() function). Then I can not attach my monitor XDP program easily on the same veth interface. This limitation makes it difficult to test my approach using OVS AF_XDP directly. I did some workarounds (write my own simple "switch" using XDP) but it would be great if the additional XDP program can be attached. I see in the README of libxdp that:

It is expected that this restriction will be lifted in an upcoming kernel version, after which xdp_program__attach() will simply add the program being loaded into the existing chain of programs running on the interface.

Is it possible to use this feature in the kernel version 5.8? Or I can test this feature by compiling the kernel source from a special testing repo? Thanks so much for your patience and answer.

With best regards,

Zuo Xiang

Update headers/linux/ to the latest?

Does it make sense to update the headers/linux/ files, or is there a specific cadence for doing this?

xdp-dump: add option to enable promiscious mode

/usr/include/linux/types.h:5:10: fatal error: 'asm/types.h' file not found

here is compiling error on Ubuntu 18.04
root@vincent-desktop:/usr/src/xdp-tools# ./configure
libbpf support: submodule
ELF support: yes
zlib support: yes
root@vincent-desktop:/usr/src/xdp-tools# make

lib

util
CC params.o
CC logging.o
CC util.o
CC stats.o

xdp-filter
CLANG xdpfilt_blk_udp.o
In file included from xdpfilt_blk_udp.c:10:
In file included from ./xdpfilt_prog.h:10:
In file included from ../lib/../headers/linux/bpf.h:11:
/usr/include/linux/types.h:5:10: fatal error: 'asm/types.h' file not found
#include <asm/types.h>
^~~~~~~~~~~~~
1 error generated.
../lib/common.mk:77: recipe for target 'xdpfilt_blk_udp.o' failed
make[1]: *** [xdpfilt_blk_udp.o] Error 1
Makefile:19: recipe for target 'all' failed
make: *** [all] Error 2

this error is similar to issue addressed in https://code.forksand.com/oisf/suricata/commit/7906c521cdde5b1d0eb3ce379b8e343c3055653f
I added diff below to fix the error, maybe similar patch needed for xdp-tools ?

diff --git a/lib/common.mk b/lib/common.mk
index dd71535..6127450 100644
--- a/lib/common.mk
+++ b/lib/common.mk
@@ -75,6 +75,7 @@ $(USER_TARGETS): %: %.c  $(OBJECT_LIBBPF) $(LIBMK) $(LIB_OBJS) $(KERN_USER_H) $(
 
 $(XDP_OBJ): %.o: %.c $(KERN_USER_H) $(EXTRA_DEPS) $(BPF_HEADERS) $(LIBMK)
        $(QUIET_CLANG)$(CLANG) -S \
+           -I/usr/include/x86_64-linux-gnu/ \
            -target bpf \
            -D __BPF_TRACING__ \
            $(BPF_CFLAGS) \

xdp-dump tests failing...

     [test_capt_pcap]              FAIL
	PING fc42:dead:cafe:1::2(fc42:dead:cafe:1::2) 56 data bytes
	64 bytes from fc42:dead:cafe:1::2: icmp_seq=1 ttl=64 time=0.068 ms
	
	--- fc42:dead:cafe:1::2 ping statistics ---
	1 packets transmitted, 1 received, 0% packet loss, time 0ms
	rtt min/avg/max/mdev = 0.068/0.068/0.068/0.000 ms
	PING fc42:dead:cafe:1::2(fc42:dead:cafe:1::2) 56 data bytes
	64 bytes from fc42:dead:cafe:1::2: icmp_seq=1 ttl=64 time=0.047 ms
	
	--- fc42:dead:cafe:1::2 ping statistics ---
	1 packets transmitted, 1 received, 0% packet loss, time 0ms
	rtt min/avg/max/mdev = 0.047/0.047/0.047/0.000 ms
	../lib/testing/test_runner.sh: line 109: kill: (-22436) - No such process
	reading from file -, link-type EN10MB (Ethernet), snapshot length 262144
	23:24:35.836630 IP6 :: > ff02::1:ff63:4b64: ICMP6, neighbor solicitation, who has fe80::f496:a8ff:fe63:4b64, length 32
	23:24:36.380195 IP6 fc42:dead:cafe:1::2 > fc42:dead:cafe:1::1: ICMP6, echo reply, id 174, seq 1, length 64
	tcpdump: pcap_loop: error reading dump file: Interrupted system call
	ERROR: IPv6 packet not received
     [test_capt_pcapng]            FAIL
	PING fc42:dead:cafe:1::2(fc42:dead:cafe:1::2) 56 data bytes
	64 bytes from fc42:dead:cafe:1::2: icmp_seq=1 ttl=64 time=0.075 ms
	
	--- fc42:dead:cafe:1::2 ping statistics ---
	1 packets transmitted, 1 received, 0% packet loss, time 0ms
	rtt min/avg/max/mdev = 0.075/0.075/0.075/0.000 ms
	PING fc42:dead:cafe:1::2(fc42:dead:cafe:1::2) 56 data bytes
	64 bytes from fc42:dead:cafe:1::2: icmp_seq=1 ttl=64 time=0.047 ms
	
	--- fc42:dead:cafe:1::2 ping statistics ---
	1 packets transmitted, 1 received, 0% packet loss, time 0ms
	rtt min/avg/max/mdev = 0.047/0.047/0.047/0.000 ms
	../lib/testing/test_runner.sh: line 109: kill: (-22465) - No such process
	reading from file -, link-type EN10MB (Ethernet), snapshot length 262144
	23:24:39.359278 IP6 fc42:dead:cafe:1::2 > fc42:dead:cafe:1::1: ICMP6, echo reply, id 176, seq 1, length 64
	tcpdump: pcap_loop: error reading dump file: Interrupted system call
	ERROR: IPv6 packet not received

Do not load dispatcher when only xsk program is active

The XDP dispatcher has a significant performance impact (up to 11% decrease) on the performance of an AF_XDP socket. One way to get around this is to not load the dispatcher when the default xsk program is the only one active. The dispatcher is not needed in this case.

Toke suggested that we add some special handling of the xsk program and just load it when there is no other XDP program there. If more programs are added, we simply add the dispatcher and in that case the performance drop should be justified, since you are getting added functionality. The question now is: why do we not do this for any program, i.e. only load the dispatcher only if there are two or more XDP programs that want to share an interface? The only reason I can come up with is that it is much faster to add or remove a program if the dispatcher is already there. Adding the dispatcher plus two programs might result in dropped packets due to this latency, but I do not know. Other reasons?

Plan from Toke:

When attaching XSK program, if nothing is attached already, just
attach it directly to the interface.
When attaching XSK program and another program is already installed,
do the regular thing and attach it as a secondary program with a
dispatcher.
When attaching another program and an XSK program is already
attached, generate a new dispatcher with a new copy of the XSK program
as one of the sub-programs, but reusing the map from the program
already attached to the interface; then replace the existing program
with that dispatcher.

xdp-dump: No BTF found for program

I dont have BTF enabled in my kernel. Is there any way to use xdp-dump without BTF?

Update symbol versioning script

The symbol versioning script is a bit of a work-in-progress, so we need to check that it is working correctly and bump the initial symbol version for a 1.0 release

Build fails on missing linux/if_xdp.h

On another machine, build fails due to missing linux/if_xdp.h

$ make
[...]
make[3]: Entering directory '/home/jbrouer/git/xdp-tools/lib/util'
cc -I../libbpf/src -Wall -I../../headers -c -o params.o params.c
params.c:13:10: fatal error: linux/if_xdp.h: No such file or directory
 #include <linux/if_xdp.h>
          ^~~~~~~~~~~~~~~~
compilation terminated.

In the XDP-tutorial this issue is already solved, and we actually have two copies of this file:

[xdp-tutorial]$ find . | grep if_xdp
./libbpf/include/uapi/linux/if_xdp.h
./headers/linux/if_xdp.h

And in this XDP-tools repo it's indirectly available via libbpf submodule.

Make tests runnable from installed package

Fix up the selftests to make them runnable from the installed version, and amend the RPM to install and run them.

It possible to use xdp-filter on many interfaces?

If I use NIC bonding and I want to protect all the slave interfaces, can I use xdp-filter?

xdpdump tests fail on new Wireshark versions

Seems the regexes testing capinfos output are a bit too specific?

$ capinfos --version
Capinfos (Wireshark) 3.2.4 (Git commit 893b5a5e1e3e)

test xdp-dump
    Running tests from test-xdpdump.sh
     [test_help]		PASS
     [test_interfaces]		PASS
     [test_capt_pcap]		PASS
     [test_capt_pcapng]		FAIL
	PING fc00:dead:cafe:1::2(fc00:dead:cafe:1::2) 56 data bytes
	64 bytes from fc00:dead:cafe:1::2: icmp_seq=1 ttl=64 time=0.033 ms
	
	--- fc00:dead:cafe:1::2 ping statistics ---
	1 packets transmitted, 1 received, 0% packet loss, time 0ms
	rtt min/avg/max/mdev = 0.033/0.033/0.033/0.000 ms
	PING fc00:dead:cafe:1::2(fc00:dead:cafe:1::2) 56 data bytes
	64 bytes from fc00:dead:cafe:1::2: icmp_seq=1 ttl=64 time=0.129 ms
	
	--- fc00:dead:cafe:1::2 ping statistics ---
	1 packets transmitted, 1 received, 0% packet loss, time 0ms
	rtt min/avg/max/mdev = 0.129/0.129/0.129/0.000 ms
	PING fc00:dead:cafe:1::2(fc00:dead:cafe:1::2) 56 data bytes
	64 bytes from fc00:dead:cafe:1::2: icmp_seq=1 ttl=64 time=0.176 ms
	
	--- fc00:dead:cafe:1::2 ping statistics ---
	1 packets transmitted, 1 received, 0% packet loss, time 0ms
	rtt min/avg/max/mdev = 0.176/0.176/0.176/0.000 ms
	REGEX: (File type:           Wireshark\/\.\.\. - pcapng.*Capture hardware:    unknown.*Capture oper-sys:    Linux alrua-x1 5\.7\.1-arch1-1 #1 SMP PREEMPT Sun, 07 Jun 2020 12:28:01 \+0000.*Capture application: xdpdump v[0-9]+\.[0-9]+\.[0-9]+.*Interface #0 info:.*Name = xdptest@fentry.*Description = xdptest:xdp_dispatcher\(\)@fentry.*Hardware = driver: "veth", version: "1\.0", fw-version: "", rom-version: "", bus-info: "".*Time precision = nanoseconds \(9\).*Interface #1 info:.*Name = xdptest@fexit.*Description = xdptest:xdp_dispatcher\(\)@fexit.*Hardware = driver: "veth", version: "1\.0", fw-version: "", rom-version: "", bus-info: "".*Time precision = nanoseconds \(9\))
	File name:           /tmp/xdptest_PID_63744_30064.pcap
	File type:           Wireshark/... - pcapng
	File encapsulation:  Ethernet
	File timestamp precision:  nanoseconds (9)
	Packet size limit:   file hdr: (not set)
	Number of packets:   2
	File size:           896 bytes
	Data size:           236 bytes
	Capture duration:    0,000045889 seconds
	First packet time:   2020-06-10 15:19:20,394685883
	Last packet time:    2020-06-10 15:19:20,394731772
	Data byte rate:      5.155 kBps
	Data bit rate:       41 Mbps
	Average packet size: 118,00 bytes
	Average packet rate: 43 kpackets/s
	SHA256:              3d46499267c66e226b0705b8535ee1dd2d08cf0f08b6233a2ff6b277a9a9ad32
	RIPEMD160:           0ea91f28fb3635b518ac7277ffbdbb3369b4b6c5
	SHA1:                32187461abbb5c0208c2f5abc1f82f83b3c0361d
	Strict time order:   True
	Capture hardware:    x86_64
	Capture oper-sys:    Linux alrua-x1 5.7.1-arch1-1 #1 SMP PREEMPT Sun, 07 Jun 2020 12:28:01 +0000
	Capture application: xdpdump v0.0.3
	Number of interfaces in file: 2
	Interface #0 info:
	                     Name = xdptest@fentry
	                     Description = xdptest:xdp_dispatcher()@fentry
	                     Encapsulation = Ethernet (1 - ether)
	                     Hardware = driver: "veth", version: "1.0", fw-version: "", rom-version: "", bus-info: ""
	                     Speed = 10000000000
	                     Capture length = 0
	                     Time precision = nanoseconds (9)
	                     Time ticks per second = 1000000000
	                     Time resolution = 0x09
	                     Number of stat entries = 0
	                     Number of packets = 1
	Interface #1 info:
	                     Name = xdptest@fexit
	                     Description = xdptest:xdp_dispatcher()@fexit
	                     Encapsulation = Ethernet (1 - ether)
	                     Hardware = driver: "veth", version: "1.0", fw-version: "", rom-version: "", bus-info: ""
	                     Speed = 10000000000
	                     Capture length = 0
	                     Time precision = nanoseconds (9)
	                     Time ticks per second = 1000000000
	                     Time resolution = 0x09
	                     Number of stat entries = 0
	                     Number of packets = 1
	ERROR: Failed capinfos content
     [test_capt_term]		FAIL
	PING fc00:dead:cafe:1::2(fc00:dead:cafe:1::2) 56 data bytes
	64 bytes from fc00:dead:cafe:1::2: icmp_seq=1 ttl=64 time=0.066 ms
	
	--- fc00:dead:cafe:1::2 ping statistics ---
	1 packets transmitted, 1 received, 0% packet loss, time 0ms
	rtt min/avg/max/mdev = 0.066/0.066/0.066/0.000 ms
	XDP program already loaded on ifindex 36; replacing not yet supported
	Couldn't attach XDP program on iface 'xdptest': File exists(-17)
     [test_exitentry]		FAIL
	XDP program already loaded on ifindex 36; replacing not yet supported
	Couldn't attach XDP program on iface 'xdptest': File exists(-17)
     [test_snap]		FAIL
	XDP program already loaded on ifindex 36; replacing not yet supported
	Couldn't attach XDP program on iface 'xdptest': File exists(-17)
     [test_multi_pkt]		FAIL
	XDP program already loaded on ifindex 36; replacing not yet supported
	Couldn't attach XDP program on iface 'xdptest': File exists(-17)
     [test_perf_wakeup]		FAIL
	XDP program already loaded on ifindex 36; replacing not yet supported
	Couldn't attach XDP program on iface 'xdptest': File exists(-17)
     [test_none_xdp]		PASS
make[1]: *** [../lib/common.mk:136: test] Error 1
make: *** [Makefile:66: test] Error 2

Couldn't find a BPF file when load xdp-filter

Dear developers,

I'm a newbie of xdp, when I tried to use the xdp-tools, some error occurred. It's really appreciated that you could take a look at this issue and kindly reply it.

When I make in the /xdp-tools, xdp-filter, loader is ok, but when it goes to xdp-dump, errors popped up:

xdpdump_bpf.c:25:18: error: unknown attribute 'preserve_access_index' ignored [-Werror,-Wunknown-attributes]
} __attribute__((preserve_access_index));
                 ^
xdpdump_bpf.c:33:18: error: unknown attribute 'preserve_access_index' ignored [-Werror,-Wunknown-attributes]
} __attribute__((preserve_access_index));
                 ^
xdpdump_bpf.c:42:18: error: unknown attribute 'preserve_access_index' ignored [-Werror,-Wunknown-attributes]
} __attribute__((preserve_access_index));
                 ^
3 errors generated.
make[1]: *** [../lib/common.mk:88: xdpdump_bpf.o] Error 1
make: *** [Makefile:26: xdp-dump] Error 2

I supposed that this won't affect the usage of xdp-filter, but when I issue ./xdp-filter load -m skb <$nic_name> , another three errors poped up:

Couldn't find a BPF file with name xdp-dispatcher.o
Couldn't open BPF file
Couldn't attach XDP program on iface 'enp68s0f0': No such file or directory(-2)

I think I must miss something that caused this issue, but I have no clue...

Thanks in advance!!

lib/util/logging.c line 40-42 I was puzzled about the meaning of this code.

Couldn't load BPF program: Unknown error 4001

Hi,

I'm trying to load xdp-filter on my interfaces. I'm able to attach xdp programs that I've written and then loaded with ip link set dev eth11 xdp obj block.o and they do work properly.

However, when I try to run xdp-filter, I get this:

./xdp-filter load eth11
Couldn't load BPF program: Unknown error 4001

I'll note that before I ran that, I did remove any running xdp programs (ip link set dev eth11 xdp off)

generic XDP mode

XDP programs can be loaded in several modes:

Generic – Driver doesn’t have support for XDP, but the kernel fakes it. XDP program works, but its not nearly as efficient as the other modes.

Native – Driver has XDP support and can hand then to XDP without kernel stack interaction

Offloaded – XDP can be loaded and executed directly on the NIC

I've got two NICs that can do 'native' XDP, and one that can only do 'generic'. I'm able to attach XDP programs that I've written to the 'generic' NIC, and it does function. However, when I try to load with xdp-filter, it is not attached:

libxdp: Failed to load dispatcher: Operation not supported
libxdp: Falling back to loading single prog without dispatcher
Couldn't attach XDP program on iface 'eth3': Operation not supported(-95)

When I manually add my own xdp program, as follows, it does get loaded:

ip link set dev eth3 xdp obj block.o

I can see the interface has the program loaded:

3: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 xdpgeneric qdisc mq state UP mode DEFAULT group default qlen 1000 link/ether 90:e2:ba:31:82:c5 brd ff:ff:ff:ff:ff:ff prog/xdp id 62 tag 2892ef4bc3a9547c

note the 'xdpgeneric' flag and the program id and tag.

On an interface with 'native' support look like this when programs are loaded have 'xdp' listed instead:

eth10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 xdp/id:444 qdisc mq state UP group default qlen 1000

Shouldn't xdp-filter support 'generic' mode as well, because its just using libpf?

thanks!

Selftests for xdpdump

We need at least basic selftests for xdpdump

xdp-dump: allow tracing one or more specific programs in a multiprogram environment

xdp-filter: old man page LOAD option about "-s"

xdp-filter man page about LOAD has the old option -s while the xdp-filter load --help has the new one -m

root@netqe2 ~]# xdp-filter load -h

Usage: xdp-filter load [options] <ifname>

 Load xdp-filter on an interface

Required parameters:
  <ifname>                  Load on device <ifname>

Options:
 -F, --force                Force loading of XDP program
 -m, --mode <mode>          Load XDP program in <mode>; default native (valid values: native,skb,hw)
 -w, --whitelist            Use filters in whitelist mode (default blacklist)
 -f, --features <feats>     Features to enable; default all (valid values: tcp,udp,ipv6,ipv4,ethernet,all)
 -v, --verbose              Enable verbose logging (-vv: more verbose)
 -h, --help                 Show this help

[root@netqe2 ~]#
[root@netqe2 ~]# man xdp-filter | cat
(...)
The LOAD command
       To use xdp-filter, it must first be loaded unto an interface. This is accomplished with the load command, which takes the name of the interface as a parameter, and optionally allows  specify-
       ing  the  features  that  should  be  included.  By default all features are loaded, but de-selecting some features can speed up the packet matching, and increase performance by a substantial
       amount.

       The syntax for the load command is:

       xdp-filter load [options] <ifname>

       Where <ifname> is the name of the interface to load xdp-filter unto, and must be specified. The supported options are:

   -F, --force
       Specifying this option causes xdp-filter to unload any XDP program already loaded on the interface.

   -s, --skb-mode
       Specifying this option causes the XDP program to be loaded in the so-called skb mode (also known as generic XDP). This is a compatibility mode that results in lower performance,  but  can  be
       used on all network interfaces (whereas the default native XDP mode requires specific driver support).

Build dependency on bsd/string.h

When building xdp-tools, it fails with the following errors:

$ make
[...]
make[3]: Entering directory '/home/hawk/git/xdp-tools/lib/util'
cc -I../libbpf/src -Wall -I../../headers -c -o params.o params.c
cc -I../libbpf/src -Wall -I../../headers -c -o logging.o logging.c
cc -I../libbpf/src -Wall -I../../headers -c -o util.o util.c
util.c:9:10: fatal error: bsd/string.h: No such file or directory
 #include <bsd/string.h>
          ^~~~~~~~~~~~~~
compilation terminated.

Are there any specific reason for depending on bsd/string.h ?

xdp-dump: Use perf-ring bulking to speedup performance

The current xdp-dump implementation is not taking advantage of "bulking" through the perf-ring buffer, instead it will get wakeup for each packet. I have measured significant performance improvements when enabling this.

In struct perf_event_attr the member wakeup_events specify getting an fd notification for X events.

Example see: https://github.com/netoptimizer/prototype-kernel/blob/master/kernel/samples/bpf/xdp_tcpdump_user.c#L236-L238

The example from prototype-kernel is not complete, because it does not handle flushing/consuming frames on exit. It is actually disabled in the code, as it was loosing packets on exit.

xdp-dump: Attach own XDP prog for dumping packets

I found it very confusing that a program named xdpdump will use the tcpdump hook to dump packets, when there isn't any other XDP-prog already attached on the interfaces.

I do like that the xdpdump program warns the end-user about this behaviour:

$ sudo ./xdpdump -i mlx5p1
WARNING: Specified interface does not have an XDP program loaded, capturing
         in legacy mode!

I think xdpdump should at-least have an option to load it's own XDP program that does the dumping (via perf-ring and returns XDP_PASS). An an end-user, I want to "see" packets and timestamps at XDP-layer. (If I wanted to see it at tcpdump hook, I would use tcpdump instead. I don't see a reason for xdpdump to replicated what tcpdump tools already does).

xdp-dump: add more clear error message when BTF information is not found.

See output below, i.e. fix ERROR: Can't find function '' on interface! to be more specific as -vv is not used by default.

# ./xdpdump -i enp3s0f1 --rx-capture entry,exit -w capture.pcap -vv 
Current rlimit 67108864 already >= minimum 1048576
libxdp: No BTF for prog ID 11
libxdp: No BTF found for program
libxdp: Found legacy program with id 11 and 0 component progs
No BTF found for program
ERROR: Can't find function '' on interface!

PCAPng support for xdpdump

The addition to the PCAPng spec was approved: IETF-OPSAWG-WG/draft-ietf-opsawg-pcap@587eb9f

So we can go ahead and add the support to xdpdump.

xdp_filter feature request: more complicated combination possibilities

I'd love to be able to use xdp-filter to be able to say things like: block all UDP to the entire 192.168.1.0/24 network (but not any other network), with the exception of 192.168.1.10 port 53, and 192.168.1.11 port 53.

At the moment, there is no netmask support in the ip command, and its also not possible (at least from my reading of the README) to be able to have IP:port combinations.

I realize that this can complicate things quickly, but it would be an amazing improvement!

Unable to load xdp-filter

After compiling i tried to load xdp-filter with defaults, but it always fails to load with the below error.

sudo ./xdp-filter load -m skb enp0s8

libxdp: Failed to load program xdpfilt_alw_all: Argument list too long
Couldn't attach XDP program on iface 'enp0s8': Argument list too long(-7)

xdp-filter: Consistency of add/remove options

There are some inconsistencies in how xdp-filter handles addition/removal of the
different objects it can filter on. We should make this consistent, and add
selftests to make sure they stay that way.

[root@netqe2 ~]# xdp-filter port -p tcp,udp -m src,dst 54321
[root@netqe2 ~]# xdp-filter status
CURRENT XDP-FILTER STATUS:

Aggregate per-action statistics:
  XDP_DROP                                    461 pkts          30 KiB
  XDP_PASS                                3571195 pkts     5279647 KiB

Loaded on interfaces:
                                           Enabled features
xdpfilt_blk_all
  enp68s0f0 (skb mode)                     tcp,udp,ipv6,ipv4,ethernet,blacklist

Filtered ports:
                                           Mode             Hit counter
  54321                                    src,dst,tcp,udp  0

Filtered IP addresses:
                                           Mode             Hit counter

Filtered MAC addresses:
                                           Mode             Hit counter

[root@netqe2 ~]# xdp-filter port -p tcp -r 54321
[root@netqe2 ~]# xdp-filter status
CURRENT XDP-FILTER STATUS:

Aggregate per-action statistics:
  XDP_DROP                                    461 pkts          30 KiB
  XDP_PASS                                3571195 pkts     5279647 KiB

Loaded on interfaces:
                                           Enabled features
xdpfilt_blk_all
  enp68s0f0 (skb mode)                     tcp,udp,ipv6,ipv4,ethernet,blacklist

Filtered ports:
                                           Mode             Hit counter
  54321                                    src,udp          0

Filtered IP addresses:
                                           Mode             Hit counter

Filtered MAC addresses:
                                           Mode             Hit counter

[root@netqe2 ~]# xdp-filter port -p tcp -m dst 54321
[root@netqe2 ~]# xdp-filter status
CURRENT XDP-FILTER STATUS:

Aggregate per-action statistics:
  XDP_DROP                                    461 pkts          30 KiB
  XDP_PASS                                3571195 pkts     5279647 KiB

Loaded on interfaces:
                                           Enabled features
xdpfilt_blk_all
  enp68s0f0 (skb mode)                     tcp,udp,ipv6,ipv4,ethernet,blacklist

Filtered ports:
                                           Mode             Hit counter
  54321                                    src,dst,tcp,udp  0

Filtered IP addresses:
                                           Mode             Hit counter

Filtered MAC addresses:
                                           Mode             Hit counter

[root@netqe2 ~]# xdp-filter port -p udp -r 54321
[root@netqe2 ~]# xdp-filter status
CURRENT XDP-FILTER STATUS:

Aggregate per-action statistics:
  XDP_DROP                                    461 pkts          30 KiB
  XDP_PASS                                3571195 pkts     5279647 KiB

Loaded on interfaces:
                                           Enabled features
xdpfilt_blk_all
  enp68s0f0 (skb mode)                     tcp,udp,ipv6,ipv4,ethernet,blacklist

Filtered ports:
                                           Mode             Hit counter
  54321                                    src,tcp          0

Filtered IP addresses:
                                           Mode             Hit counter

Filtered MAC addresses:
                                           Mode             Hit counter

[root@netqe2 ~]# xdp-filter port -p udp -m dst 54321
[root@netqe2 ~]# xdp-filter status
CURRENT XDP-FILTER STATUS:

Aggregate per-action statistics:
  XDP_DROP                                    461 pkts          30 KiB
  XDP_PASS                                3571195 pkts     5279647 KiB

Loaded on interfaces:
                                           Enabled features
xdpfilt_blk_all
  enp68s0f0 (skb mode)                     tcp,udp,ipv6,ipv4,ethernet,blacklist

Filtered ports:
                                           Mode             Hit counter
  54321                                    src,dst,tcp,udp  0

Filtered IP addresses:
                                           Mode             Hit counter

Filtered MAC addresses:
                                           Mode             Hit counter


[root@netqe2 ~]# xdp-filter port -m src -r 54321
[root@netqe2 ~]# xdp-filter status
CURRENT XDP-FILTER STATUS:

Aggregate per-action statistics:
  XDP_DROP                                    461 pkts          30 KiB
  XDP_PASS                                3571195 pkts     5279647 KiB

Loaded on interfaces:
                                           Enabled features
xdpfilt_blk_all
  enp68s0f0 (skb mode)                     tcp,udp,ipv6,ipv4,ethernet,blacklist

Filtered ports:
                                           Mode             Hit counter

Filtered IP addresses:
                                           Mode             Hit counter

Filtered MAC addresses:
                                           Mode             Hit counter

[root@netqe2 ~]# xdp-filter port -p tcp,udp -m src,dst 54321
[root@netqe2 ~]# xdp-filter status
CURRENT XDP-FILTER STATUS:

Aggregate per-action statistics:
  XDP_DROP                                    461 pkts          30 KiB
  XDP_PASS                                3571195 pkts     5279647 KiB

Loaded on interfaces:
                                           Enabled features
xdpfilt_blk_all
  enp68s0f0 (skb mode)                     tcp,udp,ipv6,ipv4,ethernet,blacklist

Filtered ports:
                                           Mode             Hit counter
  54321                                    src,dst,tcp,udp  0

Filtered IP addresses:
                                           Mode             Hit counter

Filtered MAC addresses:
                                           Mode             Hit counter

[root@netqe2 ~]# xdp-filter port -m dst -r 54321
[root@netqe2 ~]# xdp-filter status
CURRENT XDP-FILTER STATUS:

Aggregate per-action statistics:
  XDP_DROP                                    461 pkts          30 KiB
  XDP_PASS                                3571195 pkts     5279647 KiB

Loaded on interfaces:
                                           Enabled features
xdpfilt_blk_all
  enp68s0f0 (skb mode)                     tcp,udp,ipv6,ipv4,ethernet,blacklist

Filtered ports:
                                           Mode             Hit counter

Filtered IP addresses:
                                           Mode             Hit counter

Filtered MAC addresses:
                                           Mode             Hit counter

[root@netqe2 ~]#

Selftests for xdp-filter

Add selftests for xdp-filter

Detect libbpf .so-version in configure and export as define

We track upstream libbpf features pretty closely, but we do need to maintain backwards compatibility with released versions. So let's add detection of the libbpf .so version in configure and use that in ifdefs.

man page for xdp-loader

We are missing a man page for xdp-loader

xdp-dump: Add XDP multi-buffer support once it gets added

XDP multi-buffer support is currently in the works, once this is added XDPdump should be updated to support these multi-buffer frames.

However, this might not require an XDPdump change, but a change to the kernel's bpf_xdp_output() helper.

xdpdump: handle the case where only an XDP_MODE_HW program is loaded.

If only XDP_ MODE_HW is loaded, fall back to legacy or XDP load based capture.

xdp-loader/xdp-filter can't unload generic/offloaded XDP programs

The xdp-loader and xdp-filter programs have an option -S for load command to attach XDP programs in SKB mode, however, no such option is provided for the unload command. Besides that, no option is provided to load/unload programs in HW offload mode.

I think that the -S option should be replaced with -M mode option to choose between xdp, xdpgeneric, and xdpoffload. (Tried to implement this fast, but was confused in how options are parsed ;) Probably will have time to do this next week if not be fixed before.)

Map key size mismatch when using xdp-loader.

Dear developers of xdp-tools,

Thanks so much for your great effort! I am a PhD student and want to implement something using XDP. Therefore, I successfully set up the development environment using xdp-tool and started to follow xdp-tutorial. I mainly want to use the xdp-loader provided by xdp-tool to make loading/unloading stuff easier. When tried basic04 in xdp-tutorial, the xdp-loader does not work properly in pinning the map.

I run following command to load the XDP program on lo interface with a given pin path. It works fine without error (I have checked with xdp-loader status).

xdp-loader load -m skb -p /sys/fs/bpf lo ./xdp_prog_kern.o

Then I can see the map file in /sys/fs/bpf/programs/lo/xdp_pass. Then I modified the xdp-stats.c to use /sys/fs/bpf/programs as pin_dir and xdp_pass as the map name. Then I recompile the xdp_stats and run it with following command:

./xdp_stats --dev lo

It shows the following error:

ERR: check_map_fd_info() Map key size(-677305523) mismatch expected size(4)
ERR: map via FD not compatible

So the user space program can find the map file but the key size is wrong when calling check_map_fd_info. This example works fine when I use the xdp_loader.c instead of the xdp-loader utility provided by the xdp-tools(xdp_loader.c pins the maps in /sys/fs/bpf/lo/xdp_stats_map).

Am I doing something wrong here? Could you please give me some hints? Thanks so much for your help.

xdp-tool version: release: 0.0.3
xdp-tutorial version: master:b13cfb8e6bef6c85017bb1ffdf6f2922a87fdb21

Need python bindings for libxdp

Currently bcc provides python libs for writing bpf user space programs in python and loading, attaching or detaching xdp programs. A similar python lib or package for libxdp will be beneficial.

Split out libxdp logging

Currently, libxdp uses the logging helpers from logging.h, which relies on some setup and also messes with the libbpf output. This is not ideal for including libxdp in other applications, so we should change this to work more like libbpf with a separate library function for applications to configure logging

We also need to decide whether libxdp should (maybe optionally?) configure libbpf logging; otherwise, applications will need to configure libbpf logging themselves even if they are not using anything from libbpf directly.

Support appending programs to a multiprog

If we switch the dispatcher config to not be read-only, we could support appending XDP programs to it as long as they are loaded in the right order. This would mean we could support loading them one at a time and actually running as multiprog, instead of requiring them to all be loaded at the same time.

It would still only be partial support until freplace multi-attach lands in the kernel; and we would also lose the dead code elimination the verifier currently does with the parts of the dispatcher that are unused. So not sure if this is work supporting?

Add simple usage examples to man pages

We should have a few simple usage examples in man pages for all the tools.

Cleanup background PIDs after tests

If a test exits after using one of the start_background() functions, but before
calling stop_background(), it can leave the background process running. We
should add tracking of background PIDs in test_runner.sh and kill them all
before exiting.

OpenWrt Support

I try to cross-compile your tool for OpenWrt. Since u use reallocarray and similar functions you enable #define _GNU_SOURCE. I would assume that everyhing is correct, but it still fails to compile.

params.c: In function 'handle_multistring':
params.c:58:8: error: implicit declaration of function 'reallocarray'; did you mean 'realloc'? [-Werror=implicit-function-declaration]
  ptr = reallocarray(opt_set->strings, sizeof(*opt_set->strings),
        ^~~~~~~~~~~~
        realloc
params.c:58:6: error: assignment to 'void *' from 'int' makes pointer from integer without a cast [-Werror=int-conversion]
  ptr = reallocarray(opt_set->strings, sizeof(*opt_set->strings),

ERROR: Can't load eBPF object: No such process(-3)

I am very new to eBPF/XDP. I am capturing packets by using XDP dump. Without XDP loading, it is working fine.

But when I run

sudo ./xdp-filter load -m skb eno1
libxdp: Got verifier error while loading dispatcher.
libxdp: Falling back to loading single prog without dispatcher

when I check

sudo ./xdpdump -D
Interface Prio Program name Mode ID Tag Chain actions

lo xdp_prog_simple skb 92 3b185187f1855c4c
eno1 10 xdpfilt_alw_all 10669 0b394f43ab24501c XDP_PASS

actually, on interface lo I have loaded XDP program from XDP-tutorial https://github.com/xdp-project/xdp-tutorial/tree/master/basic01-xdp-pass and eno1 with Xdp-dump instructions. But at this time, I only want eno1 Xdp program should work. (for interface lo I run the below command, not interested but mentioning)

sudo ./xdp_pass_user --dev lo --skb-mode --force
Success: Loading XDP prog name:xdp_prog_simple(id:108) on device:eno1(ifindex:2)

but when I run the below command from Xdp-dump page instructions I get the below error

sudo ./xdpdump -i eno1 -w -
ERROR: Can't load eBPF object: No such process(-3)
I cannot understand what I am missing or what I could not understand, without Xdp program loading this command works fine like tcp-dump, but, with loading of XDP program I get the error. I want to capture the packets with loading above Xdp program.

xdpdump fails when XDP function BPF-prog names are too long

xdpdump fails indicating that it cannot find the eBPF object file, which first of all is a VERY confusing error message, because it is not the real problem.

$ sudo ./xdpdump -v -i mlx5p1
Looking for './xdpdump_bpf.o'
Loading bpf file 'xdpdump_bpf.o' from './xdpdump_bpf.o'
ERROR: Can't load eBPF object: No such file or directory(-2)

The real problem is that xdpdump identify the function name as xdp_tailgrow_fu(), while the real function name is xdp_tailgrow_func4(). Knowing the kernel, this is because the kernel only stores 15 chars of the original ELF function name (details: in kernel struct bpf_prog_aux->name[BPF_OBJ_NAME_LEN])

$ sudo ./xdpdump -D
if_index  if_name           XDP program entry function
--------  ----------------  --------------------------------------------------
1         lo                <No XDP program loaded!>
[... cut ...]
7         i40e2             <No XDP program loaded!>
8         mlx5p1            xdp_tailgrow_fu()
9         mlx5p2            <No XDP program loaded!>

Workaround:
Keeping function names (not SEC section names) below 15 chars in the BPF-prog is a workaround to the issue.

Help/hints to solve:
I think the xdpdump program can find the associated full-length name via BTF info. The struct bpf_prog_aux have a (const char *) member named attach_func_name, that I think we can extract somehow (according to comment this is "function name for valid attach_btf_id").

Ubuntu dependencies seem incorrect

Building with Ubuntu (18.04), dependencies in docs are listed as:

sudo apt install clang llvm libelf-dev libpcap-dev gcc-multilib build-essential
sudo apt install linux-tools-$(uname -r) -y
sudo apt install linux-headers-$(uname -r) -y
sudo apt install linux-tools-common linux-tools-generic -y

Installing these and attempting to configure, it's missing m4. Install m4 configures and make results in this error:

    CLANG    xdp-dispatcher.o
In file included from xdp-dispatcher.c:5:
../libbpf/src/root/usr/include/bpf/bpf_helpers.h:99:10: error: unknown register name 'r0' in asm
                     : "r0", "r1", "r2", "r3", "r4", "r5");
                       ^
1 error generated.

Am I missing something? It seems like I've missed a step but I don't see it. Any help is appreciated.