xdp-project / xdp-tools Goto Github PK
View Code? Open in Web Editor NEWUtilities and example programs for use with XDP
License: Other
Utilities and example programs for use with XDP
License: Other
We are missing a man page for xdp-loader
Hello,
I seem to be having a problem loading xdp programs using xdp-loader.
When attempting to load the basic01-xdp-pass XDP program from the xdp-tutorial examples, I get the following error's
xdp-loader load -v --mode skb lo basic01-xdp-pass/xdp_pass_kern.o
Current rlimit is infinity or 0. Not raising
Loading 1 files on interface 'lo'.
XDP program 0: Run prio: 50. Chain call actions: XDP_PASS
libxdp: Couldn't open BPF file
Couldn't attach XDP program on iface 'lo': Kernel verifier blocks program loading(-4007)
Notable errors from -vv
libbpf: added 60 insn from .text to prog xdp/dispatcher
libbpf: load bpf program failed: Invalid argument
libbpf: -- BEGIN DUMP LOG ---
libbpf:
Validating prog0() func#1...
btf_vmlinux is malformed
Arg#0 type PTR in prog0() is not supported yet.
processed 0 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0
libbpf: -- END LOG --
libbpf: failed to load program 'xdp/dispatcher'
libbpf: failed to load object '/usr/lib64/bpf/xdp-dispatcher.o'
libxdp: Couldn't open BPF file
Couldn't attach XDP program on iface 'lo': Kernel verifier blocks program loading(-4007)
Testing Environment
Compiling the tool from source produces the same errors.
Loading xdp program's using iproute works fine.
Thanks in advance for the help.
If only XDP_ MODE_HW is loaded, fall back to legacy or XDP load based capture.
Allow filtering like tcpdump (maybe just a subset to start with).
Fix up the selftests to make them runnable from the installed version, and amend the RPM to install and run them.
The symbol versioning script is a bit of a work-in-progress, so we need to check that it is working correctly and bump the initial symbol version for a 1.0 release
I have a cluster setup, I am using docker image of hping3 and place its pod (with IP 10.32.0.2) on one Node-1 (container), Secondly, I am using Nginx server image and place its pod (with IP 10.46.0.1) on another Node-2 (container).
Through first `Node-1 pod I am flooding packets to the container of second Node (means Nginx server node).
On Node-2 (means Nginx server node), in its container Interface vethwe-datapath I have loaded an XDP program by
xdp-filter load --mode skb vethwe-datapath , As you can see,
The problem is that it captures ONLY ONE side traffic, As you can see
with sudo ./xdpdump -i vethwe-datapath -w - | tee ab.pcap | tcpdump -r - -n
but without loading an XDP-program It captures both sides ingoing outgoing as you can see
with sudo ./xdpdump -i vethwe-datapath -w - | tee ab.pcap | tcpdump -r - -n
Why why why? Without loading it works fine and give me traffic of both directions but with loading It captures only one side of traffic? I want to capture both sides by loading an XDP-program
Thank you,
I found it very confusing that a program named xdpdump
will use the tcpdump
hook to dump packets, when there isn't any other XDP-prog already attached on the interfaces.
I do like that the xdpdump
program warns the end-user about this behaviour:
$ sudo ./xdpdump -i mlx5p1
WARNING: Specified interface does not have an XDP program loaded, capturing
in legacy mode!
I think xdpdump should at-least have an option to load it's own XDP program that does the dumping (via perf-ring and returns XDP_PASS). An an end-user, I want to "see" packets and timestamps at XDP-layer. (If I wanted to see it at tcpdump hook, I would use tcpdump
instead. I don't see a reason for xdpdump to replicated what tcpdump tools already does).
If we switch the dispatcher config to not be read-only, we could support appending XDP programs to it as long as they are loaded in the right order. This would mean we could support loading them one at a time and actually running as multiprog, instead of requiring them to all be loaded at the same time.
It would still only be partial support until freplace multi-attach lands in the kernel; and we would also lose the dead code elimination the verifier currently does with the parts of the dispatcher that are unused. So not sure if this is work supporting?
Add selftests for xdp-filter
XDP multi-buffer support is currently in the works, once this is added XDPdump should be updated to support these multi-buffer frames.
However, this might not require an XDPdump change, but a change to the kernel's bpf_xdp_output() helper.
We should have a few simple usage examples in man pages for all the tools.
Does it make sense to update the headers/linux/ files, or is there a specific cadence for doing this?
With the addition of the AF_XDP code we will be attaching XDP programs on behalf of the socket owner to redirect the packets. We should make sure this is synchronised so multiple processes can attach sockets to the same interface without stepping on each others' toes.
Since we already have a synchronisation point when building the dispatcher, this could likely be reused.
See also this upstream discussion about doing this in libbpf using bpf_link (which is an approach that will not work with multi-prog dispatchers today, sadly):
https://lore.kernel.org/bpf/[email protected]/T/#t
It would be good to actually run the selftests from the CI. Now that the 5.7 kernel has been released it should actually be possible to do, but the travis script will need to spin up a VM to run the tests. We can probably borrow some of the setup from the libbpf github repo.
We need at least basic selftests for xdpdump
xdpdump fails indicating that it cannot find the eBPF object file, which first of all is a VERY confusing error message, because it is not the real problem.
$ sudo ./xdpdump -v -i mlx5p1
Looking for './xdpdump_bpf.o'
Loading bpf file 'xdpdump_bpf.o' from './xdpdump_bpf.o'
ERROR: Can't load eBPF object: No such file or directory(-2)
The real problem is that xdpdump identify the function name as xdp_tailgrow_fu()
, while the real function name is xdp_tailgrow_func4()
. Knowing the kernel, this is because the kernel only stores 15 chars of the original ELF function name (details: in kernel struct bpf_prog_aux->name[BPF_OBJ_NAME_LEN]
)
$ sudo ./xdpdump -D
if_index if_name XDP program entry function
-------- ---------------- --------------------------------------------------
1 lo <No XDP program loaded!>
[... cut ...]
7 i40e2 <No XDP program loaded!>
8 mlx5p1 xdp_tailgrow_fu()
9 mlx5p2 <No XDP program loaded!>
Workaround:
Keeping function names (not SEC
section names) below 15 chars in the BPF-prog is a workaround to the issue.
Help/hints to solve:
I think the xdpdump program can find the associated full-length name via BTF info. The struct bpf_prog_aux
have a (const char *
) member named attach_func_name
, that I think we can extract somehow (according to comment this is "function name for valid attach_btf_id").
The XDP dispatcher has a significant performance impact (up to 11% decrease) on the performance of an AF_XDP socket. One way to get around this is to not load the dispatcher when the default xsk program is the only one active. The dispatcher is not needed in this case.
Toke suggested that we add some special handling of the xsk program and just load it when there is no other XDP program there. If more programs are added, we simply add the dispatcher and in that case the performance drop should be justified, since you are getting added functionality. The question now is: why do we not do this for any program, i.e. only load the dispatcher only if there are two or more XDP programs that want to share an interface? The only reason I can come up with is that it is much faster to add or remove a program if the dispatcher is already there. Adding the dispatcher plus two programs might result in dropped packets due to this latency, but I do not know. Other reasons?
Plan from Toke:
When attaching XSK program, if nothing is attached already, just
attach it directly to the interface.
When attaching XSK program and another program is already installed,
do the regular thing and attach it as a secondary program with a
dispatcher.
When attaching another program and an XSK program is already
attached, generate a new dispatcher with a new copy of the XSK program
as one of the sub-programs, but reusing the map from the program
already attached to the interface; then replace the existing program
with that dispatcher.
The current xdp-dump implementation is not taking advantage of "bulking" through the perf-ring buffer, instead it will get wakeup for each packet. I have measured significant performance improvements when enabling this.
In struct perf_event_attr
the member wakeup_events
specify getting an fd notification for X events.
Example see: https://github.com/netoptimizer/prototype-kernel/blob/master/kernel/samples/bpf/xdp_tcpdump_user.c#L236-L238
The example from prototype-kernel is not complete, because it does not handle flushing/consuming frames on exit. It is actually disabled in the code, as it was loosing packets on exit.
Building with Ubuntu (18.04), dependencies in docs are listed as:
sudo apt install clang llvm libelf-dev libpcap-dev gcc-multilib build-essential
sudo apt install linux-tools-$(uname -r) -y
sudo apt install linux-headers-$(uname -r) -y
sudo apt install linux-tools-common linux-tools-generic -y
Installing these and attempting to configure, it's missing m4. Install m4 configures and make
results in this error:
CLANG xdp-dispatcher.o
In file included from xdp-dispatcher.c:5:
../libbpf/src/root/usr/include/bpf/bpf_helpers.h:99:10: error: unknown register name 'r0' in asm
: "r0", "r1", "r2", "r3", "r4", "r5");
^
1 error generated.
Am I missing something? It seems like I've missed a step but I don't see it. Any help is appreciated.
See output below, i.e. fix ERROR: Can't find function '' on interface!
to be more specific as -vv
is not used by default.
# ./xdpdump -i enp3s0f1 --rx-capture entry,exit -w capture.pcap -vv
Current rlimit 67108864 already >= minimum 1048576
libxdp: No BTF for prog ID 11
libxdp: No BTF found for program
libxdp: Found legacy program with id 11 and 0 component progs
No BTF found for program
ERROR: Can't find function '' on interface!
I am very new to eBPF/XDP. I am capturing packets by using XDP dump. Without XDP loading, it is working fine.
But when I run
sudo ./xdp-filter load -m skb eno1
libxdp: Got verifier error while loading dispatcher.
libxdp: Falling back to loading single prog without dispatcher
when I check
lo xdp_prog_simple skb 92 3b185187f1855c4c
eno1 10 xdpfilt_alw_all 10669 0b394f43ab24501c XDP_PASS
actually, on interface lo I have loaded XDP program from XDP-tutorial https://github.com/xdp-project/xdp-tutorial/tree/master/basic01-xdp-pass and eno1 with Xdp-dump instructions. But at this time, I only want eno1 Xdp program should work. (for interface lo I run the below command, not interested but mentioning)
sudo ./xdp_pass_user --dev lo --skb-mode --force
Success: Loading XDP prog name:xdp_prog_simple(id:108) on device:eno1(ifindex:2)
but when I run the below command from Xdp-dump page instructions I get the below error
sudo ./xdpdump -i eno1 -w -
ERROR: Can't load eBPF object: No such process(-3)
I cannot understand what I am missing or what I could not understand, without Xdp program loading this command works fine like tcp-dump, but, with loading of XDP program I get the error. I want to capture the packets with loading above Xdp program.
If a test exits after using one of the start_background() functions, but before
calling stop_background(), it can leave the background process running. We
should add tracking of background PIDs in test_runner.sh and kill them all
before exiting.
On another machine, build fails due to missing linux/if_xdp.h
$ make
[...]
make[3]: Entering directory '/home/jbrouer/git/xdp-tools/lib/util'
cc -I../libbpf/src -Wall -I../../headers -c -o params.o params.c
params.c:13:10: fatal error: linux/if_xdp.h: No such file or directory
#include <linux/if_xdp.h>
^~~~~~~~~~~~~~~~
compilation terminated.
In the XDP-tutorial this issue is already solved, and we actually have two copies of this file:
[xdp-tutorial]$ find . | grep if_xdp
./libbpf/include/uapi/linux/if_xdp.h
./headers/linux/if_xdp.h
And in this XDP-tools repo it's indirectly available via libbpf submodule.
Dear XDP developers,
Thanks so much for your work. May I ask about the possibility to try attaching/appending another XDP program to an already-attached list of programs or replacing an attached XDP program?
I am currently developing a XDP-based traffic monitoring algorithm (as a PhD student π ). I wrote the XDP program but face the problem when I want to integrate it with OVS's AF_XDP datapath and DPDK's AF_XDP PMD. Both of them use the static int xsk_setup_xdp_prog(struct xsk_socket *xsk)
function (provided by libbpf
) to setup the AF_XDP socket and there is a "dummy" built-in XDP program be loaded inside this function (by calling xsk_load_xdp_prog()
function). Then I can not attach my monitor XDP program easily on the same veth
interface. This limitation makes it difficult to test my approach using OVS AF_XDP directly. I did some workarounds (write my own simple "switch" using XDP) but it would be great if the additional XDP program can be attached. I see in the README of libxdp
that:
It is expected that this restriction will be lifted in an upcoming kernel version, after which xdp_program__attach() will simply add the program being loaded into the existing chain of programs running on the interface.
Is it possible to use this feature in the kernel version 5.8? Or I can test this feature by compiling the kernel source from a special testing repo? Thanks so much for your patience and answer.
With best regards,
Zuo Xiang
The xdp-loader
and xdp-filter
programs have an option -S
for load
command to attach XDP programs in SKB mode, however, no such option is provided for the unload
command. Besides that, no option is provided to load/unload programs in HW offload mode.
I think that the -S
option should be replaced with -M mode
option to choose between xdp
, xdpgeneric
, and xdpoffload
. (Tried to implement this fast, but was confused in how options are parsed ;) Probably will have time to do this next week if not be fixed before.)
When building xdp-tools, it fails with the following errors:
$ make
[...]
make[3]: Entering directory '/home/hawk/git/xdp-tools/lib/util'
cc -I../libbpf/src -Wall -I../../headers -c -o params.o params.c
cc -I../libbpf/src -Wall -I../../headers -c -o logging.o logging.c
cc -I../libbpf/src -Wall -I../../headers -c -o util.o util.c
util.c:9:10: fatal error: bsd/string.h: No such file or directory
#include <bsd/string.h>
^~~~~~~~~~~~~~
compilation terminated.
Are there any specific reason for depending on bsd/string.h
?
Currently, libxdp uses the logging helpers from logging.h, which relies on some setup and also messes with the libbpf output. This is not ideal for including libxdp in other applications, so we should change this to work more like libbpf with a separate library function for applications to configure logging
We also need to decide whether libxdp should (maybe optionally?) configure libbpf logging; otherwise, applications will need to configure libbpf logging themselves even if they are not using anything from libbpf directly.
The addition to the PCAPng spec was approved: IETF-OPSAWG-WG/draft-ietf-opsawg-pcap@587eb9f
So we can go ahead and add the support to xdpdump.
We track upstream libbpf features pretty closely, but we do need to maintain backwards compatibility with released versions. So let's add detection of the libbpf .so version in configure and use that in ifdefs.
Seems the regexes testing capinfos output are a bit too specific?
$ capinfos --version
Capinfos (Wireshark) 3.2.4 (Git commit 893b5a5e1e3e)
test xdp-dump
Running tests from test-xdpdump.sh
[test_help] PASS
[test_interfaces] PASS
[test_capt_pcap] PASS
[test_capt_pcapng] FAIL
PING fc00:dead:cafe:1::2(fc00:dead:cafe:1::2) 56 data bytes
64 bytes from fc00:dead:cafe:1::2: icmp_seq=1 ttl=64 time=0.033 ms
--- fc00:dead:cafe:1::2 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.033/0.033/0.033/0.000 ms
PING fc00:dead:cafe:1::2(fc00:dead:cafe:1::2) 56 data bytes
64 bytes from fc00:dead:cafe:1::2: icmp_seq=1 ttl=64 time=0.129 ms
--- fc00:dead:cafe:1::2 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.129/0.129/0.129/0.000 ms
PING fc00:dead:cafe:1::2(fc00:dead:cafe:1::2) 56 data bytes
64 bytes from fc00:dead:cafe:1::2: icmp_seq=1 ttl=64 time=0.176 ms
--- fc00:dead:cafe:1::2 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.176/0.176/0.176/0.000 ms
REGEX: (File type: Wireshark\/\.\.\. - pcapng.*Capture hardware: unknown.*Capture oper-sys: Linux alrua-x1 5\.7\.1-arch1-1 #1 SMP PREEMPT Sun, 07 Jun 2020 12:28:01 \+0000.*Capture application: xdpdump v[0-9]+\.[0-9]+\.[0-9]+.*Interface #0 info:.*Name = xdptest@fentry.*Description = xdptest:xdp_dispatcher\(\)@fentry.*Hardware = driver: "veth", version: "1\.0", fw-version: "", rom-version: "", bus-info: "".*Time precision = nanoseconds \(9\).*Interface #1 info:.*Name = xdptest@fexit.*Description = xdptest:xdp_dispatcher\(\)@fexit.*Hardware = driver: "veth", version: "1\.0", fw-version: "", rom-version: "", bus-info: "".*Time precision = nanoseconds \(9\))
File name: /tmp/xdptest_PID_63744_30064.pcap
File type: Wireshark/... - pcapng
File encapsulation: Ethernet
File timestamp precision: nanoseconds (9)
Packet size limit: file hdr: (not set)
Number of packets: 2
File size: 896 bytes
Data size: 236 bytes
Capture duration: 0,000045889 seconds
First packet time: 2020-06-10 15:19:20,394685883
Last packet time: 2020-06-10 15:19:20,394731772
Data byte rate: 5.155 kBps
Data bit rate: 41 Mbps
Average packet size: 118,00 bytes
Average packet rate: 43 kpackets/s
SHA256: 3d46499267c66e226b0705b8535ee1dd2d08cf0f08b6233a2ff6b277a9a9ad32
RIPEMD160: 0ea91f28fb3635b518ac7277ffbdbb3369b4b6c5
SHA1: 32187461abbb5c0208c2f5abc1f82f83b3c0361d
Strict time order: True
Capture hardware: x86_64
Capture oper-sys: Linux alrua-x1 5.7.1-arch1-1 #1 SMP PREEMPT Sun, 07 Jun 2020 12:28:01 +0000
Capture application: xdpdump v0.0.3
Number of interfaces in file: 2
Interface #0 info:
Name = xdptest@fentry
Description = xdptest:xdp_dispatcher()@fentry
Encapsulation = Ethernet (1 - ether)
Hardware = driver: "veth", version: "1.0", fw-version: "", rom-version: "", bus-info: ""
Speed = 10000000000
Capture length = 0
Time precision = nanoseconds (9)
Time ticks per second = 1000000000
Time resolution = 0x09
Number of stat entries = 0
Number of packets = 1
Interface #1 info:
Name = xdptest@fexit
Description = xdptest:xdp_dispatcher()@fexit
Encapsulation = Ethernet (1 - ether)
Hardware = driver: "veth", version: "1.0", fw-version: "", rom-version: "", bus-info: ""
Speed = 10000000000
Capture length = 0
Time precision = nanoseconds (9)
Time ticks per second = 1000000000
Time resolution = 0x09
Number of stat entries = 0
Number of packets = 1
ERROR: Failed capinfos content
[test_capt_term] FAIL
PING fc00:dead:cafe:1::2(fc00:dead:cafe:1::2) 56 data bytes
64 bytes from fc00:dead:cafe:1::2: icmp_seq=1 ttl=64 time=0.066 ms
--- fc00:dead:cafe:1::2 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.066/0.066/0.066/0.000 ms
XDP program already loaded on ifindex 36; replacing not yet supported
Couldn't attach XDP program on iface 'xdptest': File exists(-17)
[test_exitentry] FAIL
XDP program already loaded on ifindex 36; replacing not yet supported
Couldn't attach XDP program on iface 'xdptest': File exists(-17)
[test_snap] FAIL
XDP program already loaded on ifindex 36; replacing not yet supported
Couldn't attach XDP program on iface 'xdptest': File exists(-17)
[test_multi_pkt] FAIL
XDP program already loaded on ifindex 36; replacing not yet supported
Couldn't attach XDP program on iface 'xdptest': File exists(-17)
[test_perf_wakeup] FAIL
XDP program already loaded on ifindex 36; replacing not yet supported
Couldn't attach XDP program on iface 'xdptest': File exists(-17)
[test_none_xdp] PASS
make[1]: *** [../lib/common.mk:136: test] Error 1
make: *** [Makefile:66: test] Error 2
There are some inconsistencies in how xdp-filter handles addition/removal of the
different objects it can filter on. We should make this consistent, and add
selftests to make sure they stay that way.
[root@netqe2 ~]# xdp-filter port -p tcp,udp -m src,dst 54321
[root@netqe2 ~]# xdp-filter status
CURRENT XDP-FILTER STATUS:
Aggregate per-action statistics:
XDP_DROP 461 pkts 30 KiB
XDP_PASS 3571195 pkts 5279647 KiB
Loaded on interfaces:
Enabled features
xdpfilt_blk_all
enp68s0f0 (skb mode) tcp,udp,ipv6,ipv4,ethernet,blacklist
Filtered ports:
Mode Hit counter
54321 src,dst,tcp,udp 0
Filtered IP addresses:
Mode Hit counter
Filtered MAC addresses:
Mode Hit counter
[root@netqe2 ~]# xdp-filter port -p tcp -r 54321
[root@netqe2 ~]# xdp-filter status
CURRENT XDP-FILTER STATUS:
Aggregate per-action statistics:
XDP_DROP 461 pkts 30 KiB
XDP_PASS 3571195 pkts 5279647 KiB
Loaded on interfaces:
Enabled features
xdpfilt_blk_all
enp68s0f0 (skb mode) tcp,udp,ipv6,ipv4,ethernet,blacklist
Filtered ports:
Mode Hit counter
54321 src,udp 0
Filtered IP addresses:
Mode Hit counter
Filtered MAC addresses:
Mode Hit counter
[root@netqe2 ~]# xdp-filter port -p tcp -m dst 54321
[root@netqe2 ~]# xdp-filter status
CURRENT XDP-FILTER STATUS:
Aggregate per-action statistics:
XDP_DROP 461 pkts 30 KiB
XDP_PASS 3571195 pkts 5279647 KiB
Loaded on interfaces:
Enabled features
xdpfilt_blk_all
enp68s0f0 (skb mode) tcp,udp,ipv6,ipv4,ethernet,blacklist
Filtered ports:
Mode Hit counter
54321 src,dst,tcp,udp 0
Filtered IP addresses:
Mode Hit counter
Filtered MAC addresses:
Mode Hit counter
[root@netqe2 ~]# xdp-filter port -p udp -r 54321
[root@netqe2 ~]# xdp-filter status
CURRENT XDP-FILTER STATUS:
Aggregate per-action statistics:
XDP_DROP 461 pkts 30 KiB
XDP_PASS 3571195 pkts 5279647 KiB
Loaded on interfaces:
Enabled features
xdpfilt_blk_all
enp68s0f0 (skb mode) tcp,udp,ipv6,ipv4,ethernet,blacklist
Filtered ports:
Mode Hit counter
54321 src,tcp 0
Filtered IP addresses:
Mode Hit counter
Filtered MAC addresses:
Mode Hit counter
[root@netqe2 ~]# xdp-filter port -p udp -m dst 54321
[root@netqe2 ~]# xdp-filter status
CURRENT XDP-FILTER STATUS:
Aggregate per-action statistics:
XDP_DROP 461 pkts 30 KiB
XDP_PASS 3571195 pkts 5279647 KiB
Loaded on interfaces:
Enabled features
xdpfilt_blk_all
enp68s0f0 (skb mode) tcp,udp,ipv6,ipv4,ethernet,blacklist
Filtered ports:
Mode Hit counter
54321 src,dst,tcp,udp 0
Filtered IP addresses:
Mode Hit counter
Filtered MAC addresses:
Mode Hit counter
[root@netqe2 ~]# xdp-filter port -m src -r 54321
[root@netqe2 ~]# xdp-filter status
CURRENT XDP-FILTER STATUS:
Aggregate per-action statistics:
XDP_DROP 461 pkts 30 KiB
XDP_PASS 3571195 pkts 5279647 KiB
Loaded on interfaces:
Enabled features
xdpfilt_blk_all
enp68s0f0 (skb mode) tcp,udp,ipv6,ipv4,ethernet,blacklist
Filtered ports:
Mode Hit counter
Filtered IP addresses:
Mode Hit counter
Filtered MAC addresses:
Mode Hit counter
[root@netqe2 ~]# xdp-filter port -p tcp,udp -m src,dst 54321
[root@netqe2 ~]# xdp-filter status
CURRENT XDP-FILTER STATUS:
Aggregate per-action statistics:
XDP_DROP 461 pkts 30 KiB
XDP_PASS 3571195 pkts 5279647 KiB
Loaded on interfaces:
Enabled features
xdpfilt_blk_all
enp68s0f0 (skb mode) tcp,udp,ipv6,ipv4,ethernet,blacklist
Filtered ports:
Mode Hit counter
54321 src,dst,tcp,udp 0
Filtered IP addresses:
Mode Hit counter
Filtered MAC addresses:
Mode Hit counter
[root@netqe2 ~]# xdp-filter port -m dst -r 54321
[root@netqe2 ~]# xdp-filter status
CURRENT XDP-FILTER STATUS:
Aggregate per-action statistics:
XDP_DROP 461 pkts 30 KiB
XDP_PASS 3571195 pkts 5279647 KiB
Loaded on interfaces:
Enabled features
xdpfilt_blk_all
enp68s0f0 (skb mode) tcp,udp,ipv6,ipv4,ethernet,blacklist
Filtered ports:
Mode Hit counter
Filtered IP addresses:
Mode Hit counter
Filtered MAC addresses:
Mode Hit counter
[root@netqe2 ~]#
I dont have BTF enabled in my kernel. Is there any way to use xdp-dump without BTF?
here is compiling error on Ubuntu 18.04
root@vincent-desktop:/usr/src/xdp-tools# ./configure
libbpf support: submodule
ELF support: yes
zlib support: yes
root@vincent-desktop:/usr/src/xdp-tools# make
lib
util
CC params.o
CC logging.o
CC util.o
CC stats.o
xdp-filter
CLANG xdpfilt_blk_udp.o
In file included from xdpfilt_blk_udp.c:10:
In file included from ./xdpfilt_prog.h:10:
In file included from ../lib/../headers/linux/bpf.h:11:
/usr/include/linux/types.h:5:10: fatal error: 'asm/types.h' file not found
#include <asm/types.h>
^~~~~~~~~~~~~
1 error generated.
../lib/common.mk:77: recipe for target 'xdpfilt_blk_udp.o' failed
make[1]: *** [xdpfilt_blk_udp.o] Error 1
Makefile:19: recipe for target 'all' failed
make: *** [all] Error 2
this error is similar to issue addressed in https://code.forksand.com/oisf/suricata/commit/7906c521cdde5b1d0eb3ce379b8e343c3055653f
I added diff below to fix the error, maybe similar patch needed for xdp-tools ?
diff --git a/lib/common.mk b/lib/common.mk index dd71535..6127450 100644 --- a/lib/common.mk +++ b/lib/common.mk @@ -75,6 +75,7 @@ $(USER_TARGETS): %: %.c $(OBJECT_LIBBPF) $(LIBMK) $(LIB_OBJS) $(KERN_USER_H) $( $(XDP_OBJ): %.o: %.c $(KERN_USER_H) $(EXTRA_DEPS) $(BPF_HEADERS) $(LIBMK) $(QUIET_CLANG)$(CLANG) -S \ + -I/usr/include/x86_64-linux-gnu/ \ -target bpf \ -D __BPF_TRACING__ \ $(BPF_CFLAGS) \
[test_capt_pcap] FAIL
PING fc42:dead:cafe:1::2(fc42:dead:cafe:1::2) 56 data bytes
64 bytes from fc42:dead:cafe:1::2: icmp_seq=1 ttl=64 time=0.068 ms
--- fc42:dead:cafe:1::2 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.068/0.068/0.068/0.000 ms
PING fc42:dead:cafe:1::2(fc42:dead:cafe:1::2) 56 data bytes
64 bytes from fc42:dead:cafe:1::2: icmp_seq=1 ttl=64 time=0.047 ms
--- fc42:dead:cafe:1::2 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.047/0.047/0.047/0.000 ms
../lib/testing/test_runner.sh: line 109: kill: (-22436) - No such process
reading from file -, link-type EN10MB (Ethernet), snapshot length 262144
23:24:35.836630 IP6 :: > ff02::1:ff63:4b64: ICMP6, neighbor solicitation, who has fe80::f496:a8ff:fe63:4b64, length 32
23:24:36.380195 IP6 fc42:dead:cafe:1::2 > fc42:dead:cafe:1::1: ICMP6, echo reply, id 174, seq 1, length 64
tcpdump: pcap_loop: error reading dump file: Interrupted system call
ERROR: IPv6 packet not received
[test_capt_pcapng] FAIL
PING fc42:dead:cafe:1::2(fc42:dead:cafe:1::2) 56 data bytes
64 bytes from fc42:dead:cafe:1::2: icmp_seq=1 ttl=64 time=0.075 ms
--- fc42:dead:cafe:1::2 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.075/0.075/0.075/0.000 ms
PING fc42:dead:cafe:1::2(fc42:dead:cafe:1::2) 56 data bytes
64 bytes from fc42:dead:cafe:1::2: icmp_seq=1 ttl=64 time=0.047 ms
--- fc42:dead:cafe:1::2 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.047/0.047/0.047/0.000 ms
../lib/testing/test_runner.sh: line 109: kill: (-22465) - No such process
reading from file -, link-type EN10MB (Ethernet), snapshot length 262144
23:24:39.359278 IP6 fc42:dead:cafe:1::2 > fc42:dead:cafe:1::1: ICMP6, echo reply, id 176, seq 1, length 64
tcpdump: pcap_loop: error reading dump file: Interrupted system call
ERROR: IPv6 packet not received
Dear developers,
I'm a newbie of xdp, when I tried to use the xdp-tools, some error occurred. It's really appreciated that you could take a look at this issue and kindly reply it.
When I make
in the /xdp-tools
, xdp-filter, loader is ok, but when it goes to xdp-dump
, errors popped up:
xdpdump_bpf.c:25:18: error: unknown attribute 'preserve_access_index' ignored [-Werror,-Wunknown-attributes]
} __attribute__((preserve_access_index));
^
xdpdump_bpf.c:33:18: error: unknown attribute 'preserve_access_index' ignored [-Werror,-Wunknown-attributes]
} __attribute__((preserve_access_index));
^
xdpdump_bpf.c:42:18: error: unknown attribute 'preserve_access_index' ignored [-Werror,-Wunknown-attributes]
} __attribute__((preserve_access_index));
^
3 errors generated.
make[1]: *** [../lib/common.mk:88: xdpdump_bpf.o] Error 1
make: *** [Makefile:26: xdp-dump] Error 2
I supposed that this won't affect the usage of xdp-filter, but when I issue ./xdp-filter load -m skb <$nic_name>
, another three errors poped up:
Couldn't find a BPF file with name xdp-dispatcher.o
Couldn't open BPF file
Couldn't attach XDP program on iface 'enp68s0f0': No such file or directory(-2)
I think I must miss something that caused this issue, but I have no clue...
Thanks in advance!!
I'd love to be able to use xdp-filter to be able to say things like: block all UDP to the entire 192.168.1.0/24 network (but not any other network), with the exception of 192.168.1.10 port 53, and 192.168.1.11 port 53.
At the moment, there is no netmask support in the ip command, and its also not possible (at least from my reading of the README) to be able to have IP:port combinations.
I realize that this can complicate things quickly, but it would be an amazing improvement!
Currently bcc provides python libs for writing bpf user space programs in python and loading, attaching or detaching xdp programs. A similar python lib or package for libxdp will be beneficial.
Issue: After running git submodule update
the xdp-tools Makefile system does not detect that it needs to rebuild libbpf.
The full sequence of commands below
First make
command compile everything, and warns me that I should run git submodule update
:
$ make
** WARNING **: git submodule SHA-1 out-of-sync
consider running: git submodule update
lib
util
CC params.o
CC logging.o
CC util.o
CC stats.o
libxdp
CC staticobjs/libxdp.o
LINK libxdp.a
CC sharedobjs/libxdp.o
LINK libxdp.so.0.0.3
M4 xdp-dispatcher.c
CLANG xdp-dispatcher.o
LLC xdp-dispatcher.o
xdp-filter
[... cut ...]
xdp-loader
make[2]: Nothing to be done for 'all'.
CC xdp-loader
LINK xdp-dispatcher.o
xdp-dump
make[2]: Nothing to be done for 'all'.
CC xdpdump
CLANG xdpdump_bpf.o
LLC xdpdump_bpf.o
LINK xdp-dispatcher.o
Next I run git submodule update
which finds a new libbpf version:
$ git submodule update
Submodule path 'lib/libbpf': checked out '3b239425426e4fa1c204ea3c708d36ec3f509702'
Then I run make
to get everything compile against the new libbpf, but that does not happen:
$ make
lib
util
make[2]: Nothing to be done for 'all'.
libxdp
make[2]: Nothing to be done for 'all'.
xdp-filter
make[2]: Nothing to be done for 'all'.
xdp-loader
make[2]: Nothing to be done for 'all'.
xdp-dump
make[2]: Nothing to be done for 'all'.
After compiling i tried to load xdp-filter with defaults, but it always fails to load with the below error.
sudo ./xdp-filter load -m skb enp0s8
libxdp: Failed to load program xdpfilt_alw_all: Argument list too long
Couldn't attach XDP program on iface 'enp0s8': Argument list too long(-7)
I try to cross-compile your tool for OpenWrt. Since u use reallocarray and similar functions you enable #define _GNU_SOURCE
. I would assume that everyhing is correct, but it still fails to compile.
params.c: In function 'handle_multistring':
params.c:58:8: error: implicit declaration of function 'reallocarray'; did you mean 'realloc'? [-Werror=implicit-function-declaration]
ptr = reallocarray(opt_set->strings, sizeof(*opt_set->strings),
^~~~~~~~~~~~
realloc
params.c:58:6: error: assignment to 'void *' from 'int' makes pointer from integer without a cast [-Werror=int-conversion]
ptr = reallocarray(opt_set->strings, sizeof(*opt_set->strings),
If I use NIC bonding and I want to protect all the slave interfaces, can I use xdp-filter?
Hi,
I'm trying to load xdp-filter on my interfaces. I'm able to attach xdp programs that I've written and then loaded with ip link set dev eth11 xdp obj block.o
and they do work properly.
However, when I try to run xdp-filter, I get this:
./xdp-filter load eth11
Couldn't load BPF program: Unknown error 4001
I'll note that before I ran that, I did remove any running xdp programs (ip link set dev eth11 xdp off
)
XDP programs can be loaded in several modes:
Generic β Driver doesnβt have support for XDP, but the kernel fakes it. XDP program works, but its not nearly as efficient as the other modes.
Native β Driver has XDP support and can hand then to XDP without kernel stack interaction
Offloaded β XDP can be loaded and executed directly on the NIC
I've got two NICs that can do 'native' XDP, and one that can only do 'generic'. I'm able to attach XDP programs that I've written to the 'generic' NIC, and it does function. However, when I try to load with xdp-filter, it is not attached:
libxdp: Failed to load dispatcher: Operation not supported
libxdp: Falling back to loading single prog without dispatcher
Couldn't attach XDP program on iface 'eth3': Operation not supported(-95)
When I manually add my own xdp program, as follows, it does get loaded:
ip link set dev eth3 xdp obj block.o
I can see the interface has the program loaded:
3: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 xdpgeneric qdisc mq state UP mode DEFAULT group default qlen 1000 link/ether 90:e2:ba:31:82:c5 brd ff:ff:ff:ff:ff:ff prog/xdp id 62 tag 2892ef4bc3a9547c
note the 'xdpgeneric' flag and the program id and tag.
On an interface with 'native' support look like this when programs are loaded have 'xdp' listed instead:
eth10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 xdp/id:444 qdisc mq state UP group default qlen 1000
Shouldn't xdp-filter support 'generic' mode as well, because its just using libpf?
thanks!
Dear developers of xdp-tools,
Thanks so much for your great effort! I am a PhD student and want to implement something using XDP. Therefore, I successfully set up the development environment using xdp-tool and started to follow xdp-tutorial. I mainly want to use the xdp-loader
provided by xdp-tool to make loading/unloading stuff easier. When tried basic04 in xdp-tutorial, the xdp-loader
does not work properly in pinning the map.
I run following command to load the XDP program on lo interface with a given pin path. It works fine without error (I have checked with xdp-loader status
).
xdp-loader load -m skb -p /sys/fs/bpf lo ./xdp_prog_kern.o
Then I can see the map file in /sys/fs/bpf/programs/lo/xdp_pass
. Then I modified the xdp-stats.c to use /sys/fs/bpf/programs
as pin_dir
and xdp_pass
as the map name. Then I recompile the xdp_stats
and run it with following command:
./xdp_stats --dev lo
It shows the following error:
ERR: check_map_fd_info() Map key size(-677305523) mismatch expected size(4)
ERR: map via FD not compatible
So the user space program can find the map file but the key size is wrong when calling check_map_fd_info
. This example works fine when I use the xdp_loader.c instead of the xdp-loader
utility provided by the xdp-tools(xdp_loader.c
pins the maps in /sys/fs/bpf/lo/xdp_stats_map
).
Am I doing something wrong here? Could you please give me some hints? Thanks so much for your help.
Tool man pages are exported from README.org in each directory, but this is currently done automatically. We should create a Makefile rule to automate this.
xdp-filter man page about LOAD has the old option -s
while the xdp-filter load --help
has the new one -m
root@netqe2 ~]# xdp-filter load -h
Usage: xdp-filter load [options] <ifname>
Load xdp-filter on an interface
Required parameters:
<ifname> Load on device <ifname>
Options:
-F, --force Force loading of XDP program
-m, --mode <mode> Load XDP program in <mode>; default native (valid values: native,skb,hw)
-w, --whitelist Use filters in whitelist mode (default blacklist)
-f, --features <feats> Features to enable; default all (valid values: tcp,udp,ipv6,ipv4,ethernet,all)
-v, --verbose Enable verbose logging (-vv: more verbose)
-h, --help Show this help
[root@netqe2 ~]#
[root@netqe2 ~]# man xdp-filter | cat
(...)
The LOAD command
To use xdp-filter, it must first be loaded unto an interface. This is accomplished with the load command, which takes the name of the interface as a parameter, and optionally allows specify-
ing the features that should be included. By default all features are loaded, but de-selecting some features can speed up the packet matching, and increase performance by a substantial
amount.
The syntax for the load command is:
xdp-filter load [options] <ifname>
Where <ifname> is the name of the interface to load xdp-filter unto, and must be specified. The supported options are:
-F, --force
Specifying this option causes xdp-filter to unload any XDP program already loaded on the interface.
-s, --skb-mode
Specifying this option causes the XDP program to be loaded in the so-called skb mode (also known as generic XDP). This is a compatibility mode that results in lower performance, but can be
used on all network interfaces (whereas the default native XDP mode requires specific driver support).
A declarative, efficient, and flexible JavaScript library for building user interfaces.
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. πππ
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google β€οΈ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.