Hi, I compiled the <a href="https://github.com/vincentmli/XDPeriment

./bpftool --version bpftool v7.2.0 <p

bpftool from kernel 4.15 <div class="snippet-clipboard-content notranslate positio

4.15: This is because that version does not have <a href="https:/

bpftool v6.7.0 load xdp program on Ubuntu 16.04 kernel 4.15.0-99-generic,about libbpf/bpftool

Comments (23)

qmonnet commented on August 15, 2024 1

Could be that the 72 as attr size passed to the syscall does not play well with the kernel, I don't remember if it ignores additional fields in attr or complains if it's longer than expected. Probably complains if those fields are non-zero (not sure if strace would show that if it also expects fewer fields in attr that modern bpftool uses.

I'd recommend trying with the version of bpftool that was in the kernel repo at the time of 4.15, maybe?

from bpftool.

vincentmli commented on August 15, 2024 1

./bpftool --version
bpftool v7.2.0

This is the current version, not the one from kernel 4.15?

sorry I messed it up, it is from current upstream version :)

from bpftool.

qmonnet commented on August 15, 2024 1

bpftool GitHub mirror works both when libbfd is enabled or disabled

Yes, I've spent quite some time recently to make sure that it works properly on the GH mirror, and I've got some CI checks to detect if it breaks. So yeah, I do hope it works on the mirror.

in summary, from upstream kernel bpf-next git repo, when libbfd is on and with static link to libbfd.a, undefined reference error occurs

With `CFLAGS=--static`

OK, I see the same on my side. The error is due to the feature detection test for libbfd passing on static builds when it should fail, because we need feature-libbfd-liberty-z instead on Ubuntu. I could fix the build with this quick hack in the Makefile:

diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
index 681fbcc5ed50..ebcc46b1e161 100644
--- a/tools/bpf/bpftool/Makefile
+++ b/tools/bpf/bpftool/Makefile
@@ -153,7 +153,7 @@ ifeq ($(feature-llvm),1)
 else
   # Fall back on libbfd
   ifeq ($(feature-libbfd),1)
-    LIBS += -lbfd -ldl -lopcodes
+    LIBS += -lbfd -ldl -lopcodes -liberty -lz
   else ifeq ($(feature-libbfd-liberty),1)
     LIBS += -lbfd -ldl -lopcodes -liberty
   else ifeq ($(feature-libbfd-liberty-z),1)

But I think this is simply because CFLAGS are not made to be used with feature detection. tools/build/Makefile.feature has the following:

define feature_check_code
  feature-$(1) := $(shell $(MAKE) OUTPUT=$(OUTPUT_FEATURES) CC="$(CC)" CXX="$(CXX)" CFLAGS="$(EXTRA_CFLAGS) $(FEATURE_CHECK_CFLAGS-$(1))" CXXFLAGS="$(EXTRA_CXXFLAGS) $(FEATURE_CHECK_CXXFLAGS-$(1))" LDFLAGS="$(LDFLAGS) $(FEATURE_CHECK_LDFLAGS-$(1))" -C $(feature_dir) $(OUTPUT_FEATURES)test-$1.bin >/dev/null 2>/dev/null && echo 1 || echo 0)
endef

Note the CFLAGS="$(EXTRA_CFLAGS)": the variable just gets overwritten.

With `EXTRA_CFLAGS=--static`

This works for me on bpf-next. Make sure to clean up the artefacts from feature detection first:

$ make -C tools/build/feature clean
$ make -C tools/bpf/bpftool clean
$ EXTRA_CFLAGS=--static make -C tools/bpf/bpftool

I have not tested with older versions, though.

from bpftool.

vincentmli commented on August 15, 2024

here is the strace, is prog_type=BPF_PROG_TYPE_SOCKET_FILTER correct type for XDP program?

write(2, "libbpf: map 'state_map' (legacy): legacy map definitions are deprecated, use BTF-defined maps instead\n", 102) = 102
write(2, "libbpf: map 'state_map_v6' (legacy): legacy map definitions are deprecated, use BTF-defined maps instead\n", 105) = 105
close(3)                                = 0
bpf(BPF_PROG_LOAD, {prog_type=BPF_PROG_TYPE_SOCKET_FILTER, insn_cnt=2, insns=0x7ffe2d6f4330, license="GPL", log_level=0, log_size=0, log_buf=0, kern_version=0}, 144) = 3
close(3)                                = 0
bpf(BPF_PROG_LOAD, {prog_type=BPF_PROG_TYPE_SOCKET_FILTER, insn_cnt=2, insns=0x7ffe2d6f3fc0, license="GPL", log_level=0, log_size=0, log_buf=0, kern_version=0}, 144) = 3
close(3)                                = 0
bpf(BPF_MAP_CREATE, {map_type=0x5 /* BPF_MAP_TYPE_??? */, key_size=4, value_size=16, max_entries=1000000}, 72) = -1 EINVAL (Invalid argument)
write(2, "libbpf: map 'state_map': failed to create: Invalid argument(-22)\n", 65) = 65
write(2, "libbpf: failed to load object 'xdp_rrl_per_ip.o'\n", 49) = 49
write(2, "Error: ", 7)                  = 7
write(2, "failed to load object file", 26) = 26
write(2, "\n", 1)                       = 1
exit_group(4294967295)                  = ?
+++ exited with 255 +++

from bpftool.

qmonnet commented on August 15, 2024

Hi Vincent!

Ubuntu 16.04 is very old (by BPF standards at least). We've tried to keep bpftool backward compatible, but I'm afraid that it's not been completely the case.

This being said, taking a very brief look on the phone:

The type doesn't look correct for an XDP program, but I don't think this is bpftool trying to load your program. I think it's libbpf probing for the availability of basic things like the bpf() syscall. So this is probably OK.
So there's something wrong when it tries to create the map. Strace does not recognize the map type. I don't remember, what's map type number 5, and is it supported on kernel 4.15?

from bpftool.

vincentmli commented on August 15, 2024

it is map type BPF_MAP_TYPE_PERCPU_HASH supported since kernel 4.6 https://github.com/iovisor/bcc/blob/master/docs/kernel-versions.md, the map type enum start with 1, right? so 5 is BPF_MAP_TYPE_PERCPU_HASH

from bpftool.

vincentmli commented on August 15, 2024

bpftool from kernel 4.15

./bpftool --version
bpftool v7.2.0
using libbpf v1.2
features:

./bpftool prog load xdp_rrl_per_ip.o xdp dev lo
libbpf: elf: legacy map definitions in 'maps' section are not supported by libbpf v1.0+
Error: failed to open object file

this is back to where I need to change the map to BTF map definition and requires loading external BTF file.
I have a simple code like below to use btf_custom_path to load the external BTF file, but I could not figure out how to compile it statically and run on Ubuntu 16.04 userspace

run clang -lelf -lz loader.c -o loader libbpf.a in Ubuntu 20.04 and run loader on Ubuntu 16.04 still get glibc/libc version error.

// SPDX-License-Identifier: GPL-2.0                                              

#include <linux/bpf.h>
#include <stdio.h>
#include <errno.h>
#include <string.h>
#include <stdlib.h>
#include <unistd.h>

#include "bpf/bpf.h"
#include "bpf/libbpf.h"

static int bpfverbose = 0;

int
libbpf_print_fn(enum libbpf_print_level level, const char *format, va_list args)
{
        if (level == LIBBPF_DEBUG && !bpfverbose)
                return 0;

        return vfprintf(stderr, format, args);
}

int main () {
        char *btf_file;
        struct bpf_object *obj = NULL;
        struct bpf_program *program = NULL;
        struct bpf_object_open_opts openopts = {};
        int err = 0;

        libbpf_set_print(libbpf_print_fn);

        openopts.sz = sizeof(struct bpf_object_open_opts);
        btf_file = getenv("EXAMPLE_BTF_FILE");

        if (btf_file != NULL)
                openopts.btf_custom_path = strdup(btf_file);

                // create bpf object from file

        obj = bpf_object__open_file("xdp_rrl_per_ip.o", &openopts);
        err = libbpf_get_error(obj);

        if (err) {
                fprintf(stderr, "ERROR: failed to open bpf object file: %d\n", err);
                goto cleanup;
        }

        // load program(s)

        err = bpf_object__load(obj);

        if (err) {
                fprintf(stderr, "ERROR: failed to load bpf object file: %d\n", err);
                goto cleanup;
        }

cleanup:
        if (obj)
                bpf_object__close(obj);
        return 0;
}

from bpftool.

qmonnet commented on August 15, 2024

./bpftool --version
bpftool v7.2.0

This is the current version, not the one from kernel 4.15?

from bpftool.

vincentmli commented on August 15, 2024

run into compiling error with both upstream 4.15 and master branch on ubuntu 20.04, it appears related to bfd and static build only

$ git branch
  master
* v4.15

$ CFLAGS=-static make
  CC       common.o
  CC       main.o
main.c:36:10: fatal error: bfd.h: No such file or directory
   36 | #include <bfd.h>
      |          ^~~~~~~

$ sudo apt-get install binutils-dev

$ CFLAGS=-static make
  CC       jit_disasm.o
jit_disasm.c: In function ‘disasm_print_insn’:
jit_disasm.c:110:29: error: incompatible type for argument 1 of ‘disassembler’
  110 |  disassemble = disassembler(bfdf);
      |                             ^~~~
      |                             |
      |                             bfd * {aka struct bfd *}
In file included from jit_disasm.c:21:
/usr/include/dis-asm.h:322:63: note: expected ‘enum bfd_architecture’ but argument is of type ‘bfd *’ {aka ‘struct bfd *’}
  322 | extern disassembler_ftype disassembler (enum bfd_architecture arc,
      |                                         ~~~~~~~~~~~~~~~~~~~~~~^~~
jit_disasm.c:110:16: error: too few arguments to function ‘disassembler’
  110 |  disassemble = disassembler(bfdf);
      |                ^~~~~~~~~~~~
In file included from jit_disasm.c:21:
/usr/include/dis-asm.h:322:27: note: declared here
  322 | extern disassembler_ftype disassembler (enum bfd_architecture arc,
      |                           ^~~~~~~~~~~~
make: *** [Makefile:72: jit_disasm.o] Error 1

upstream master branch

$ git branch
* master
  v4.15

$ CFLAGS=-static make

Auto-detecting system features:
...                         clang-bpf-co-re: [ on  ]
...                                    llvm: [ OFF ]
...                                  libcap: [ OFF ]
...                                  libbfd: [ on  ]
...snip...

  LINK    bpftool
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(plugin.o): in function `try_load_plugin':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/plugin.c:273: warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(elf64-x86-64.o): in function `elf_x86_64_output_arch_local_syms':
...snip...

from bpftool.

qmonnet commented on August 15, 2024

4.15: This is because that version does not have the commit to fix compilation with recent versions of libbfd. You would need to compile against an older version, or to disable linking against binutils/libbfd in the Makefile (you'd lose the disassembler for JIT-compiled programs).
master branch: What's wrong? I don't see any error message in your output?

from bpftool.

vincentmli commented on August 15, 2024

master branch static build at LINK stage, maybe my environment are missing something for static build?

LINK bpftool
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(plugin.o): in function try_load_plugin': /build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/plugin.c:273: warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(elf64-x86-64.o): in function elf_x86_64_output_arch_local_syms':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/elf64-x86-64.c:4638: undefined reference to htab_traverse' /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(elf.o): in function _bfd_elf_is_local_label_name':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/elf.c:8975: undefined reference to _sch_istable' /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(elflink.o): in function elf_link_add_object_symbols':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/elflink.c:5289: undefined reference to objalloc_free_block' /usr/bin/ld: /build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/elflink.c:5539: undefined reference to _sch_istable'

from bpftool.

vincentmli commented on August 15, 2024

and here is the change I made to 4.15 branch to remove the disassembler feature, maybe I removed too much code :-)

diff

diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
index ec3052c0b004..cad70705ed59 100644
--- a/tools/bpf/bpftool/Makefile
+++ b/tools/bpf/bpftool/Makefile
@@ -53,17 +53,14 @@ CC = gcc
 CFLAGS += -O2
 CFLAGS += -W -Wall -Wextra -Wno-unused-parameter -Wshadow
 CFLAGS += -D__EXPORTED_HEADERS__ -I$(srctree)/tools/include/uapi -I$(srctree)/tools/include -I$(srctree)/tools/lib/bpf -I$(srctree)/kernel/bpf/
-LIBS = -lelf -lbfd -lopcodes $(LIBBPF)
+LIBS = -lelf $(LIBBPF)
 
 include $(wildcard *.d)
 
 all: $(OUTPUT)bpftool
 
 SRCS=$(wildcard *.c)
-OBJS=$(patsubst %.c,$(OUTPUT)%.o,$(SRCS)) $(OUTPUT)disasm.o
-
-$(OUTPUT)disasm.o: $(srctree)/kernel/bpf/disasm.c
-       $(QUIET_CC)$(COMPILE.c) -MMD -o $@ $<
+OBJS=$(patsubst %.c,$(OUTPUT)%.o,$(SRCS))
 
 $(OUTPUT)bpftool: $(OBJS) $(LIBBPF)
        $(QUIET_LINK)$(CC) $(CFLAGS) -o $@ $^ $(LIBS)
diff --git a/tools/bpf/bpftool/jit_disasm.c b/tools/bpf/bpftool/jit_disasm.c
deleted file mode 100644
index 1551d3918d4c..000000000000
--- a/tools/bpf/bpftool/jit_disasm.c
+++ /dev/null

diff --git a/tools/bpf/bpftool/main.c b/tools/bpf/bpftool/main.c
index d294bc8168be..62f534d42489 100644
--- a/tools/bpf/bpftool/main.c
+++ b/tools/bpf/bpftool/main.c
@@ -33,7 +33,6 @@
 
 /* Author: Jakub Kicinski <[email protected]> */
 
-#include <bfd.h>
 #include <ctype.h>
 #include <errno.h>
 #include <getopt.h>
@@ -327,7 +326,6 @@ int main(int argc, char **argv)
        if (argc < 0)
                usage();
 
-       bfd_init();
 
        ret = cmd_select(cmds, argc, argv, do_help);
 
diff --git a/tools/bpf/bpftool/prog.c b/tools/bpf/bpftool/prog.c
index dded77345bfb..224c9e9c819f 100644
--- a/tools/bpf/bpftool/prog.c
+++ b/tools/bpf/bpftool/prog.c
@@ -402,231 +402,6 @@ static int do_show(int argc, char **argv)
        return err;
 }
 
-static void print_insn(struct bpf_verifier_env *env, const char *fmt, ...)
-{
-       va_list args;
-
-       va_start(args, fmt);
-       vprintf(fmt, args);
-       va_end(args);
-}
-
-static void dump_xlated_plain(void *buf, unsigned int len, bool opcodes)
-{
-       struct bpf_insn *insn = buf;
-       bool double_insn = false;
-       unsigned int i;
-
-       for (i = 0; i < len / sizeof(*insn); i++) {
-               if (double_insn) {
-                       double_insn = false;
-                       continue;
-               }
-
-               double_insn = insn[i].code == (BPF_LD | BPF_IMM | BPF_DW);
-
-               printf("% 4d: ", i);
-               print_bpf_insn(print_insn, NULL, insn + i, true);
-
-               if (opcodes) {
-                       printf("       ");
-                       fprint_hex(stdout, insn + i, 8, " ");
-                       if (double_insn && i < len - 1) {
-                               printf(" ");
-                               fprint_hex(stdout, insn + i + 1, 8, " ");
-                       }
-                       printf("\n");
-               }
-       }
-}
-
-static void print_insn_json(struct bpf_verifier_env *env, const char *fmt, ...)
-{
-       unsigned int l = strlen(fmt);
-       char chomped_fmt[l];
-       va_list args;
-
-       va_start(args, fmt);
-       if (l > 0) {
-               strncpy(chomped_fmt, fmt, l - 1);
-               chomped_fmt[l - 1] = '\0';
-       }
-       jsonw_vprintf_enquote(json_wtr, chomped_fmt, args);
-       va_end(args);
-}
-
-static void dump_xlated_json(void *buf, unsigned int len, bool opcodes)
-{
-       struct bpf_insn *insn = buf;
-       bool double_insn = false;
-       unsigned int i;
-
-       jsonw_start_array(json_wtr);
-       for (i = 0; i < len / sizeof(*insn); i++) {
-               if (double_insn) {
-                       double_insn = false;
-                       continue;
-               }
-               double_insn = insn[i].code == (BPF_LD | BPF_IMM | BPF_DW);
-
-               jsonw_start_object(json_wtr);
-               jsonw_name(json_wtr, "disasm");
-               print_bpf_insn(print_insn_json, NULL, insn + i, true);
-
-               if (opcodes) {
-                       jsonw_name(json_wtr, "opcodes");
-                       jsonw_start_object(json_wtr);
-
-                       jsonw_name(json_wtr, "code");
-                       jsonw_printf(json_wtr, "\"0x%02hhx\"", insn[i].code);
-
-                       jsonw_name(json_wtr, "src_reg");
-                       jsonw_printf(json_wtr, "\"0x%hhx\"", insn[i].src_reg);
-
-                       jsonw_name(json_wtr, "dst_reg");
-                       jsonw_printf(json_wtr, "\"0x%hhx\"", insn[i].dst_reg);
-
-                       jsonw_name(json_wtr, "off");
-                       print_hex_data_json((uint8_t *)(&insn[i].off), 2);
-
-                       jsonw_name(json_wtr, "imm");
-                       if (double_insn && i < len - 1)
-                               print_hex_data_json((uint8_t *)(&insn[i].imm),
-                                                   12);
-                       else
-                               print_hex_data_json((uint8_t *)(&insn[i].imm),
-                                                   4);
-                       jsonw_end_object(json_wtr);
-               }
-               jsonw_end_object(json_wtr);
-       }
-       jsonw_end_array(json_wtr);
-}
-
-static int do_dump(int argc, char **argv)
-{
-       struct bpf_prog_info info = {};
-       __u32 len = sizeof(info);
-       unsigned int buf_size;
-       char *filepath = NULL;
-       bool opcodes = false;
-       unsigned char *buf;
-       __u32 *member_len;
-       __u64 *member_ptr;
-       ssize_t n;
-       int err;
-       int fd;
-
-       if (is_prefix(*argv, "jited")) {
-               member_len = &info.jited_prog_len;
-               member_ptr = &info.jited_prog_insns;
-       } else if (is_prefix(*argv, "xlated")) {
-               member_len = &info.xlated_prog_len;
-               member_ptr = &info.xlated_prog_insns;
-       } else {
-               p_err("expected 'xlated' or 'jited', got: %s", *argv);
-               return -1;
-       }
-       NEXT_ARG();
-
-       if (argc < 2)
-               usage();
-
-       fd = prog_parse_fd(&argc, &argv);
-       if (fd < 0)
-               return -1;
-
-       if (is_prefix(*argv, "file")) {
-               NEXT_ARG();
-               if (!argc) {
-                       p_err("expected file path");
-                       return -1;
-               }
-
-               filepath = *argv;
-               NEXT_ARG();
-       } else if (is_prefix(*argv, "opcodes")) {
-               opcodes = true;
-               NEXT_ARG();
-       }
-
-       if (argc) {
-               usage();
-               return -1;
-       }
-
-       err = bpf_obj_get_info_by_fd(fd, &info, &len);
-       if (err) {
-               p_err("can't get prog info: %s", strerror(errno));
-               return -1;
-       }
-
-       if (!*member_len) {
-               p_info("no instructions returned");
-               close(fd);
-               return 0;
-       }
-
-       buf_size = *member_len;
-
-       buf = malloc(buf_size);
-       if (!buf) {
-               p_err("mem alloc failed");
-               close(fd);
-               return -1;
-       }
-
-       memset(&info, 0, sizeof(info));
-
-       *member_ptr = ptr_to_u64(buf);
-       *member_len = buf_size;
-
-       err = bpf_obj_get_info_by_fd(fd, &info, &len);
-       close(fd);
-       if (err) {
-               p_err("can't get prog info: %s", strerror(errno));
-               goto err_free;
-       }
-
-       if (*member_len > buf_size) {
-               p_err("too many instructions returned");
-               goto err_free;
-       }
-
-       if (filepath) {
-               fd = open(filepath, O_WRONLY | O_CREAT | O_TRUNC, 0600);
-               if (fd < 0) {
-                       p_err("can't open file %s: %s", filepath,
-                             strerror(errno));
-                       goto err_free;
-               }
-
-               n = write(fd, buf, *member_len);
-               close(fd);
-               if (n != *member_len) {
-                       p_err("error writing output file: %s",
-                             n < 0 ? strerror(errno) : "short write");
-                       goto err_free;
-               }
-       } else {
-               if (member_len == &info.jited_prog_len)
-                       disasm_print_insn(buf, *member_len, opcodes);
-               else
-                       if (json_output)
-                               dump_xlated_json(buf, *member_len, opcodes);
-                       else
-                               dump_xlated_plain(buf, *member_len, opcodes);
-       }
-
-       free(buf);
-
-       return 0;
-
-err_free:
-       free(buf);
-       return -1;
-}
-
 static int do_pin(int argc, char **argv)
 {
        int err;
@@ -663,7 +438,6 @@ static int do_help(int argc, char **argv)
 static const struct cmd cmds[] = {
        { "show",       do_show },
        { "help",       do_help },
-       { "dump",       do_dump },
        { "pin",        do_pin },
        { 0 }
 };

compiling error:

$ sudo CFLAGS=-static make
  CC       common.o
  CC       main.o
  CC       json_writer.o
  CC       prog.o
  CC       map.o
make[1]: Entering directory '/usr/src/bpf-next/tools/lib/bpf'

Auto-detecting system features:
...                        libelf: [ on  ]
...                           bpf: [ on  ]

  CC       libbpf.o
  CC       bpf.o
  LD       libbpf-in.o
  LINK     libbpf.a
make[1]: Leaving directory '/usr/src/bpf-next/tools/lib/bpf'
  LINK     bpftool
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libelf.a(elf_compress.o): in function `__libelf_compress':
(.text+0x10a): undefined reference to `deflateInit_'
/usr/bin/ld: (.text+0x1ce): undefined reference to `deflate'
/usr/bin/ld: (.text+0x23c): undefined reference to `deflateEnd'
/usr/bin/ld: (.text+0x25e): undefined reference to `deflateEnd'
/usr/bin/ld: (.text+0x2fc): undefined reference to `deflateEnd'
/usr/bin/ld: (.text+0x324): undefined reference to `deflateEnd'
/usr/bin/ld: (.text+0x380): undefined reference to `deflate'
/usr/bin/ld: (.text+0x426): undefined reference to `deflateEnd'
/usr/bin/ld: (.text+0x456): undefined reference to `deflateEnd'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libelf.a(elf_compress.o): in function `__libelf_decompress':
(.text+0x52e): undefined reference to `inflateInit_'
/usr/bin/ld: (.text+0x55c): undefined reference to `inflate'
/usr/bin/ld: (.text+0x569): undefined reference to `inflateReset'
/usr/bin/ld: (.text+0x57d): undefined reference to `inflateEnd'
collect2: error: ld returned 1 exit status
make: *** [Makefile:66: bpftool] Error 1

from bpftool.

qmonnet commented on August 15, 2024

master branch static build at LINK stage, maybe my environment are missing something for static build?

Ah but this is from the GitHub mirror right? You need to use EXTRA_CFLAGS instead of CFLAGS, see instructions on the README.md of the repo

from bpftool.

qmonnet commented on August 15, 2024

and here is the change I made to 4.15 branch to remove the disassembler feature, maybe I removed too much code :-)

Maybe :)
Just remove libbfd and maybe related options from FEATURE_TESTS in the Makefile, and you should be good

from bpftool.

vincentmli commented on August 15, 2024

master branch static build at LINK stage, maybe my environment are missing something for static build?

Ah but this is from the GitHub mirror right? You need to use EXTRA_CFLAGS instead of CFLAGS, see instructions on the README.md of the repo

no this is from upstream git clone https://kernel.googlesource.com/pub/scm/linux/kernel/git/bpf/bpf-next
, not GitHub mirror.

I tried both EXTRA_CFLAGS=--static make and CFLAGS=--static make this time, here is the difference, CFLAGS=--static make detects libbfd and enable libbfd, result in LINK error, EXTRA_CFLAGS=--static make does not detect libbfd and disable libbfd, LINK succeeded

$ sudo  CFLAGS=--static make

Auto-detecting system features:
...                         clang-bpf-co-re: [ on  ]
...                                    llvm: [ OFF ]
...                                  libcap: [ OFF ]
...                                  libbfd: [ on  ] <===libbfd is on

and result in error

  LINK    bpftool
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(plugin.o): in function `try_load_plugin':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/plugin.c:273: warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(elf64-x86-64.o): in function `elf_x86_64_output_arch_local_syms':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/elf64-x86-64.c:4638: undefined reference to `htab_traverse'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(elf.o): in function `_bfd_elf_is_local_label_name':

try EXTRA_CFLAGS=--static make

$ sudo  EXTRA_CFLAGS=--static make

Auto-detecting system features:
...                         clang-bpf-co-re: [ on  ]
...                                    llvm: [ OFF ]
...                                  libcap: [ OFF ]
...                                  libbfd: [ OFF ] <====libbfd off

LINK bpftool

from bpftool.

vincentmli commented on August 15, 2024

bpftool GitHub mirror works both when libbfd is enabled or disabled

libbfd on, works

$ sudo CFLAGS=--static make
...                        libbfd: [ on  ]
...               clang-bpf-co-re: [ on  ]
...                          llvm: [ on  ]
...                        libcap: [ OFF ]
...snip...
  LINK     bpftool
/usr/bin/ld: /usr/local/lib/libLLVMSupport.a(Path.cpp.o): in function `llvm::sys::fs::expandTildeExpr(llvm::SmallVectorImpl<char>&)':
Path.cpp:(.text._ZN4llvm3sys2fsL15expandTildeExprERNS_15SmallVectorImplIcEE+0x1a0): warning: Using 'getpwnam_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/bin/ld: /usr/local/lib/libLLVMSupport.a(Path.cpp.o): in function `llvm::sys::path::home_directory(llvm::SmallVectorImpl<char>&) [clone .localalias]':
Path.cpp:(.text._ZN4llvm3sys4path14home_directoryERNS_15SmallVectorImplIcEE+0x10e): warning: Using 'getpwuid_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking

libbfd off, works

$ sudo EXTRA_CFLAGS=--static make
...                        libbfd: [ OFF ]
...               clang-bpf-co-re: [ on  ]
...                          llvm: [ on  ]
...                        libcap: [ OFF ]
...snip...
  LINK     bpftool
/usr/bin/ld: /usr/local/lib/libLLVMSupport.a(Path.cpp.o): in function `llvm::sys::fs::expandTildeExpr(llvm::SmallVectorImpl<char>&)':
Path.cpp:(.text._ZN4llvm3sys2fsL15expandTildeExprERNS_15SmallVectorImplIcEE+0x1a0): warning: Using 'getpwnam_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/bin/ld: /usr/local/lib/libLLVMSupport.a(Path.cpp.o): in function `llvm::sys::path::home_directory(llvm::SmallVectorImpl<char>&) [clone .localalias]':
Path.cpp:(.text._ZN4llvm3sys4path14home_directoryERNS_15SmallVectorImplIcEE+0x10e): warning: Using 'getpwuid_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking

from bpftool.

vincentmli commented on August 15, 2024

in summary, from upstream kernel bpf-next git repo, when libbfd is on and with static link to libbfd.a, undefined reference error occurs

output

vincent@srv-bk-gw:~/go/src/github.com/bpf-next/tools/bpf/bpftool$ sudo CFLAGS=--static make

Auto-detecting system features:
...                         clang-bpf-co-re: [ on  ]
...                                    llvm: [ OFF ]
...                                  libcap: [ OFF ]
...                                  libbfd: [ on  ]
  LINK    bpftool
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(plugin.o): in function `try_load_plugin':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/plugin.c:273: warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(elf64-x86-64.o): in function `elf_x86_64_output_arch_local_syms':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/elf64-x86-64.c:4638: undefined reference to `htab_traverse'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(elf.o): in function `_bfd_elf_is_local_label_name':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/elf.c:8975: undefined reference to `_sch_istable'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(elflink.o): in function `elf_link_add_object_symbols':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/elflink.c:5289: undefined reference to `objalloc_free_block'
/usr/bin/ld: /build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/elflink.c:5539: undefined reference to `_sch_istable'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(elflink.o): in function `bfd_elf_size_dynamic_sections':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/elflink.c:6750: undefined reference to `lbasename'
/usr/bin/ld: /build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/elflink.c:6523: undefined reference to `lbasename'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(archures.o): in function `bfd_default_scan':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/archures.c:1195: undefined reference to `_sch_istable'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(bfd.o): in function `bfd_errmsg':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/bfd.c:795: undefined reference to `xstrerror'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(bfd.o): in function `_bfd_doprnt_scan':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/bfd.c:1162: undefined reference to `_sch_istable'
/usr/bin/ld: /build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/bfd.c:1178: undefined reference to `_sch_istable'
/usr/bin/ld: /build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/bfd.c:1215: undefined reference to `_sch_istable'
/usr/bin/ld: /build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/bfd.c:1190: undefined reference to `_sch_istable'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(bfd.o): in function `_bfd_doprnt':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/bfd.c:921: undefined reference to `_sch_istable'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(bfd.o):/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/bfd.c:938: more undefined references to `_sch_istable' follow
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(bfd.o): in function `bfd_demangle':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/bfd.c:2482: undefined reference to `cplus_demangle'
/usr/bin/ld: /build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/bfd.c:2482: undefined reference to `cplus_demangle'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(hash.o): in function `bfd_hash_table_free':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/hash.c:426: undefined reference to `objalloc_free'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(hash.o): in function `bfd_hash_table_init_n':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/hash.c:385: undefined reference to `objalloc_create'
/usr/bin/ld: /build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/hash.c:392: undefined reference to `_objalloc_alloc'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(hash.o): in function `bfd_hash_insert':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/hash.c:536: undefined reference to `_objalloc_alloc'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(hash.o): in function `bfd_hash_lookup':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/hash.c:486: undefined reference to `_objalloc_alloc'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(hash.o): in function `bfd_hash_allocate':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/hash.c:623: undefined reference to `_objalloc_alloc'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(opncls.o): in function `_bfd_delete_bfd':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/opncls.c:126: undefined reference to `objalloc_free'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(opncls.o): in function `find_separate_debug_file':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/opncls.c:1507: undefined reference to `lrealpath'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(opncls.o): in function `_bfd_new_bfd':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/opncls.c:74: undefined reference to `objalloc_create'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(opncls.o): in function `_bfd_free_cached_info':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/opncls.c:143: undefined reference to `objalloc_free'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(opncls.o): in function `bfd_alloc':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/opncls.c:985: undefined reference to `_objalloc_alloc'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(opncls.o): in function `bfd_create_gnu_debuglink_section':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/opncls.c:1715: undefined reference to `lbasename'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(opncls.o): in function `bfd_fill_in_gnu_debuglink_section':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/opncls.c:1809: undefined reference to `lbasename'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(opncls.o): in function `bfd_release':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/opncls.c:1079: undefined reference to `objalloc_free_block'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(binary.o): in function `mangle_name':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/binary.c:139: undefined reference to `_sch_istable'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(ihex.o): in function `ihex_bad_byte':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/ihex.c:221: undefined reference to `_sch_istable'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(ihex.o): in function `ihex_object_p':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/ihex.c:509: undefined reference to `_hex_value'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(ihex.o): in function `ihex_init':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/ihex.c:168: undefined reference to `hex_init'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(ihex.o): in function `ihex_read_section':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/ihex.c:578: undefined reference to `_hex_value'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(srec.o): in function `srec_init':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/srec.c:186: undefined reference to `hex_init'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(srec.o): in function `srec_read_section':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/srec.c:748: undefined reference to `_hex_value'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(srec.o): in function `srec_bad_byte':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/srec.c:251: undefined reference to `_sch_istable'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(srec.o): in function `srec_scan':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/srec.c:469: undefined reference to `_hex_value'
/usr/bin/ld: /build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/srec.c:414: undefined reference to `_hex_value'
/usr/bin/ld: /build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/srec.c:373: undefined reference to `_sch_istable'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(srec.o): in function `srec_object_p':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/srec.c:661: undefined reference to `_hex_value'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(srec.o): in function `srec_init':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/srec.c:186: undefined reference to `hex_init'
/usr/bin/ld: /build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/srec.c:186: undefined reference to `hex_init'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(tekhex.o): in function `getvalue':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/tekhex.c:279: undefined reference to `_hex_value'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(tekhex.o): in function `getsym':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/tekhex.c:304: undefined reference to `_hex_value'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(tekhex.o): in function `first_phase':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/tekhex.c:379: undefined reference to `_hex_value'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(tekhex.o): in function `tekhex_init':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/tekhex.c:213: undefined reference to `hex_init'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(tekhex.o): in function `pass_over':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/tekhex.c:540: undefined reference to `_hex_value'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(tekhex.o): in function `tekhex_object_p':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/tekhex.c:614: undefined reference to `_hex_value'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(verilog.o): in function `verilog_init':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/verilog.c:329: undefined reference to `hex_init'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(elfxx-x86.o): in function `elf_x86_link_hash_table_free':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/elfxx-x86.c:734: undefined reference to `htab_delete'
/usr/bin/ld: /build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/elfxx-x86.c:736: undefined reference to `objalloc_free'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(elfxx-x86.o): in function `_bfd_elf_x86_get_local_sym_hash':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/elfxx-x86.c:622: undefined reference to `htab_find_slot_with_hash'
/usr/bin/ld: /build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/elfxx-x86.c:635: undefined reference to `_objalloc_alloc'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(elfxx-x86.o): in function `_bfd_x86_elf_link_hash_table_create':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/elfxx-x86.c:821: undefined reference to `htab_try_create'
/usr/bin/ld: /build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/elfxx-x86.c:825: undefined reference to `objalloc_create'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(elfxx-x86.o): in function `_bfd_x86_elf_size_dynamic_sections':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/elfxx-x86.c:1107: undefined reference to `htab_traverse'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(elf-eh-frame.o): in function `cie_compute_hash':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/elf-eh-frame.c:261: undefined reference to `iterative_hash'
/usr/bin/ld: /build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/elf-eh-frame.c:262: undefined reference to `iterative_hash'
/usr/bin/ld: /build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/elf-eh-frame.c:263: undefined reference to `iterative_hash'
/usr/bin/ld: /build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/elf-eh-frame.c:264: undefined reference to `iterative_hash'
/usr/bin/ld: /build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/elf-eh-frame.c:265: undefined reference to `iterative_hash'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(elf-eh-frame.o):/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/elf-eh-frame.c:266: more undefined references to `iterative_hash' follow
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(elf-eh-frame.o): in function `find_merged_cie':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/elf-eh-frame.c:1303: undefined reference to `htab_find_slot_with_hash'
/usr/bin/ld: /build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/elf-eh-frame.c:1299: undefined reference to `htab_try_create'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(elf-eh-frame.o): in function `_bfd_elf_discard_section_eh_frame_hdr':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/elf-eh-frame.c:1630: undefined reference to `htab_delete'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(dwarf2.o): in function `hash_abbrev':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/dwarf2.c:965: undefined reference to `htab_hash_pointer'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(dwarf2.o): in function `read_abbrevs':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/dwarf2.c:1018: undefined reference to `htab_find_slot'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(dwarf2.o): in function `_bfd_dwarf2_find_symbol_bias':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/dwarf2.c:4742: undefined reference to `xcalloc'
/usr/bin/ld: /build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/dwarf2.c:4742: undefined reference to `htab_create_alloc'
/usr/bin/ld: /build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/dwarf2.c:4750: undefined reference to `htab_find_slot'
/usr/bin/ld: /build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/dwarf2.c:4769: undefined reference to `htab_find'
/usr/bin/ld: /build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/dwarf2.c:4780: undefined reference to `htab_delete'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(dwarf2.o): in function `_bfd_dwarf2_cleanup_debug_info':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/dwarf2.c:5147: undefined reference to `htab_delete'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(dwarf2.o): in function `_bfd_dwarf2_slurp_debug_info':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/dwarf2.c:4504: undefined reference to `htab_create_alloc'
/usr/bin/ld: /build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/dwarf2.c:4509: undefined reference to `htab_create_alloc'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(dwarf2.o): in function `hash_asymbol':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/dwarf2.c:4710: undefined reference to `htab_hash_string'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(elf32-i386.o): in function `elf_i386_output_arch_local_syms':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/elf32-i386.c:4093: undefined reference to `htab_traverse'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(cofflink.o): in function `coff_link_add_symbols':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/cofflink.c:538: undefined reference to `_sch_istable'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(pei-x86_64.o): in function `pex64_bfd_print_pdata_section':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/pei-x86_64.c:622: undefined reference to `xmalloc'
/usr/bin/ld: /build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/pei-x86_64.c:698: undefined reference to `xmalloc'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(plugin.o): in function `build_plugin_list':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/plugin.c:423: undefined reference to `make_relative_prefix'
/usr/bin/ld: /build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/plugin.c:446: undefined reference to `concat'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(archive.o): in function `adjust_relative_path':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/archive.c:1380: undefined reference to `getpwd'
/usr/bin/ld: /build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/archive.c:1384: undefined reference to `lrealpath'
/usr/bin/ld: /build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/archive.c:1387: undefined reference to `lrealpath'
/usr/bin/ld: /build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/archive.c:1401: undefined reference to `filename_ncmp'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(archive.o): in function `_bfd_look_for_bfd_in_cache':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/archive.c:311: undefined reference to `htab_find'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(archive.o): in function `_bfd_add_bfd_to_archive_cache':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/archive.c:372: undefined reference to `htab_find_slot'
/usr/bin/ld: /build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/archive.c:361: undefined reference to `htab_create_alloc'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(archive.o): in function `_bfd_generic_read_ar_hdr_mag':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/archive.c:537: undefined reference to `_sch_istable'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(archive.o): in function `_bfd_append_relative_path':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/archive.c:623: undefined reference to `lbasename'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(archive.o): in function `find_nested_archive':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/archive.c:406: undefined reference to `filename_cmp'
/usr/bin/ld: /build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/archive.c:416: undefined reference to `filename_cmp'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(archive.o): in function `_bfd_construct_extended_name_table':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/archive.c:1552: undefined reference to `filename_cmp'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(archive.o): in function `normalize':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/archive.c:1341: undefined reference to `lbasename'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(archive.o): in function `_bfd_construct_extended_name_table':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/archive.c:1598: undefined reference to `filename_ncmp'
/usr/bin/ld: /build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/archive.c:1646: undefined reference to `filename_cmp'
/usr/bin/ld: /build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/archive.c:1598: undefined reference to `filename_ncmp'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(archive.o): in function `normalize':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/archive.c:1341: undefined reference to `lbasename'
/usr/bin/ld: /build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/archive.c:1341: undefined reference to `lbasename'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(archive.o): in function `_bfd_bsd44_write_ar_hdr':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/archive.c:1761: undefined reference to `_sch_istable'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(archive.o): in function `normalize':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/archive.c:1341: undefined reference to `lbasename'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(archive.o): in function `bfd_bsd_truncate_arname':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/archive.c:2018: undefined reference to `lbasename'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(archive.o): in function `normalize':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/archive.c:1341: undefined reference to `lbasename'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(archive.o): in function `bfd_gnu_truncate_arname':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/archive.c:2050: undefined reference to `lbasename'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(archive.o): in function `_bfd_unlink_from_archive_parent':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/archive.c:2806: undefined reference to `htab_find_slot'
/usr/bin/ld: /build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/archive.c:2810: undefined reference to `htab_clear_slot'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(archive.o): in function `_bfd_archive_close_and_cleanup':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/archive.c:2835: undefined reference to `htab_traverse_noresize'
/usr/bin/ld: /build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/archive.c:2836: undefined reference to `htab_delete'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(cache.o): in function `bfd_open_file':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/cache.c:631: undefined reference to `unlink_if_ordinary'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(compress.o): in function `bfd_is_section_compressed_with_header':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/compress.c:454: undefined reference to `_sch_istable'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(compress.o): in function `bfd_compress_section_contents':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/compress.c:127: undefined reference to `compressBound'
/usr/bin/ld: /build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/compress.c:174: undefined reference to `compress'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(corefile.o): in function `generic_core_file_matches_executable_p':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/corefile.c:188: undefined reference to `filename_cmp'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(stabs.o): in function `_bfd_link_section_stabs':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/stabs.c:352: undefined reference to `_sch_istable'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(syms.o): in function `bfd_decode_symclass':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/syms.c:715: undefined reference to `_sch_toupper'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(syms.o): in function `_bfd_stab_section_find_nearest_line':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/syms.c:1421: undefined reference to `filename_ncmp'
/usr/bin/ld: /build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/syms.c:1422: undefined reference to `filename_cmp'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libopcodes.a(disassemble.o): in function `remove_whitespace_and_extra_commas':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/opcodes/../../opcodes/disassemble.c:784: undefined reference to `_sch_istable'
/usr/bin/ld: /build/binutils-UqFHUN/binutils-2.34/builddir-single/opcodes/../../opcodes/disassemble.c:792: undefined reference to `_sch_istable'
collect2: error: ld returned 1 exit status
make: *** [Makefile:244: bpftool] Error 1

from bpftool.

vincentmli commented on August 15, 2024

bpftool GitHub mirror works both when libbfd is enabled or disabled

Yes, I've spent quite some time recently to make sure that it works properly on the GH mirror, and I've got some CI checks to detect if it breaks. So yeah, I do hope it works on the mirror.

in summary, from upstream kernel bpf-next git repo, when libbfd is on and with static link to libbfd.a, undefined reference error occurs

With CFLAGS=--static

OK, I see the same on my side. The error is due to the feature detection test for libbfd passing on static builds when it should fail, because we need feature-libbfd-liberty-z instead on Ubuntu. I could fix the build with this quick hack in the Makefile:
diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
index 681fbcc5ed50..ebcc46b1e161 100644
--- a/tools/bpf/bpftool/Makefile
+++ b/tools/bpf/bpftool/Makefile
@@ -153,7 +153,7 @@ ifeq ($(feature-llvm),1)
 else
   # Fall back on libbfd
   ifeq ($(feature-libbfd),1)
-    LIBS += -lbfd -ldl -lopcodes
+    LIBS += -lbfd -ldl -lopcodes -liberty -lz
   else ifeq ($(feature-libbfd-liberty),1)
     LIBS += -lbfd -ldl -lopcodes -liberty
   else ifeq ($(feature-libbfd-liberty-z),1)

ok I made above changes, but got error /usr/bin/ld: cannot find -liberty, what is -liberty? a library I need to install on Ubuntu?

  LINK    bpftool
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libbfd.a(plugin.o): in function `try_load_plugin':
/build/binutils-UqFHUN/binutils-2.34/builddir-single/bfd/../../bfd/plugin.c:273: warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/bin/ld: cannot find -liberty
collect2: error: ld returned 1 exit status
make: *** [Makefile:244: bpftool] Error 1

since you put lot of effort to make bpftool GitHub mirror build successfully, I guess the recommendation is to compile and use bpftool from GitHub mirror, not from the kernel repo?

from bpftool.

qmonnet commented on August 15, 2024

ok I made above changes, but got error /usr/bin/ld: cannot find -liberty, what is -liberty? a library I need to install on Ubuntu?

You should preferably use EXTRA_CFLAGS, rather. But yes, in both case you need to apt install libiberty-dev. I would expect it to be installed though, given that you managed to compile already, even if it was not a static build 🤔.

since you put lot of effort to make bpftool GitHub mirror build successfully, I guess the recommendation is to compile and use bpftool from GitHub mirror, not from the kernel repo?

Static build works on the kernel repo as well, so no, I don't think it's particularly more stable here. I'd usually recommend building from the mirror because it's simpler, you don't have to download the full kernel repo.

But in your case, if you want to load programs on older Ubuntu versions, the bpftool versions in the mirror might be too recent (that was you initial report, we've gone full circle :) ). So if you do want to make it work I think picking the version that was in the kernel repo for 4.15:

Without libbfd (losing the ability to bpftool prog dump jited, but you can still bpftool prog dump xlated) because of #91 (comment)
Or by compiling from 4.16 sources which should have the commit to fix that; although depending on your libbfd version you might also hit a subsequent issue, addressed in 6.0
Or by using an older libbfd

from bpftool.

vincentmli commented on August 15, 2024

Thanks, I am about to give up building static bpftool from mainline kernel repo for old Ubuntu 16.04 kernel, as you said, it is too old in BPF world. I think if I know enough about bpftool and the build system and do as you suggested, I probably could make it work, but it could take too much time if I continue to bug you :). I had come up alternative solution which is to put a most recent Ubuntu machine as bridge with XDP program loaded to protect the legacy Ubuntu 16.04 machine. we are good to close this issue report, thank you a lot for your help!

from bpftool.

qmonnet commented on August 15, 2024

You're welcome! Let me know how it goes :)

from bpftool.

vincentmli commented on August 15, 2024

@qmonnet FYI, I am able to compile https://github.com/cilium/iproute2 libbpf-static-data branch on Ubuntu 16.04 and run the ip command to load XDP program on Unbuntu 16.04 with 4.15 kernel.

I did try bpf-next 4.15 kernel as you suggested, removed disassemble feature and able to compile bpftool, but the bpf-next 4.15 kernel bpftool lack of load feature so I am unable to use bpftool

from bpftool.

qmonnet commented on August 15, 2024

Ah right, not being able to load is certainly limiting for actually loading programs :).

Glad you found a solution, and thank you for reporting back!

from bpftool.

bpftool v6.7.0 load xdp program on Ubuntu 16.04 kernel 4.15.0-99-generic about bpftool HOT 23 CLOSED

Comments (23)

With `CFLAGS=--static`

With `EXTRA_CFLAGS=--static`

With `CFLAGS=--static`

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent

Comments (23)

With CFLAGS=--static

With EXTRA_CFLAGS=--static

With CFLAGS=--static

Related Issues (20)

Recommend Projects

Recommend Topics

Recommend Org

With `CFLAGS=--static`

With `EXTRA_CFLAGS=--static`

With `CFLAGS=--static`