solo-io / bumblebee Goto Github PK
View Code? Open in Web Editor NEWGet eBPF programs running from the cloud to the kernel in 1 line of bash
License: Apache License 2.0
Get eBPF programs running from the cloud to the kernel in 1 line of bash
License: Apache License 2.0
No response
No response
It would be nice if the CLI would print the help message for the command if there is an invalid number of arguments passed.
For example:
$ ~/go/bin/bee package hello
Error: accepts 2 arg(s), received 1
2024/02/15 16:59:00 exiting: accepts 2 arg(s), received 1
I would prefer if it printed this:
bumblebee/pkg/cli/internal/commands/package/package.go
Lines 39 to 50 in c2422b5
No response
No response
No response
0.0.13
5.15.0-46-generic
The Bumblebee endpoint cannot be moved to a port other than the default port 9091.
In our architecture. Bumblebee is not directly scraped by Prometheus because we require data enrichment with user space information. To achieve this, we have wrapped the Bumblebee endpoint.
The idea: While Bumblebee streamlines user space development and is hidden from developers, this approach also introduces some lack of flexibility when dealing with user-space data. However, this "wrapper-based" approach can mitigate Bumblebee's lack of control on the user space side. (see also: this other issue)
Here's how it works:
Prometheus scrapes our wrapper endpoint, which is on port 9091.
The wrapper endpoint calls the Bumblebee endpoint, adds additional information to the metrics, and returns the enriched metric to Prometheus.
Both Bumblebee and the wrapper share the same POD, which is why we cannot use the default 9091 port for Bumblebee to avoid conflicts.
Add a -port flag to override the default port
split the two containers adding two different pod.
However, this is not a good approach for us.
Before we also considered an alternative way to send "messages" from user space to kenel spaces. We attached a probe on file open and used the TOUCH and CAT operation on files (and paths) under a specific folder to "communicate" some filter to the kernel space. E.G. we sent the cgroups of running pods throug touching a filename with the corresponding name. TOUCH a file with a certain CGROUP means "POD STARTED". CAT the same file means "POD STOPPED".
We used this information to keep the shared map as clean as possible from cgroups that no logner belongs to a running pod.
It workes, but this is not ideal, the C code is much more complicated than the plain C we actually need, so we moved to the "wrapper" pattern. However we need to restart bumblebee once a day to free the map.
No response
Hi all, I have written several bpf programs running them with Bumblebee and I found his approach very simple. But now I'm stuck with a user-space code error and I can't figure out the problem origin.
Actually, the error is quite clear and well defined:
panic: counter cannot decrease in goroutine 116 [running]:
github.com/prometheus/client_golang/prometheus.(*counter).Add(0xc001ad8060, 0xc008000000000000)
github.com/solo-io/bumblebee/pkg/stats.(*setCounter).Set(0xc00269c7c0, {0xf320f0, 0xc001acf880}, 0xd, 0xc000ff8180)
github.com/solo-io/bumblebee/pkg/loader.(*loader).startHashMap(0xc000012270, {0xf320f0, 0xc001acf880}, 0xc00029d7c0, 0xc0000c5c20, {0xf22220, 0xc00269c7c0}, {0xc0001140e0, 0x1d}, {0xf32748, ...})
/home/runner/work/bumblebee/bumblebee/pkg/loader/loader.go:400 +0x843
github.com/solo-io/bumblebee/pkg/loader.(*loader).WatchMaps.func2()
/home/runner/work/bumblebee/bumblebee/pkg/loader/loader.go:286 +0x18c
golang.org/x/sync/errgroup.(*Group).Go.func1()
/home/runner/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:57 +0xb4
created by golang.org/x/sync/errgroup.(*Group).Go
/home/runner/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:54 +0xb3
This happens because the value for the counter map in my ebpf is going to decrease. Just to confirm the problem I moved the map to gauge and the error disappears.
I guess using a gauge is not the solution, in fact I'm expecting a monotonic data, because I just increment the counter by 1 for each ebpf event.
On the other hand, I cannot understand the origin of this kind of race condition, in fact the ebpf code contains a specific check on that:
// pointer_to_key is a pointer to a struct representing the object I'm tracking down by incrementing the counter.
// the struct is allocated into a PER_CPU_ARRAY
u64 *cntp;
cntp = bpf_map_lookup_elem(&my_bpf_hash_map_counter, pointer_to_key); // declared as SEC(".maps.counter");
if (cntp)
{
__sync_fetch_and_add(cntp, 1);
}
else
{
u64 cnt = 1;
bpf_map_update_elem(&my_bpf_hash_map_counter, pointer_to_key, &cnt, BPF_NOEXIST);
}
I'm not an eBpf expert, however I cannot see how the counter value could be lower that the previous value here.
I think the bpf_map_update_elem function is the only one that could "decrease" some value. However both the if-else than the BPF_NOEXIST flag should be preventing from updating an higher existing record.
Any idea would be very apreciated!
Thanks!
[EDIT]
I have an update on that issue: the problem had nothing to do with concurrency.
After debugging a while, I've found that my_bpf_hash_map_t keys content was wrong.
In fact I populated in a very bad way these keys, and happened that keys I was expecting as EQUALS were actually DIFFERENT for the bpf map. I know that thanks to the bpf_trace_printk, showing me the return value == 0 for
bpf_map_update_elem(&my_bpf_hash_map, pointer_to_key, &cnt, BPF_NOEXIST);
I guess something like that happened: immagine this is key A:
char val_1 = ['f','o','o','\0' ..... (something more I ignore causing bpf consider this different)];
...
char val_6 = ['b','a','r','\0' ..... (something more I ignore causing bpf consider this different)];
u32 number = 1234;
...and this is key B:
char val_1 = ['f','o','o','\0' ..... (something more I ignore causing bpf consider this different)];
...
char val_6 = ['b','a','r','\0' ..... (something more I ignore causing bpf consider this different)];
u32 number = 1234;
However, this still cannot explain the client code error. An so, this is what was going on on the (GOlang) user-space:
Keys equality was correctly detected by the user space code (while reading the copy of values from the same shared map), and so when a new ebpf key is created, the respective counter's value appears to be as a drop to the value of 1 (on the user-side point of view).
Writing good content into keys this behavior was solved and now ebpf maps creates a new key only if the content actually differs for meaningful differences.
No response
No response
No response
Currently Docker containers allow signing for security purposes. Bumblebee
should allow users to verify the provenance of their programs.
No response
No response
0.0.9
ubuntu 18.04 lts , Kernel version : 5.4.0-91-generic
I successfully build tcpconnect example using bee build tcpconnect.c tcp:v2
But while running it using bee run tcp:v2
getting below error:
SUCCESS Fetching program from registry: tcp:v2
ERROR Loading BPF program and maps into Kernel
Error: map events_ring: map create: invalid argument
2022/01/11 14:42:58 exiting: map events_ring: map create: invalid argument
I also tested a hash map and ring buffer test ebpf codes separately. I saw that the code using hash_map ran successfully but the one with ring buffer is giving the same error while running.
It shall run successfully.
No response
vagrant@impish:~$ bee build /tmp/hello.c hello:v1
▄ Compiling BPF program locally (58s)Unable to find image 'ghcr.io/solo-io/bumblebee/builder:0.0.9' locally
0.0.9: Pulling from solo-io/bumblebee/builder
0e29546d541c: Pulling fs layer
fca6a5226638: Pulling fs layer
afcfaa23af5a: Pulling fs layer
d35920e59df0: Pulling fs layer
4f601b52c7f8: Pulling fs layer
fc2c674a0c0b: Pulling fs layer
9feed63100ed: Pulling fs layer
bed84b2b6d89: Pulling fs layer
fb7b09ef9b9e: Pulling fs layer
2999dd9432b1: Pulling fs layer
bcf743229f06: Pulling fs layer
9feed63100ed: Waiting
bed84b2b6d89: Waiting
fb7b09ef9b9e: Waiting
d35920e59df0: Waiting
4f601b52c7f8: Waiting
fc2c674a0c0b: Waiting
2999dd9432b1: Waiting
bcf743229f06: Waiting
afcfaa23af5a: Download complete
d35920e59df0: Verifying Checksum
d35920e59df0: Download complete
4f601b52c7f8: Verifying Checksum
4f601b52c7f8: Download complete
fca6a5226638: Verifying Checksum
fca6a5226638: Download complete
0e29546d541c: Verifying Checksum
0e29546d541c: Download complete
9feed63100ed: Verifying Checksum
9feed63100ed: Download complete
bed84b2b6d89: Verifying Checksum
bed84b2b6d89: Download complete
fb7b09ef9b9e: Verifying Checksum
fb7b09ef9b9e: Download complete
2999dd9432b1: Verifying Checksum
2999dd9432b1: Download complete
bcf743229f06: Download complete
0e29546d541c: Pull complete
fca6a5226638: Pull complete
afcfaa23af5a: Pull complete
d35920e59df0: Pull complete
4f601b52c7f8: Pull complete
fc2c674a0c0b: Verifying Checksum
fc2c674a0c0b: Download complete
fc2c674a0c0b: Pull complete
9feed63100ed: Pull complete
bed84b2b6d89: Pull complete
fb7b09ef9b9e: Pull complete
2999dd9432b1: Pull complete
bcf743229f06: Pull complete
Digest: sha256:4c4f21da350cc95acd0e1b3f0e3a4b5cc1a8b7aa4a197c097bbdb470abb83305
Status: Downloaded newer image for ghcr.io/solo-io/bumblebee/builder:0.0.9
clang: error: no such file or directory: '/tmp/hello.c'
clang: error: no input files
ERROR Failed to compile BPF program
Error: exit status 1
2022/01/06 10:26:24 exiting: exit status 1
any
No response
The second line of the install file uses shopt, and when running this as a script per the README - getting the error sh: 4: shopt: not found
curl -sL https://run.solo.io/bee/install | sh
Installs correctly
It works properly by executing through bash instead ( not taking credit for this, whoever was in the Open Source Summit called this out, I'm just writing it down )
curl -sL https://run.solo.io/bee/install | bash
0.0.10
5.13.0-28-generic
No response
Bumblebee currently does not allow the user any control on the user space program and only allows writing of some kernel programs. It would be very useful to allow some user space level control.
This could range from
(1) Generate a user space stub program that goes with the kernel program generated and shares the bpf map struct definitions etc
(2) Document any safe workarounds until then such as using the bpftools utility directly to interface with Bee's kernel space program. Document any race scenarios/ concurrency issues that developers may need to be aware of, or use of bpf maps vs global variables etc.
Basic use case example might be setting up network filters in the data plane based on user configured filter/ policy keys.
No response
No response
main
N/A
When iterating builds with the build container (not as sudo
), the BPF ELF generated is owned by root
, so successive builds fail until the ELF is manually removed e.g. via sudo
$ ./mybee build examples/tcpconnect/tcpconnect.c tcpconnect
Error: open examples/tcpconnect/tcpconnect.o: permission denied
2022/01/11 16:51:00 exiting: open examples/tcpconnect/tcpconnect.o: permission denied
ls -al examples/tcpconnect/tcpconnect.o
-rw-r--r-- 1 root root 731424 Jan 11 02:17 examples/tcpconnect/tcpconnect.o
UX allowing iteration without manual cleanup necessary
No response
Unfortunately this is due to the fact that we currently do not support ARM.
We do have adding multi-arch builds on our roadmap, but until that gets implemented we should probably be more clear of our support (and maybe even fail the installation script if it detects non-AMD architecture)
Originally posted by @lgadban in #86 (comment)
No response
No response
Does bumblebee support chaining 1+ ebpf programs? Is this on the roadmap?
If I have 2 ebpf programs (they are not related), do i need 2 bumblebee runner to run them (for example 2 daemonset yamls in k8s)? Anyway this can be optimized?
Could I pass in simple params to bee runner, for example, I am only interested in a particular pid vs capturing every pid info?
No response
No response
No response
dev
5.11.0-49-generic
There is no such image in the repository.
sudo bee run ghcr.io/solo-io/bumblebee/tcpconnect:$(bee version)
The tag is dev
, but there is no image with such tag:
Run sudo bee run ghcr.io/solo-io/bumblebee/tcpconnect:$(bee version)
after install bee
Run success
No response
The current BumbleBee workflow is:
bee build
bee run
So, in order to run your BPF programs, the bee
CLI must be present on any system you wish to run them on.
To make the entire process portable, we can package the bee
runner/loader along with the built BPF programs in a container to allow it to run anywhere without prior work necessary to e.g. install the bee
CLI.
cc: @aantn -- thanks for the suggestion!
When users are prompted with BPF Program File Location
hitting enter should default to probe.c
in the current directory if the C language is picked
Here are my results based on the example provided in the README:
% vagrant up
Bringing machine 'impish' up with 'vmware_desktop' provider...
==> impish: Checking if box 'generic/ubuntu2110' version '3.6.4' is up to date...
==> impish: Verifying vmnet devices are healthy...
==> impish: Preparing network adapters...
==> impish: Starting the VMware VM...
An error occurred while executing `vmrun`, a utility for controlling
VMware machines. The command and output are below:
Command: ["start", "/Users/nschuetz/Projects/bumblebee/.vagrant/machines/impish/vmware_desktop/577e0bde-07ab-41e0-b964-2a3a9092f21a/generic-ubuntu2110-vmware.vmx", "nogui", {:notify=>[:stdout, :stderr], :timeout=>45}]
Stdout: 2022-01-06T11:01:00.524| ServiceImpl_Opener: PID 28446
Error: The operation was canceled
Stderr:
Then I tried using this setup with an actual arm64 box: https://gist.github.com/sbailliez/f22db6434ac84eccb6d3c8833c85ad92
Here are my results:
vmrun version 1.17.0 build-18656771
Vagrant File:
Vagrant.configure("2") do |config|
config.vm.box = "spox/ubuntu-arm"
config.vm.box_version = "1.0.0"
config.vm.define :impish
config.vm.hostname = "impish"
config.vm.synced_folder ".", "/source"
config.vm.provision "shell", inline: <<-SHELL
# install llvm:
export DEBIAN_FRONTEND=noninteractive
export LLVM_VERSION=13
wget https://apt.llvm.org/llvm.sh
bash ./llvm.sh "$LLVM_VERSION"
rm ./llvm.sh
apt-get -qq update
# bpf related deps:
apt-get -qq install linux-headers-$(uname -r) linux-tools-$(uname -r) libbpf-dev
# dev tools:
apt-get -qq install -y golang docker.io make
usermod -aG docker vagrant
# add headers:
bpftool btf dump file /sys/kernel/btf/vmlinux format c > /usr/local/include/vmlinux.h
cp /source/builder/solo_types.h /usr/local/include/
SHELL
end
% vagrant up
Bringing machine 'impish' up with 'vmware_desktop' provider...
==> impish: Cloning VMware VM: 'spox/ubuntu-arm'. This can take some time...
==> impish: Checking if box 'spox/ubuntu-arm' version '1.0.0' is up to date...
==> impish: Verifying vmnet devices are healthy...
==> impish: Preparing network adapters...
WARNING: The VMX file for this box contains a setting that is automatically overwritten by Vagrant
WARNING: when started. Vagrant will stop overwriting this setting in an upcoming release which may
WARNING: prevent proper networking setup. Below is the detected VMX setting:
WARNING:
WARNING: ethernet0.pcislotnumber = "160"
WARNING:
WARNING: If networking fails to properly configure, it may require this VMX setting. It can be manually
WARNING: applied via the Vagrantfile:
WARNING:
WARNING: Vagrant.configure(2) do |config|
WARNING: config.vm.provider :vmware_desktop do |vmware|
WARNING: vmware.vmx["ethernet0.pcislotnumber"] = "160"
WARNING: end
WARNING: end
WARNING:
WARNING: For more information: https://www.vagrantup.com/docs/vmware/boxes.html#vmx-allowlisting
==> impish: Starting the VMware VM...
==> impish: Waiting for the VM to receive an address...
==> impish: Forwarding ports...
impish: -- 22 => 2222
==> impish: Waiting for machine to boot. This may take a few minutes...
impish: SSH address: 127.0.0.1:2222
impish: SSH username: vagrant
impish: SSH auth method: private key
impish:
impish: Vagrant insecure key detected. Vagrant will automatically replace
impish: this with a newly generated keypair for better security.
impish:
impish: Inserting generated public key within guest...
impish: Removing insecure key from the guest if it's present...
impish: Key inserted! Disconnecting and reconnecting using new SSH key...
==> impish: Machine booted and ready!
==> impish: Setting hostname...
==> impish: Configuring network adapters within the VM...
==> impish: Waiting for HGFS to become available...
==> impish: Enabling and configuring shared folders...
impish: -- /Users/nschuetz/Projects/bumblebee: /source
==> impish: Running provisioner: shell...
impish: Running: inline script
impish: --2022-01-06 16:54:35-- https://apt.llvm.org/llvm.sh
impish: Resolving apt.llvm.org (apt.llvm.org)... 199.232.6.49, 2a04:4e42:3e::561
impish: Connecting to apt.llvm.org (apt.llvm.org)|199.232.6.49|:443... connected.
impish: HTTP request sent, awaiting response... 200 OK
impish: Length: 3751 (3.7K) [application/octet-stream]
impish: Saving to: ‘llvm.sh’
impish:
impish: 0K ... 100% 4.48M=0.001s
impish:
impish: 2022-01-06 16:54:36 (4.48 MB/s) - ‘llvm.sh’ saved [3751/3751]
impish:
impish: + needed_binaries=(lsb_release wget add-apt-repository)
impish: + missing_binaries=()
impish: + for binary in "${needed_binaries[@]}"
impish: + which lsb_release
impish: + for binary in "${needed_binaries[@]}"
impish: + which wget
impish: + for binary in "${needed_binaries[@]}"
impish: + which add-apt-repository
impish: + [[ 0 -gt 0 ]]
impish: + LLVM_VERSION=13
impish: + '[' 1 -eq 1 ']'
impish: + LLVM_VERSION=13
impish: ++ lsb_release -is
impish: + DISTRO=Ubuntu
impish: ++ lsb_release -sr
impish: + VERSION=20.04
impish: + DIST_VERSION=Ubuntu_20.04
impish: + [[ 0 -ne 0 ]]
impish: + declare -A LLVM_VERSION_PATTERNS
impish: + LLVM_VERSION_PATTERNS[9]=-9
impish: + LLVM_VERSION_PATTERNS[10]=-10
impish: + LLVM_VERSION_PATTERNS[11]=-11
impish: + LLVM_VERSION_PATTERNS[12]=-12
impish: + LLVM_VERSION_PATTERNS[13]=-13
impish: + LLVM_VERSION_PATTERNS[14]=
impish: + '[' '!' _ ']'
impish: + LLVM_VERSION_STRING=-13
impish: + case "$DIST_VERSION" in
impish: + REPO_NAME='deb http://apt.llvm.org/focal/ llvm-toolchain-focal-13 main'
impish: + apt-key add -
impish: + wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key
impish: --2022-01-06 16:54:36-- https://apt.llvm.org/llvm-snapshot.gpg.key
impish: Resolving apt.llvm.org (apt.llvm.org)... 199.232.6.49, 2a04:4e42:3e::561
impish: Connecting to apt.llvm.org (apt.llvm.org)|199.232.6.49|:443... connected.
impish: HTTP request sent, awaiting response... Warning: apt-key output should not be parsed (stdout is not a terminal)
impish: 200 OK
impish: Length: 3145 (3.1K) [application/octet-stream]
impish: Saving to: ‘STDOUT’
impish:
impish: 0K ... 100% 221M=0s
impish:
impish: 2022-01-06 16:54:36 (221 MB/s) - written to stdout [3145/3145]
impish:
impish: OK
impish: + add-apt-repository 'deb http://apt.llvm.org/focal/ llvm-toolchain-focal-13 main'
impish: Hit:1 http://us.ports.ubuntu.com/ubuntu-ports focal InRelease
impish: Hit:2 http://us.ports.ubuntu.com/ubuntu-ports focal-updates InRelease
impish: Hit:3 http://us.ports.ubuntu.com/ubuntu-ports focal-backports InRelease
impish: Hit:4 http://us.ports.ubuntu.com/ubuntu-ports focal-security InRelease
impish: Get:5 https://apt.llvm.org/focal llvm-toolchain-focal-13 InRelease [6,821 B]
impish: Get:6 https://apt.llvm.org/focal llvm-toolchain-focal-13/main arm64 Packages [9,891 B]
impish: Fetched 16.7 kB in 2s (10.7 kB/s)
impish: Reading package lists...
impish: + apt-get update
impish: Hit:2 http://us.ports.ubuntu.com/ubuntu-ports focal InRelease
impish: Hit:3 http://us.ports.ubuntu.com/ubuntu-ports focal-updates InRelease
impish: Hit:1 https://apt.llvm.org/focal llvm-toolchain-focal-13 InRelease
impish: Hit:4 http://us.ports.ubuntu.com/ubuntu-ports focal-backports InRelease
impish: Hit:5 http://us.ports.ubuntu.com/ubuntu-ports focal-security InRelease
impish: Reading package lists...
impish: + apt-get install -y clang-13 lldb-13 lld-13 clangd-13
impish: E: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 2548 (unattended-upgr)
impish: E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?
impish: E: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 2548 (unattended-upgr)
impish: E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?
impish: E: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 2548 (unattended-upgr)
impish: E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?
impish: usermod: group 'docker' does not exist
impish: /tmp/vagrant-shell: line 14: bpftool: command not found```
⏚ [id:~] $ go install github.com/solo-io/bumblebee/bee
go install: version is required when current directory is not in a module
Try 'go install github.com/solo-io/bumblebee/bee@latest' to install the latest version
⏚ [id:~] 1 $ go install github.com/solo-io/bumblebee/bee@latest
go install: github.com/solo-io/bumblebee/bee@latest (in github.com/solo-io/[email protected]):
The go.mod file for the module providing named packages contains one or
more replace directives. It must not contain directives that would cause
it to be interpreted differently than if it were the main module.
⏚ θ61° [id:~] 1 $
because the only 1 line override in your deps here (for ebpf
):
https://github.com/solo-io/bumblebee/blob/main/go.mod#L90
This problem I have seen it before... is more concisely put by these fellow developers:
https://hacker-news.news/post/27630625
dear me
N/A
N/A
This passage in the documentation cannot be right. ipv6 as a u32?
https://github.com/solo-io/bumblebee/blob/main/docs/concepts.md
// A basic ipv4 address represented as a u32
typedef u32 ipv4_addr;
// A basic ipv6 address represented as a u32
typedef u32 ipv6_addr;
// A duration in NS stored as a u64
typedef u64 duration;
N/A
ipv4 should be u128
N/A
0.0.14
5.15.0-1022-aws
Hi solo.io, great job!
Example from link https://github.com/solo-io/bumblebee/tree/main/examples/tcpconnlat doesn't work.
ERROR Failed to load OCI image
github.com/containerd/containerd/remotes/docker.(*request).do
/home/runner/go/pkg/mod/github.com/containerd/[email protected]/remotes/docker/resolver.go:551
github.com/containerd/containerd/remotes/docker.(*request).doWithRetries
/home/runner/go/pkg/mod/github.com/containerd/[email protected]/remotes/docker/resolver.go:576
github.com/containerd/containerd/remotes/docker.(*request).doWithRetries
/home/runner/go/pkg/mod/github.com/containerd/[email protected]/remotes/docker/resolver.go:589
github.com/containerd/containerd/remotes/docker.(*dockerResolver).Resolve
/home/runner/go/pkg/mod/github.com/containerd/[email protected]/remotes/docker/resolver.go:280
oras.land/oras-go/pkg/oras.Copy
/home/runner/go/pkg/mod/oras.land/[email protected]/pkg/oras/copy.go:63
github.com/solo-io/bumblebee/pkg/spec.TryFromLocal
/home/runner/work/bumblebee/bumblebee/pkg/spec/utils.go:37
github.com/solo-io/bumblebee/pkg/cli/internal/commands/run.getProgram
/home/runner/work/bumblebee/bumblebee/pkg/cli/internal/commands/run/run.go:180
github.com/solo-io/bumblebee/pkg/cli/internal/commands/run.run
/home/runner/work/bumblebee/bumblebee/pkg/cli/internal/commands/run/run.go:95
github.com/solo-io/bumblebee/pkg/cli/internal/commands/run.Command.func1
/home/runner/work/bumblebee/bumblebee/pkg/cli/internal/commands/run/run.go:76
github.com/spf13/cobra.(*Command).execute
/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:856
github.com/spf13/cobra.(*Command).ExecuteC
/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:974
github.com/spf13/cobra.(*Command).Execute
/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:902
github.com/spf13/cobra.(*Command).ExecuteContext
/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:895
main.main
/home/runner/work/bumblebee/bumblebee/bee/main.go:13
runtime.main
/opt/hostedtoolcache/go/1.17.6/x64/src/runtime/proc.go:255
runtime.goexit
/opt/hostedtoolcache/go/1.17.6/x64/src/runtime/asm_amd64.s:1581
Error: failed to authorize: failed to fetch anonymous token: unexpected status: 401 Unauthorized
2023/01/14 11:48:26 exiting: failed to authorize: failed to fetch anonymous token: unexpected status: 401 Unauthorized
There isn't the tcpconnlat
image in the packages on GitHub: https://github.com/orgs/solo-io/packages
bee run ghcr.io/solo-io/bumblebee/tcpconnlat:$(bee version)
Program is started
No response
Latest version
20.04.1-Ubuntu kernel 5.13.0-1029-Azure
While using go to install bumblebee, I ran the command go install ./bee, which produced the following error:
build github.com/solo-io/bumblebee/bee: cannot load embed: malformed module path "embed": missing dot in first path element
Using go
I expected Bumblebee to install successfully without errors.
No response
Fails during build
Ubuntu 20
I cloned the code from github and ran 'go install ./bee
git clone https://github.com/solo-io/bumblebee.git
sudo go install ./bee
The build failed with the error:
build github.com/solo-io/bumblebee/bee: cannot load embed: malformed module path "embed": missing dot in first path element
issue.txt
git clone https://github.com/solo-io/bumblebee.git
sudo go install ./bee
That I could use the bee command
No response
n/a
Linux 5.4.0-1060-gcp #64~18.04.1-Ubuntu SMP Fri Jan 7 04:02:54 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
setcap
on README doesn't work correctly
$ sudo setcap cap_sys_resource,cap_sys_admin,cap_bpf+eip $(which bee)
fatal error: Invalid argument
usage: setcap [-q] [-v] (-r|-|<caps>) <filename> [ ... (-r|-|<capsN>) <filenameN> ]
Note <filename> must be a regular (non-symlink) file.
need to doc correct way to setcap on a default ubuntu 18.04 LTS (this example from GCP)
No response
0.0.13
5.15.0-46-generic
Working with "tracepoint/syscalls/sys_enter_openat"
and a very simple MAP
struct {
__uint(type, BPF_MAP_TYPE_HASH);
__uint(max_entries, MAX_CONTAINERS_NUM);
__type(key, struct my_struct_t);
__type(value, u64);
} k8s_containers SEC(".maps.counter");
struct my_struct_t{
u64 cgroup;
char my_char[MAX_LEN];
}__attribute__((packed));
If I put a key/value pair into a BPF_MAP_TYPE_HASH with the .map.counter flag, then I can see the corresponding value both from bee cli and Prometheus endpoint.
For a very simple test just add a new key using bpf_map_update_elem with BPF_NOEXIST flag.
(Using BPF_NOEXIST you will get an error if you try to create a key that already exists. To create an item with the same key you need to delete that key. This is usefull if you want to be sure that key has been deleted before).
Using __sync_fetch_and_add I can easily increment the counter for that key as well.
Unfortunately it seems I cannot get rid of that key.
In fact if I use the
bpf_map_delete_elem
primitive the key is still visible into the userspace stack (both bee CLI and Prometheus endpoint).
I'm sure I correctly deleted the key from kernel space program, because if I try to query for that key using bpf_map_lookup_elem I got a null result.
I can add the same key one more time after deletion (still using BPF_NOEXIST). This is a proof of the fat that on the kernel space creation and deletion are working just fine.
It seems the user space program is not reflecting that deletions.
The corresponding key/value shoud disappear
No response
Currently, the following ones are lacking:
list
login Log in so you can push images to the remote server.
pull
push
run Run a BPF program file or OCI image.
tag
cc @lgadban
Note that this is a VM on a Mac M1 running Ubuntu 21.10 (GNU/Linux 5.13.0-23-generic aarch64). I compiled this bee binary on the same vm it's running on.
$ file /usr/local/bin/bee
/usr/local/bin/bee: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, Go BuildID=7d6I66htAooEyvvjafcO/4smkmG1QZgmhRnqbyK1N/AcgWG-x3EfW5StWmnIUX/_YCuHdBV4WDMGPDjTZML, not stripped
sudo bee run ghcr.io/solo-io/bumblebee/tcpconnect:0.0.9
SUCCESS Fetching program from registry: ghcr.io/solo-io/bumblebee/tcpconnect:0.0.9
ERROR Loading BPF program and maps into Kernel
Error: program tcp_v4_connect: load program: invalid argument: trace type programs with run-time allocated hash maps are unsafe. Switch to preallocated hash maps.
R1 type=ctx expected=fp
; int BPF_KPROBE(tcp_v4_connect, struct sock *sk)
0: (85) call unknown#195896080
invalid func unknown#195896080
processed 1 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0
2022/01/07 00:17:28 exiting: program tcp_v4_connect: load program: invalid argument: trace type programs with run-time allocated hash maps are unsafe. Switch to preallocated hash maps.
R1 type=ctx expected=fp
; int BPF_KPROBE(tcp_v4_connect, struct sock *sk)
0: (85) call unknown#195896080
invalid func unknown#195896080
processed 1 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0
This same example runs on amd64 w/o issue.
Hello bumblebee team,
I'm one of the co-chairs of the CNCF TAG-Runtime, I'm reaching out and think it would be great for you to present/discuss the project at one of our meetings. An overview of the project would be great.
Let me know if this something you'd be interested in doing. If yes, please feel free to add it to our agenda or reach out to me (raravena80 at gmail.com)
Thanks!
latest
N/A
Probes will need to be recompiled on different architectures, but the bee
tooling currently is hardcoded for x86
Multi-arch support for the builder tools and the built images
No response
detach and free
Hello,
I am attempting to use the tcpconnect.c
example from the repository and I'm encountering an issue with the creation of the counter_events_ring
ring buffer map. The specific error message I'm encountering is as follows:
Error: map counter_events_ring: map create: invalid argument
2023/08/11 14:47:03 exiting: map counter_events_ring: map create: invalid argument
I have reviewed my code and the map definition, and I believe I have followed the instructions correctly. Here is the relevant map definition from my code:
struct {
__uint(max_entries, 1 << 24);
__uint(type, BPF_MAP_TYPE_RINGBUF);
__type(value, struct dimensions_t);
} counter_events_ring SEC(".maps");
Could you please provide some guidance on what might be causing this "invalid argument" error during map creation? Are there any specific considerations I should be aware of when using ring buffer map?
Thank you for your assistance.
Shay Reuven
v0.0.12
5.15.0-39-generic
When following the Installation description, the latest version of bee is installed, but it always downloads the amd64 version of the binary (see here).
On an ARM64 machine the downloaded binary will not work.
The installation script shall take the outcome of uname -m
into account to decide what binary shall be downloaded.
No response
No response
0.0.12
5.15.x
No response
Much like Sysdig's Falco, it would be useful, for security instrumentation to format the output as json events, such as:
{"time":"2022-06-11T09:53:40.734542-0700","bpf":"opensnoop.o","event":"/proc/123/env"}
Then this output can be piped to fluentd or else.
Additionally, for instrumentation in general, event sampling (at the source) might also be valuable, i.e.:
define rate: -r 0.50 will drop every other log event.
this might be useful if you only want a sample of logs, i.e. if instrumenting TCP payloads, one could only care about the "ratio" of SYN packets over SYN+ACK.
No response
No response
I understand that it's not really possible to run these BPF programs in rootless mode so how do I provide Podman the necessary permissions so that it can build the bumblebee programs
Steps to reproduce:
sudo dnf install podman-docker
bee build examples/tcpconnect/tcpconnect.c tcpconnect
▀ Compiling BPF program (0s)Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
+ CFLAGS=
+ clang-13 -g -O2 -target bpf -D__TARGET_ARCH_x86 -Wall -c examples/tcpconnect/tcpconnect.c -o examples/tcpconnect/tcpconnect.o
clang: error: no such file or directory: 'examples/tcpconnect/tcpconnect.c'
clang: error: no input files
ERROR Failed to compile BPF program
Error: exit status 1
2023/06/21 11:57:32 exiting: exit status 1
OS Details
NAME="Fedora Linux"
VERSION="38 (Workstation Edition)"
ID=fedora
VERSION_ID=38
VERSION_CODENAME=""
PLATFORM_ID="platform:f38"
PRETTY_NAME="Fedora Linux 38 (Workstation Edition)"
ANSI_COLOR="0;38;2;60;110;180"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:38"
DEFAULT_HOSTNAME="fedora"
HOME_URL="https://fedoraproject.org/"
DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/f38/system-administrators-guide/"
SUPPORT_URL="https://ask.fedoraproject.org/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=38
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=38
SUPPORT_END=2024-05-14
VARIANT="Workstation Edition"
VARIANT_ID=workstation
0.0.10
No response
I think there is a slight error in https://github.com/solo-io/bumblebee/blob/c2422b5bab66754b286d062317e244f02a431dac/builder/build.sh. Shoudn't this be:
clang -g -O2 -target bpf -D__TARGET_ARCH_x86 ${CFLAGS} -Wall -c $1 -o $2
$ bee build -b podman -i ghcr.io/solo-io/bumblebee/builder:0.0.10 --local probe.c my_probe:v1
INFO + CFLAGS=
+ clang-13 -g -O2 -target bpf -D__TARGET_ARCH_x86 -Wall -c probe.c -o probe.o
sh: line 6: clang-13: command not found
ERROR Failed to compile BPF program locally
Error: exit status 127
2023/12/23 13:38:19 exiting: exit status 127
Should compile fine
No response
$ go run -exec sudo ./bee/main.go build -l examples/tcpconnect/tcpconnect.c 2-bad-tcpconnect.o
▄ Compiling BPF program locally (0s)Successfully compiled "examples/tcpconnect/tcpconnect.c" and wrote it to "examples/tcpconnect/tcpconnect.o"
SUCCESS Successfully compiled "examples/tcpconnect/tcpconnect.c" and wrote it to "examples/tcpconnect/tcpconnect.o"
SUCCESS Saved BPF OCI image to 2-bad-tcpconnect.o
need to fix or correctly doc how to build locally without docker and storing in image, output flag may be important here
$ go run -exec sudo ./bee/main.go run -f="events_hash,daddr,1.1.1.1" -f="events_ring,daddr,1.1.1.1" ghcr.io/solo-io/bumblebee/tcpconnect:$(./mybee version)
ERROR Failed to load OCI image
github.com/containerd/containerd/remotes/docker.(*dockerResolver).Resolve
/home/.../go/pkg/mod/github.com/containerd/[email protected]/remotes/docker/resolver.go:391
oras.land/oras-go/pkg/oras.Copy
/home/.../go/pkg/mod/oras.land/[email protected]/pkg/oras/copy.go:63
github.com/solo-io/bumblebee/pkg/spec.TryFromLocal
/home/.../bumblebee/pkg/spec/utils.go:37
github.com/solo-io/bumblebee/pkg/cli/internal/commands/run.getProgram
/home/.../bumblebee/pkg/cli/internal/commands/run/run.go:168
github.com/solo-io/bumblebee/pkg/cli/internal/commands/run.run
/home/.../bumblebee/pkg/cli/internal/commands/run/run.go:94
github.com/solo-io/bumblebee/pkg/cli/internal/commands/run.Command.func1
/home/.../bumblebee/pkg/cli/internal/commands/run/run.go:70
github.com/spf13/cobra.(*Command).execute
/home/.../go/pkg/mod/github.com/spf13/[email protected]/command.go:856
github.com/spf13/cobra.(*Command).ExecuteC
/home/.../go/pkg/mod/github.com/spf13/[email protected]/command.go:974
github.com/spf13/cobra.(*Command).Execute
/home/.../go/pkg/mod/github.com/spf13/[email protected]/command.go:902
github.com/spf13/cobra.(*Command).ExecuteContext
/home/.../go/pkg/mod/github.com/spf13/[email protected]/command.go:895
main.main
/home/.../bumblebee/bee/main.go:13
runtime.main
/usr/local/go/src/runtime/proc.go:255
runtime.goexit
/usr/local/go/src/runtime/asm_amd64.s:1581
Error: ghcr.io/solo-io/bumblebee/tcpconnect:dev: not found
2021/12/09 19:03:43 exiting: ghcr.io/solo-io/bumblebee/tcpconnect:dev: not found
exit status 1
0.0.13
5.4
seems there is no opensnoop:0.0.13 image.
sample from https://github.com/solo-io/bumblebee/tree/main/examples/opensnoop can't be executed.
tcpconnect is available
$ bee describe ghcr.io/solo-io/bumblebee/tcpconnect:$(bee version)
┌─ ghcr.io/solo-io/bumblebee/tcpconnect:0.0.13 ─┐
| ┌──────────────────────┐ |
| | No description found | |
| └──────────────────────┘ |
| ┌──────────────────┐ |
| | No Authors found | |
| └──────────────────┘ |
| ┌─ Platform ─┐ |
| | unknown | |
| └────────────┘ |
| |
└───────────────────────────────────────────────┘
$ bee describe ghcr.io/solo-io/bumblebee/opensnoop:$(bee version)
Error: failed to authorize: failed to fetch anonymous token: unexpected status: 403 Forbidden
2022/10/30 13:54:46 exiting: failed to authorize: failed to fetch anonymous token: unexpected status: 403 Forbidden
$ bee describe ghcr.io/solo-io/bumblebee/opensnoop:$(bee version)
┌─ ghcr.io/solo-io/bumblebee/opensnoop:0.0.13 ─┐
| ┌──────────────────────┐ |
| | No description found | |
| └──────────────────────┘ |
| ┌──────────────────┐ |
| | No Authors found | |
| └──────────────────┘ |
| ┌─ Platform ─┐ |
| | unknown | |
| └────────────┘ |
| |
└───────────────────────────────────────────────┘
No response
also make sure vagrant info linked early in readme
Tried joining the Slack using the link given in README: https://soloio.slack.com/ and https://solo-io.slack.com/ (given in GitHub issues) but in my email it shows account not found so I'm not completely sure where the account creation process is supposed to happen for this.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.