stevelorenz / comnetsemu Goto Github PK

A virtual emulator/testbed designed for the book: Computing in Communication Networks: From Theory to Practice (2020)

Home Page: https://stevelorenz.github.io/comnetsemu/

License: MIT License

Makefile 0.25% Python 48.97% Shell 3.53% Dockerfile 0.41% HTML 34.37% CSS 0.97% Jupyter Notebook 9.91% JavaScript 1.41% Pug 0.05% Roff 0.14%

sdn nfv emulation computing-in-network teaching research-project mininet container

comnetsemu's Introduction

ComNetsEmu

A virtual emulator/testbed designed for the book: Computing in Communication Networks: From Theory to Practice

This project has a online Github page for Python API documentation and other useful documentation. Please check it when you develop or use ComNetsEmu's Python APIs.

INFO: This project is currently still under development [beta]. Version 1.0.0 has not yet been released. We try to make it stable but breaking changes might happen.

The repository is hosted both on the internal Gitlab server of ComNets TUD and Github. The GitLab ComNets TUD is read-only for public users. For all public users, if you have a question or want to contribute, please create an issue or send a pull request on Github.

Description
- Main Features
Installation
Upgrade ComNetsEmu and Dependencies
Run the Docker-in-Docker example
Project Structure
Development Guide and API Documentation
FAQ
Contributing
Contact

Description

ComNetsEmu is a testbed and network emulator designed for the NFV/SDN teaching book "Computing in Communication Networks: From Theory to Practice". The design focuses on emulating all examples and applications on a single computer, for example on a laptop. ComNetsEmu extends the famous Mininet network emulator to support better emulation of versatile Computing In The Network (COIN) applications. It extends and puts forward the concepts and work in the Containernet project. It uses a slightly different approach to extend the Mininet compared to Containernet. It's main focus is to use "sibling containers" to emulate network systems with computing. See a more detailed comparison between upstream Mininet and Containernet here.

Common facts about ComNetsEmu:

Emulation accuracy is highly considered, but it but can not be guaranteed for arbitrary topology. All emulated nodes (processes) share the same underlying compute, storage and network resources when running it on a single system. ComNetsEmu is heavier than vanilla Mininet due to stricter node isolation. Choosing a reasonable emulation parameters is required for correct simulation results. RT-Tests is installed on the test VM. RT-Tests can be used to evaluate the real-time performance of the current emulation system.
ComNetsEmu is mainly developed with Python3.8. To reduce the complexity of dependencies (third-party packages, frameworks etc.), ComNetsEmu tries to leverage as much of the powerful Python standard library as possible, and prefers simple third-party dependencies when necessary.
Examples and applications in this repository are mainly developed with high-level script language for simplicity. These programs are not performance optimized. Please contact us if you want highly optimized implementation of the concepts introduced in this book. For example, we had a DPDK-accelerated version of the low-latency (sub-millisecond) Random Linear Network Coding (RLNC) network function.

Main Features

Use Docker hosts in Mininet topologies.
Manage application Docker containers deployed inside Docker hosts. "Docker-in-Docker" (sibling containers) is used as a lightweight emulation of nested virtualization. A Docker host with multiple internal Docker containers deployed is used to mimic an actual physical host running Docker containers (application containers).
A collection of application examples for "Computing In Communication Networks" with sample codes and detailed documentation. All examples can be easily reproduced and extended.

Check the Roadmap for planed and WIP features.

Installation

Currently, only the latest Ubuntu 20.04 LTS is supported. Supporting multiple Linux distributions and versions is not the goal of this project.

It is highly recommended to run ComNetsEmu inside a virtual machine (VM). Root privileges are required to run the ComNetsEmu/Mininet applications. ComNetsEmu also uses privileged Docker containers by default. It's also safer to play it inside a VM. ComNetsEmu's installation script is a wrapper of an Ansible playbook. This playbook uses Mininet's install script to install Mininet natively from source. As described in Mininet's doc, the install script is a bit intrusive and may possible damage your OS and/or home directory. ComNetsEmu runs smoothly in a VM with 2 vCPUs and 2GB RAM. (Host Physical CPU: Intel i7-7600U @ 2.80GHz). Some more complex applications require more resources. For example, the YOLO object detection application requires a minimum of 5GB of memory.

The recommended and easiest way to install ComNetsEmu is to use Vagrant and Virtualbox. Assuming that the directory where ComNetsEmu is stored is "~/comnetsemu" in your home directory, just run the following commands to get a fully configured VM using vagrant with Virtualbox provider:

$ cd ~
$ git clone https://git.comnets.net/public-repo/comnetsemu.git
$ cd ./comnetsemu
$ vagrant up comnetsemu
# Take a coffee and wait about 15 minutes

# SSH into the VM when it's up and ready (The ComNetsEmu banner is printed on the screen)
$ vagrant ssh comnetsemu

Mainly due to performance and special feature requirements, some examples and applications can only run on virtual machines with KVM as the hypervisor. The built-in Vagrantfile provided by ComNetsEmu supports libvirt provider. Please check the detailed documentation of Option 1 here if you want to use the libvirt provider for Vagrant.

Congratulations! The installation is done successfully! You can now run the tests, examples, and skip the rest of the documentation in this section. Continue reading only if you are interested in the details of the installation or want other installation options.

For users running Windows as the host OS:

Warning: Main developers of ComNetsEmu does not use Windows and does not have a Windows machine to test on.

If you are using Windows, we recommend using Mobaxterm as the console. This should solve problems opening xterm in the emulator.

ComNetsEmu's installer will try to install the dependencies using a package manager (apt, pip, etc.) if the desired version is available. Unavailable dependencies (e.g. the latest Mininet) and dependencies that require patching are installed directly from source code. By default, the dependency source codes are downloaded into "~/comnetsemu_dependencies". You can modify the Ansible playbook based on your needs.

Please see the detailed installation guide here for additional installation options.

Upgrade ComNetsEmu and Dependencies

ComNetsEmu's installer can only upgrade when ComNetsEmu's underlying Linux distribution is not changed/upgraded. For example, you can use this upgrade function when Ubuntu 20.04 LTS is used as the base VM. When the base VM is upgraded to the next LTS version, the upgrade function is not guaranteed to work since many packages are upgraded. Therefore, it's suggested to vagrant destroy and vagrant up again when a new Ubuntu LTS is used as the base VM. Thanks to Vagrant and Docker packaging, it should be not too difficult to re-create the environment after rebuild the VM.

Example screenshots for running the upgrade process in terminal:

The master branch contains stable/tested sources for ComNetsEmu's Python package, utility scripts, examples and applications. It is recommended to upgraded to latest published tag of the master branch.

The installer script has a function to upgrade ComNetsEmu automatically. And the installer script also needs to be updated firstly. Therefore, it takes three steps to upgrade everything. It is assumed here the ComNetsEmu is installed using option 1 with Vagrant.

Step 1: Upgrade source code of ComNetsEmu Python package, examples and applications

Use git to pull (or fetch+merge) the latest tag (or commit) in master branch:

$ cd ~/comnetsemu
$ git checkout master
$ git pull origin master

Step 2: Automatically upgrade ComNetsEmu Python modules and all dependencies

The installer script is used to perform this step. Run following commands inside the VM to upgrade automatically:

$ cd ~/comnetsemu/util
$ bash ./install.sh -u

The script may ask you to input yes or no several times, please read the terminal output for information.

Step 3: Check if the upgrade is successful

Run following commands inside the VM to run tests:

$ cd ~/comnetsemu/
$ sudo make test && sudo make test-examples

If all tests pass without any errors or exceptions, the upgrading was successful. Otherwise, it is recommended to redo the upgrade process or just rebuild the Vagrant VM if the situation is too bad...

Run the Docker-in-Docker example

$ cd $TOP_DIR/comnetsemu/examples/
$ sudo python3 ./dockerindocker.py

See the README to get information about all built-in examples.

Project Structure

To keep the VMs small, Vagrantfile and test_containers contain only minimal dependencies to start the VMs and be able to run all the built-in examples. Dependencies of specific applications (e.g. Python packages like numpy, scipy etc.) should be installed by the script or instructions provided in the corresponded application folder. Therefore, the user need to install them only if she or he wants to run that application.

app: All application programs are classified in this directory. Each subdirectory contains a brief introduction, source codes, Dockerfiles for internal containers and utility scripts of the application
bin: Commands and binaries provided by ComNetsEmu
comnetsemu: Source codes of ComNetsEmu's Python packages
doc: Markdown files and sources to generate ComNetsEmu Sphinx documentation
examples: Example programs for functionalities of the ComNetsEmu emulator
patch: Patches for external dependencies that are installed from source code via installer
test_containers: Contains Dockerfiles for essential Docker images for tests and built-in examples
utils: Utility and helper scripts
Vagrantfile: Vagrant file to setup development/experiment VM environment

Development Guide and API Documentation

Please check the online documentation page.

FAQ

Check faq

Contributing

This project exists thanks to all people who contribute. List of known contributors.

For all public users, please create issues or send pull requests on Github.

Contact

Project main maintainers:

Zuo Xiang: [email protected] (office), [email protected] (personal)

comnetsemu's People

Contributors

Stargazers

Watchers

comnetsemu's Issues

Use Sysbox runc to run software such as systemd, Docker, Kubernetes, legacy apps in DockerHost

Recently, I have found a project (a container runtime) called Sysbox which can run "VM workloads" inside a Docker container. So the sys container managed by the Sysbox can run e.g. systemd, Kubernetes inside a Docker container. This startup is recently acquired by the Docker company. I would like to check this runc and use it as the runtime to run outside Docker containers in the comnetsemu. This can replace the current very limited and insecure sibling container mechanism used by comnetsemu. With this new runtime, it is possible to use comnetsemu to emulate a network of e.g. multiple Kubernetes clusters without using e.g. nested VM virtualization.

Installation is broken due to new GitHub security policies

Due to new GitHub security policies (see https://github.blog/2021-09-01-improving-git-protocol-security-github) the installation is broken and most of the tools/resources/components (e.g Docker) are not installed due to error(s)

Hope it will be fixed as soon as possible.

Flowvisor bash image can not be called from both DockerFile

Hello,

I was trying to simulate the multi-tenant-sdn-slicing and I am having problem with runing the flowvisor. When I try to build the FlowVisor image: vagrant@comnetsemu:~/comnetsemu/SVMN_project/flowvisor $ ./build_flowvisor_image.sh
It is giving error for both Dockerfile.flowvisor and Dockerfile.flowvisor.2 that is recently modified.
For Dockerfile.flowvisor.2 is giving this error:
curl: (6) Couldn't resolve host 'updates.onlab.us'
The command '/bin/sh -c curl -LO http://updates.onlab.us/GPG-KEY-ONLAB' returned a non-zero code: 6

For Dockerfile.flowvisor is giving this error:
Error: Cannot retrieve repository metadata (repomd.xml) for repository: base. Please verify its path and try again
The command '/bin/sh -c yum update -y && yum install wget sudo nano -y' returned a non-zero code:

Thank you for your understanding.

Docker images

Hi
I'm very interested in comnetsemu and it's fantastic features. But I have a problem in building docker images for chapter 26 of the book. It's very amazing if i can have built image from somewhere. Is it possible for you to push the images in a public repository?

name missmatch in service_migration example

I'm trying to run the service_migration example located at comnetsemu/examples/service_migration/ but there is something weird with it.

The README.md states to run the following two commands:

sudo ./build_docker_images.sh
sudo python3./topology.py

The first sh is just a docker build command that tags the output image as "service_migration".
The second command should run the topology and perform tests, but it fails.

Reading the output, the failure reason is:

"pull access denied for dev_test, repository does not exist or may require 'docker login': denied: requested access to the resource is denied"

Investigating through the topology.py, I noticed that it is trying to run a docker image named "dev_test", but it obviously does not exist on my system.

I tried to change "dev_test" into "service_migration" in all of its occurrencies.
In this case, no error is shown as the service_migration image is present on my sistem after running the build_docker_images.sh. The problem now is that it stucks after the first container run and it does not go on.

I tried to run manually an interactive shell on build_docker_images container with the command:

docker run -it service_migration /bin/sh

Then I manually run the command present in the Dockerfile:

python /home/server.py

and I got the following error:

usage: server.py [-h] [--get_state] hostname
server.py: error: the following arguments are required: hostname

I finally tried to pass as argument an IP address or an hostname, but it always fail with the following error:

Traceback (most recent call last):
  File "/home/server.py", line 85, in <module>
    run(args.hostname, args.get_state)
  File "/home/server.py", line 60, in run
    sock.bind((SERVICE_IP, SERVICE_PORT))
OSError: [Errno 99] Address not available

Any suggestion?

SSH Agent issue, and install.sh has windows line endings

Issue 1:
Was attempting to install using Vagrant on Windows with VirtualBox as the provider, kept receiving an error about SSH agent not working. I ended up having to run:
vagrant halt
vagrant destroy -f
vagrant up comnetsemu

When none of those options worked, I ended up having to open VirtualBox, and then running vagrant up commnetsemu, and the SSH error went away and I was able to vagrant ssh comnetsemu

Issue 2:
when running install.sh from ~/comnetsemu/util/
I kept getting an error:
./install.sh: line 9: $'\r'

Basically, the install.sh script does not have unix endings.
Had to run: sed $'s/\r$//' ./install.sh > ./install.Unix.sh
then I ran: bash ./install.Unix.sh -u
everything worked...I guess I'm asking to please be careful editing bash files in windows! :)

Can comnetsemu works with vagrant+hyper-v in Win10?

I check the vagrantfile and dont find the support for Hyper-v.

VirtualBox still have problem to work with Hyper-v,
<ExtraDataItem name="VBoxInternal/NEM/UseRing0Runloop" value="0"/>
this setting also doesn't work with my PC( Win10 2004 21H1).

it would be better if comnetsemu Hyper-v support.

vagrant up ssh is slow

Launching comnetsemu using vagrant up takes a lot of time due to comnetsemu: SSH auth method: private key

mininet-ryu

hello
is need to install mininet and Ryu
when I run sudo python3 network.py
File "network.py", line 3, in
from mininet.topo import Topo
ModuleNotFoundError: No module named 'mininet'
as well as when I run flowvisor shows error

[centos@centos7 flowvisor]$ sudo ./run_flowvisor_container.sh
[root@centos7 ~]# cd slicing_scripts
[root@centos7 slicing_scripts]# ./1st-flowvisor_slicing.sh
bash: ./1st-flowvisor_slicing.sh: No such file or directory
[root@centos7 slicing_scripts]#

and can I use flowvisor on ubuntu and the same structure?
please help me
thank you

Containernet.addDockerHost hang at buffered read

I'm creating a Docker host with the following code (a small excerpt from the whole topology):

net = Containernet(
    controller=Controller, ipBase="10.0.0.0/8", link=TCLink
)
net.addController("c0")

switch_core = net.addSwitch("s1", ip="10.80.95.0/24")

epc = net.addDockerHost(
    "srsepc",
    dcmd="srsepc --mme.mme_bind_addr=10.80.95.10 --spgw.gtpu_bind_addr=10.80.95.10",
    dimage="srsran",
    docker_args={
        "devices": ["/dev/net/tun"],
        "cap_add": ["SYS_NICE", "NET_ADMIN"]
    },
)
net.addLink(switch_core, epc, intf=TCIntf, ip="10.80.95.10", bw=1000, delay="1ms")

The code hangs indefinitely at addDockerHost, when sending a SIGINT the stacktrace is as follows:

Click to expand (stacktrace)

Traceback (most recent call last):
  File "src/4g.py", line 170, in <module>
    net = Simple4G(daemon=False)
  File "src/4g.py", line 73, in __init__
    self._net.addDockerHost(
  File "/home/vagrant/comnetsemu/comnetsemu/net.py", line 60, in addDockerHost
    return self.addHost(name, cls=DockerHost, **params)
  File "/usr/local/lib/python3.8/dist-packages/mininet/net.py", line 232, in addHost
    h = cls( name, **defaults )
  File "/home/vagrant/comnetsemu/comnetsemu/node.py", line 123, in __init__
    super(DockerHost, self).__init__(name, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/mininet/node.py", line 123, in __init__
    self.startShell()
  File "/home/vagrant/comnetsemu/comnetsemu/node.py", line 174, in startShell
    data = self.read(1024)
  File "/usr/local/lib/python3.8/dist-packages/mininet/node.py", line 249, in read
    data = os.read( self.stdout.fileno(), size - count )
KeyboardInterrupt

Using setLogLevel("debug"), the logs stop with these last lines:

Command:  srsepc --mme.mme_bind_addr=10.80.95.10 --spgw.gtpu_bind_addr=10.80.95.10
Created docker container object srsepc
image: srsran
Before creating the container
Docker container srsepc started. ID:c7415be20351fed52820e079a99c3ad49e86278d7c00944c83fa4e861749951a
*** errRun: ['which', 'mnexec'] 
/usr/bin/mnexec
  0*** errRun: ['which', 'ifconfig'] 
/usr/sbin/ifconfig
  0Insert interactive shell bin and argsDocker host master:12, slave:13
_popen ['docker', 'exec', '-it', 'srsepc', 'env', 'PS1=\x7f', 'bash', '--norc', '-is', 'mininet:srsepc'] 42829 
^^^ (hangs here) ^^^

Click to expand (full debug logs)

*** errRun: ['grep', '-c', 'processor', '/proc/cpuinfo'] 
4
  0*** Setting resource limits
*** errRun: ['which', 'mnexec'] 
/usr/bin/mnexec
  0*** errRun: ['which', 'ifconfig'] 
/usr/sbin/ifconfig
  0_popen ['mnexec', '-cd', 'env', 'PS1=\x7f', 'bash', '--norc', '--noediting', '-is', 'mininet:c0'] 42725*** c0 : ('unset HISTFILE; stty -echo; set +m',)
unset HISTFILE; stty -echo; set +m
*** errRun: ['which', 'telnet'] 
/usr/bin/telnet
  0*** c0 : ('echo A | telnet -e A 127.0.0.1 6653',)
Telnet escape character is 'A'.
Trying 127.0.0.1...
telnet: Unable to connect to remote host: Connection refused
*** errRun: ['which', 'ovs-vsctl'] 
/usr/bin/ovs-vsctl
  0*** errRun: ['ovs-vsctl', '-t', '1', 'show'] 
37247a23-d0d6-4a19-ad09-b8e880b10579
    ovs_version: "2.13.3"
  0*** errRun: ['ovs-vsctl', '--version'] 
ovs-vsctl (Open vSwitch) 2.13.3
DB Schema 8.2.0
  0_popen ['mnexec', '-cd', 'env', 'PS1=\x7f', 'bash', '--norc', '--noediting', '-is', 'mininet:s1'] 42733*** s1 : ('unset HISTFILE; stty -echo; set +m',)
unset HISTFILE; stty -echo; set +m

added intf lo (0) to node s1
*** s1 : ('ifconfig', 'lo', 'up')
_popen ['mnexec', '-cd', 'env', 'PS1=\x7f', 'bash', '--norc', '--noediting', '-is', 'mininet:s2'] 42736*** s2 : ('unset HISTFILE; stty -echo; set +m',)
unset HISTFILE; stty -echo; set +m

added intf lo (0) to node s2
*** s2 : ('ifconfig', 'lo', 'up')

Command:  srsepc --mme.mme_bind_addr=10.80.95.10 --spgw.gtpu_bind_addr=10.80.95.10
Created docker container object srsepc
image: srsran
Before creating the container
Docker container srsepc started. ID:c7415be20351fed52820e079a99c3ad49e86278d7c00944c83fa4e861749951a
*** errRun: ['which', 'mnexec'] 
/usr/bin/mnexec
  0*** errRun: ['which', 'ifconfig'] 
/usr/sbin/ifconfig
  0Insert interactive shell bin and argsDocker host master:12, slave:13
_popen ['docker', 'exec', '-it', 'srsepc', 'env', 'PS1=\x7f', 'bash', '--norc', '-is', 'mininet:srsepc'] 42829

I tried replicating the topology in docker-compose and everything works (only relevant lines):

services:
  srsepc:
    container_name: srsepc
    image: srsran
    networks:
      core:
        ipv4_address: 10.80.95.10
    cap_add:
      - NET_ADMIN
      - SYS_NICE
    devices:
      - /dev/net/tun
    command:
      - srsepc
      - --mme.mme_bind_addr=10.80.95.10
      - --spgw.gtpu_bind_addr=10.80.95.10

And running the single Docker CLI command in the same way comnetsemu does still works fine, with docker exec not hanging and successfully attaching to the running container.

$ docker run --privileged -itd \
    --network bridge \
    --name srsepc -v $(pwd)/config:/etc/srsran:ro \
    --device /dev/net/tun \
    srsran \
    srsepc --mme.mme_bind_addr=127.0.0.1 --spgw.gtpu_bind_addr=127.0.0.1

$ docker exec -it srsepc bash --norc -is mininet:srsepc

Notes:

I'm running everything inside the stock Vagrant VM with just the addition of my files and docker-compose v2 installed as standalone binary (not through apt)
I cannot test other containers in the topology because they depend on this first one