nsg-ethz / mini_internet_project Goto Github PK

Dear all,

I am working with a test system for the current Mini-Internet implementation based on a Debian 12.5 system. I am using the config from the 2022 assignment of the communication network course for testing purposes. I have two issues with this setting:

1. When starting the Mini-Internet with startup.sh I get "RTNETLINK answers: File exists" errors in host_links_setup.sh, which I am not able to locate.
2. In layer2_setup.sh I get an error in line 81 "row_l[6]: unbound variable". It seems that layer2_setup.sh is expecting a 7th row (buffer) in the l2_links.txt config which is not present or documented somewhere.

Can you please help me with these problems? Thanks for the wonderful software, I am using an older version of the Mini-Internet in my lectures and it works like a charm!

Best regards
Markus Zeilinger

Hi,

we are running a deployment of the Mini Internet and for a reason I haven't figured out yet all docker containers died 9 hours ago.

Is there a way to recover from here without setting up a new Mini Internet (which resets passwords as well)?

Kind regards
Raphael

Currently, when the mini-internet is built, the ssh key located in /groups is only readable by root, hence is only usable to ssh into the student containers using sudo.

I find it a bit odd and easy to fix. But I wonder: is there a (security?other?) reason behind it?

On the configuration page, the link pointing to the explanation for the config generation does not work.

ℹ️ We automatically generate this file using a script, which we will explain in a following section.

Here, https://github.com/nsg-ethz/mini_internet_project/ASlevel_configuration is missing wiki in its path. The correct link is probably: https://github.com/nsg-ethz/mini_internet_project/wiki/ASlevel_configuration.

I hoping reporting this here is ok. (I did not see an option to open a PR for the wiki.)

Hello there!

I'm packing up for this new year. I just git pull on the last year folder and start with the new config.

But it seems that the docker images are not updated at startup. I had to remove them all and then relaunch the startup script to get them updated.

I had hard time with matrix and website tshoot. Now everything look good.

Before:

root@totoro:/home/totoro/mini_internet_project/platform/config_2023# docker images -a
REPOSITORY                      TAG       IMAGE ID       CREATED         SIZE
miniinterneteth/d_router        latest    ea9cc6b00413   11 months ago   166MB
traefik                         v2.6      22c6901de2be   15 months ago   102MB
miniinterneteth/d_ssh           latest    2eb541a6d6be   15 months ago   87.7MB
miniinterneteth/d_webserver     latest    b6e35504214f   17 months ago   245MB
miniinterneteth/d_routinator    latest    5ed07c7e7010   17 months ago   150MB
miniinterneteth/d_matrix        latest    7ca9d2c08308   17 months ago   124MB
miniinterneteth/d_host          latest    c68d046354c9   17 months ago   83MB
miniinterneteth/d_krill         latest    068f3e926daa   17 months ago   236MB
miniinterneteth/d_switch        latest    2c1da238440d   18 months ago   143MB
miniinterneteth/d_dns           latest    b9ac82d14c7b   18 months ago   101MB
miniinterneteth/d_measurement   latest    c017f24cba95   18 months ago   346MB
miniinterneteth/d_ixp           latest    55e7e7232eff   18 months ago   276MB

After:

root@totoro:/home/totoro/mini_internet_project/platform/config_2023# docker image ls
REPOSITORY                     TAG       IMAGE ID       CREATED         SIZE
miniinterneteth/d_webserver     latest    47113e0cb1a7   4 months ago    283MB
miniinterneteth/d_matrix        latest    5b0ccc2fb29d   5 months ago    122MB
miniinterneteth/d_ssh           latest    df7011f96dbd   6 months ago    87.8MB
miniinterneteth/d_router        latest    ea9cc6b00413   11 months ago   166MB
traefik                         v2.6      22c6901de2be   15 months ago   102MB
miniinterneteth/d_routinator    latest    5ed07c7e7010   17 months ago   150MB
miniinterneteth/d_host          latest    c68d046354c9   17 months ago   83MB
miniinterneteth/d_krill         latest    068f3e926daa   17 months ago   236MB
miniinterneteth/d_switch        latest    2c1da238440d   18 months ago   143MB
miniinterneteth/d_dns           latest    b9ac82d14c7b   18 months ago   101MB
miniinterneteth/d_measurement   latest    c017f24cba95   18 months ago   346MB
miniinterneteth/d_ixp           latest    55e7e7232eff   18 months ago   276MB

Maybe i'm doing things wrong! I'm not familiar with dockerish stuff.

Best regards,

Hello,

First of all, thank you for this incredible platform! I'm using it with student, they like it!

I used old version (2019 or 2020) and I just updated it to the last one to have the website, rpki ...
I had an issue with config files that cause me headache.

It seems that the Krill host should be put in first in the l3_routers config file. Otherwise the key is not created into the group folder.

This works:

GENE    DNS krill:miniinterneteth/d_krill       vtysh
LOND    MATRIX	routinator:miniinterneteth/d_routinator	vtysh
ZURI    MATRIX_TARGET host:miniinterneteth/d_host	vtysh

This does not work (for me):

LOND    MATRIX	routinator:miniinterneteth/d_routinator	vtysh
ZURI    MATRIX_TARGET host:miniinterneteth/d_host	vtysh
GENE    DNS krill:miniinterneteth/d_krill       vtysh

I get
./setup/container_setup.sh: line 180: /home/xxxx/groups/g1/krill/kill_token.txt: No such file or directory
and, juste before, during the rpki_config.sh script, it creates only the groups/rpki/root.key

That's quite strange. It didn't get through the code since it's way above my knowledge.

I hope I can help to make this platform even better with my little contribution :)

Have a nice day

Hi, I'm using the mini internet right now and found that the link delay is not working anymore.
It seems to not work anymore because tc-netem needs the unit ms for the command to succeed.
At the moment the command is run like that tc qdisc add dev "${PORTNAME}"_l root netem delay "${DELAY}" which doesn't include the unit ms.

3 possible ways to fix it

1. Change all the config files where a delay is specified to include the unit ms. So if the config contained 100 before it should now be 100ms.
2. Change the code to add the unit automatically. This would be in the file mini_internet_project/platform/setup/ovs-docker.sh line 193 and 194. It would need to be changed from:

if [ -n "$DELAY" ]; then
         echo "tc qdisc add dev "${PORTNAME}"_l root netem delay "${DELAY}" " >> groups/delay_throughput.sh
         echo "  tc qdisc add dev "${PORTNAME}"_l root netem delay "${DELAY}" " >> groups/restart_container.sh
fi

to:

if [ -n "$DELAY" ]; then
         echo "tc qdisc add dev "${PORTNAME}"_l root netem delay "${DELAY}ms" " >> groups/delay_throughput.sh
         echo "  tc qdisc add dev "${PORTNAME}"_l root netem delay "${DELAY}ms" " >> groups/restart_container.sh
fi

3. The third option would be to add the ms to the config file but have a test in the mini_internet_project/platform/setup/ovs-docker.sh file that will add the "ms" if it is missing to keep it compatible with previous configs.

If you decide on one of the options that change the config files then the wiki entries also need to be updated to reflect the changes.

On another note thanks a lot for providing access to the mini internet. It is a really amazing tool.

In the branch commnet-23 the script platform/utils/gif/history_config.sh uses a hard coded username to perform the git commit & push. The variable for the username ($USERNAME) is available, but not used here.

Hi,

is there are some kind of integrity check to verify that the network is setup correctly without going through the whole exercise to test everything?

Kind regards,
Raphael

I am using a 40 AS Mini-Internet here, four regions with 10 ASes each, 2 tier-1, 2 stub (fully configured) and 6 tier-2 ASes (managed by the students). So a "classic" setup I would suppose. Find all config files attached:

40_as_config.zip

Currently the whole intra-domain stuff is done, eBGP session are configured and running, business relationships and IXPs are setup too. The connection matrix shows full connectivity, some paths are still invalid due to route leaks based on mishandled business relationships. RPKI stuff is not done at the moment.

Now I ran into some serious troubles. I observe heavy and rising load on the virtual machine (VM) running the Mini-Internet. The VM has 16 CPU cores, all are up between 95 and 100 % load, load average is between 55 and 70. Memory consumption is at around 44 GB of 64 GB in total. Here's the current output of htop on this VM:

Some deeper analysis shows that a big part of this heavy load seems to originate from the routinator processes in the 40 ASes (look at the TIME column in the ps output):

The ones with the most CPU time are the ones in the fully configured tier-1 and stub ASes. Looking at one of the affected containers (group 12, tier-1, routinator running on the host at router GRZ) shows the following for ps:

Using strace on one of the routinator process on the VM shows that, if I am right, the routinator process is spawning a lot of new processes "doing things". I attach a file with strace output here:

g18_grz_host_routinator_trace.txt

Any ideas what is going wrong here?

Thanks for your help in advance!
Markus

Dear all,

I am using your Mini Internet Software for a lecture on Advanced Routing. It is just great, thanks for your efforts!

I currently have some troubles with the Connectivity Matrix feature. I start a Mini Internet with 24 ASes all fully configured ("Config" in C3 in "AS_config.txt"). The MATRIX container is working (unpause) and collecting data (I can see the files when connecting directly to the container). In the web frontend I get the message "Sorry, no data is available! Please check in later." when accessing the Connectivity Matrix feature (this originates from the Flask [?] template for this page). When looking at the webserver logs ("docker logs web") I get the following error all the time:

Traceback (most recent call last):
File "/server/routing_project_server/app.py", line 280, in loop
raise RuntimeError(
RuntimeError: Worker prepare_matrix crashed! Restarting.
Traceback (most recent call last):
File "/server/routing_project_server/app.py", line 277, in loop
function(*args, **kwargs)
File "/server/routing_project_server/app.py", line 320, in prepare_matrix
config['LOCATIONS']['matrix_stats']
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
KeyError: 'matrix_stats'

AFAIK "matrix_stats" refers to a stats.txt file, which does exist on the MATRIX container but nowhere else. Any help would be much appreciated!

Hello,

We decided to run the mini-internet on a virtual machine with a small amount of storage (~30GB). Everything was going perfectly well until the virtual machine crashed. Upon investigation, I discovered that the file system was saturated with logs generated by the various services running the mini-internet:

• The mini_internet_project/platform/docker_images/base_supervisor/logger.sh is used to gather all logs to the docker daemon. However, docker is not configured by default to limit the size of the logs and rotate them when they exceed the maximum size.
• The supervisord daemon also generates many logs for the daemons it supervises. By default, it rotates the logs when the log file size exceeds 50 MB. Logs for router daemons tend to be a bit verbose. The frr and looking_glass processes can generate more than 50 MB after 2 weeks. If the mini-internet is configured to run a large network of 70ASes, each with 8 routers like the configuration given on the wiki, a large amount of storage is required.

It would be nice to configure the docker daemon to limit the logs recorded with logger.sh. This can be done either in the startup.sh script or indicated in the wiki that docker should be manually configured to limit logs.

It could be also a good idea to limit the size of logs kept and the number of files rotated per container.

If keeping these logs is intentional, it would be worth to indicate in the wiki that a large amount of storage is required to run the mini-internet.