containernetworking / cni Goto Github PK

Config and result JSONs should carry version info. Will need to spell out a strategy on how to also version plugin specific portions of the configs.

As long as port forwarding is not dealt within CNI there's a lot of work left for the runtime alone.
CNI could make this easier by returning the names/indexes of the created interfaces.

It may be useful to return more/other information than interface names, so an implementation could just add a generic dictionary to the result and let the content depend on the plugins.

I create a netcat server that looks up hostname, and a netcat client. I kept the client up, and deleted/recreated the server. It kept coming up with the same ip. However, on the host:

root@e2e-test-beeps-minion-jzy6:/home/beeps# arp -a
? (10.245.1.5) at 02:e6:f9:ed:a0:8b [ether] on cbr0
? (10.245.1.3) at 1a:20:f0:ed:ec:cd [ether] on cbr0
? (10.245.1.4) at de:30:22:0a:5c:24 [ether] on cbr0
? (10.240.0.1) at 42:01:0a:f0:00:01 [ether] on eth0

---[duration T1]---

? (10.245.1.5) at <incomplete> on cbr0
? (10.245.1.3) at 1a:20:f0:ed:ec:cd [ether] on cbr0
? (10.245.1.4) at de:30:22:0a:5c:24 [ether] on cbr0
? (10.240.0.1) at 42:01:0a:f0:00:01 [ether] on eth0

if during duration T1 I send some udp packets to the server through nc -u, tcpdump on the eth0 of the server (inside the container), shows:

Client:

# nc -u 10.245.1.5 8081
hostName
hostName
hostName

--- 3 failures

hostName
netserver-4

Server:

15:16:17-beeps~/goproj/src/k8s.io/kubernetes] (kubelet_plugins)$ ./repro.sh 
pod "netserver-4" deleted
pod "netserver-4" created
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
23:16:34.978072 IP (tos 0x0, ttl 62, id 42388, offset 0, flags [DF], proto UDP (17), length 37)
    10.245.2.6.35824 > 10.245.1.5.8081: [udp sum ok] UDP, length 9
23:16:41.778665 IP (tos 0x0, ttl 62, id 42414, offset 0, flags [DF], proto UDP (17), length 37)
    10.245.2.6.35824 > 10.245.1.5.8081: [udp sum ok] UDP, length 9
23:16:46.928938 IP (tos 0x0, ttl 62, id 43415, offset 0, flags [DF], proto UDP (17), length 37)
    10.245.2.6.35824 > 10.245.1.5.8081: [udp sum ok] UDP, length 9


-----3 failures




23:18:03.738968 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.245.1.5 tell 10.245.1.1, length 28
23:18:04.738960 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.245.1.5 tell 10.245.1.1, length 28
23:18:05.738956 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.245.1.5 tell 10.245.1.1, length 28
23:18:07.873561 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.245.1.5 tell 10.245.1.1, length 28
23:18:07.873586 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.245.1.5 is-at 02:1a:0e:6b:ba:36, length 28

------now another one 

23:18:07.873600 IP (tos 0x0, ttl 62, id 53565, offset 0, flags [DF], proto UDP (17), length 37)
    10.245.2.6.35824 > 10.245.1.5.8081: [udp sum ok] UDP, length 9
23:18:07.873811 IP (tos 0x0, ttl 64, id 6569, offset 0, flags [DF], proto UDP (17), length 39)
    10.245.1.5.8081 > 10.245.2.6.35824: [bad udp cksum 0x1919 -> 0xd673!] UDP, length 11
23:18:12.874950 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.245.1.1 tell 10.245.1.5, length 28
23:18:12.874980 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.245.1.1 is-at 52:17:41:2a:8b:a4, length 28

This is with v0.1.0, @eyakubovich thoughts?

Edit: net.conf

{
  "bridge":"cbr0",
  "ipam":{
    "routes":[{"dst":"0.0.0.0/0"}],
    "subnet":"10.245.1.0/24",
    "type":"host-local"
  },
  "isGateway":true,
  "name":"kubenet",
  "type":"bridge"
}

We could reorganize the current implementation which could be done right now without losing compatibility with existing configurations.

• ipam
  • host-local.go (implements combination of disk and sequential)
  • store
    • store.go (package, interface)
    • disk/...
  • allocator
    • allocator.go (package, interface)
    • sequential/...

However, the next step could be to configure the store and allocator in the configuration file, so that any combination could be chosen. We can probably still allow predefined combinations for known types as host-local.

/cc @aanm

On Del command, bridge plugin does not remove the installed IP masquerade rules.

I have a bash prototype plugin with two functions activate_network (for when the container starts) and deactivate_network for when then container exits. My environment is as follows:-

rkt version 0.8.0
appc version 0.6.1

and I have the following block at the end of the plugin:-

if [[ "${CNI_COMMAND}" == "ADD" ]]; then
        activate_network
elif [[ "${CNI_COMMAND}" == "DEL" ]]; then
        deactivate_network
fi

When the container starts, $CNI_COMMAND is equal to ADD so networking is setup fine. However on exit, deactivate_network is never run. Am I missing something?

The error message from the IPAM plugin search is confusing particularly if the IPAM entry is missing from the config but required.

We should add functional for all combinations of plugins and their configuration.
rkt has partial functional tests for networking, some of which probably fit better in here.

This issue is for overview and tracking purposes.

The spec is currently being drafted here. It'll be moved into an .md file in this repository shorly.

https://docs.google.com/document/d/1pGJiQhA_5lZsCeswsKDm5v_brkeqaT1Bmn4zANCynGg/edit?usp=sharing

CNI should have something to say about how to limit cross-talk between networks.
This is tricky to specify as a particular plugin might not know what kind of isolation even exists in the infrastructure. For example, if "bridge" is configured in non-gateway mode (bridged to host's interface), it might not know if it is isolated at L2 from other networks.

In a lot of cases you can trivially isolate via routes in a container's network namespace. However this requires the container to be ran with CAP_NET_ADMIN revoked to ensure the container is not able to change the routes. A cleaner solution provides network isolation outside of the container's namespace.

Option 6 is used to communicate the DNS server. The dhcp plugin should output this in "dns" field of the result JSON.

• Add DNS field to result (#99)
• Populate DNS field with DHCP information

If details of the network change while the pod is alive there needs to be some way to run the CNI plugin to potentially update the pod's network setup for those changes. To be clear, this would be for an existing network that changes some metadata or internal implementation details that the CNI plugin could determine and act upon. For example, in our simple OpenShift multitenant plugin, the network's VNID can change based on administrator action and we need to run the pod network config plugin to updates some pod-specific rules on our OVS vswitch.

Since CNI plugins are executables/scripts that only run on ADD/DEL and quit after their work is done, they have no mechanism for watching for network/config changes. The container runtime (the thing calling ADD/DEL) is the most likely candidate for the UPDATE request, based on internal information it gets from somewhere (a plugin of its own, or other events).

I wouldn't consider this a full ADD/DEL because the network object itself isn't changing, just some arbitrary/internal attributes of the network object. Second, ADD/DEL would obviously interrupt connectivity and might change the IP address of the pod during its lifetime which is undesirable.

Thoughts?

We need to define the where in the returned JSON metadata a plugin can tell the caller what the DNS configuration should be. This will essentially be a priority ordered list of IP addresses:

"dns-resolvers": ["8.8.8.8", "8.8.4.4"]

I think that we should have a documentation for the implemented plugins in this repository. I'd imagine to write about (in)compatibilities, specific example configurations and also use-case suggestions.

Currently the host-local IPAM doesn't allow to have a pool of IPv4 and IPv6 addresses.
Does it make sense to have that?

http://linux.die.net/man/5/resolv.conf has some more options than nameserver.
I suggest to add search and domain. I'm not sure if we want/need the others.

/cc @alban

When I run build, it will meet the error as log in the screen catch as below.

my environment:
Linux zenlin-pc 3.13.0-53-generic #89-Ubuntu SMP Wed May 20 10:34:39 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

Can anyone help me to fix it, or tell me to how to fix it ,I just want to use the cni, and I am going to contribute to it.THX.

ipvlan did not work for @thockin. Ask him for details.

As discovered and discussed in rkt/rkt#851 - there are problems with the ways that some of the plugins utilise the CNI_NETNS parameter:

• the veth plugin uses it as a source of entropy for generating interface names for the host. Unfortunately, rkt supplies a (consistent) relative path to this file (which is useful for other reasons and not strictly prohibited by the current wording of the spec). Consequently, multiple invocations of the veth plugin for different containers end up using conflicting "entropy".
• macvlan has a similar issue
• the host-local IPAM allocator uses the (raw) netns name as an identifier for its local store; in this case, multiple allocations happen to succeed but removals will conflict

Some of these problems would be addressed by mandating that the CNI_NETNS parameter must be an absolute path (which should by definition mean it is unique to that container), but this introduces other issues (for example, in rkt this path might change between when its networking is being set up and destroyed).

A better solution is probably going to be to make CNI_CONTAINERID required, rather than optional as it currently is in the latest draft of the spec, and have plugins use this as a unique container identifier (and safe source of entropy) instead.

Happening on my laptop since a recent system upgrade. It's unclear what's causing this, and no error is produced when e.g. using the scripts to enter a new namespace created by CNI.

The journal shows the following error messages during the CNI plugin setup:

Nov 11 18:26:32 steveej-laptop NetworkManager[955]: <warn>  (veth9a2ffe01): failed to find device 63 'veth9a2ffe01' with udev

My current NetworkManager version 1.0.6

Hi,

Can you please let me know if CNI currently support multi-host (overlay) network so that containers in different hosts can communicate in the same network? Thanks!

I am creating this issue to summarize @thockin's comments on #2. In particular, Tim's proposed to make the spec less opinionated. Current spec defines a network as a group of containers that are able to talk to each other. The user defines which networks a given container needs to join. The container runtime executes a plugin for each network and the job of said plugin is to join the container to the network. As such, there's an assumption that a plugin will create an interface, assign it an IP, etc.

Tim proposes to have the container runtime call out to a plugin (or maybe multiple) and the plugin can do whatever it wants. For example, you can have a plugin to setup IP Masquerade or to block some traffic or change an IP of an existing interface.

Tim's proposal is quite flexible but also a fundamental shift from the current spec. I'd like to discuss this further and also schedule a hangout for Fri or next week: http://doodle.com/arqu4y9rgt9ybfp3

I think it would be nice if appc supports OVS. Is there any plan to support OVS?

Hello guys, I was wondering if there are any thoughts/implementations on how to add multiple interfaces inside a POD and connect them to different, for example bridges.

Example:

{
    "interface": [{
        "name": "mynet",
        "type": "bridge",
        "bridge": "cni0",
        "isGateway": true,
        "ipMasq": true,
        "ipam": {
            "type": "host-local",
            "subnet": "10.22.0.0/16",
            "routes": [{
                "dst": "0.0.0.0/0"
            }]
        }
    }, {
        "name": "mynet2",
        "type": "bridge",
        "bridge": "cni1",
        "isGateway": true,
        "ipMasq": true,
        "ipam": {
            "type": "host-local",
            "subnet": "10.23.0.0/16",
            "routes": [{
                "dst": "0.0.0.0/0"
            }]
        }
    }]
}

If not would it be easy for someone to point at the right files to start hacking ? I am pretty new to open source communities but I would love to contribute here. I also understand that a POD is designed to have one IP address, but the the use case I am experimenting with adds this sort of complexity.

Thanks again

Which plugins should support configurable attributes?
What should these be?

• main
  • bridge
    • MAC
  • macvlan
    • MAC
• ipam
  • host-local(-ptp)
    • IP

Config

{
    "name": "bridge1_ipv4",
    "type": "bridge",
    "master": "enp0s20u3u1u3",
    "ipMasq": true,
    "isGateway": true,
    "ipam": {
            "type": "host-local",
            "subnet": "10.0.0.0/24",
            "rangeStart": "10.0.0.230",
            "rangeEnd": "10.0.0.250",
            "routes": [
                {"dst": "0.0.0.0/0"}
            ]
    }
}

Situation

• veth host end is DOWN
• cni0 bridge on host is DOWN (not a problem)

# bridge link show
...
2: veth53d096e0 state DOWN @(null): <BROADCAST,MULTICAST> mtu 1500 master cni0 state disabled priority 32 cost 2

Hey All,

On the host-bridge plugin the loopback device is down by default. Is there any particular reason for that or could we set it to up as part of the plugin itself? Maybe we introduce a conf value to handle whether or not to up the loopback device in the namespace?

The spec does allow for IPv6 but the plugins only support IPv4 (with the exception of host-local).

Tracking:

• bridge
• ptp
• macvlan
• ipvlan
• host-local #131

I could not simultaneously use both on the same ethernet NIC when testing manually using the ip util. This needs to be investigated further using the plugins, since applications like rkt will eventually want to use both at the same time.

Support for ipvlan devices would be nice to have. This should not be much different from macvlan.

I don't think it's really possible to get IP masquerade to work with macvlan as it operates at L2 and probably does not call into netfilter code. In that case, support for it should be removed. ipvlan maybe can work (maybe in one of the modes) but needs to be investigated.

Current design requires the same set of args/data to be passed to both the ADD and DEL commands. This is slightly tricky b/c the net conf (JSON) is often stored on disk. In that case, the container runtime needs to save a copy of it in case it gets changed by the user prior to DEL command execution.

Since the runtime must be saving data to disk, we can clean the API a bit by having the ADD command return a dictionary with arbitrary key/values for the runtime to persist. It would then pipe it back to DEL command. It is more general b/c it allows the plugin to not have to serialize state to disk itself (at least for a subset of cases).

Example ADD:

$plugin_type < /etc/cni/mynet.conf
{
    "ip": "1.2.3.4/24",
    "data": {
        "masqChain": "xyz123",
        "bridgeName": "abc"
    }
}

Followed by DEL:

$plugin_type <<EOF
{
    "masqChain": "xyz123",
    "bridgeName": "abc"
}
EOF

I would expect for the kernel to take care of this, but somehow it doesn't seem to work if IPs are reused. This issue stems from rkt/rkt#1765.

The old p-t-p dialup links were set up so that all interfaces had the same IP on the server. I think we can use the same idea here. This should allow us to use only one IP per container in the case of "ptp" plugin.

Hi guys, there are some lint warnings in CNI code, I want to make a PR to fix them, is it OK:-)

$ golint ./...
cnitool/cni.go:27:2: exported const EnvCNIPath should have comment (or a comment on this block) or be unexported
libcni/api.go:24:6: exported type RuntimeConf should have comment or be unexported
libcni/api.go:31:6: exported type NetworkConfig should have comment or be unexported
libcni/api.go:36:6: exported type CNI should have comment or be unexported
libcni/api.go:41:6: exported type CNIConfig should have comment or be unexported
libcni/api.go:45:1: exported method CNIConfig.AddNetwork should have comment or be unexported
libcni/api.go:50:1: exported method CNIConfig.DelNetwork should have comment or be unexported
libcni/conf.go:26:1: exported function ConfFromBytes should have comment or be unexported
libcni/conf.go:34:1: exported function ConfFromFile should have comment or be unexported
libcni/conf.go:42:1: exported function ConfFiles should have comment or be unexported
libcni/conf.go:65:1: exported function LoadConf should have comment or be unexported
pkg/invoke/args.go:22:6: exported type CNIArgs should have comment or be unexported
pkg/invoke/args.go:32:1: receiver name should not be an underscore
pkg/invoke/args.go:36:1: exported function ArgsFromEnv should have comment or be unexported
pkg/invoke/args.go:40:6: exported type Args should have comment or be unexported
pkg/invoke/args.go:50:1: exported method Args.AsEnv should have comment or be unexported
pkg/invoke/exec.go:44:1: exported function ExecPluginWithResult should have comment or be unexported
pkg/invoke/exec.go:55:1: exported function ExecPluginWithoutResult should have comment or be unexported
pkg/invoke/find.go:23:1: exported function FindInPath should have comment or be unexported
pkg/ip/ipforward.go:21:1: exported function EnableIP4Forward should have comment or be unexported
pkg/ip/ipforward.go:25:1: exported function EnableIP6Forward should have comment or be unexported
pkg/ipam/ipam.go:28:1: exported function ExecAdd should have comment or be unexported
pkg/ipam/ipam.go:35:1: exported function ExecDel should have comment or be unexported
pkg/types/args.go:24:1: exported function LoadArgs should have comment or be unexported
pkg/types/types.go:23:1: comment on exported type IPNet should be of the form "IPNet ..." (with optional leading article)
pkg/types/types.go:38:1: exported method IPNet.MarshalJSON should have comment or be unexported
pkg/types/types.go:42:1: exported method IPNet.UnmarshalJSON should have comment or be unexported
pkg/types/types.go:72:1: exported method Result.Print should have comment or be unexported
pkg/types/types.go:83:6: exported type Route should have comment or be unexported
pkg/types/types.go:88:6: exported type Error should have comment or be unexported
pkg/types/types.go:98:1: exported method Error.Print should have comment or be unexported
pkg/types/types.go:117:1: exported method IPConfig.MarshalJSON should have comment or be unexported
pkg/types/types.go:127:1: exported method IPConfig.UnmarshalJSON should have comment or be unexported
pkg/types/types.go:139:1: exported method Route.UnmarshalJSON should have comment or be unexported
pkg/types/types.go:150:1: exported method Route.MarshalJSON should have comment or be unexported
plugins/ipam/dhcp/daemon.go:40:6: exported type DHCP should have comment or be unexported
plugins/ipam/dhcp/lease.go:51:6: exported type DHCPLease should have comment or be unexported
plugins/ipam/dhcp/lease.go:279:1: exported method DHCPLease.IPNet should have comment or be unexported
plugins/ipam/dhcp/lease.go:291:1: exported method DHCPLease.Gateway should have comment or be unexported
plugins/ipam/dhcp/lease.go:295:1: exported method DHCPLease.Routes should have comment or be unexported
plugins/ipam/dhcp/lease.go:306:16: should omit type time.Duration from declaration of var baseDelay; it will be inferred from the right-hand side
plugins/ipam/host-local/allocator.go:26:6: exported type IPAllocator should have comment or be unexported
plugins/ipam/host-local/allocator.go:33:1: exported function NewIPAllocator should have comment or be unexported
plugins/ipam/host-local/allocator.go:71:1: comment on exported method IPAllocator.Get should be of the form "Get ..."
plugins/ipam/host-local/allocator.go:136:1: comment on exported method IPAllocator.Release should be of the form "Release ..."
plugins/ipam/host-local/config.go:37:6: exported type IPAMArgs should have comment or be unexported
plugins/ipam/host-local/config.go:41:6: exported type Net should have comment or be unexported
plugins/ipam/host-local/config.go:46:1: comment on exported function LoadIPAMConfig should be of the form "LoadIPAMConfig ..."
plugins/ipam/host-local/backend/store.go:19:6: exported type Store should have comment or be unexported
plugins/ipam/host-local/backend/disk/backend.go:26:6: exported type Store should have comment or be unexported
plugins/ipam/host-local/backend/disk/backend.go:31:1: exported function New should have comment or be unexported
plugins/ipam/host-local/backend/disk/backend.go:44:1: exported method Store.Reserve should have comment or be unexported
plugins/ipam/host-local/backend/disk/backend.go:65:1: exported method Store.Release should have comment or be unexported
plugins/ipam/host-local/backend/disk/backend.go:69:1: comment on exported method Store.ReleaseByID should be of the form "ReleaseByID ..."
plugins/main/bridge/bridge.go:36:6: exported type NetConf should have comment or be unexported
plugins/main/ipvlan/ipvlan.go:32:6: exported type NetConf should have comment or be unexported
plugins/main/macvlan/macvlan.go:32:6: exported type NetConf should have comment or be unexported
plugins/main/ptp/ptp.go:42:6: exported type NetConf should have comment or be unexported
plugins/meta/flannel/flannel.go:42:6: exported type NetConf should have comment or be unexported

Hey All,

Great work going on here, much appreciated (not said enough):

Wanted to ask - why an inline struct was defined here and not a type?

https://github.com/appc/cni/blob/master/pkg/types/types.go#L61

We're integration this into a project and want to be able to configure the IPAM (using your type) upfront. Would you mind if I PR'ed this into being a concrete type or was there a specific design reason for it being as shown above?

Cheers!

After bumping the netlink library, the following configuration doesn't work as expected:

{
        "name": "default",
        "type": "ptp",
        "ipMasq": true,
        "ipam": {
                "type": "host-local-ptp",
                "subnet": "172.16.28.0/24",
                "routes": [
                        { "dst": "0.0.0.0/0" }
                ]
        }
}

The host can't be reached from within the new network namespace anymore. Diagnostics:

# ip -4 a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default 
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
3: eth0@if19: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state LOWERLAYERDOWN group default  link-netnsid 0
    inet 172.16.28.5/31 scope global eth0
       valid_lft forever preferred_lft forever
# ip -4 r
default via 172.16.28.4 dev eth0 
172.16.28.4/31 dev eth0  proto kernel  scope link  src 172.16.28.5 
# ping -c1 172.16.28.4
PING 172.16.28.4 (172.16.28.4) 56(84) bytes of data.
From 172.16.28.5: icmp_seq=1 Destination Host Unreachable

Implementation starts to get messy when different orchestrators use different paths to the CNI_NETNS. To configure the netns, we need the absolute path, so for rkt, we do something like this:

netns_path = '%s/%s/%s' % (RKT_NETNS_ROOT, env['CNI_CONTAINERID'], env['CNI_NETNS'])

This will be different for other orchestrators using CNI, so it's currently not clear how to write a generic CNI plugin if the prefix of CNI_NETNS isn't known.

There's no reason for the IP assignments on the host to survive a reboot.

As a convenience feature, bridge plugin should have an ability to connect host interface to the bridge (when the bridge is created).

(moved from rkt/rkt#477)

Under systemd, socket activation support would be highly valuable to save on resource and eliminate startup races.

In https://github.com/appc/cni/blob/master/Documentation/flannel.md example subnet.env does not have FLANNEL_NETWORK variable, without which https://github.com/appc/cni/blob/master/plugins/meta/flannel/flannel.go#L205 will panic.

Btw. there is no error checking if required variables are in subnet.env file.
If this variables could be missing, then in https://github.com/appc/cni/blob/master/plugins/meta/flannel/flannel.go#L72 pointer struct members should be initialized with empty values like:

se := &subnetEnv{
  nw: &net.IPNet{},
  sn: &net.IPNet{},
}

With ipvlan, all interfaces will share the MAC address so the DHCP server will return the same lease. Need to add support for ClientID (DHCP option 61). The value can be ContainerID by default but over-rideable via CNI_ARGS.

Add test script and integrate with TravisCI.

Seems to be happening with both macvlan and ipvlan. Might be that the interface is DOWN when IPAM is invoked.

Name of the interface inside the container. This is the name that should be assigned to the interface created inside the container (network namespace); consequently it must comply with the standard Linux restrictions on interface names.

The spec doesn't have anything related to that. So what is the procedure? Are the plugins allowed to remove, make the necessary modifications, and then recreate the interface with the same name?

First raised in rkt/rkt#862

bridge does correctly sets the IP on on the bridge (if isGateway is true) but not on the route.

Some containers need to have port mapping between them and the host. The current stance of CNI is that it is outside the scope (and rkt handles port mapping itself). The reasoning behind this is that port mapping is not associated with any particular network but rather is "global" to the container. At the same time port mapping is implemented as a DNAT rule that maps to a particular IP in the container and thus into a network. Therefore it may also make sense to make that a part of the plugin's responsibilities.

Currently the CNI works best if there are two levels of plugins: main and ipam.
Fewer levels (i.e. just main plugin) works fine but more levels are a bit problematic. Consider how the ipam plugin finds its configuration. The JSON that it receives is the same one that was fed to the main plugin. E.g.

{
    "name": "foo",
    "type": "bridge",
    "ipam": {
        "type": "host-local",
        "subnet": "10.1.2.0/24"
    }
}

The host-local IPAM plugin looks for its config under the ipam key. It will also look for the network name under the name key. But what happens if we wanted to have 3 layers? For example, what if we wanted to add another plugin that will install firewall rules to make sure the network can't cross talk to another network:

{
    "name": "foo",
    "type": "isolator",
    "deny": [ "10.0.0.0/8" ],
    "net": {
        "type": "bridge",
        "ipam": {
            "type": "host-local",
            "subnet": "10.1.2.0/24"
        }
    }
}

(If this is actually a config file that a user enters, the UX is horrible but we'll ignore it for now).

The problem with this approach is that both bridge and host-local plugins now need to look for their configs in a slightly different place (net and net.ipam, respectively). There's of course nothing to tell them that. The network name is still at name key.

How should we handle this? One approach is to have the invoking plugin feed only the JSON that is specific for the plugin. For example, an ipam plugin would just receive:

{
    "type": "host-local",
    "subnet": "10.1.2.0/24"
}

The network name would then be passed via an environment variable: CNI_NETNAME. I think this approach would work well but the nested JSON does not present the best UX, in the cases where these configs are entered by the user and stored on disk. Ideas for better handling UX are welcomed.