Comments (8)
I need to work through all the edge cases, but this sounds intriguing -- I'm particularly concerned about traffic that bounces off of kube-proxy in the default namespace and either ends up going to another pod on the same EC2 instance or exits back out the originating pod to hit another EC2 instance.
from cni-ipvlan-vpc-k8s.
How about we move to having all additional ENIs be created on the same subnet as the boot ENI and rip out all existing checks and support for multiple subnets? @theatrus and I are fine with this approach.
We can run the changes through conformance tests, but we're not expecting issues. Note that for deploys using large subnets, ARP table limits will need to be raised. We currently operate our legacy stack with large subnets and it's fine in AWS, particular given the lack of broadcast traffic.
Regarding l3/l3s support on AWS, what's the use case?
from cni-ipvlan-vpc-k8s.
I like the idea, it would be simpler
The only limit is that the primary interface and the pod interfaces will be in the same subnet (not a big deal but depending on architecture choices it could be a limitation: we would not be able to use the secondary CIDR block for pods for instance)
We have no use case new for l3/l3s but since there is a "mode" option in the plugin I wanted to make sure not to break anything
In any case, I can help and work on the design we will choose
from cni-ipvlan-vpc-k8s.
I'm also fine if there's an option for pods to be on a different subnet from the control plane/boot subnet. tldr; let's simplify the architecture, but if you all have network design constraints, we're happy to accomodate.
from cni-ipvlan-vpc-k8s.
After thinking about this and discussing it internally, we also think that using the boot ENI and removing support for multiple subnets is a good solution (nice simplification)
We may need other subnets in the future but I think going with the simplest solution is the best for now
I can PR this quickly if you want
from cni-ipvlan-vpc-k8s.
If you have time to PR, that would be great.
We've also been testing the stack with Calico in policy only mode to add Network Policy support to cni-ipvlan-vpc-k8s. So far so good on testing, but out of an abundance of caution, current Calico documentation https://docs.projectcalico.org/v3.1/getting-started/kubernetes/requirements#ip-pool-configuration mentions that instance IPs must be distinct from pod IPs -- I believe this is just a documentation error and doesn't apply to policy only mode. cc @bpownow
from cni-ipvlan-vpc-k8s.
One of the engineers at Tigera said it's fine as long as a pod is never given an address that a node already uses (per Kubernetes requirements).
from cni-ipvlan-vpc-k8s.
@paulnivin Sure I'll work on it next week and let you know how it goes
from cni-ipvlan-vpc-k8s.
Related Issues (20)
- Clarify configuration and setup instructions
- The CNI plugin only works on OS where interfaces are called ethX
- Plugin does not work with containerd runtime HOT 6
- missing routes on first pod for an interface HOT 3
- Nodeports not working properly HOT 8
- Incorrect handling of errors in IPAM HOT 1
- Inconsistency between ENI allocated IPs and OS configuration HOT 16
- CNI plugin: `portmap` when chained with cni-ipvlan-vpc-k8s does not work
- Listing VPC CIDRs to create routes in pods does not filter on association state
- New release HOT 1
- Disabling ipMasq
- Cannot connect to pod outside VPC HOT 4
- Pods loosing connection to each other on particular node from time to time HOT 13
- Build binaries for 0.5.0 release HOT 1
- aws-vpc-cni-k8s vs lyft cni HOT 3
- IP Batching changes the default behavior HOT 1
- Brother, I compile for a day. Compilation is unsuccessful. Please find a compiled package.
- Binary Release for 0.6.1 HOT 3
- support for containerd
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cni-ipvlan-vpc-k8s.