Network Service Mesh's exclude prefix mechanism is designed to protect the IPAM/Route Assignments an NSE makes from colliding with the prefixes already in use by the workloads environment (like the K8s cluster prefixes) and with the prefixes used by other NSEs to which the workload is connected.
Normally this means that if a workload connects to NSE-1 which assigns the workload an IP address and some routes, those should be added to the exclude prefix list for Requests to future NSEs.
One side effect of this is that it preclude the same IP address from being assigned to the interfaces from two different NSEs.
This is done because NSEs are presumed to be mutually ignorant of one another. There is no safe way to allow IP space overlap between two mutually ignorant NSEs.
In #119 a really good use case was raised for mutually aware NSEs in the form of multihoming a loadbalancer that assigns the same VIP address coming from multiple NSEs. It's important to support this use case.
In order for NSM to safely allow mutually aware NSEs to have overlapping IP space, it is necessary for NSEs to have a way to credibly signal that they are part of a group of mutually aware NSEs.
My initial idea for accomplishing this would be to add a field to ConnectionContext to hold a key unique to each 'awareness group' to indicate membership in that group.
There is some additional thought necessary around how those 'awareness group' ids associate to exclude prefixes so that the correct entities in the chain (like forwarders) can know its OK to allow overlapping prefixes, even if different connections are handled by different forwarwders.
@szvincze Thoughts?