ietf-wg-masque / draft-ietf-masque-connect-ip Goto Github PK

I continue to struggle with our difference in how we consider IP addressing and assumption on IP topology. I can't really determine which functionality is necessary to support the use cases.

I think it would actually help if we actually detailed a couple of IP addressing examples for some use cases as an appendix to the draft. I am willing to help write such text.

                        Target Address --+
                                          \
   +--------+           +--------+         \ +--------+
   |        |  Path #1  |        | Path #2  V|        |
   | Client |<--------->|  Proxy |<--------->| Target |
   |        |          ^|        |^          |        |
   +--------+         / +--------+ \         +--------+
                     /              \
                    /                +-- Proxy's external address
                   /
                  +-- Proxy's service address

In this figure, are there any cases where we need to provide an proxy's external address to the client, rather than having the proxy act as a router of all addresses assigned to client's for packet arriving on its right side interface. The reason I ask, is in many tunnel protocols one can actually ping so to say the remote side of the tunnel, we haven't discussed if that should be possible.

Also the discussion around the default route, in many cases that corresponds to all addresses except the subnet's IP range for a leaf network.

I do think we would all be helped by actually drawing a couple of pictures and include subnet addresses in these to clarify the assumptions of the roles.

So lets assume a client would do the following two HTTP Masque requests:

Stream 4: Connect-IP: Iproto=TCP
Stream 8: Connect-IP:ipproto=UDP

Can the server now send the same IP address in Address-assign capsules on both streams?
I would think this works as they are not requesting overlapping spaces.

Can a client send yet another request:
Stream 12: Connect-IP
I no limitations but it is on the same QUIC connection. I guess the answer is no as there are no hard binding that guarantees that this request is from the same underlying client, and there might be HTTP intermediaries combining requests into one QUIC connection. What happens if it request the same address using the Address-request capsule that was assigned in the first streams?

I think this issue contains two questions, first what is the authority model here. Are we assuming a single client so that assigning a resource multiple times with overlapping space on different tunnel streams are not an issue from a security perspective?

Secondly, even if we know it is a single client can we ensure that it works and have text that makes it so. I think more specific filter has its points as it enables future extensions like header compression to be applied to a subset of traffic for a larger assignment where it functions. However, I think some rules about precedence here will be required.

Implementing this on the client I was wondering if there should be a way for the client to ask for the server for an address without specifying what address it wants. This could allow a VPN client to mention that it wants an IPv6 address or not.

If we think this could be useful, we could encode it as ADDRESS_ASSIGN(IPv4-0.0.0.0/32) or ADDRESS_ASSIGN(IPv6-::/128).

Connect-IP enables the Masque client to set the source address for any IP packets it requests the MASQUE proxy to forward to the external network. This enables the potential for source address spoofing by the MASQUE client or an entity prior to it. For proxies that assign addresses to the client and allows general access to Internet I think it would be really good to recommend or require the masque server to perform source address validation. However, there clearly exist some use cases where the MASQUE will be used to establish a virtual path between two networks and where source address validation is hard as neither network may be stubs networks.

However, I think there are several things to address in the document.

1. Include in security description a discussion of the risks here and when it would be useful to perform source address validation. Especially for the assigned address(es) use case where rejecting all but the provided address range would be possible.
2. Consider if the security risks for certain deployments are such that source address validation is RECOMMENDED or REQUIRED

Even if PR #60 was an improvement to the description of the IPv6 minimal link MTU issue the text still leaves an open issue in that it doesn't make clear which of the endpoints that are responsible for HTTP intermediaries or if is truly both which has another implication.

So a Connect-IP endpoint will be aware of the HTTP datagram supported MTU and if its HTTP/3 connection can support sending at least 1280 bytes of IPv6 packets without fragmentation. However, if there is two or more HTTP intermediaries in between the client and the server then there are HTTP intermediaries that might have an HTTP/3 connection which has less than 1280 bytes MTU for HTTP datagram payloads in-between the client and server. Thus, arises the issue of how to detect this.

PR #60 states:

• if HTTP intermediaries are in use, CONNECT-IP endpoints can enter in an out of
  band agreement with the intermediaries to ensure that endpoints and
  intermediaries pad QUIC INITIAL packets.

From my perspective the big question is how a client or a server knows that there are intermediaries, or the most important aspect, that there are no unknown HTTP intermediaries. To my understanding both the client and the server can potentially bring in HTTP intermediaries unknown to the other part. Thus resulting in a HTTP intermediary to intermediary connection where neither endpoint knows if the requirement is fulfilled.

Thus my interpretation is that the only way to be sure that it works it to actually execute on the third bullet in the PR:

• CONNECT-IP endpoints can also send ICMPv6 echo requests with 1232 bytes of
  data to ascertain the link MTU and tear down the tunnel if they do not receive
  a response. Unless endpoints have an out of band means of guaranteeing that
  one of the two previous techniques is sufficient, they MUST use this method.

I would also note that this only verifies from the acting endpoint towards its peer, not the reverse path. For that to be verified the peer need to send ICMP probes the other direction.

Thus, my conclusion is that Connect-IP still don't ensure that the minimal MTU is fulfilled.

• We need to clearly state that ADDRESS_ASSIGN defines a set of addresses the receiver of the capsule can use as a source IP address on packets it generates
• We need to clearly state that ROUTE_ADVERTISEMENT defines a set of addresses the receiver of the capsule can use as a destination IP address on packets it generates
• Should we rename the capsules to make this relationship more clear in all use cases?

In the examples section (8), ADDRESS_ASSIGN capsules do not have individual Assigned Address entries wrapped in parenthesis, unlike the otherwise identically spec'd IP Address Range entries in a ROUTE_ADVERTISEMENT capsule. This feels especially odd in example 8.3 where two addresses are assigned. Should we just wrap them everywhere?

The example Assigned Address entries do not contain a Request ID. We should include this value, and perhaps add another example with an ADDRESS_REQUEST capsule so every example address doesn't carry a zero-value Request ID.

This is the current text in the draft

If an endpoint sends ROUTE_ADVERTISEMENT capsules, its routes SHOULD include an allowance for receiving ICMP messages. If an endpoint does not send ROUTE_ADVERTISEMENT capsules, such as a client opening an IP flow through a proxy, it SHOULD process proxied ICMP packets from its peer in order to receive these errors. Note that ICMP messages can originate from a source address different from that of the CONNECT-IP peer.

However, this seems to be mixing two things: ICMP messages directly from the proxy or other ICMP messages from any host behind the proxy.

I think the main thing we want to say here is that even when he client didn't request proxying of ICMP messages, it might get ICMP messages directly from the proxy and should process them.

If the proxy should forward any ICMP messages from other hosts to the client is a separate question. I would actually say the answer should be no, expect if requested by the client.

Looking at the current text in the draft, we have support for limiting scope to

• target, which is a hostname or IP address
• IP protocol number

Target clearly makes sense, as that limits the route advertisements and other components that we need to worry about, and allows multiplexing at least the 2-tuple (src, dst) IP addresses.

IP protocol number being supported in this draft is less clear to me. In conjunction with target, it expands our scope to multiplexing 3-tuples (ipproto, src, dst), but that doesn't seem like enough utility over 2-tuple, since the protocols involved are almost always going to be ICMP, TCP, and UDP and I can imagine most users would want to do all of these together.

Given that we're already not supporting TCP/UDP port numbers in this draft, would it make sense to move ipproto out?

The protocol contains a mechanism that allows limiting the scope of the request, but doesn't explain why it would be desirable to do so. Can we add text explaining when a client would want to limit itself, and how a server can leverage this information?

Based on the discussion in issue #43 we recommend to use ICMP for error signal. ICMP works for MTU issue or destination unreachable, however, there is no appropriate message type if the IP header of the packet provided by the client contains an IP that wasn't assigned to the client. So we need a different kind of error handling for this case?

I'm confused by the use of SETTINGS_ENABLE_CONNECT_[..] in the example. It makes me wonder if this document is defining a new setting that I don't know about. Can we instead just spell out SETTINGS_ENABLE_CONNECT_PROTOCOL?

There is no normative specification for the target IP address formats that can be used.

I don't understand how the UDP Listen Flow Example works. Unless I'm missing something, for this example to work you would need a way to convey the client's UDP port number. Since we decided that this was out of scope, perhaps we should remove this example?

If one is using IPPROTO to select a particular protocol to be tunneled, one may still received ICMP in the context of this request. To my understanding there are two reasons for this.

a) ICMP generated by the Connect-IP peer endpoint
b) ICMP sent back by node beyond the Connect-IP that is directly related to forward traffic sent by this endpoint and which the tunnel endpoint maps back to the Connect-IP request.

I think we could add a clarification of this in #69, or we could do it later.

As per Section 5 of RFC 8200, the IPv6 minimum link MTU is 1280 bytes. However, Section 8.1 of RFC 9000 states that it is possible to have a functional QUIC connection on a path that only allows 1200 bytes of UDP payloads. Such a path would only allow HTTP Datagrams with payloads up to 1180 bytes. If we only send IPv6 packets over QUIC DATAGRAM frames, such a setup would result in violating the IPv6 minimum link MTU. We should describe this problem and how to react to it. Possible solutions include failing the CONNECT-IP request, using the DATAGRAM capsule, or different solutions altogether.

As discussed in #33, turning a sequence of ROUTE_ADVERTISEMENT and ROUTE_WITHDRAWAL capsules into a functional routing table isn't as simple as initially expected. This is because two separate capsules can describe prefixes that are distinct but overlap. We should simplify the design or better explain the caveats.

If an endpoint have received first ADDRESS_ASSIGN capsule with some address and then receives a subsequent ADDRESS_ASSIGN capsule the document does not explain the meaning of this. There are several cases:

A) Just additional addresses are provided
B) Some removal
C) Some information are updated
D) Combinations

I think A) is straight forward and likely to occur. I think B) will happen when some connect-ip requests have been closed, and the need for an address does not any longer exist while a new request has come in. Can C) occur? I guess combinations are possible due to timing.

I also think the intention here is to have the new capsule completely replace the old one, and that is also not explicit in the text.

I think some clarification on this will be needed to ensure robust implementations.

The URI target example seems to have a copy paste error from connect UDP that has a port instead of a protocol

I would like to discuss this question a bit more before we submit the -00 version. Route advertising (and requesting) makes sense as a general concept to limit the destination address space, similarly as address requesting defines the source address space. However it is less clear to me if we need to support dynamic handling of routes in the base spec. I would assume that in many deployments the proxy has a fixed set of routes that are supported and that can be preconfigured and might be provided to the client together with the proxy address/proxy discovery. What exactly is the use case for handling this dynamically? And is that use case in scope for the base spec?

What is the behavior when an endpoint sends a packet for which it didn’t have an assigned source address or a valid destination route. Is this cause for dropping the packet silently, sending an error capsule, or closing the request stream with an error?

Closing the stream too harshly may be problematic if the packet was valid in some previous routing table, but now is not.

Splitting {ADDRESS_REQUEST|ADDRESS_ASSIGN|ROUTE_ADVERTISEMENT} into ..._V4 and ..._V6 would result in smaller capsules that have fewer error conditions. It would also keep IPv4 and IPv6 logic distinct.

The current text in this draft needs to be updated to match the changes from ietf-wg-masque/draft-ietf-masque-h3-datagram#97. I'll write up a PR once existing conflicting PRs are merged.

"The ADDRESS_REQUEST capsule contains a sequence of zero or more Requested Addresses."

In cases there are zero Requested Addresses what does this mean? It appears that this is an even less determined form than the all zero block , that at least indicates an address family support or preference as well as prefix length by including one or two all-zero address of the families the endpoint want assigned?

Should this be explicitly stated?

Instead of just an address / hostname, could we ask to open a tunnel to a prefix?

The target variable states the following:

"The server SHOULD send a ROUTE_ADVERTISEMENT capsule that includes routes for all usable resolved addresses for the requested hostname."

To me that implies that the server if providing both IPv4 and IPv6 routes to the target that it also MUST assign corresponding IP version to the client. The text should be explicit about this requirement. And if the server is not willing to assign an address then it should not provide a ROUTE_ADVERTISEMENT either?

So the description of target says:

The variable "target" contains a hostname or IP prefix of a specific host to which the client wants to proxy packets. If the "target" variable is not specified or its value is "*", the client is requesting to communicate with any allowable host.

"target" supports using DNS names, IPv6 literals and IPv4 literals.

Further down the the ABNF is stating:
target = IPv6prefix / IPv4prefix / reg-name / ""
IPv6prefix = IPv6address ["%2F" 13DIGIT]
IPv4prefix = IPv4address ["%2F" 12DIGIT]
ipproto = 13DIGIT / "*"

Thus, the text is inconsistent where the first indicates hostnames and prefixes. Second indicates hostnames (as DNS names) and IPv4/6 Literals, but not prefixes. Then the ABNF indicates Prefixes and reg-name.

So if it is only the prefix construct and hostnames then the text about literals directly needs to be addressed.

If it is hostnames, IPvX literals, and IPprefix the ABNF and description need to be addressed.

I think there are two option here, 1) close/deny the connect request, and 2) use streams instead. I think we should consider both option, especially given stream are just there to use.

Section 3.1 (Limiting Request Scope) uses the term path but then it talks about variables which are part of the query, not the path. Do we mean query here, or am I misunderstanding? As a reminder, the ":path" pseudo-header in h2 and h3 contains both the path and the query separated by a question mark.

ADDRESS_ASSIGN, aside from being too many characters to type, is apparently now cumulative. There is also no direct tie to ADDRESS_REQUEST, which is also too many characters to type.

ADDRESS_ASSIGN can also be spontaneous.

Take the case where the client asks for two addresses, but only gets one. How does the client distinguish between this being a spontaneous assignment or a response to its request. If it is spontaneous, it might want to keep waiting in case the server will eventually give it the other address that it asked for.

This is not a great situation for clients.

So there appear to exist some downside with the timing of the interactions when one want to use Address-Request capsule. So the basic problem is simply that the proxy will assign an address prior to the Address-Request capsule has been received, thus preventing the proxy from processing the request in the right context.

So lets assume that the client has already created one connect-IP request with a limited scope and been assigned IP-A. Then it want to make an additional request for another limited scope proxying and would like to get the same address assigned. Due to the separation between the HTTP request for the Connect-IP and the capsule Address-Request a server can actually process the whole HTTP request prior to receiving the Address-Request. Resulting potentially in that another IP address assignment (IP-B) be provided to the client, despite it asking for IP-A to be assigned also to the request.

So does the HTTP request needs an indicator parameter to tell the proxy to wait for one or more "request" capsules from the client before processing the request?

(This is related to #2, and here I'm assuming that Section 3.1 meant query and not path.)

What does the URI path represent? The examples use a path of /vpn and /proxy. Is the intention to let deployments decide what a path means? That makes sense to me but we should mention it.

We should be clear that when a proxy resolves a hostname, it SHOULD give all results back as routes to the client (unless there is some policy to restrict it).

Credit to @bemasc for proposing this on the list

There might be ICMP messages that are relevant to the proxy and should be handled by the proxy, mainly in flow mode. This needs additional discussion in the draft.

I've been noodling a bunch of scenarios where I think the draft could use a bit more clarification, and maybe a rev of the wire format around Request IDs for ADDRESS_REQUEST / ADDRESS_ASSIGN. They largely fall under the question of how does an endpoint NAK an address request? The draft says an endpoint receiving an ADDRESS_REQUEST capsule SHOULD respond with an ADDRESS_ASSIGN capsule, so I suppose not responding is allowed, but figuring out how to respond with a NAK is worthwhile.

For this, I'm going to use the term client to refer to the party requesting addresses/prefixes, and server to refer to the party assigning addresses/prefixes.

The client has received an (unsolicited?) address.
The client then requests a different address.
The server has chosen to not assign that address.

A suggestion I've heard: server sends ADDRESS_ASSIGN capsule with existing assignment and new request's Request ID.

The client sends a capsule requesting multiple addresses
The server has chosen to only assign a subset of those addresses (or a different address).

Using the suggestion from (1) involves replying with sufficiently many ADDRESS_ASSIGN capsules so as to exhaust the Request IDs sent. Maybe this means sending back n capsules all with the same one assignment, or n/2 capsules with the same two assignments, etc?

The client sends a capsule requesting an IP version not supported by the server.

This could be a request for addresses for IPv4 and IPv6 from a server that supports only one, or a simpler case of asking for an IPv4 address from an IPv6-only server.

It feels odd to use the suggestion from (1) to respond to a request for an IPv4 address with an IPv6 address or vice versa.

An alternative, which requires a wire format revision, is to include a list of NAK'd Request IDs in the ADDRESS_ASSIGN capsule, which could be as simple as a varint count of IDs, followed by that many more varints. In the happy path, this is just one extra 00 byte in the capsule.

Align with CONNECT-UDP to use the well-known default template

From ekr:

S 4.2.
Is the server required to send ADDRESS_ASSIGN? If not, what is the
client supposed to use as it's source address? Also, what happens
if you only get a v6 address but need to talk to a v4 endpoint,
or vice versa.

As pointed by @gloinul is #46 (comment), there is currently no non-empty way to convey "any ipproto" or "any IP address of any address family". That can cause issues when using some path templates. It would be simple to define a non-empty way to convey "any". I propose using * for this purpose.

(I wouldn't use ipproto=0 to mean "any ipproto" because 0 means IPv6 Hop-by-Hop Options)

I think the field description needs updates. The IP adress is after all an prefix that is masked by the specified number of bits. That is not quite clear yet.

However, I think the question if this range should support excluding one or more sub-ranges needs to be sorted out also.

The lifetime of the IP forwarding tunnel is tied to the IP proxying request stream. Closing that stream (in HTTP/3 via the FIN bit on a QUIC STREAM frame, or a QUIC RESET_STREAM frame) closes the associated IP tunnel.

I think it would be helpful to review the lifetime text in https://www.rfc-editor.org/rfc/rfc9298#section-3.1-5 more closely align with it where it makes sense. What stuck out to me first was speaking about HTTP/3 specifically sticks out because all versions are supported. Second, there's no text about if a proxy can timeout itself and close the stream.

realize that we need to remember this text from RFC 9000:

"Packet and frame diagrams in this document use a custom format. The purpose of this format is to summarize, not define, protocol elements. Prose defines the complete semantics and details of structures."

The issue is with at least the adderss assigned capsule:

The ADDRESS_ASSIGN capsule contains a sequence of zero or more Assigned Addresses.¶

Assigned Address {
Request ID (i),
IP Version (8),
IP Address (32..128),
IP Prefix Length (8),
}
Figure 4: Assigned Address Format
Request ID:
If this address assignment is in response to an Address Request (see Section 4.6.2), then this field SHALL contain the value of the corresponding field in the request. Otherwise, this field SHALL be zero.¶
IP Version:
IP Version of this address assignment. MUST be either 4 or 6.¶
IP Address:
Assigned IP address. If the IP Version field has value 4, the IP Address field SHALL have a length of 32 bits. If the IP Version field has value 6, the IP Address field SHALL have a length of 128 bits.¶
IP Prefix Length:
The number of bits in the IP Address that are used to define the prefix that is being assigned. This MUST be less than or equal to the length of the IP Address field, in bits. If the prefix length is equal to the length of the IP Address, the receiver of this capsule is only allowed to send packets from a single source address. If the prefix length is less than the length of the IP address, the receiver of this capsule is allowed to send packets from any source address that falls within the prefix.¶

So the actual data format is missing for IP version and IP prefix, and only implicit for IP Address.
I would guess they are both unsinged int, but that is not specified.
Also the ones that use variable integer encoding from QUIC should be explicit about that.

Same corrections needs to be applied to the other capsules.

Have we checked with the expert @mnot what the preferred way of documenting that multiple document specify different part of the sub-path after the initial prefix?

I would feel that we need to be clearer that we want IANA to add this information to what is already existing. The use of just update can easily mean replace which would not be correct. I also wonder if this should be an additional entry line or just kept in the same entry is unclear to me. I could see that one actually split the current masque entry into two, one for masque/udp and one for masque/ip

It is unclear to me if ADDRESS_ASSIGN only support assigning only a single address or a whole range. And if there is a difference between assigning the whole range to a client's usage versus specific addresses that will actually be processed out of this range, by using future requests?

Is it legal to send a CONNECT-IP request where :path = /proxy?target=target.example.com&ipproto=1&foo=bar ?

The more I think about this, the more I think that the target and ipproto would be better conveyed by two new HTTP headers instead of query parameters. Is there value in having these present in the URI?

The current version of the draft contains the ROUTE_ADVERTISEMENT capsule but is missing the ROUTE_REJECTION capsule. It's very common for networks to want to advertise routes such as "I can handle all of IPv4 space minus RFC 1918 space" and that becomes incredibly painful if you don't have a way to subtract prefixes. An alternate solution is to use an address start/address end encoding like IKEv2 traffic selectors, but if we want to use a prefix-based encoding we need a way to cut out holes from prefixes.

The draft currently mentions an "IP forwarding path" (or elsewhere a "path for proxying IP packets") without defining what that term means. It's ambiguous because "path" is a well-defined term in HTTP. I'd propose we replace this use of the word "path" with the word "tunnel" which is well-defined for CONNECT.

There should be a reference providing an URL for the Assigned Internet Protocol Number registry in this document, not just the name, even if this is formally sufficient. The URL is: https://www.iana.org/assignments/protocol-numbers/protocol-numbers.xhtml

I understand the value of the IP Protocol field on the ROUTE_ADVERTISEMENT capsule, but not on ADDRESS_ASSIGN or ADDRESS_REQUEST. What purpose does it serve?

I think the capsules description needs to be explicit if multiple capsules are allowed. I think there are different considerations depending on what type of capsules are used.

The authors have already discussed and concluded that multiple route_advertisement will be needed to allow to declare reachability of multiple prefix.

I don''t know if multiple address_assign capsules are allowed. Allowing multiple allows multiple address ranges to be declared. However, that results that we also get the question if one need to withdraw source address prefixes..