fixtradingcommunity / fixp-specification Goto Github PK

Section 7.3.1.1 states that

"The Sequence, Context, EstablishmentAck and Retransmission messages are sequence forming. They turn the message flow into a sequenced mode since they have the next implicit sequence number."

The Establish message is not on that list. Is that omission deliberate? If it is, then the client (with a recoverable flow) would need to send a Sequence after sending the Establish message, but before sending the first application message.

In 7.3.1, it says that

"Over TCP – a Client could (emphasis mine) send a Sequence message at the very beginning of the session upon establishment. The counterparty would not use it initially as it is provided in the EstablishmentAck message."

First, is that sentence correct in referring to the EstablishmentAck message, or should it refer to the Establish message?

Second, assuming that sentence was about the Establish message, if the Sequence message isn't required, then the Establish message must have been sequence forming.

Please clarify.

I believe that the table describing the Establish message in table 4.3.1 has an error: The "Description" field for the NextSeqNo should be "For re-establishment of a recoverable client flow only...".

First sentence of section on idempotent flow: should it read "guaranteed to be applied"?

On one of the past projects I used TestRequest message to measure network latency between Trader and Exchange. Would be great to have such opportunity in FIXP as well.

Multicast Session Protocol section 5 is vague about Sequence or Context messages for sequencing. It would seem to be parallel to simplex versus multiplexed point-to-point sessions. If only a single Topic is published, then Sequence can be used since there is no context switch. If multiple topics are published on a shared multicast transport, then Context must be used.

It the wait takes longer than KeepaliveInterval for the flow

I guess 'if' meant to be used instead of 'it'

From one side peer may terminate transport:

A session may terminate its transport if there are no more messages to send but it intends to restart at a later time.

From another side peer must close transport:

On a connection-oriented transport such as TCP, when the initiator receives the Terminate response, it must disconnect the transport from its end. When the acceptor receives the TCP peer reset signal, it completes closing of the transport.

So, must peer close transport?

Also in case of multiplexing, peer cannot terminate transport if there are more than one Sessions bound to Transport.

Need to explore the feasibility of exchanging the SBE schema inline within the context of a FIXP session, not sure if this could be accomplished through FIXP messaging or application layer.

Currently no mechanism defined in FIXP or application layer to do this.

Closest possibilities appear to be XMLnonFIX message (35=n) in FIXT or the MsgTypGrp component block in FIXT logon message (35=A)

This is little bit of a chicken and egg situation since the SBE schema will have to be provided out of band for the first time or delivered via existing FIX ASCII encoding as a 35=n message and then future versions could also be publicised as such

Upon receiving a Terminate message, the acceptor must respond to the initiator with a Terminate message before disconnecting the transport.

While being reading that, I was sure acceptor is a Server side.
I guess it worth to clarify "initiator of terminate" and "acceptor of the terminate request".

01Introduction.md
A method of sending datagrams from one producer to multiple

... multiple producers?

The provider of a recoverable flow need not retransmit all requested messages in a single batch. Rather, retransmission should be executed as an iterative process. It is the requester's responsibility to determine whether the current batch fills the original gap. If not, it should send another RetransmitRequest for the remainder.

Did not get. If Requester sends request for 20 messages and Provider sends 5 messages in first batch, should Requester issue new RestransmitRequest for 15 messages?

If yes, protocol violation can occur:

Only one RetransmitRequest is allowed in-flight at a time per session. Another RetransmitRequest must not be sent until a response has been received from a previous request.

To support mutual authentication, add a Credentials field to the NegotiationResponse message. It would convey identification of the server back to the client. As for the Credentials field in the Negotiate message, the format would be determined by agreement of counterparties.

Clarify rules for message framing:
If a framing header is used, the same framing header must be used for all sessions on a multiplexed transport. There would be no practical way to delimit messages with mixed framing policies.

A multicast publisher sends heartbeats at regular intervals when idle, as for a point-to-point session.

During the lifetime of a multicast session, its publisher should send Sequence or Context messages as a heartbeat at regular intervals when the session is otherwise inactive. This allows a receiver to tell whether a session is live and has not reached the end of its logical flow.

The problem is that the interval is not communicated to a consumer. There is no way for it to know when the interval is exceeded.

KeepaliveInterval could be added to the Topic message to serve the same purpose it does in Establish, to tell the expected interval.

For the Data Types section, use the terminology of ISO/IEC 11404 General Purpose Datatypes to give precise characteristics according to that standard.

I believe that the "Server Flow" value in the 7.1.4 table should be "Recoverable"

To facilitate out of band recovery for previously used UUID's we might need to enhance the RetransmitRequest to add an additional field such as OnBehalfOfSessionID since we cannot expect the same UUID to be used to Negotiate another FIXP session to recover messages sent previously with that same UUID as the sequence stream will overlap.

What if remote peer sends Finished Receiving message while local peer did not send Finished Sending message. Is this Protocol Violation case?

Could you please explain how should RequestTimestamp be used? What is the use case for this field?

Publish an addendum to the FIXP specification with a message schema (XML file) for Simple Binary Encoding wire format.

Clarify that the keepalive interval is the outer limit of silence, but hearbeats may be sent more frequently than required. In other words, sending a Sequence or Context message more frequently than required is not a protocol violation. A firm may wish to send an explicit sequence number before initiating a trading strategy, for example, even when it is not required by the time interval since the last application message.

To guarantee idempotence, a unique sequential identifier msut be allocated

I guess 'must' meant to be used instead of 'msut'

I have a question raised in https://stackoverflow.com/questions/68434287/fix-performance-session-layer-fixp-and-sbe
Wondering whether you people can give some suggestion.

The current FIXP specification proposes finalization of a FIXP session at the end of the day or trading session after which the TCP socket connection is severed with the expectation that a new FIXP session will be Negoitated.

How will this work for markets that need to operate on a 24 hour basis? (either 24x5 or 24x7)

Do we need to be able to finalize a FIXP session without termination in this case since we don't want the FIXP session to live on contiguously for all time (24x7) or even for a whole week (24x5).

Third sentence of idempotent flows says "The response flow must identify which operations that have been carried out...". Should "that" simply be dropped?

Consider adding a field to Negotiate to request cancel-on-disconnect or similar actions that are triggered by unexpected session disconnection. Cancel-on-disconnect is a common feature of exchanges to cancel open orders if they cannot be controlled or monitored due to a connection break. It may simply be an on/off switch or may involve parameters to configure actions.

Of course, not all implementers of FIXP would use cancel-on-disconnect, so there should be a mechanism to extend Negotiate with optional features to be negotiated between client and server.

Should Terminate Initiator handle Retransmit Request, business level messages and other incoming messages while waiting for Terminate Ack?

To populate FIXP messages in FIXimate and other reference tools, add FIXP message IDs and field tags to the FIX Repository.

If we decide to use only application level sequencing and application resend then is FIXP sequencing still required? Would it be okay to forgo FIXP sequencing altogether i.e. operate in unsequenced mode from exchange to customer and unsequenced from customer to exchange? If even customer to exchange uses unsequenced flow and application sequencing then could we apply NotApplied message to their application sequencing gap as well since it is supposed to be an application layer message?

Also we are thinking to change the concept of application sequence to application message ID instead so we might have to modify the NotApplied message accordingly.

Update resources for new message type MessageTemplate (issue #25).

• FixRepositoryForFIXP.xml

• SBEschemaForFIXP.xml

State explicitly that Idempotent flow over multicast with Sequence or Context message per package allows for loss detection, but NotApplied cannot be returned to the producer. If there is any recovery of lost packets, it must be through another channel.

Upon receiving Terminate message, acceptor of Terminate should not wait for TCP FIN message from initiator. It can send TCP FIN just after Terminate message. So sequence will be following:

Initiator

1. Send Terminate
2. Wait for Terminate acknowledge
3. Close socket

Acceptor:

1. Receive Terminate
2. Send Terminate
3. Shutdown socket (send TCP FIN message)
4. Close socket

Does it make sense?

04PointToPointSessionProtocol.md
The server should evaluate NextSeqNo to determine whether it missed any messages after re-establishment of a recoverable flow. If so, it may immediately send a RetransmitRequest after sending EstablishAck.

Why do not send NextExpectedSeqNum in Establishment and EstablishmentAck messages and send missed messages automatically?

3.5.1 Session Identification
Each session must be identified by a unique Session ID encoded as a UUID version 4 (RFC 4122) assigned by the client. The benefit of using an UUID is that it is effortless to allocate in a distributed system. It is also simple and efficient to hash and therefore easy to look up at the endpoints. The downside is a larger size overhead. The identifier however does not appear in the stream except once at the start of each datagram, when using UDP, or when sessions are multiplexed, regardless of the underlying transport that is used. For a non-multiplexed TCP session, the identifier therefore appears only once during the lifetime of the TCP session. A session identifier must be unique, not only amongst currently active sessions, but for all time. Reusing a session ID is a protocol violation.

It may not be appropriate to classify reusing session ID as a protocol violation since we don't want the exchange to shoulder the burden of having to validate this. If the customer does not ensure uniqueness then their FIXP sessions and sequence streams will converge and collapse into a single stream and retransmission of messages will not be reliable anymore.

A receiver of sessions that are multiplexed on a shared transport must examine SessionID of each session message in order to demultiplex. Currently, SessionID is in a position dependent on message type. Therefore, the receiver must first examine the message type to determine which decoder to apply to the message. This lookup of decoder type could be optimized out of the demultiplex operation by designing all session messages to have SessionID in the same position. Ideally, it should be the first field in the message.

According to specification, field SessionId in the RestransmitRequest message is required. I guess it should be required only if transport is shared between several sessions.

In the section describing finalization of a session, the FIXP specification refers to the ability of a customer to terminate a FIXP session after sending a FinishedSending message to the exchange and receiving a corresponding FinishedReceiving message from it. However this could result in a situation where the exchange still has messages to send to the customer and these will not be recoverable in band any longer since the new FIXP session will have to be negotiated with a new UUID and with inbound & outbound sequence numbers reset back to 1.

Finalization of a FIXP session should ideally be contingent upon two way exchange of FinishedSending/FinishedReceving messages so that both parties know that they are done for the day similar to TCP half-close concept described here at --> http://www.vorlesungen.uni-osnabrueck.de/informatik/networking-programming/notes/22Nov96/3.html

For multicast publishers, the spec currently states:

To support late joiners, Topic messages should be repeated at regular intervals on a session. This specification does not dictate a specific interval, but the shorter the interval, the less time it takes for a late joiner to identify flows.

I suggest sending the Topic along with every heartbeat when the session is otherwise idle. The heartbeat consists of a Sequence message for single topic session or Context when multiple topics are multiplexed on a multicast transport.

In the FIXT specification stated:

Whenever possible it is recommended that FIX engine attempt to send a Logout message with a text message of “MsgSeqNum too low, expecting X but received Y”

Does FIXP specification cover this case?

On a point-to-point session, either peer may initiate termination. Upon receiving a Terminate message, the acceptor must respond to the initiator with a Terminate message before disconnecting the transport.

I believe this sentence should be revised. Currently it means that Acceptor must send Terminate if he is going to close socket. If Acceptor does not want to close socket, he should not send Terminate.

03CommonFeatures.md
Sending a Sequence message on an Unsequenced or None (one-way session) flow is a protocol violation.

Why is sending Sequence message inside the UDP multicast message is a violation?

Reading the example in #42, where there is

 RetransmitRequest FromSeqNo=1000 Count=20
 Retransmission FromSeqno=1000 Count=5, followed by 5 application messages

Would it be correct for the re-transmitting side to issue

 Retransmission FromSeqno=1005 Count=5, followed by 5 application messages

without a second RetransmitRequest?

Given the example in 7.3.4.2, where the re-transmitting side satisfies a single RetransmitRequest with two Retransmissions, it seems like that behavior is allowed.

However, if that is allowed, I'm confused, how would the requesting side know if it would need to make a second request or not?

FIXP is transport independent, and the specification already provides for stream-oriented (TCP), point-to-point packet-oriented (UDP), and multicast transports. WebSocket Protocol (IETF RFC 6455) should also be supported since it is gaining popularity in the financial industry and elsewhere.

WebSocket is initiated with HTTP, and optionally TLS, handshakes. Therefore, it my be run over a secure pipe with authentication and privacy via cipher suites. After a session is established, either side may push messages asynchronously. WebSocket acts as a message framing protocol over a reliable TCP transport.

FIXP would enhance a WebSocket transport to provide these features:

• Durable sessions that survive a transport disconnection
• Configurable message delivery gurantees
• Configurable keep-alive intervals

Project Conga has already demonstrated feasibility.

Also tracked as GTC Management issue GM-187.

In example 7.1.10, the Negotiate message specifies that the ClientFlow is "Idempotent". The subsequent Establish message has NextSeqNo set to "1".

In 4.3.1, in the schema/table for the Establish message, the description of NextSeqNo states that it is "For re-establishment of a recoverable [see #70] flow only...".

It seem that one of these is in error: either the example in 7.1.10 should not have NextSeqNo set on the Establish message, or the description in 4.3.1 should say "For re-establishment of a recoverable or idempotent flow...".

If the schema in 4.3.1 needs to be changed, then the last paragraph in that section needs adjustment too: if the flow is idempotent, the server will not be sending RetransmitRequests (would it send NotApplied?). If those change are made to 4.3.1, then similar changes would need to be made with 4.3.2 as well.

In example 7.3.1.1, the client sends a NewOrderSingle with (implicit) sequence number of 100, which the server presumably responds to. The client then sends a Sequence message setting the NextSeqNo to 200, and follows that with a NewOrderSingle with (implicit) sequence number of 200.

In response, the server sends a NotApplied message, with FromSeqNo 101, and Count 100.

In 4.8.2, the description of NotApplied states:

When a receiver on an idempotent flow recognizes a sequence number gap, it should send the NotApplied message immediately but continue to accept messages with a higher sequence number after the gap.

Given that text, and given that the server sent an ExecutionReport at T6 (though it isn't explicit that it is for the second order), it appears that the sever applied the message with sqn 200. If that is indeed the case, then the range of not applied messages was from 101-199 inclusive (i.e. a count of 99). Therefore, shouldn't the NotApplied message have FromSeqNo 101 and Count 99?

I have few questions regarding subj:

• How should FIXP Acceptor handle any Session Level message message while processing Negotiate message?
• How should FIXP Acceptor handle any Session Level message message while processing Establish message?

Update the FIXP specification to use terminology of IETF RFC 2119 "Key words for use in RFCs to Indicate Requirement Levels" to precisely distinguish required versus optional or recommended features.

The FIXP spec should be updated to be consistent with FIX security recommendations, now under development.

• The security standard will recommend that all FIX traffic (at least over public networks) should be protected by TLS, using preferred algorithms and key strength.
• Clarify the role of Credentials in authentication. With TLS, the server is always authenticated. However, FIX credentials may be used to authenticate clients at the application layer.

As far as FinishedSending message is the Session Level message, it does not have assigned Sequence Number.

What if remote peer sends FinishedSending but local peer does not receive it, because connection is dropped in the middle? This message cannot be retransmitted...

Section on idenpotent flow says "The sequence number is implicit and is defined using a Sequence message." Suggest to add "with the explicit field NextSeqNo" to make explicit the fact that only the Sequence message itself and not the first message after it has the field "NextSeqNo". Maybe also change the next sentence to read "The first message after Sequence has the implicit sequence number NextSeqNo."