Let's first look at the message structure in V2

Message structure in V2 conversations

payload = encoded content,
header = {topic, timestamp}
digest = Hash(header || payload)
packet = {payload, sender address, sender's signature using prekey (digest)}
encrypted = encrypt(packet, shared_key, header)
packet = {encrypted, header}

Question

The primary question we want to address in this issue is
Does the message sender need to sign the digest of the header and payload?

By including a signature on header+payload, the sender is attesting to the message contents and context (because it's used as AAD in the encryption). In addition, the attestation is verifiable by both the intended recipients and parties external to the communication.

Properties

Each function in the message structure defined above results in different properties

1. Encryption gets confidentiality that only intended recipients can read the plaintext message
2. MAC gets the integrity of the message
3. Signature with the private key enables public verifiability

We focus on the last property here

Definitions

Deniable authentication

Say Alice and Bob are communicating via a secure pairwise channel

The shared key they establish for communication is called a DAKE (Deniable Authenticated Key Exchange) iff

1. Alice and Bob are each convinced that they established this secret with the counterparty
2. Alice (resp Bob) can convince Judge Judy that Bob(resp Alice) participated in a key exchange session with them.

There are two types of deniability depending on the role of the Judge in 2 above

Offline deniability

In this version, Judy is not observing actively observing the key establishment session.

Online deniability

In this version, Alice colludes with Judy to frame Bob. This is similar to Alice wearing a wire.

Establishing deniability

A protocol is said to have deniability properties if a simulator can produce a communication transcript similar to the transcript for true interaction between the parties. The defendant could deny it in this case by arguing that the transcript is fabricated.

Implications of non-deniability

Without deniability every message is on-the-record. This is potentially harmful to the senders as the messages can be used against them (now or in the future). For example, a whistleblower using a non-deniable communication system will have their identity leaked. Deniability is a complex issue requiring legal, social sciences, and cryptography inputs.

• Legal: In a legal setting, a Judge relies on a wide range of evidence. A deniable communication by itself doesn't provide enough evidence. At the same time, there is no precedent where courts admitted cryptographic deniability as the key evidence. However, the non-deniability property makes digital communication less private compared to real-world conversation. Without deniability, the Judge will consider A) Who is making the statements and B) the Digital trace of the communication.
• Social sciences: There is an argument that the deniability property favors the powerful more than it favors the less powerful participant in a conversation.
• Technical: Deniability is easy to achieve in a pairwise setting. But, it's more challenging in a group setting. Although, there is some recent work showing deniability can be achieved without sacrificing scalability.

Deniability in existing secure communication systems

Most of the existing secure message services have deniable communications by default. In Signal protocol, it can be shown that X3DH is an instance of DAKE. It follows that the communications are deniable too. The same holds for OTR and HMQV protocols.

Deniability in pairwise vs group communications

Pairwise communications

As mentioned earlier, deniability is easy and default in most of the pairwise communication protocols including Signal, OTR, etc. Things get more complicated in a group messaging scenario.

Group communications

Most of the existing deployments of group messaging sacrifice scalability to support deniability. Signal App uses pairwise channels that have linear message complexity to achieve authentication+deniability. Multiparty OTR (mpOTR) has similar complexity to achieve deniability. Tree-based methods (ART + TreeKEM) sacrifice the deniability property to achieve sublinear (log n) message complexity. More recently, Epochal Signatures have been proposed that can be integrated with tree-based methods to achieve deniability.

My current thoughts

1. Similar to widely deployed messaging Apps, pairwise communication should be deniable. In this setting, the receiver doesn't need a digital signature from the sender. A secret key is already shared between the sender and receiver. Without deniability, communication is less private compared to physical communication in the real world.
2. Even in a group setting, deniability should be the default behavior with support for adding non-deniability in individual groups. For example, a signature might be required after the fact to prove the authenticity of a message.
3. It's possible to have different deniability properties depending on the size of the group.

References

Epochal signatures: https://eprint.iacr.org/2020/1138
OTR: https://otr.cypherpunks.ca/otr-wpes.pdf
Deniability of Signal protocol: https://eprint.iacr.org/2021/642.pdf
Deniable Email: https://eprint.iacr.org/2019/390.pdf
Time deniable signatures : https://eprint.iacr.org/2022/1018

PublicKey signature is I think the only case where we don't capture the signed bytes. Consequently signature verification requires remarshaling the key contents (without the signature), which we know can be fraught with non-determinism in protobuf. We may want to consider replacing the current PublicKey definition

message PublicKey {
    // The key bytes
    message Secp256k1Uncompressed {
        // uncompressed point with prefix (0x04) [ P || X || Y ], 65 bytes
        bytes bytes = 1; 
    }
    uint64 timestamp = 1;
    optional Signature signature = 2;
    oneof union {
        Secp256k1Uncompressed secp256k1_uncompressed = 3;
    }
}

With something like this

message PublicKey {
    // The key bytes
    message Secp256k1Uncompressed {
        // uncompressed point with prefix (0x04) [ P || X || Y ], 65 bytes
        bytes bytes = 1; 
    }
    uint64 timestamp = 1;
    oneof union {
        Secp256k1Uncompressed secp256k1_uncompressed = 3;
    }
}

message SignedPublicKey {
    bytes key_bytes = 1;  // embeds a PublicKey
    Signature signature = 2;
}

Upside:

• safe from protobuf issues
• removes the optional field which seems to be a hassle
• saves remarshaling for the verifier

Downside:

• this would be a major breaking change,
  • all the contact bundles would need to be re-exported
  • all the message headers would be broken too

Thoughts?

Context

Progress

This change is complete when it has been implemented in all our SDKs:

• xmtp-js Link to tracking issue
• xmtp-ios Link to tracking issue
• xmtp-android Link to tracking issue
• xmtp-flutter Link to tracking issue
• Update PROTOCOL.md if necessary

Describe the bug

While using the JS or React SDK the app functions as expected, but I receive a recurring error in the console regarding a missing file. The error is as follows:

This was originally reported by @fabriguespe here: xmtp/xmtp-js#489

Expected behavior

No response

Steps to reproduce the bug

Use the JS or React SDK in a browser with source maps enabled

Context

Developers are experiencing an issue where there are duplicate conversations for a single ConversationId. This leads to messages being lost as they are split into different topics.

The root cause is:

• If multiple conversations are started with the same participants/ConversationId in a short period of time, this results in multiple invites being sent.
• SDK currently treats every invite as a distinct Conversation and returns the duplicates to developers

Solution

1. SDK/Keystore should generate new invites with deterministic parameters, such that multiple invites point to the same topicName and use the same topicKey.
   1. TopicName and TopicKey should be derived from the symmetric encryption key generated in the initial keyExchange. e.g. TopicKey = hkdf(symmetric_aes_key), TopicName = Encrypt(symmetric_aes_key, {participants, conversationId})
2. listConversations should be updated to remove duplicate entries with the same conversationId.
   1. This ought to occur in the same page, which could be a simplifying assumption.
3. SDK must deterministically resolve which invite to use when sending new messages
   1. This ensures that all messages end up in the same place.
   2. Keystore should use the most recent invite to to maximize forward compatibility
4. Existing duplicates will still exist, and will not be addressed at the SDK level. This should be a finite set of messages which will be less and less relevant and time progresses.
   1. New conversations will be fixed by A and B
   2. Once stabilized DM participants will use the same Topic for new messages
   3. If really needed; Apps can merge ‘Conversations’ with the same ConverstationID.

Full recommendation can be found in this RAPID.

Progress

This change is complete when it has been implemented in all our SDKs:

• xmtp-js: xmtp/xmtp-js#316
• xmtp-ios: xmtp/xmtp-ios#86
• xmtp-android: xmtp/xmtp-android#60
• xmtp-flutter: xmtp/xmtp-flutter#62
• Update PROTOCOL.md if necessary

Edit: michaelx11

1. Get xmtpv3 protos into xmtp-proto generation step and reference them in libxmtp-core
2. Create initial definitions for device contact bundles and pre-keys