Ximin Luo <email@example.com> writes: the issue is *not* merely a matter of sticking a recovery system on top of mpCAT, as you argue below. If you are not aware of *how recovery systems work*, then you may end up with a mpCAT that does things that it *impossible* to add such a system onto it.
In my opinion mpCAT should be like SSL and IPSEC, the messaging transport control protocol (MTCP) should act like TCP. The transport protocol should try to build the message tree you had in mind then a tree can be trun into a global order which can be passed to mpCat to just affirming the consistency of the global order. So basically like IP header over TCP header over SSL, we have "MTCP:blablah:3blah2blah2" then, MTCP will put the ":3blah2blah2" in correct order then pass it to mpCAT.
> for the end-to-end clients to be able to recover from message dropping, the server needs to add its own *sequence number* to messages;
I think assuming extra ordinary config/storage/abilities for the server, defeat the purpose of end-to-endness. If some participants are malicious probably you can't do much bar failing to democratic decisions or admin dictatorship. But under assumption of honesty, it shouldn't be impossible to recover the tree collectively.
> timestamps are not enough.
All I'm saying using time stamps in contrast to lexicographical ordering required in original mpOTR, makes us able to detect order tampering, in optimistic way. It wasn't about recovery.
> if he is told "all future messages may not be consistent".
No by reset, I meant it says some line between message x and y isn't consistence and we are restarting consistency check so you all are consistence after message y.". (an easily implementable compromise is to reset consistency digest at each group key exchange, I'll add that if nobody objects).
> Perhaps there is some confusion over one of our goals. One nice goals is to end up with a transport agnostic protocol, that
Again, in my opinion the MTCP should run in a lower layer than mpCat like TCP, that is how we can be transport agnostic. However, I think there is a general consensus that async is off the table for now.
> In my opinion mpCAT should be like SSL and IPSEC, the messaging transport control protocol (MTCP) should act like TCP...
We are working on end-to-end security principles, which applies for reliability too. TCP reliability is on the transport layer, not end-to-end between clients working on separate TCP connections. XMPP MUC reliability helps, but is not general enough: as I noted in the other thread, typically public services do not have the incentive, and cannot afford (due to threat of DoS) to offer unconditional reliability - they can only offer limited reliability, i.e. finite-time buffer or finite-message buffer, as experienced on real-world MUC servers.
As I explain in msg-notes, consistency is closely related to reliability; therefore it's best done within the end-to-end reliability layer. For example, your proposed scheme fails badly in the general case where the server cannot provide end-to-end reliability.
I agree that deniable authenticity and forward-secure confidentiality is best done in a separate layer, though. The idea of separation is good, but I disagree with how you're choosing to separate things, based on my research into the consistency problem.
> I think assuming extra ordinary config/storage/abilities for the server, defeat the purpose of end-to-endness. If some participants are malicious...
Context for others: this is for a server-dictated order that supports end-to-end reliability.
Participants might innocently drop for longer periods than the server is willing to buffer (due to potential maliciousness). However, you as the end client implicitly trusts your friend, so you are happy to buffer your own messages for them. Large public servers cannot make this assumption for all their users.
The server adding a sequence number, is zero-cost and (not considering migration costs) should be easy and reasonable to implement. It will enable end-to-end reliability, not guarantee it - the actual recovery behaviour would be implemented on the clients.
> .. we are restarting consistency check so you all are consistence after message y.". (an easily implementable compromise is to reset consistency digest at each group key exchange, I'll add that if nobody objects).
I think it would be better to just automatically re-run the GKE when inconsistency is detected. Why wait until the next group key exchange?
But I think this sort of solution is not ideal, and results much greater overal complexity; consistency and reliability should be done as part of the same layer, because it is simpler and more natural that way.
> .. Again, in my opinion the MTCP should run in a lower layer than mpCat like TCP, that is how we can be transport-agnostic. However, I think there is a general consensus that async is off the table for now.
To explain why this viewpoint is naive and narrow: your layer mpCAT, makes assumptions that do not apply in the more general case. The way you calculate consistency, assumes global-totally-ordered (linear) reliable delivery; it cannot work in a less strict scenario such as serverless broadcast, even though theoretically we can achieve consistency in those scenarios (e.g. via causal ordering). So, general (i.e. transport-agnostic) reliability systems do not try to achieve this property, so you will not be able to "layer" mpCat on top of it.
The only way a reliability layer can achieve this property (i.e. to ensure your assumption holds), is to require the server changes I mentioned (global explicit sequence numbers) and implement end-to-end recovery in the clients based on this information. This is a fine path to go down, but no reasonable person should call this "transport-agnostic", since it requires server changes which other protocols may not want to adopt (if the protocol even has servers).
TL;DR: the currently-proposed protocol is not transport-agnostic due to an assumption that is fundamentally transport-specific (global total-ordering). No "layering" strategy will enable the protocol to be transport-agnostic; only changes to the protocol itself that don't make this assumption will enable it to achieve this.