[Fixed]-How to make sure data over UNX Sockets gets sent in order using Twisted Python

1👍

SOCK_STREAM connections are always ordered. Data on one connection comes out in the same order it went in (or the connection breaks).

THe only obvious problem with the code you shared is that you shouldn’t override dataReceived on a LineOnlyReceiver subclass. All your logic belongs in lineReceived.

That wouldn’t cause out-of-order data problems but it could lead to framing issues (like partial JSON messages being processed, or multiple messages being combined) which would probably cause json.loads to raise an exception.

So, to answer your question: data is delivered in order. If you are seeing out-of-order operation, it’s because the data is being sent in a different order than you expect or because there is a divergence between the order of data delivery and the order of observable side-effects. I don’t see any way to provide a further diagnosis without seeing more of your code.

Seeing your sending code, the problem is that you’re using a new connection for every perform_create operation. There is no guarantee about delivery order across different connections. Even if your program does:

  • establish connection a
  • send data on connection a
  • establish connection b
  • send data on connection b
  • close connection a
  • close connection b

The receiver may decide to process data on connection b before data on connection a. This is because the underlying event notification system (select, epoll_wait, etc) doesn’t (ever, as far as I know) preserve information about the ordering of the events it is reporting on. Instead, results come out in a pseudo-random order or a boring deterministic order (such as ascending by file descriptor number).

To fix your ordering problem, make one UnixClient and use it for all of your perform_create calls.

Leave a comment