• Dominik Charousset's avatar
    Make sure BASP nodes agree on a single connection · 83c43935
    Dominik Charousset authored
    This change fixes a race on the routing tables when spinning up multiple
    connections between two nodes simultaneously. With 2 nodes A and B, this
    is the sequence causing an error:
    
    - A publish an actor on port x.
    - A publish an actor on port y.
    - B connects to A:x with connection handle b1.
    - B connects to A:y with connection handle b2.
    - A sees a new connection handle a1 for the connection to A:x.
    - A writes server handshake A:x.
    - A reads client handshake A:x, adds (B, a1) to the routing table.
    - A sees a new connection handle a2 for the connection to A:y.
    - A writes server handshake A:y.
    - A reads client handshake A:y and drops it.
    - B reads server handshake on b2 and adds (A, b2) to the routing table.
    - B reads server handshake on b1 and closes the redundant connection.
    
    After that point, B uses the connection `a2 <-> b2`. However, the server
    believes `a1 <-> b1` is the active connection. When receiving data via
    `a2`, the server has no mapping for the handle in its routing table.
    Consequently, it cannot attribute incoming data on this connection to
    node B.
    
    To fix this dilemma, this change adds additional steps:
    
    - The server now adds *all* connection handles to its routing table. The
      first handle is going to be the default route. However, additional
      handles get added as alternatives. When receiving data on any of these
      sockets, the server is able to attribute it to right node ID.
    - The client dictates which connection gets used. After reading the
      server handshake and setting a direct route to the server, the client
      repeats its handshake with the new select connection flag.
    - When the server receives a handshake where the select connection flag
      is set, it updates it routing table to have the node ID resolve to the
      right connection handle.
    
    This patch is backwards compatible. Older versions of CAF in the network
    simply ignore the unrecognized flag.
    83c43935
header.hpp 4.1 KB