Multipath TCP for Linux

Multipath TCP for LinuxSkip to main content LinkMenuExpand(external link)DocumentSearchCopyCopied

Multipath TCP or MPTCP is an extension to the standard TCP and is described in RFC 8684. It allows a device to make use of multiple interfaces at once to send and receive TCP packets over a single MPTCP connection. MPTCP can aggregate the bandwidth of multiple interfaces or prefer the one with lowest latency, it also allows a fail-over if one path is down, and the traffic is seamlessly reinjected on other paths.

Technically, when a new socket is created with the IPPROTO_MPTCP protocol (Linux-specific), a subflow (or path) is created. This subflow consists of a regular TCP connection that is used to transmit data through one interface. Additional subflows can be negotiated later between the hosts. For the remote host to be able to detect the use of MPTCP, a new field is added to the TCP option field of the underlying TCP subflow. This field contains, amongst other things, a MP_CAPABLE option that tells the other host to use MPTCP if it is supported. If the remote host or any middlebox in between does not support it, the returned SYN+ACK packet will not contain MPTCP options in the TCP option field. In that case, the connection will be “downgraded” to plain TCP, and it will continue with a single path.

This behavior is made possible by two internal components:

• Path Manager: Manages subflows from creation to deletion, and also address announcements. Typically, it is the client side that initiates subflows, and the server side that announces additional addresses via the ADD_ADDR and REMOVE_ADDR options.

  Potential subflow
  Initial subflow
  Subflows creation
  Addresses announcement

As of Linux v5.19, there are two path managers, controlled by the net.mptcp.pm_type sysctl knob: the in-kernel one (type 0) where the same rules are applied for all the connections (see: ip mptcp) ; and the userspace one (type 1), controlled by a userspace daemon (i.e. mptcpd) where different rules can be applied for each connection.

• Packet Scheduler: In charge of selecting which available subflow(s) to use to send the next data packet. It can decide to maximize the use of the available bandwidth, only to pick the path with the lower latency, or any other policy depending on the configuration.

  Packets distribution between subflows
  Packet
Scheduler
  subflow 1
  subflow 2

As of Linux v6.8, there is only one packet scheduler, controlled by sysctl knobs in net.mptcp.

Features

As of Linux v6.10, major features of MPTCP include:

• Support of the IPPROTO_MPTCP protocol in socket() system calls.
• Fallback from MPTCP to TCP if the peer or a middlebox do not support MPTCP.
• Path management using either an in-kernel or userspace path manager.
• Socket options that are commonly used with TCP sockets.
• Debug features including MIB counters, diag support (used by the ss command), and tracepoints.

See the ChangeLog for more details.

Communication

• Mailing List: [email protected] (and archives)
• IRC: #mptcp on libera.chat
• Online Meetings

Projects

• Maintained by MPTCP community members
  • Kernel development on GitHub
  • Multipath TCP Daemon
    • The mptcpd daemon can do full userspace path management or control the in-kernel path manager.
    • Includes the mptcpize utility to allow legacy TCP binaries to use MPTCP.
  • Packetdrill with MPTCP support
• Projects with MPTCP-related enhancements
  • iproute2 (for the ip mptcp command)
  • Network Manager: MPTCP features are included starting with v1.40.
  • Multipath TCP applications: A project to coordinate MPTCP updates for popular TCP applications.

Kernel Development