Understanding IPFS in Depth(5/6): What is Libp2p?

Why Libp2p? : The Significance of Libp2p

In order to understand its significance, we have to go back in history.

The early research on ARPANET(predecessor of the internet) began around the question that

How do you build a network which is so resilient, so that it could even survive a NUCLEAR WAR?

The reason that this topic was of such interest in the 1960s was that most of the communication infrastructure at that time looked like this:

These were banks of humans operators connecting wires to facilitate country-wide communication. This was a highly centralized system, which could be destroyed easily in a nuclear war.

Fast-forward to today’s world, most of the internet companies today have centralized servers, most of which are hosted in the data centers outside the cities. The internet we see today is still somehow full of services similar to the bank of human operators which are highly centralized and fragile(well, we aren’t in a constant threat of a nuclear war today, but any technical or natural disaster can cause great damage).

But Why do we need to rethink about networking in 2019?

Today, we are dependent on the internet more than ever. We are dependent on it for life and death situations. But the current internet, as we see it today is quite fragile and has several design problems. Most of these problems stem from the location-addressing.

Hmm, But What’s location addressing?

This means that whenever any device(your phone, laptop, fridge, car) connects to the internet(via wifi, broadband, satellite…whatever) it gets a name(and hence the data stored in it) based on its location.

The proposed solution to this is content addressing, which allows a much resilient, peer-to-peer model.

However, in order to give content addressing properties to the web, we have to go deeper into networking and overcome many of the challenges that the current internet infrastructure has. These challenges include things like:

• NATs: NATs blind devices from each other on the network. Ever tried to host a website from your college network? I bet the experience would have sucked.
• Firewalls : Apply to restrictive filters on the network, which restricts different devices to talk to each other. Every got facebook blocked on your office networks? Yeah, that’s firewalls right there.
• High-latency networks: This is why sometimes the internet is slow, and you are mad because your favorite movie on Netflix is buffering.
• Reliability : Networks which are just not reliable and keep failing. We all have a memory of one shitty wifi that would just lose connection every time.
• Roaming: Problems with changing IP addresses while roaming due to location-based addressing. We all use free wifi of the coffee shops that we visit. While using the wifi your device is assigned an IP address(the identity of your device). When you move out of the coffee shop, your IP changes(as you disconnect from the wifi). So, your device never has a single identity that stays the same over a period of time.
• Censorship: The fact that Google is banned in China and children(well, everybody) of Turkey have never used Wikipedia for their school assignments.
• Runtimes with different properties : Lots of devices with very different connectivity properties force developers to work very hard to get around their limitations to get connected with each other, and a lot of times they are just simply not able to connect at all.
  Ever met a network engineer(IT guy) having a bad day? Well, pretty much every day.
• Innovation is super slow: Innovation in networks land is really slowwww…It takes years or even a decade for a new protocol to go down the OSI stack and to be deployed on a large scale. Even with a huge amount of funding, it might take years to decades to deploy new fantastic ideas to the internet’s fabric.
• And the list goes on…

These problems are partly because of the location-addressing model. The proposed solution to these problems is content-addressing .

Libp2p is not the first project that is working on the peer-to-peer content-addressing model. There have been a ton of projects like eMule , Kazaa , LimeWire , Napster , BitTorrent , some of which even exist until today. There was a lot of research done on algorithms like Kademlia , Pastry which powered these projects. Even though all these projects existed and got popular, it is really hard to leverage the work done in the past. Most of these projects have the classic software usability problems.

These problems include:

• Lack of good documentation, or none at all.
• Restrictive Licensing or no licensing to be found.
• Old with last update more than a decade away.
• No easy to reach, point of contact.
• Closed Source(Product) or the source doesn’t exist anymore.
• No specification provided.
• The implementation doesn’t expose a friendly API.
• Projects that are tightly coupled with a specific use-case.
• Make a lot of assumptions about how they work.

And this is something that we still see today.

A good example of this is WebRTC, the peer-to-peer transport of the web platform. Most of its opensource implementations are build by hacking the Chrome browser and are buggy and unreliable.

So, there is a need for a future-proof framework, that can efficiently encapsulate all the standards, which would allow:

• faster innovation, as the things are modular as-well-as compatible.
• future-proof system; so that things do not break as we add new things in the coming future.
• cross compatible; not only this allows new things to be added but also keeps the support for previously used standards, so that we don’t kill the legacy systems.

Well, I know what you are thinking…

Are we talking about just another standard?

But, no. We are not talking about just another standard. Instead of creating just another project with another “unique” protocol, we are trying to build a toolkit that allows creating and managing these standards and protocol that do not suffer from the classic software usability problems.

Ok. Tell me What is this “Libp2p”?

The one-liner pitch is that libp2p is a modular system of protocols , specifications, and libraries that enable the development of peer-to-peer network applications.

Peer-to-peer basics

There’s a lot to unpack in that one-liner! Let’s start with the last bit, “peer-to-peer network applications.” You may be here because you’re knee-deep in the development of a peer-to-peer system and are looking for help. Likewise, you may be here because you’re just exploring the world of peer-to-peer networking for the first time. Either way, we ought to spend a minute defining our terms upfront, so we can have some shared vocabulary to build on.

A peer-to-peer network is one in which the participants (referred to as peers or nodes) communicate with one another directly, on more or less “equal footing”. This does not necessarily mean that all peers are identical; some may have different roles in the overall network. However, one of the defining characteristics of a peer-to-peer network is that they do not require a privileged set of “servers” which behave completely differently from their “clients”, as is the case in the predominant client/server model .

Because the definition of peer-to-peer networking is quite broad, many different kinds of systems have been built that all fall under the umbrella of “peer-to-peer”. The most culturally prominent examples are likely the file-sharing networks like BitTorrent, and, more recently, the proliferation of blockchain networks that communicate in a peer-to-peer fashion.

Before going further I would strongly recommend going through the core concepts of networking, which are explained beautifully here.

What problems can libp2p solve?

While peer-to-peer networks have many advantages over the client/server model, there are also challenges that are unique and require careful thought and practice to overcome. In the process of overcoming these challenges, while building IPFS , we(contributors of IPFS) took care to build the solutions in a modular, composable way, into what is now libp2p. Although libp2p grew out of IPFS, it does not require or depend on IPFS, and today many projects(including Libra aka ZuckBucks) use libp2p as their network transport layer. Together we can leverage our collective experience and solve these foundational problems in a way that benefits an entire ecosystem of developers and a world of users.

Here I’ll try to briefly outline the main problem areas that are addressed by libp2p today (early 2019). This is an ever-growing space, so don’t be surprised if things change over time.

Transport

At the foundation of libp2p is the transport layer, which is responsible for the actual transmission and receipt of data from one peer to another. There are many ways to send data across networks in use today, with more in development and still more yet to be designed. libp2p provides a simple interface that can be adapted to support existing and future protocols, allowing libp2p applications to operate in many different runtime and networking environments.

Identity

In a world with billions of networked devices, knowing who you’re talking to is key to secure and reliable communication. libp2p uses public key cryptography as the basis of peer identity, which serves two complementary purposes. First, it gives each peer a globally unique “name”, in the form of a PeerId . Second, the PeerId allows anyone to retrieve the public key for the identified peer, which enables secure communication between peers.

Security

It’s essential that we be able to send and receive data between peers securely , meaning that we can trust the identity of the peer we’re communicating with and that no third-party can read our conversation or alter it in-flight.

libp2p supports “upgrading” a connection provided by a transport into a securely encrypted channel. The process is flexible and can support multiple methods of encrypting communication. The current default is secio , with support for TLS 1.3 under development.

Peer Routing

When you want to send a message to another peer, you need two key pieces of information: their PeerId , and a way to locate them on the network to open a connection.

There are many cases where we only have the PeerId for the peer we want to contact, and we need a way to discover their network address. Peer routing is the process of discovering peer addresses by leveraging the knowledge of other peers.

In a peer routing system, a peer can either give us the address we need if they have it or else send our inquiry to another peer who’s more likely to have the answer. As we contact more and more peers, we not only increase our chances of finding the peer we’re looking for, we build a more complete view of the network in our own routing tables, which enables us to answer routing queries from others.

The current stable implementation of peer routing in libp2p uses a distributed hash table to iteratively route requests closer to the desired PeerId using the Kademlia routing algorithm.

Here is the best slide deck I ever encountered for understanding Kademlia.

Content Discovery

In some systems, we care less about who we’re speaking with than we do about what they can offer us. For example, we may want some specific piece of data, but we don’t care who we get it from since we’re able to verify its integrity.

libp2p provides a content routing interface for this purpose, with the primary stable implementation using the same Kademlia -based DHT as used in peer routing.

Messaging / PubSub

Sending messages to other peers is at the heart of most peer-to-peer systems, and pubsub (short for publish/subscribe) is a very useful pattern for sending a message to groups of interested receivers.

libp2p defines a pubsub interface for sending messages to all peers subscribed to a given “topic”. The interface currently has two stable implementations; floodsub uses a very simple but inefficient “network flooding” strategy, and gossipsub defines an extensible gossip protocol. There is also active development in progress on episub , an extended gossipsub that is optimized for single source multicast and scenarios with a few fixed sources broadcasting to a large number of clients in a topic.

An Infographic to sum it up

libp2p is the networking layer of IPFS

Source

a collection of p2p protocols

Source

modules that satisfy interfaces (roles)

Source

The whole of IPFS is made up of libp2p modules

Source

An analogy to compare IPFS vs Libp2p

Source

So, as we saw above, Libp2p is a collection of building blocks that expose a very well-defined , documented and tested interfaces so they are composable , swappable and hence upgradable .

In other words, Libp2p takes the current “vertical” OSI model and makes it horizontal.

Current “Vertical” OSI model

This current networking model has a few shortcomings:

• Repeated actions throughout multiple layers, hence a lot of duplicated work and wasting resources.
• Information is hidden between the layers, missing significant improvement opportunities.

This is mainly due to the fact that internet protocols are spread across multiple layers(as shown in the diagram) and are very tightly coupled which makes them hard to change, thus discouraging innovation.

Libp2p changes this and just gives you a frame in which all of these protocols can co-exist and co-operate.

Libp2p: A modular suite of protocols for a better web

If you are new to this idea(OSI model), I strongly recommend you to go through the following RFCs for much better understanding:

• RFC 1958 : Architectural Principles of the Internet
• RFC 3439 : Some Internet Architectural Guidelines and Philosophy

Implementations

Libp2p is available in the following official implementations:

• go-libp2p in Go
• js-libp2p in Javascript, for Node and the Browser
• rust-libp2p in Rust
• py-libp2p in Python

There are also a number of other community built implementations which you can use.