Technical overview of Awala

This is an introduction to Awala aimed at a diverse audience with at least a basic understanding of networking and cryptography, such as:

If you’re looking for a less technical introduction, you may prefer to read how Awala works in simple terms instead.

We highly recommend familiarising yourself with the way Awala works from the end user’s perspective before reading this document. If you haven’t done so already, try it out with Letro or Awala Ping, or watch the demo video.

Introduction

Awala is a delay-tolerant, overlay network: It runs on top of the Internet when it’s available, but it can switch to a secure sneakernet when the Internet is unavailable. With and without the Internet, users get end-to-end encryption and pseudonymity.

App developers can make existing Internet-based services (e.g., social networks) work on Awala, or build Awala-native apps that unlock additional benefits (e.g., spam protection, decentralisation).

The network itself is decentralised – or, more precisely, it’s federated: End user devices must be paired with an Awala-Internet Gateway, which is a server that relays messages between its users and other Awala-compatible Internet hosts. By default, users are paired with a gateway operated by Relaycorp (e.g., frankfurt.relaycorp.cloud), but this can be changed.

For example, the following video illustrates how Alice would post the message “Hi!” on Facebook via Awala whilst she’s connected to the Internet – assuming that Meta or a third party has built an Awala-compatible Facebook app for Android:

In the hypothetical scenario above, the Facebook Android app would produce a JSON document containing Alice’ post, and it’d send it to the Awala Gateway for Android using the Awala endpoint library. Internally, this library wraps that JSON document in an Awala parcel, which is encrypted with the public key of Facebook’s Awala endpoint middleware. Next, Alice’ Awala Gateway for Android sends the parcel to its Internet peer (frankfurt.relaycorp.cloud in this case), which then passes the parcel on to Facebook’s Internet endpoint (e.g., awala.facebook.com). Finally, Facebook’s endpoint middleware verifies and decrypts the parcel, and forwards the encapsulated JSON document to the Facebook app server.

Conversely, when the Facebook app server wants to send a message to a user, it first sends the message to its middleware, which wraps it into a parcel and then sends it to the user’s Internet gateway. The Internet gateway then forwards the parcel to the user’s gateway, which forwards the parcel to the Facebook app. Finally, the Facebook app verifies and decrypts the parcel using the endpoint library, and processes the encapsulated data. The following video builds on the previous example to illustrate this scenario:

Note that in the animation above, Bob is using a different Internet gateway (london.relaycorp.cloud) than Alice (frankfurt.relaycorp.cloud). As a federated network, users can choose their own Internet gateway independently of the service they use and the people they communicate with – Think of it as the Awala equivalent of an ISP.

Sneakernet

When the Internet is unavailable, users can have their data physically transported by couriers, who effectively form a secure sneakernet.

Most of the protocol remains the same as when the Internet is available, but gateways don’t exchange parcels: instead, they exchange cargo, which is also encrypted and contains one or more parcels. This way, an eavesdropper wouldn’t be able to see which services are being used (e.g., awala.facebook.com). The following video illustrates the transport of cargo to the Internet:

Collecting cargo from the Internet gateway is almost the same process in reverse, except that the courier must start by providing a Cargo Collection Authorisation (CCA), which the user’s private gateway would’ve issued to the courier when they synchronised. The CCA is encrypted with the Internet gateway’s public key, and it contains the Cargo Delivery Authorisation (see delivery authorisations below) that the Internet gateway must use to sign the cargo.

Messaging

Both parcels and cargoes are serialised using the Awala Abstract Message Format (RAMF for historic reasons), which is a binary format based on ASN.1 Distinguished Encoding Rules (DER) and the Cryptographic Message Syntax (CMS). The protocol suite defines additional RAMF types.

A RAMF message is a CMS SignedData message that encapsulates the following in a DER-encoded structure:

Additionally, the SignedData structure contains the sender’s certificate, which will be self-issued if the recipient is an Internet node. If the recipient is private, the sender will have to sign the message with the appropriate Delivery Authorisation, or else the network will drop the message.

Note that the unencrypted metadata above doesn’t include any personally-identifiable information. Additionally, only parcels bound for an Internet endpoint can be attributed to a specific service (e.g., awala.facebook.com), which is something that only the gateways that relay the parcels can see – couriers can’t see that because parcels are encapsulated in cargo.

Messaging protocols

Awala involves three layers of messaging protocols:

The following diagram illustrates the relationship between the three layers when the sneakernet is used:

protocol-layers.svg

Message transport bindings

The messaging protocols don’t actually specify how endpoints, gateways and couriers exchange messages: That’s deferred to the message transport bindings. Awala defines three bindings:

Bindings are abstract, in the sense that the protocol is agnostic of the underlying Layer 7 protocol. As of this writing, we support one concrete binding for each abstract binding:

When a client connects to a remote server (i.e., not on 127.0.0.1), Awala requires TLS 1.2+ to be used. Cleartext HTTP or SSL connections must be rejected. DTLS 1.2+ is allowed in case a future binding uses UDP.

Nodes

Nodes are entities that produce and consume messages. Endpoints and gateways are nodes, but couriers are not.

Nodes can currently be private or Internet, depending on whether they’re Internet hosts. Future versions of the network may introduce additional node types as new underlying networks (e.g., scatternets) are supported.

Private nodes

Private endpoints and private gateways, collectively known as private nodes, run on end user devices (e.g., smartphones, laptops) and implement a subset of the bindings:

Awala requires every message bound for a private node to be signed with an appropriate Delivery Authorisation.

Internet nodes

Internet gateways and Internet endpoints, collectively known as Internet nodes, are Internet hosts that implement one or more bindings:

Awala requires that such hosts have an SRV record to “advertise” its role as an Internet node. For example, building on the example above, Facebook could deploy its endpoint middleware at awala.facebook.com (using an A or CNAME record), and then have an SRV record pointing to it from facebook.com (in which case, facebook.com would be its Internet address). This SRV record must be in an DNSSEC-signed zone.

Additionally, clients connecting to Awala Internet nodes must use DNS-over-HTTPS (DoH) or DNS-over-TLS (DoT) to resolve the both records for privacy and censorship-circumvention reasons. As of this writing, the reference implementations use Cloudflare’s DoH service.

Unlike private nodes, Internet nodes do not benefit from any type of access control like Delivery Authorisations.

Authentication and access control

Awala deploys its own Public Key Infrastructure (PKI) to completely protect private nodes from spam and minimise illegitimate traffic on the sneakernet (where the storage capacity may be limited). The PKI is based on X.509 certificates, and it’s used to authenticate all nodes.

Identity keys

Every node has a long-term, identity key pair that’s used to issue certificates (e.g., delivery authorisations) and sign RAMF messages.

Nodes use 2028-bit RSA keys by default, but 3072- and 4096-bit keys are also supported. Digital signatures are produced with the Probabilistic Signature Scheme (RSA-PSS), and SHA-256 by default (SHA-384 and SHA-512 are also supported).

We plan to migrate to Curve25519/Curve448 keys and EdDSA in the future.

Node ids

The id of a node is the SHA-256 hash of its public key, hex-encoded, and prefixed with a zero (0; denoting the first version of this format). For example, 0b5bb9d8014a0f9b1d61e21e796d78dccdf1352f23cd32812f4850b878ae4944c would be the id of a node whose identity public key has the SHA-256 digest b5bb9d8014a0f9b1d61e21e796d78dccdf1352f23cd32812f4850b878ae4944c.

Delivery Authorisations

A delivery authorisation is a certificate whereby a private node authorises another node (private or Internet) to send messages to it. There are two types of authorisations:

With the appropriate certificate chain attached to each message, the gateways and couriers transporting the message can verify whether the sender is authorised to communicate with the recipient.

End-to-end encryption

The end-to-end encryption scheme – known as the Awala Channel Session Protocol – is used to encrypt the content of RAMF messages with perfect forward secrecy, future secrecy and replay attack mitigation. It’s mostly a delay-tolerant, decentralised version of the X3DH and Double Ratchet protocols from the Signal project.

Another key difference is that X3DH requires X25519 or X448 keys, whereas Awala uses NIST curves (P-256 by default) due to the fact that one of our third-party dependencies doesn’t support X25519 or X448. However, we plan to migrate to X25519/X448 in the future. The following algorithms are also used:

Internet nodes have an initial ECDH key pair, whose public key is made available to prospective peers via the connection parameters. Because the Channel Session Protocol uses a ratchet mechanism, the initial key pair is only used by private nodes to encrypt the first message sent to an Internet peer, and subsequent messages will use different key pairs. The initial key pair is rotated periodically.

Connection parameters

Since establishing a channel between two nodes requires two public keys (a long-term identity key and a short-term encryption key) and an Internet address from each peer, Awala facilitates this process by having nodes exchange their connection parameters (a binary blob) with prospective peers.

Internet node connection parameters

Internet endpoints and Internet gateways expose their connection parameters via their parcel delivery and gateway synchronisation bindings, respectively. Such parameters are:

Connections parameters are safe to distribute publicly. In fact, we at Relaycorp distribute our private nodes with their public peers’ connection parameters embedded in the software so that they won’t need to be fetched from the Internet. For example:

Private node connection parameters

In the case of private endpoints, the connection parameters is known as connection authorisation, since its primary purpose is to authorise peers. This binary blob extends the Internet node connection parameters with the PDA for the grantee and the relevant intermediate certificates. Consequently, each authorisation is unique to a particular endpoint.

Private gateways do not have connection parameters. Instead, they use Cargo Delivery Authorisations to authorise their Internet gateways, and the private gateway’s latest ECDH public key is already shared with the Internet gateway in the CMS EnvelopedData structure encapsulated in the cargo.

Key management

Internet nodes use Key Management Services (KMSs) via a provider-agnostic library. In Relaycorp’s case, all of our keys are backed by Hardware Security Modules (HSMs) in Google Cloud KMS. Unfortunately, for encryption keys, we have to protect them with envelope encryption because ECDH keys are not natively supported yet.

Private nodes on Android use the Android Keystore, whilst desktop nodes currently store the keys in cleartext.

Certificate management

The Awala PKI requires certificates to be valid for no more than 6 months to compensate for the lack of a revocation mechanism (which wouldn’t work when the Internet is unavailable).

Certificate renewal is fully automated:

Service (de)centralisation

Awala services can be centralised, decentralised or hybrid. The examples so far illustrate a centralised service, where Facebook (the service provider) operates an Internet endpoint, which processes all the communication between users. However, it’s also possible to build decentralised services where Awala gateways transport parcels between users, with no involvement from a server operated by the service provider.

For example, the following video illustrates how Alice could send the message “Hi!” to Bob on WhatsApp via Awala whilst she’s connected to the Internet – assuming that Meta or a third party has built an Awala-compatible WhatsApp app for Android:

In this example, Alice and Bob are using different Internet gateways, but if they were using the same gateway, the process would be a bit simpler as there would be one fewer hop as shown in the following video:

Ping service

Awala defines a trivial service called Ping, which is used to test the network itself. The service has private endpoint apps on Android and desktop, and a backend app for the Internet endpoint middleware.

Learn more

If you wish to learn more, you can read the Awala specs, browse the reference implementations or ask us questions.