Edge Node

An Introduction to Edge Network

1. Background

A typical computer network or distributed computing environment uses one of the two basic architectures, client/server or peer-to-peer (P2P). The client/server architecture designates one computer or host as a server and others as clients. In this model, the server must always be online with good connectivity. On the other hand, the P2P model differs in that all computers are equal and act as both suppliers (servers) and consumers (clients) of resources or services.

Starting from the 1990s, with the widespread adoption of the World Wide Web on the Internet, the client/server architecture has become the most commonly adopted architecture for web servers and file servers, using protocols such as HTTP or FTP among others. There are some inherent shortcomings of this model, such as the need for the server to be online all the time, and facing very high loads if there are many clients seeking services. Scaling a system based on this architecture is a difficult problem and requires lots of resources and planning.

In the late 90s, we saw some early file-sharing networks based on peer-to-peer architectures, such as Napster and, later, BitTorrent. Their success raised interest in P2P architectures, and it was regarded as a solution to overcome the obvious limitations of client/server systems. In a P2P network, every node can be a provider of data and services; therefore, the system’s availability and scalability can be managed much more easily compared with client/server systems.

2. Introduction

Sending.Network (SDN) is a communication network based on a peer-to-peer (P2P) protocol. It is a completely decentralized system and contains no central server; instead, each user's client is an independent node, connected to at least two other peers. All communication between users is encrypted by their clients so that no third party can eavesdrop on it, ensuring users' privacy.

Figure 1 shows the network structure of SDN. There are two types of nodes in the network, Client Nodes, and Edge Nodes. The Client Node is the computing device (e.g., mobile, laptop or tablet) of ordinary users, with uncertainty about their state and storage. These nodes may be online or offline at any time and place, and their storage capacity often changes. An Edge Node is a node in the SDN network that assists Client Nodes in communication. It has a public IP address, which can be directly accessed by other nodes, and is online all the time. In order to compensate for the Client Node’s uncertainty in state and storage, Edge Nodes need to have large storage space and have certain requirements for hardware and bandwidth to ensure that they can provide computing services to the Client Nodes they serve.

Figure 1: The SDN’s networking architecture

The lowest level of the SDN networking architecture, shown in Figure 1, is the storage layer. SDN users can back up their social data, e.g., contact lists and social graph relations on EVM-compatible blockchains or other decentralized storage like IPFS. We are planning to support more blockchains in the near future.

3. Design Objectives

SDN aims to address the lack of functionality of P2P-based chat services due to the lack of a central server. According to the impossibility triangle principle, it is difficult to achieve the three characteristics of decentralization, security, and efficiency simultaneously. Conventional P2P-based services contain no centralized servers and sacrifice efficiency for the ultimate in decentralization and security.

However, in order to provide efficient services, a role similar to that of a centralized server is usually inevitable. In SDN, we introduce high-performance Edge Nodes to replace the traditional centralized server to provide efficient services. The cost is that we have to carefully consider the implementation of Edge Nodes to balance the demands of security and decentralization at the same time.

Specifically, the selection of Edge Nodes and the implementation of their functions must meet the requirements of a decentralized P2P-based solution. To guarantee business reliability, any Edge Node in the SDN network should be replaceable at any time, and the service should not be affected when it is replaced. In addition, due consideration should be given to protecting the data security and privacy of user data cached in an Edge Node. No malicious adversary should be able to extract the user's personal data from Edge Node. This requires a solution with a very low probability that the chosen encryption can be broken.

Edge Node communication meets the requirements of the libp2p specification and supports protocols like TCP, UDP, WS, WSS, etc. As the backbones of the P2P service, Edge Nodes provide message relays, network gateways, push notifications, and storage services to the client nodes. Figure 2 illustrates the architecture of an Edge Node.

Figure 2: The Architecture of an Edge Node

4. Design Principles

Unlike centralized services, decentralized networks operate in an environment where trust cannot be automatically assumed. In other words, services always operate in an untrustworthy environment. Therefore, in the overall design, we assume that an Edge Node is a Byzantine type of actor. That is, it may be hacked, fail or even act maliciously.

Therefore, we need to implement appropriate precautions for both user data storage security and network stability. The requirements for the SDN solution are as follows:

Security of stored data

1. Confidentiality of stored data. Any user data stored in an Edge Node shall be encrypted, and no malicious adversary should be able to recover any meaningful plaintext data from the ciphertext stored therein.

2. Verifiability of stored data. Any stored data must be signed with the user's private key and verified upon retrieval to ensure that the stored data has not been tampered with. Any tampered data shall not pass the verification process and must be rejected immediately.

High network reliability

To avoid the problem of a single point of failure due to force majeure factors such as censorship, an important design principle is to ensure that SDN does not depend on any single node. SDN has designed a certain algorithm that randomly assigns Edge Nodes to Client Nodes for message relaying and data storage. The randomness of the assignment distributes these nodes in different geographical areas around the world. Thus, user data has multiple backups on different Edge Nodes. Even if some of these nodes fail, there are still other distributed nodes that can continue to provide services.

5. Function classification and implementation Guidelines

An Edge Node is an auxiliary node in the P2P network, providing support for the main function of SDN – the peer-to-peer communication between nodes. With a public IP address and an access port, any server that meets the hardware and network bandwidth requirements can be configured as an Edge Node.

They are designed to mainly take the role of data storage and message relay. Depending on the performance of the server, individual Edge Node nodes can decide which auxiliary services they wish to support. By setting different configuration parameters, an Edge Node can take on the following different functions.

Verifiable information storage

For any storage method, data integrity and security are always paramount. As mentioned earlier, any stored data is protected by its signature and verified upon retrieval or request. There are two types of file storage at Edge Nodes. One is based on the libp2p transfer protocol to store files on an Edge Node selected by a Client Node, and the other is through calling the IPFS API to store files on an IPFS storage node, which is deployed on a Client Node and some of the Edge Nodes. The reason for using two protocols is that using the libp2p transfer protocol can control the transfer rate and the target storage edge node more conveniently; while the IPFS-based method has greater versatility. Many decentralized systems are based on the implementation of IPFS. In the future, we hope to allow users to freely choose the storage method for storing local files.

At this time, IPFS is only used for storing user social graph data, but with the ability to also store other files which we will discuss in the following sections. Most file storage uses the libp2p transfer protocol to store data directly on the Edge Node to which the client is connected.

The libp2p-based transfer protocol saves user data files in the Edge Node. First, the Client Node sends an upload request with the file’s md5 hash to the Edge Node it is connected to. The Edge Node checks whether the file has been successfully uploaded based on the md5 and, if so, updates the file's storage timestamp. If the local storage space is insufficient, it returns an upload failure message.

The Edge Node scans its local hard drive periodically to check disk usage and deletes the files with the longest timestamp when the disk usage exceeds a set threshold. The Edge Node stores chat messages in a local database. The current message capacity management has two dimensions: the number of stored messages for a single topic, and the time stamps of the messages. The maximum number of entries per topic is calculated based on the local disk space and the maximum number of topics listened to. When this maximum value is reached, the message cleanup co-routine deletes the earliest message records. The current default message storage time is 30 days, and any message stored past that time is deleted. Users can also deploy their own Edge Node(s) to set up persistent storage.

An Edge Node needs to solve the problem of persistent storage for the P2P chat service. When users are offline, Client Nodes cannot store messages. Therefore, Edge Nodes (associated with their Client Nodes) need to take responsibility for storing these messages. Specifically, these messages fall into the following three categories.

1) Offline Message Storage

These are the messages received by a user when they are offline. An Edge Node caches these until the user comes back online and pulls all cached messages for various conversations from the Edge Node. This function is implemented based on the pub-sub mechanism. Chat parties find each other by subscribing to the same topic and establishing a chat room. After the user joins the room, his/her Edge Node will save the messages that the topic listens to in the Edge Node’s local database. When the Client Node comes online again, it will pull the cached messages from the Edge Node's database.

Besides, the Edge Node stores any invitation messages received by a user when they are offline. Similar to the storage of offline messages, if user B invites offline user A to join a group chat, user B's invitation will be sent to the Edge Node serving user A. The Edge Node will cache the request. When user A comes online, he will pull the offline invitation from the Edge Node and join the group chat.

This is illustrated in Figure 3.

Figure 3: Offline messaging

2) File Storage

Pictures, attachments, and other files sent by users during a chat will be cached on the Edge Node. The Edge Node's file storage is based on the IPFS service. Its basic process is shown in Figure 4, and described below:

1. First, the sender packages the file content and attribute information, including file name and md5 hash, and publishes them locally through IPFS and obtains the IPFS Content Identifier (CID).

2. Then, the sender sends a file storage request to the Edge Node. After receiving the request, the Edge Node pulls the file using its CID, pins the file to the local storage, and informs the sender that the file was successfully stored.

3. Finally, the sender packages the Edge Node's information and CID into a message and sends it to the target chat room. After other members receive the message, they can use the CID to download the file anytime and anywhere using the IPFS protocol.

There is still ongoing work to improve the storage function. We will include additional storage methods in the future to suit different needs.

Figure 4: Verifiable File Storage

3) User Profile Storage

In SDN, personal profile information includes DID documents, wallet information, chat room information, etc. Such information is stored publicly in the Edge Node and can be obtained by other users when needed, regardless of whether the owner is online or not. For example, when a user verifies the signature of the sender of an offline message, the Edge Node is queried for the sender’s public key.

Message relay

An Edge Node needs to take on the function of message forwarding during peak traffic times. The user's client is often unable to send messages quickly due to bandwidth limitations. Also, it is unable to maintain a large routing table, requiring messages to be forwarded more times than necessary to reach their destinations. An efficient Edge Node can forward user messages quickly and efficiently, thus improving the quality of service.

To address this issue, users can offload the server forwarding function for chat services to an Edge Node to improve the quality of service. Specifically, the Edge Node takes on the following functions

1) Intranet penetration

In SDN, messages are sent and received through a decentralized P2P network. Two different intranet users or users whose networks are not accessible from each other need access to servers – Edge Nodes, as shown in Figure 5 – with fixed addresses for message relaying. The underlying P2P protocol used by SDN already provides support for the relay function. Therefore, users only need to turn on the relevant settings of the Edge Nodes to which they are connected to set it as the message-forwarding agent and realize the successful delivery of messages across network boundaries. In addition to normal text messages, the Edge Node can also act as a WebRTC streaming server to provide relay services for voice and video calls.

Figure 5: Message relaying across network boundaries

2) Group chat optimization

Usual P2P protocols deliver messages to peers by relaying these through multiple nodes. This leads to higher network traffic overhead and latency compared to centralized protocols. It can even lead to broadcast storms in some cases.

SDN optimizes the network topology for the broadcasting of group chat messages. Group chat messages are processed and delivered mainly among specified Edge Nodes, with other Edge Nodes serving only in an auxiliary role to ensure network security and liveliness. This avoids too much repetitive flooding of message delivery and reduces the burden on other network nodes.

In addition, users or organizations may deploy their own Edge Nodes and solely rely on such specific nodes for messaging. Users may deploy their Edge Node inside the firewall for security concerns so that all internal Client Nodes communicate safely in the intranet but are still able to reach the entire network through the privately hosted Edge Node. They can also designate these trusted, dedicated Edge Nodes to handle group chats, which then assume a role similar to a centralized server. It will forward all group chat messages from its users, which greatly improves the efficiency of group chats.

3) Notification

When a user is offline, such as when the phone is locked, the OS needs to notify the user that a message has been received. In the decentralized scenario without a central server, this feature cannot be implemented just by the client participating in the chat. Therefore, a user needs to select their corresponding Edge Node (which can be the same one) for each chat relationship, including group chat and single chat, and receive notifications to the user's device from that Edge Node. For third-party software integrated with SDN, the Edge Node can send the information to the push server of the third-party software provider, with the push function implemented by that server.

Peer Discovery

Similar to a general P2P protocol, SDN uses the Kademlia protocol based on discrete hash tables to maintain the connection relationships between nodes. Given the characteristics of instant messaging scenarios, we have made some adjustments, as described below.

When a new node attempts to join the SDN, it obtains a list of long-running stable nodes in the network from the bootstrap node and establishes connections to these nodes. A bootstrap node is a node in an overlay network that provides initial configuration information to newly joining nodes so that they may successfully join the overlay network. SDN officially provides some default bootstrap nodes. Users can access their IP addresses directly or resolve them through the DNS service. Users can also customize the bootstrap nodes in the source code as needed, for example, in private deployments.

After establishing a connection with the nodes in the SDN network, the user exchanges the list of known nodes with them and selects his own inbound nodes from them. An inbound node is a node that has joined the network with an extranet address. When an Edge Node is deployed, it broadcasts its own Edge Node label to the entire SDN network through the P2P protocol. Client Nodes can search for Edge Nodes in the whole network using the Edge Node's public label, and select a number of them as Edge Nodes through a random algorithm. The selected Edge Node will then broadcast the information of its proxy nodes and mapping relationships to the whole network.

Like usual P2P protocols, nodes in SDN remove chronically inactive nodes from their peer list in order to conserve network resources. The difference is that the inbound node persists in the connection relationship with the user it belongs to. Therefore, after selecting their inbound node, users do not have to stay online for a long time to maintain the network relationship, but only need to pull from the inbound node when they are online.

Figure 6: Edge Node selection

6. Conclusions

Sending.Network has been designed as a fully decentralized, secure and high-performance communication network for Web3 applications. Edge Nodes, connected in a P2P network, provide critical functions such as message caching, storage and routing to Client Nodes, whose performance, storage capacity and status are variable and often unknown. The SDN P2P protocol ensures that applications using this infrastructure can work in both intranet deployments as well as across network boundaries.