Improving the Availability and Reliability of the Relay Network

YakiHonne is committed to creating a censorship-resistant decentralized media. A sufficiently decentralized and immutable storage layer is key to achieving this goal. The relay network based on the Nostr protocol provides an excellent censorship-resistant storage solution. Relay serves as an intermediary in message storage and broadcasting, allowing users to self-host relays and to choose which relays to broadcast information freely. If a relay refuses service or shuts down, other relays can continue propagating the information(fiatjaf.2019). This mechanism turns shadow banning into a "whack-a-mole" game, making it nearly impossible to completely block users through a specific data source, thereby providing a space for free speech for many controversial topics and creators.

Despite providing a simple and effective architecture for social media censorship resistence, existing relay networks face two major challenges to scale and sustain. First, the relay network lacks incentives, without which the stability and availability of the relay network will suffer. The censorship-resistant network relies on numerous distributed and available relays and wider usage of relay to ensure free flow of information(Rabble,2024). Second, although the core function of a relay is to store and distribute information in a decentralized manner, they cannot guarantee the immutability of information. Therefore, relay nodes are able to manipulate or delete information.

In this article, we will focus on how to solve these two issues by introducing economic incentives and attestation mechanism, thereby enhancing the availability and reliability of the relay network, ensuring that it provides solid support for decentralized media.

Nostr Protocol is Decentralized

The design of Nostr involves the separation of user accounts, relays, and clients from each other, free from any entity's control and censorship. Users can host relays, and content can be stored and retrieved across multiple relays. Even if certain relays refuse service or shut down, other relays can still store and propagate information. This differs from Fediverse applications such as Mastodon and Bluesky, and is certainly different from "Web3" social media protocols like Farcaster.
The design of Nostr involves the separation of user accounts, relays, and clients from each other, free from any entity's control and censorship. Users can host relays, and content can be stored and retrieved across multiple relays. Even if certain relays refuse service or shut down, other relays can still store and propagate information. This differs from Fediverse applications such as Mastodon and Bluesky, and is certainly different from "Web3" social media protocols like Farcaster.

  • In Mastodon, user accounts are tied to servers controlled by administrators, thus instance owners can ban users and have the authority to block other instances, implementing censorship (Rozenshtein,2023).
  • Although Bluesky promotes itself as an open and decentralized network, it is misleading(Fiatjaf:Bluesky, 2023): Bluesky directly controls the atproto protocol, which allows Bluesky to change the protocol at any time; Bluesky's identity system relies on a central server to maintain and authenticate global IDs, allowing Bluesky to control and potentially ban any user; even if users can host their own content, all content must be distributed through Bluesky's central server. Moreover, Bluesky's design does not encourage or support effective interoperability with other clients. This means if users are dissatisfied with Bluesky, their options are very limited.
  • Farcaster relies on large Hubs to store all user data, and these Hubs will grow increasingly larger. These Hubs' power to censor and disseminate data cannot be underestimated. As the network expands, storage demands and costs surge dramatically. It is estimated that if Farcaster's daily active users grow by 5% per week, the cost of running a Hub will reach $3,500 per year by 2024, and will soar to $6.9 million per year by 2027 (Varun,2022). The high operating costs mean that only a few companies can manage Hubs, leading to a decrease in the number of Hubs and an increased risk of network centralization. Additionally, Hub operators may collude to lower the priority of certain content or to censor it (Varun,2022). In contrast, Nostr encourages a mix of large Hubs and smaller relays built for specific purposes,as shown in Figure 1. These small relays can be established by large publishers, small organizations, or hackers, maintaining the network’s decentralization and openness (Hodlbod:Outbox,2024).
Figure 1. Farcaster Hub vs Nostr Relay

This is indeed the case. Gareth Tyson et al., using a dataset from July 1, 2023, to December 31, 2023, comprising 17.8 million posts, 1.5 million pubkeys, and 712 relays, analyzed the decentralization of Nostr and found that the distribution of posts and users on relays and relay hosting exhibited a high degree of decentralization. This demonstrates that the technical architecture of the Nostr protocol is superior to that of all existing decentralized media protocols.

  • Posts and users are not highly concentrated on individual relays but are widely distributed, As shown in Figure 2. 93% of posts can be found across multiple relays, with 178 relays, or 25% of all relays, hosting more than 5% of the posts each. According to user count statistics, even if the top 50 relays were shut down, 90% of the content would still be accessible. Similarly, based on post count statistics, shutting down the top 30 relays would still maintain accessibility to over 90% of the content. Even removing the top 50 relays would still leave 71% of the content accessible, as shown in Figure 3.1.
Figure 2. The percentage of relays, posts, and users in the top 15 regions and ASes, ranked by the number of relays. Source: Gareth Tyson et al. (2024), "Exploring the Nostr Ecosystem: A Study of Decentralization and Resilience," arXiv preprint arXiv:2402.05709.
  • The decentralization of relay hosting across regions and autonomous systems(AS). Relays are distributed across 50 countries and 151 autonomous systems(ASes), as shown in Figure 4. Surprisingly, no single country or autonomous system hosts over 25% of relays. Over 80% of posts remain available after removing the top 10 ASes, as shown in Figure 3.2. Taking Mastodon as an example, post availability drops to less than 10% after removing the top 10 ASes hosting instances (Raman et al., 2019). This is mainly due to the more even distribution of relays across different ASes, making them more resilient to failures in individual AS.
Figure 3.1 Top X Relays Removed; Figure 3.2 Top X ASes Removed. Source: See Figure 2.
Figure 4. The distribution of Relay numbers across different countries. Data from Nostr.Watch

Analysis of Relay Availability

Relay is the soul of the Nostr protocol's decentralization. To build a truly usable censorship-resistant relay network, two conditions must be met: relay nodes must be sufficiently distributed and available to ensure the free storage and dissemination of information; even small-scale relay nodes should be widely discovered and utilized.

As of April 23. 2024, there are only 639 relays online globally, a two-thirds reduction from the same period last year, predominantly distributed in North America and Europe, which together host 80% of these relays. Additionally, due to differences in network conditions, the performance of relays varies significantly across regions. For instance, tests in Singapore have shown notable differences in response times among relays in Asia, North America, and Europe, as shown in Figure 5. Moreover, a pronounced head effect is evident , with the top relay hosting 73% of the posts. Although these posts are available across multiple relays, Nostr remains highly decentralized (fiatjaf:Nostr,2024). However, this concentration of usage does not favor the wide discovery and use of smaller relay nodes or the visibility of users, reducing the incentive to build small relay nodes, especially in an ecosystem lacking incentives.

Figure.5 Relay availability testing. Data from Nostr Watch.
  • The reduction in the number of relays and their instability are primarily due to the lack of effective economic incentives (Shinobi:Nostr Scale,2023). Within the Nostr ecosystem, because clients often lack stable income or financial support, it becomes difficult to provide effective incentives for relays. Most relays rely on personal interest or restrictive paid models to maintain operations. These paid models limit specific users' write or even read access, contradicting the initial anti-censorship intent and weakening economic interactions between clients and relays. Currently, 95% of relays struggle to cover operational costs, and 20% have experienced significant downtime due to lack of financial support (Gareth Tyson et al., 2024).
  • The discovery and follow mechanisms of relays cause those with higher usage to be more easily discovered by users. These relays are often operated by well-known clients or developers, thereby attracting more users. Ensuring that more small relays are widely discovered and used is a key factor in maintaining the censorship resistance and activity of the relay network. The Nostr ecosystem is working to improve the discovery and follow mechanisms of relays through the Gossip Model, Outbox model, and Blastr. The challenge or key aspect of these models is optimizing the discoverability and coverage among relay users without over-replicating and redundantly retrieving posts, ensuring broad content dissemination (Hodlbod:Outbox,2024). However, achieving this goal requires better collaboration and consensus among Nostr developers. Currently, due to the lack of sufficient incentives, Nostr developers focus more on their own client design ideas, neglecting efficiency and compatibility with various existing user types, and even causing confusion in the development of other clients.

Potential Incentive Measures for Relays

The key to resolving the availability issues of the relay network lies in clearly identifying who will continuously pay for its operational costs. Only when the relay network can be profitable or at least cover its operational costs can it maintain long-term scalability and prosperity. The primary cost of operating a relay comes from server storage expenses. To ensure content availability, content is simultaneously published and repeatedly retrieved across multiple relays(Shinobi:Relay,2023), which further increases traffic consumption and operational costs. However, this is crucial for decentralization and ensuring data reliability. Currently, the main sources of income for relays are donations and paid posts, but 95% of relays struggle to sustain their operational costs through publishing donations. Therefore, this section will primarily explore potential solutions to cover the operational costs of relays.

1. Client Pays for Storage Costs

Clients are encouraged to create paid products or cover relay’s storage costs through their financial budgets. This approach helps in exploring the diversity of monetizing decentralized media and facilitates the formation of economic consensus between relay and its clients, thereby establishing incentive mechanisms for relay, as illustrated in Figure 6.

Figure 6. Incentive mechanisms between clients and the relays

In a previous article, we discussed two pillars of decentralized media of YakiHonne: decentralized publishing and decentralized review. The former guarantees that content will never be lost; the latter creates a new cost-incentive model to ensure that when content and moderation become permissionless, the platform can still maintain truthfulness and cost-effectiveness. Currently, YakiHonne supports various forms of decentralized publishing, including articles, flash news, curations, videos, and uncensored notes. Content review through uncensored notes further promotes YakiHonne's monetization and decentralization. In YakiHonne, publishing flash news requires a minimum payment of 800 sats, with some revenue used to incentivize relay operations and support uncensored notes. For long-form content, payment of relay's storage fees can be made through subscriptions, advertisements, or even the client's financial budget, as shown in Figure 7.

Figure 7. Incentive mechanisms between YakiHonne and the relays

2. Direct Income for Relays

In Nostr, the development of relays focuses more on performance optimization rather than the development of new features. Relays could potentially enhance revenue by offering specific relay functions and storing various types of events. Earning income through specific functions requires widespread adoption by clients and relays, otherwise, it may lead to the risk of centralization or implementation failure (Hodlbod: relay Function,2023 ). Earning income by storing different types of events could be a viable option. While maintaining openness to various content, relays can use keyword filters during data retrieval to display specific topics, thus promoting a subscription-based payment model. Additionally, relays and clients can form revenue-sharing agreements on subscription fees, which not only helps increase income but also promotes economic consensus between the two parties.

Attesting content on relays

The redundancy in content storage ensures high availability of content but does not fully guarantee its immutability. In practice, it is possible for relays to tamper with contents or delete information by influencing certain relay nodes. To address this issue, we need to introduce a complementary attestation mechanism to enhance the reliability of the current relay systems.

There are various methods to implement this proof. As the most decentralized social media base-layer protocol, Nostr is well recognized within the Bitcoin community. However, to really use the Bitcoin network to validate information in technical sense has not been disccussed yet. If content hosted on the Nostr network should ever be attested, it should happen on the Bitcoin network.

To attest and validate large amount of content on the Nostr relay network, it's not economically viable to attest every post / every NIP-23 article on the Bitcoin network. Data should be compressed and organized efficiently before being attested. The merkle tree is an efficient and secure data verification mechanism. The accuracy of the data can be verified using only the merkle root and the related hash paths, as shown in Figure 8. Therefore, a final attestation over a large amount of content can be one merkle root submission to the Bitcoin network. Once submitted, all content represented under this merkle root becomes immutable.

Figure 8. Hashed content on Merkle Tree

When it is necessary to verify whether the content stored on a relay has been unaltered, the content is first retrieved from the relay and its hash value recalculated. Then, using this hash value and other related intermediate hash values, verification proceeds up the Merkle tree until reaching the Merkle root recorded on the Bitcoin blockchain. Finally, by comparing this computed Merkle root with the root hash recorded on the blockchain, the integrity and authenticity of the content are confirmed. The process is depicted in Figure 9.

Figure 9. Attesting content on relays

Repeating this process will eventually make sure all attested content is genuine. As a result, relays will provide content availability and redundancy, while the Bitcoin network validate Nostr content in batches.

Conclusion

The relay network is one of the core infrastructures that YakiHonne uses to build decentralized media. Through data analysis of Nostr's decentralization features, it has been found that Nostr's technical architecture has made it the most decentralized media protocol currently available. However, due to the lack of effective economic incentives, the relay network faces challenges in availability. This article explores several potential economic incentive models for relays. Despite many uncertainties and challenges, the importance of sustained incentives in combating censorship networks is clear. Additionally, considering the current lack of immutability in the relay network, we propose an attestation mechanism to enhance the reliability of the relay system. Improving relays reliability and availability will help promote the scalability and prosperity of the entire relay ecosystem, providing solid support for building decentralized media.