schmittlauch/paper_hashtag_federation
1
0
Fork 0
Code Issues 1 Pull requests Releases Wiki Activity

add Makefile for building expose

Browse source 
also adding pdf and html version of expose
This commit is contained in:
Trolli Schmittlauch 2018-12-22 22:59:38 +01:00
parent e53199e828
commit 87dd12c39b
3 changed files with 56 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

7
Makefile Normal file
View file

@ -0,0 +1,7 @@
expose: expose_html expose_pdf
expose_html: expose.md
pandoc -s -t html5 -o expose.html expose.md
expose_pdf: expose.md
pandoc -t latex -s --variable classoption="twocolumn" --variable classoption="a4paper" -o expose.pdf expose.md

49
expose.html Normal file
View file

@ -0,0 +1,49 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<meta name="generator" content="pandoc">
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">
<meta name="author" content="@schmittlauch@toot.matereal.eu">
<title>Decentralized Hashtag Search and Subscription for Federated Social Networks</title>
<style type="text/css">code{white-space: pre;}</style>
<!--[if lt IE 9]>
<script src="//cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7.3/html5shiv-printshiv.min.js"></script>
<![endif]-->
</head>
<body>
<header>
<h1 class="title">Decentralized Hashtag Search and Subscription for Federated Social Networks</h1>
<p class="author"><span class="citation" data-cites="schmittlauch">@schmittlauch</span><span class="citation" data-cites="toot.matereal.eu">@toot.matereal.eu</span></p>
<p class="date">22.12.2018</p>
</header>
<h2 id="motivation">Motivation</h2>
<p>With the final standardization of the ActivityPub<a href="#fn1" class="footnoteRef" id="fnref1"><sup>1</sup></a> protocol by the W3C, federated social networks seem to have gained traction again with several new social servers implementing it<a href="#fn2" class="footnoteRef" id="fnref2"><sup>2</sup></a>.</p>
<p>But even these current implementations still suffer from a limitation all social networks based on push-federation still have: They can not provide a consistent network-wide view on all public posts including a certain hashtag. Such a search currently only returns posts from the instance's<a href="#fn3" class="footnoteRef" id="fnref3"><sup>3</sup></a> local database which have been delivered to the instance anyways due to other subscriptions. This limits the user experience compared to centralized mainstream social networks like Twitter, Tumblr or Facebook.</p>
<p>For the Diaspora* network there exists an implementation of a centralized federation relay, where federation servers send all their public posts to and can register to receive all messages containing a certain hashtag. But this centralized approach is against the spirit of federation and decentralization as it forms a single point of failure and potential bottleneck.</p>
<h2 id="planned-work">Planned Work</h2>
<p>The push-federation principle of current federated social networks works on the basis that user identifiers always include their home instance. Thus the server responsible for managing subscription requests and delivering new posts to all subscribers is always known. But there is no such place for all messages containing a certain hashtag, as these can originate from any instance. In federated social networks it is even not necessary for all instances to know each other.</p>
<p>I plan to provide such responsible subscription points as an additional mechanism for ActivityPub-compatible federation servers:<br />
All participating instances form an overlay network using a Distributed Hash Table (DHT) as an addressing and routing mechanism. Responsibility for a certain hashtag is then distributed over the DHT's key space to the closest n nodes. This group of nodes is then the responsible point for managing subscriptions. Participating nodes send all messages containing a certain hashtag to these responsible nodes, which then forward the posts to all subscribing instances. The DHT is only used for delegating and discovering responsibility for hashtags, while the post delivery happens directly routed between instances and subscription nodes using normal ActivityPub push mechanisms.</p>
<p>Additionally to this subscription and relay mechanism I want to provide a method for instances to query posts under a hashtags even in retrospective and without subscribing. The reasoning behind this is that users should be able to first get an impression of a hashtag's post history before deciding to subscribe to it, and also be able to get posts published before their instance subscribed to that hashtag.<br />
For this, the nodes responsible for a hashtag also need to store the history of posts containing their hashtag and provide all posts from a requested time frame to a requesting instance.</p>
<p>Although a network of server instances will not have a high fluctuation of nodes, measures need to be taken to provide some redundancy of responsible nodes per hashtag. There also needs to be a mechanism for transfering the stored message history to new nodes.</p>
<p>One possible problem that might arise is balancing the load of &quot;hot-spot&quot; nodes responsible for popular hashtags. Real-world data has to be analyzed to estimate whether load balancing and storage distribution mechanisms are necessary even within a hashtag.</p>
<p>The system performance shall at least be able to handle the throughput present in existing centralised networks.<br />
For Twitter this means dealing with a global message rate of averagely 1620 posts per second, with peaks going up to 143,199 posts per second<a href="#fn4" class="footnoteRef" id="fnref4"><sup>4</sup></a>. But only an eventually-consistent view is needed, posts are allowed to need up to a few minutes to propagate through the network.<br />
These performance requirements are going to be evaluated at least by calculations or even by simulations of such a system. It needs to be evaluated whether relaying only references to a post creates a sufficient performance, or whether relaying the posts themselves is needed.</p>
<h3 id="security-objectives">Security Objectives</h3>
<p>As this approach shall only be used to federate public posts, confidentiality is not a big issue. It is worth to be evaluated though how good deletion/ recall of posts can work in such a network.</p>
<p>Posts are signed like in current ActivityPub implementations for providing a way to verify their integrity. But the more important aspect of integrity is that an attacker must not be able to deliberately gain the responsibility for a certain hashtag and then silently drop messages. The integrity of the message history itself needs to be ensured.<br />
At a further look, depending on the stored content a way to transparently exclude messages from a node's store due to legal reasons might be necessary.</p>
<section class="footnotes">
<hr />
<ol>
<li id="fn1"><p><a href="http://activitypub.rocks" class="uri">http://activitypub.rocks</a><a href="#fnref1">↩</a></p></li>
<li id="fn2"><p><a href="https://joinmastodon.org">Mastodon</a>, <a href="https://pleroma.social">Pleroma</a>, <a href="https:pixelfed.org">PixelFed</a> are examples for general social networks, with several special-interest software like FunkWhale adopting the protocol as well<a href="#fnref2">↩</a></p></li>
<li id="fn3"><p>a federation server domain<a href="#fnref3">↩</a></p></li>
<li id="fn4"><p><a href="https://blog.twitter.com/official/en_us/a/2011/numbers.html" class="uri">https://blog.twitter.com/official/en_us/a/2011/numbers.html</a><a href="#fnref4">↩</a></p></li>
</ol>
</section>
</body>
</html>

BIN
expose.pdf Normal file
View file

Binary file not shown.