schmittlauch/Hash2Pub
1
0
Fork 1
Code Issues 47 Pull requests Releases Wiki Activity
350 commits 7 branches 1 tag 5.2 MiB
Commit graph

234 commits

Author SHA1 Message Date
Trolli Schmittlauch 4aa4667a1d kChoices cost calculations for rebalance decisions 
- loop with load queries and cost calculations on whether to do a vs
  relocation
- actual relocation still missing though
- untested
 2020-10-10 04:35:24 +02:00
Trolli Schmittlauch 6aebd982f8 make RingMap an instance of Traversable 
- some examples tested out by hand, but not thorough test case or even
  QuickCheck coverage
 2020-10-07 19:24:15 +02:00
Trolli Schmittlauch 048a6ce391 modularise VS candidate load querying into own function 2020-10-07 16:17:45 +02:00
Trolli Schmittlauch 8bd4e04bcd bootstrapQueryId doesn't need a STM'd node state 2020-10-07 15:50:44 +02:00
Trolli Schmittlauch 0cb4b6815c start implementing k-choices rebalancing, considering 1 VS each run 
only loop implemented, rebalancing not implemented
 2020-10-07 00:43:06 +02:00
Trolli Schmittlauch b111515178 add config option for k-choices rebalance interval 2020-10-06 16:01:29 +02:00
Trolli Schmittlauch ecb127e6af k-choices cost calculation for departure cost 2020-10-05 22:48:56 +02:00
Trolli Schmittlauch 5ed8a28fde refactor vservers map to RingMap to be able to index it 
- in preparation for periodical rebalancing
- makes it possible to look up the next vserver for iterating through
  it, after refreshing the map in-between
- added some necessary RingMap functions
 2020-10-05 02:27:02 +02:00
Trolli Schmittlauch bb0fb0919a refactor request sender ID spoof check to suit k-choices 
- mostly refactored the checks into its own function
- now additionally check the vserver number limit
- refactoring to pass that limit to the checking function invocations
- closes #74
 2020-09-29 02:59:42 +02:00
Trolli Schmittlauch b2b4fe3dd8 change vserver ID representation type to Word8 
- performance improvement: avoid unnecessary representation and
  conversion from/to Integer
- part of hot path: with k-choices, all possible IDs are regularly
  generated and checked
- preparation for #74
 2020-09-29 02:06:31 +02:00
Trolli Schmittlauch c208aeceaa rename isJoined to reflect its scope on a single VS 
This should be enough to close #76, as it was only used in the scope of
a single LocalNodeState anyways.
 2020-09-29 00:45:15 +02:00
Trolli Schmittlauch 0ee8f0dc43 adjust joinOnNewEntreisThread to k-choices join 2020-09-29 00:34:11 +02:00
Trolli Schmittlauch 21ecf9b041 bootstrapQueryID: try all possible node IDs of a bootstrap node 
- closes #77
- when k-choices (#2) joining, try addressing each possible node ID of
  the bootstrap node until success
- bugfix: include correct target ID of node that shall respond in
  QueryID requests
 2020-09-28 00:56:15 +02:00
Trolli Schmittlauch 9a61c186e3 start restructuring joinOnNewEntries flow 
- overview comment on possible flow
- cache query
- doesn't compile yet
 2020-09-26 22:08:09 +02:00
Trolli Schmittlauch 1a0de55b8c integrate k-choices into tryBootstrapJoin flow 
part of #2
 2020-09-25 22:07:12 +02:00
Trolli Schmittlauch 7a87d86c32 k-choices error handling: detect empty joins, finer fail granularity 2020-09-25 02:04:34 +02:00
Trolli Schmittlauch 3b6d129bfc implement k-choices join functions 
large commit, contains a number of things:
- k-choices #2 cost calculation
- k-choices parameters
- adjusting ASN.1 network messages to contain all values required for
  cost calculation #71
- adjusting stats to contain required values
- k-choices node and vserver join functions
- placeholder/ dummy invocation of k-choices join
 2020-09-25 01:41:04 +02:00
Trolli Schmittlauch 62da66aade add runtime flag for enabling k-choices or not 
any value except "off" means on

contributes to #2
 2020-09-22 23:12:30 +02:00
Trolli Schmittlauch 1ed0281417 respond to QueryLoad requests 
closes #71
closes #72
contributes to #2
 2020-09-21 18:15:40 +02:00
Trolli Schmittlauch 499c90e63a stylish run 2020-09-21 02:23:06 +02:00
Trolli Schmittlauch 1a7afed062 finish restructuring fediMainThreads 
contributes to #34
 2020-09-21 02:22:46 +02:00
Trolli Schmittlauch 8e8ea41dc4 re-structure convergenceSampleThread to work on a RealNode and iterate over all joined vservers 
contributes to #34
 2020-09-21 02:18:28 +02:00
Trolli Schmittlauch 33ae904d17 re-structure cacheVerifyThread to work on a RealNode and iterate over all joined vservers 
contributes to #34
 2020-09-21 02:11:43 +02:00
Trolli Schmittlauch 68de73d919 re-structure fediChordMessageHandler to dispatch requests to the responsible vserver 
contributes to #34
 2020-09-20 21:19:55 +02:00
Trolli Schmittlauch 0ab6ee9c8f re-strucuture fediChordInit flow to also do the bootstrapping 2020-09-20 19:30:35 +02:00
Trolli Schmittlauch 12dfc56a73 fediChordInit returns a RealNode, manages vservers as map 
- contributes to #34
 2020-09-19 23:01:55 +02:00
Trolli Schmittlauch 9bf7365a2c include target ID in request to address individual vserver 
- necessary for dispatching the request to a certain vserver
- also refactored request sending to use a common `mkRequest`
- contributes to #2
 2020-09-19 20:41:58 +02:00
Trolli Schmittlauch 5e745cd035 only specify upper key bound when querying load 
As a querying node does not always know the lower bound of the queried
segment – determined by the predecessor – let the currently responsible
node provide that bound instead.

affects #71
 2020-09-19 15:37:41 +02:00
Trolli Schmittlauch 30bf0529ed send load query request, parse result and represent it 
- sending side of #71
- introduces SegmentLoadStats to hold the response data
- contributes to #2
 2020-09-19 02:05:29 +02:00
Trolli Schmittlauch 576ea2c3f6 calculate service load rates, interface for querying loads 
- define data type for load representation
- this representation can be queried from any Service (closes #72)
- loads are periodically calculated from measured rates (contributes to #2)
 2020-09-18 20:36:46 +02:00
Trolli Schmittlauch 7dd7e96cce conversion of RingMap to key-value list 2020-09-18 20:36:35 +02:00
Trolli Schmittlauch af27cded19 adjust payload parser naming for consistency and clarity 2020-09-18 20:36:35 +02:00
Trolli Schmittlauch 41aaa8ff70 parse ASN.1 representation of load querying 
includes tests
contributes to #71
 2020-09-18 20:36:35 +02:00
Trolli Schmittlauch eee40ce4fb add log messages for failed relays as well 2020-09-17 02:17:57 +02:00
Trolli Schmittlauch f5de7601bb do not store published posts for reducing memory consumption 2020-09-16 13:49:26 +02:00
Trolli Schmittlauch a2f268d374 improve logging: line buffering, time stamps 
contributes to #60
 2020-09-16 01:54:50 +02:00
Trolli Schmittlauch c036dea7f9 periodically purge expired subscriptions 2020-09-14 15:49:44 +02:00
Trolli Schmittlauch a0e7142a7d report number of subscriptions 2020-09-14 14:57:25 +02:00
Trolli Schmittlauch 3c28cde942 catch and print all Socket bind exceptions 2020-09-12 15:45:03 +02:00
Trolli Schmittlauch 1fc264a226 manage logging via file handle 
reason: `appendFile` combined with lazy evaluation lead to exhaustion of
open file descriptors, as each file is opened again for each write and
due to lazy evaluation is kept open multiple times.
 2020-09-12 12:37:41 +02:00
Trolli Schmittlauch da579a0756 decrease logging verbosity 2020-09-11 00:39:14 +02:00
Trolli Schmittlauch e12d8ef70a properly format stats log numbers: no e-notation 2020-09-11 00:39:14 +02:00
Trolli Schmittlauch 0f9727c05a log the post rates instead of the absolute sums 2020-09-10 22:41:06 +02:00
Trolli Schmittlauch 34ecdd66e1 make stats measurement delay configurable, take speedup into account 2020-09-10 21:23:33 +02:00
Trolli Schmittlauch 8f917130c4 tag normalisation includes lower case conversion 2020-09-10 13:14:48 +02:00
Trolli Schmittlauch 3ac89d301c bugfix: subscribe as default if not subscribed yet, when posting to a tag 2020-09-10 13:14:23 +02:00
Trolli Schmittlauch 3c76544afb launch background worker threads 2020-09-10 12:00:17 +02:00
Trolli Schmittlauch f8d30d0cc4 report post fetches to statistics 2020-09-09 19:55:34 +02:00
Trolli Schmittlauch 620e998876 report incoming relay posts to statistics 2020-09-09 19:25:48 +02:00
Trolli Schmittlauch 85d10f6773 report published posts to statistics 2020-09-09 18:50:55 +02:00