schmittlauch/Hash2Pub
1
0
Fork 1
Code Issues 47 Pull requests Releases Wiki Activity
350 commits 7 branches 1 tag 5.2 MiB
Commit graph

234 commits

Author SHA1 Message Date
Trolli Schmittlauch 96c1963a4f actually check own responsibility for tags before accepting posts 2020-08-11 00:16:10 +02:00
Trolli Schmittlauch 7036867ae0 implemented first Servant client query 2020-08-03 22:50:48 +02:00
Trolli Schmittlauch 20e51ecca4 define API client functions 2020-08-02 14:59:03 +02:00
Trolli Schmittlauch 89706f688a server endpoint for tag unsubscription 2020-08-01 11:18:16 +02:00
Trolli Schmittlauch 7280f251b5 server endpoint for tag subscription 2020-08-01 11:00:38 +02:00
Trolli Schmittlauch 7d7fa3b52a fix haddock parsing 2020-07-31 17:49:52 +02:00
Trolli Schmittlauch 50044673a6 server endpoint for tag-post delivery 2020-07-31 17:46:33 +02:00
Trolli Schmittlauch e3c7faa80b properly initialise RealNode with service and vserver data, set up reference 2020-07-31 16:54:19 +02:00
Trolli Schmittlauch 5ffe1b074e add reference from RealNode to Service 
This required to make both RealNode(STM) and LocalNodeState(STM) parameterisable
polymorphic types
 2020-07-30 02:19:52 +02:00
Trolli Schmittlauch 4bf8091143 fix type signature of fediChordInit 2020-07-30 01:30:42 +02:00
Trolli Schmittlauch 98ca0ff13e service config, integrate service launch into DHT launch 
TODO: hold a reference from DHT to service
 2020-07-30 01:23:03 +02:00
Trolli Schmittlauch da47f8062f add lease time to subscription entries 2020-07-29 23:06:07 +02:00
Trolli Schmittlauch ad52a017aa add relay inbox endpoint 2020-07-29 22:15:14 +02:00
Trolli Schmittlauch bd70e2dff0 implement multiple post fetch (with placeholder content) 2020-07-29 00:06:27 +02:00
Trolli Schmittlauch 63bc06a88e implement post fetch (with placeholder content) 2020-07-28 23:45:21 +02:00
Trolli Schmittlauch 970c94ff0d set up subscription data structures and transfer subscription endpoint 2020-07-28 23:17:26 +02:00
Trolli Schmittlauch 3b65757406 worker thread for processing incoming posts in background, started together with web server 2020-07-28 02:12:15 +02:00
Trolli Schmittlauch 736815ea83 normalise hastag unicode representation of incoming posts 2020-07-27 21:49:49 +02:00
Trolli Schmittlauch daae9d0b38 process and enqueue incoming posts 2020-07-27 21:39:49 +02:00
Trolli Schmittlauch 04423171fd define data types for post and subscription storage 2020-07-27 13:20:15 +02:00
Trolli Schmittlauch 7878c67635 adjust rest of code to refactored RingMap 2020-07-27 00:37:31 +02:00
Trolli Schmittlauch 988144e9e7 further relax constrains on RingMap 
key now needs to be explicitly given at insert, instead of
deriving it from the value. This makes it possible to store values where
a key cannot be extracted from (HasKeyID)

contributes to #62, #32, #41
 2020-07-26 18:55:55 +02:00
Trolli Schmittlauch 0d1551261b adjust code to refactored and parameterisable RingMap 
is a bit ugly due to FlexibleContexts being neede at several places
 2020-07-25 23:59:48 +02:00
Trolli Schmittlauch 9a20a60222 fix type constraints after RingMap refactor 2020-07-25 23:59:48 +02:00
Trolli Schmittlauch 1ff540fd68 refactor RingMap into own module 2020-07-25 23:59:48 +02:00
Trolli Schmittlauch d55c2f1f1b define typeclasses for interfacing between PostService and DHT 2020-07-25 23:59:40 +02:00
Trolli Schmittlauch 1a085f2fe0 define typeclasses for interfacing between PostService and DHT 2020-07-25 01:53:59 +02:00
Trolli Schmittlauch c05544aa5b able to start a web server and run the mock API handlers 
contributes to #41, #32
 2020-07-23 18:29:52 +02:00
Trolli Schmittlauch f1697e5fd7 re-structure post API, placeholder handlers 
contributes to #41, #32
 2020-07-23 16:25:14 +02:00
Trolli Schmittlauch 5fedd9f87c specify post relay API endpoints 
contributes to #41, #32
 2020-07-23 00:09:45 +02:00
Trolli Schmittlauch e2ecf0739d maybeEmpty can be replaced by Data.List.listToMaybe 2020-07-22 02:56:30 +02:00
Trolli Schmittlauch ebc0d54ddc periodically purge lookup cache of expired entries 
closes #24
 2020-07-16 00:24:44 +02:00
Trolli Schmittlauch 0ea5402231 improve documentation exports 2020-07-15 01:48:27 +02:00
Trolli Schmittlauch c37fe88b35 store and initialise lookup cache 
contributes to #24
 2020-07-15 01:44:47 +02:00
Trolli Schmittlauch 42af6afb86 generalise CacheEntry type 
contributes to #24
 2020-07-15 01:32:37 +02:00
Trolli Schmittlauch 7fa6db81de make sure send sockets are bound to the correct local IP 2020-07-11 19:51:03 +02:00
Trolli Schmittlauch 61ea6ed3ff Periodically contact bootstrap nodes for convergence sampling or joining 
closes #56
 2020-07-08 01:18:53 +02:00
Trolli Schmittlauch 56ca2b53cc refactor trying to join on any bootstrap node into own function 
contributes to #56
 2020-07-07 18:07:25 +02:00
Trolli Schmittlauch d293cc05d1 data structure for RealNode holds common config and management data 
contributes to #56, #34, #2
 2020-07-07 17:36:12 +02:00
Trolli Schmittlauch df7423ce2e fixup! iteration limit for QueryID lookups 2020-07-05 18:18:51 +02:00
Trolli Schmittlauch e91f317a8e decrease logging verbosity 2020-07-05 16:52:09 +02:00
Trolli Schmittlauch 7c17e3a44d implement join-retry on new cache entries 
closes #42
 2020-07-05 16:48:57 +02:00
Trolli Schmittlauch bdb92411c6 iteration limit for QueryID lookups 
- closes #52
 2020-07-02 03:34:40 +02:00
Trolli Schmittlauch e06c53ff7c stylish fixes 2020-07-02 01:38:51 +02:00
Trolli Schmittlauch edf66e1b51 add som debug prints 2020-07-02 01:36:31 +02:00
Trolli Schmittlauch f6481996d7 Change default response to QueryID for unjoined nodes 
reason: allow unjoined nodes to find bootstrap node in local cache,
while always advertising itself to incoming requests for bootstrapping.
 2020-07-02 00:54:14 +02:00
Trolli Schmittlauch 8d349212b4 prevent cache invariant querying when not joined 2020-07-01 18:24:33 +02:00
Trolli Schmittlauch abbe664ca1 fixup! make sure that own node isn't added as neighbour 2020-06-30 00:26:26 +02:00
Trolli Schmittlauch eab5a73ead make sure that own node isn't added as neighbour 2020-06-29 13:42:39 +02:00
Trolli Schmittlauch 6313a06a83 fix time conversion for threadDelay 2020-06-29 13:41:11 +02:00
Trolli Schmittlauch f7ed0ee8d8 check cache slice invariant for predecessor slices as well 
closes #30 \0/
 2020-06-27 16:23:16 +02:00
Trolli Schmittlauch 2c3ef44064 check cache invariant for successors and lookup missing IDs 
first half of #30
 2020-06-27 16:06:43 +02:00
Trolli Schmittlauch 280d928ad7 Refactor requestQueryID to be able to send a single request 
in preparation for #30
 2020-06-27 16:06:15 +02:00
Trolli Schmittlauch 7f5dac55ea close #29: periodic cache maintenance 
periodically delete expired cache entries, check unverified ones and
potentially use them as neighbour
 2020-06-25 01:24:38 +02:00
Trolli Schmittlauch 5e8cfb0ccd mark successfully Pinged nodes as verified 
for #29
 2020-06-24 22:27:46 +02:00
Trolli Schmittlauch c9783a10cf launch stabilise thread 
closes #44

although stabilise functionality is still untested
 2020-06-24 02:51:39 +02:00
Trolli Schmittlauch 81e346db4e update responding neighbours and delete unresponding ones from cache 
contributes to #44
 2020-06-24 02:48:41 +02:00
Trolli Schmittlauch 16b46a8b0b add some comments on stabilise 2020-06-24 02:12:21 +02:00
Trolli Schmittlauch 25f44f3a45 look up additional neighbours if necessary 
for #44
 2020-06-24 01:19:53 +02:00
Trolli Schmittlauch 111c1a299d refactored stabilise: use first responding neighbour 
contributes to #44
 2020-06-23 19:34:15 +02:00
Trolli Schmittlauch d5f502c05c ping potential neighbours before adding to list 
for #44
 2020-06-20 22:28:11 +02:00
Trolli Schmittlauch 0494ddd696 stabilise periodically 
contributes to #44
 2020-06-20 21:24:31 +02:00
Trolli Schmittlauch 3482876d9b send and parse Ping requests 
contributes to #29 #44
 2020-06-19 23:03:27 +02:00
Trolli Schmittlauch 00ff2bf071 refactor adding neighbours 2020-06-19 19:15:05 +02:00
Trolli Schmittlauch e898b80762 Merge branch 'refactorSuccessorsPredecessors' into dhtNetworking 2020-06-19 18:31:39 +02:00
Trolli Schmittlauch 3f42f98443 adjust lookup to RingMap, fix #48 
- change default lookup result when not joined to FOUND
- fix determining own responsibility #48
- adjust tests
 2020-06-18 23:08:20 +02:00
Trolli Schmittlauch da0b8626cb critical bug fix: use target ID for predecessor query lookup 2020-06-18 23:06:43 +02:00
Trolli Schmittlauch f27812bcf3 give up on providing type signature for takeEntriesUntil 2020-06-17 15:13:49 +02:00
Trolli Schmittlauch 43e4ab184e adjust cache entry insertion to usage of RingMap 
#48
 2020-06-17 14:53:36 +02:00
Trolli Schmittlauch fb164dea0a fix instance declaration of RingMap 2020-06-17 14:32:26 +02:00
Trolli Schmittlauch 7e08250f8c refactor setting successors and predecessors 2020-06-17 14:29:30 +02:00
Trolli Schmittlauch 2269357ed0 deleting RingMap entries, list conversion 2020-06-17 02:21:37 +02:00
Trolli Schmittlauch 6142ee61d7 WIP: implement adding, setting and taking RingMap entries. 
contributes to #48
 2020-06-16 23:51:53 +02:00
Trolli Schmittlauch 22a6becf6b fix all previously working tests 2020-06-15 16:41:11 +02:00
Trolli Schmittlauch 061bce2b08 adjust types to refactored RingMap NodeCache 2020-06-15 15:14:11 +02:00
Trolli Schmittlauch 6a98b5c6da fix RingMap function types 2020-06-15 13:53:06 +02:00
Trolli Schmittlauch b179357ab0 generalise NodeCache implementation to make it usable for neighbour nodes as well 
contributes to #48
 2020-06-13 21:41:32 +02:00
Trolli Schmittlauch 2739b47162 send stabilise to certain successor or predecessor 2020-06-12 15:49:12 +02:00
Trolli Schmittlauch 1204457a2a make sure that predecessors are smaller and successors are larger than node ID 2020-06-10 16:53:17 +02:00
Trolli Schmittlauch 2c98d8507d implement stabilise request sending and parsing 
contributes to #44
 2020-06-09 22:11:42 +02:00
Trolli Schmittlauch f15d83baff Refactor predecessors and successors to hold RemoteNodeStates 
- neighbour nodes need to be contacted reliably
- Only holding NodeIDs requires a cache lookup for getting hostname and
  port. This is brittle as the entry could've been purged from cache.
- refactored ASN.1 scheme, types and add/ sort/ remove implementations

closes #46
 2020-06-09 15:21:22 +02:00
Trolli Schmittlauch 67cba1b69b fixup! rename responseTo -> requestID to avoid partial record accessors 2020-06-08 23:18:12 +02:00
Trolli Schmittlauch 4e359775ec add some debug output prints 2020-06-08 20:31:22 +02:00
Trolli Schmittlauch 2c827ea326 rename responseTo -> requestID to avoid partial record accessors 2020-06-08 20:31:18 +02:00
Trolli Schmittlauch 3892dc91aa Merge branch 'mainline' into dhtNetworking 2020-06-08 18:58:23 +02:00
Trolli Schmittlauch 7c87a578d3 move Main executable to app/ closes #18 2020-06-08 18:57:53 +02:00
Trolli Schmittlauch 6699237243 respond to and handle QueryID requests 
closes #28
 2020-06-08 00:36:59 +02:00
Trolli Schmittlauch eec751584c make setPredecessors and setSuccessors enforce the ordering + size limit 
closes #47
 2020-06-07 23:45:28 +02:00
Trolli Schmittlauch 0a9b0547c6 forgot handling the successors and predecessors of the leaving node 
contributes to #28
 2020-06-07 00:21:14 +02:00
Trolli Schmittlauch e00da9b84f respond to and handle Leave requests 
contributes to #28
 2020-06-06 18:02:55 +02:00
Trolli Schmittlauch 53308080db respond to Stabilise requests 2020-06-06 17:31:20 +02:00
Trolli Schmittlauch 43d72128d2 respond to Ping requests 2020-06-06 17:05:54 +02:00
Trolli Schmittlauch e32f0c9146 process and respond to join requests 
- serialiseMessage now starts numbering parts from the first number it
  gets, to be able to continue responding after having ACKed previous
  parts

contributes to #28
 2020-06-06 13:45:18 +02:00
Trolli Schmittlauch 914e07a412 change NodeCache protection to STM 
- putting the NodeCache behind an IORef had been chose because those
  could've been read non-blocking
- the same is true for TVars. The performance characteristics are likely
  worse, but at the advantage of composability within STM monads
 2020-06-05 22:07:47 +02:00
Trolli Schmittlauch dc2e399d64 protect concurrent node state access with STM 
- for allowing concurrent access to predecessors and successors, the
  whole LocalNodeState is passed wrapped into an STM TVar
- this allows keeping the tests for the mostly pure data type, compared
  to protecting only the successor and predecessor list

contributes to #28
 2020-06-04 22:31:41 +02:00
Trolli Schmittlauch f42dfb2137 dispatch incoming requests to their response functions 
- contributes to #28
 2020-06-04 14:48:05 +02:00
Trolli Schmittlauch 0660bce299 acknowledge parts when receiving partial requests 2020-05-31 23:21:55 +02:00
Trolli Schmittlauch 88104de9bf periodically purge request parts 2020-05-31 21:07:40 +02:00
Trolli Schmittlauch bcd1c34c7c manage incoming request parts before handling 2020-05-31 01:14:09 +02:00
Trolli Schmittlauch 2542091379 adjust rest of code to new message structure 2020-05-30 13:52:06 +02:00
Trolli Schmittlauch f8d444d5b6 FediChordMessage: last part has falg instead of parts number in each msg 
Motivation: Including the number of parts in each message part requires
the total number of parts to be known in advance, making dynamic
responses based on the received data difficult
 2020-05-30 13:07:28 +02:00
Trolli Schmittlauch fea9660f80 WIP: start implementing incoming request handling 2020-05-30 01:10:56 +02:00
Trolli Schmittlauch 61818c58a9 main server thread structure 2020-05-29 17:39:35 +02:00
Trolli Schmittlauch b4ecf8b0aa catch and handle more join errors 2020-05-27 23:37:32 +02:00
Trolli Schmittlauch 6ff765c63e catch and handle bootstrap join errors 2020-05-27 18:59:38 +02:00
Trolli Schmittlauch 27e5c5f9ce bracket all socket-using operations to avoid resource leakage 2020-05-27 18:08:46 +02:00
Trolli Schmittlauch b1c5c5e5f4 try all bootstrap nodes until successfully joined 2020-05-26 20:54:02 +02:00
Trolli Schmittlauch 702684b1a9 split fediChordJoin into general purpose and bootstrapping part 2020-05-26 11:02:29 +02:00
Trolli Schmittlauch 43eb04dfea preliminary passing of bootstrap nodes in Main to fediChordJoin 2020-05-26 09:39:02 +02:00
Trolli Schmittlauch ad1465c5fe use global cache adding function 2020-05-26 08:55:44 +02:00
Trolli Schmittlauch e3bfa26ddb join request + large FediChord refactoring 
- implement sending of initial join request sending, response parsing
  and cache population (untested but compiles)
- refactor basic types and their functions into Hash2Pub.FediChordTypes
  to prevent import loops, leaving Hash2Pub.FediChord to contain the
  high level actions called from Main
 2020-05-25 22:03:24 +02:00
Trolli Schmittlauch d5841d13fd extract queryID message sending function so it can be used for first join 2020-05-22 22:12:38 +02:00
Trolli Schmittlauch e8091b0a29 change all function definitions to new NodeState types 
- adjust implementation

contributes to #20
 2020-05-21 23:48:36 +02:00
Trolli Schmittlauch fe673dc255 make NodeState a typeclasse, define setters and getters on its representations 
contributes to #20
 2020-05-21 23:40:10 +02:00
Trolli Schmittlauch 99a2b0ba09 add instance deriving instructions 2020-05-20 19:27:51 +02:00
Trolli Schmittlauch f6c252d314 sending a queryID request compiles (untested) 2020-05-20 18:55:24 +02:00
Trolli Schmittlauch beffab99a0 clarify counting of FediChordMessage parts 2020-05-19 19:50:36 +02:00
Trolli Schmittlauch c31baa3635 run stylish 2020-05-19 17:55:40 +02:00
Trolli Schmittlauch 8d18f952cd implement send-receive-acknowledge-retry loop for requests 2020-05-19 17:53:13 +02:00
Trolli Schmittlauch 0e6f126b3b Merge branch 'master' into dhtNetworking 2020-05-19 16:51:43 +02:00
Trolli Schmittlauch 84bcd676ae move protocol data definitions to own module to prevent cyclic imports 2020-05-19 16:30:56 +02:00
Hécate 41e999ed99 Linting integration 
This commit brings in an HLint configuration file
and several recommended modifications such as:

* End-of-line extra spaces removal;
* Import lines ordering;
* Redundant $ removal;
* Generalisation of ++ and map to <> and fmap;
* Preferring `pure` over `return`;
* Removing extraenous extensions.

And finally, a `stylish-haskell` helper script
that detects if code files are dirty. Can be useful for CI,
although manually calling it can be nice if you would rather
first implement then beautify.
 2020-05-19 12:42:06 +02:00
Trolli Schmittlauch 60c164dbb0 remove unnecessary directory level 2020-05-19 00:01:32 +02:00
Trolli Schmittlauch 8b01ad2f37 remove unnecessary directory level 2020-05-17 21:31:44 +02:00
Trolli Schmittlauch 21fef145a0 working project structure with tests 
- restructured project again: splitting into executable and Hash2Pub
  library
- placeholder test file is successfully compiled and run
 2020-05-17 21:31:43 +02:00
Trolli Schmittlauch 30cfed81ea restructure nix development environment, now with Hoogle 2020-05-17 21:31:43 +02:00
Trolli Schmittlauch 6329f815d1 Initial packaging as a Cabal package, in preparation for #2 
- also restructures shell.nix to parse cabal file instead of directly
  specifying dependencies
 2020-05-17 21:31:43 +02:00
Trolli Schmittlauch 3b3ee09575 fix Haddock documentation, contrinutes to #8 2020-05-17 21:31:43 +02:00
Trolli Schmittlauch 08932cf80a selectively export only needed constructors 2020-05-17 21:31:43 +02:00
Trolli Schmittlauch 4a89ffe25a improve documentation, contributes to #8 2020-05-17 21:31:43 +02:00
Trolli Schmittlauch 3f9452ab7e choose and implement data structures for node state and cache 2020-02-27 22:10:57 +01:00
Trolli Schmittlauch ce4cc385b8 rename whole project and make DHT implementation as a project specific module 2020-02-26 14:11:33 +01:00
Trolli Schmittlauch 6c5faa4fc6 rename DHT implementation as it deviates from the EpiChord paper 2020-02-24 22:56:03 +01:00
Trolli Schmittlauch e83710f10b Start implementing the Epichord DHT: NodeID type 2020-02-24 21:53:47 +01:00