We thought we would share a brief pre-dev-update update
Since Thursday’s update and test, the team’s primary focus has been looking at ways to reduce the amount of network traffic we saw between nodes and also to allow for data to be gradually relocated around the network, rather than instantly when we lose a node. We have specific areas that we are concentrating on and once these are in place we shall internally test then release to the community for the next round of testing.
I shall briefly give an overview of the areas the team have been / are currently working on:
- The ability for the network to remain functional while lots of nodes are starting or stopping at the same time. This was addressed and implemented on Friday and reduces network fragmentation.
- Allow groups to repair by stopping accumulating particular messages when a group goes below quorum. Disabled for just now and the security side is being investigated.
- Churn tweaks, specifically handling data more sequentially in order to reduce node traffic. Currently being reviewed and implemented today.
- Streamline Routing Tables (RT) - to drop connections that are no longer required. This is achieved through mutual agreement between nodes - send a message to a node in my own RT that I no longer need, telling them I no longer want you in my RT and if that node agrees to drop me, a Crust disconnect is initiated. This needs to be agreed by both nodes. The motivation here is that the larger the RT = more messages, heartbeats, etc going through you, so having more relevant RTs will also reduce overall network traffic and increase efficiency. Testing today.
- Removing the threading block between Routing and Vaults, eliminating potential issues occurring from a backlog of events. Currently Routing handles its events (from Crust and Vault) in a different thread from where Vaults handles its events (from Routing). It turns out that Vaults needs a lot of information all of the time; that is only available in Routing’s event handling thread, so we are planning to have only one event handling thread.
So this is what the majority of the Dev team have been focusing on over the last few days and these will be the areas that we shall be closely monitoring for improvements during the next release iteration. Big thanks to everyone that took part in the rolling release test last Thursday, this real world testing helps us a lot and is really appreciated. Expect some quick changes and more short test phases like this to come.
Thanks guys - until tomorrow.