Launch of a community safe network

tfa · May 19, 2019, 8:51pm

I was talking about max possible size used by an account. This size will not be reached by a regular usage of WHM. But it could be reached by a user storing some videos (that happened in the past).

bytes · May 19, 2019, 9:07pm

I wonder if there is a way to:

Determine how much space any given account has consumed.
Selectively grant more PUTs to accounts.

If so, perhaps someday the authentication/quota component could be modified to renew users’ PUT quotas periodically. So if an account is using under X megabytes, give them Y PUTs (up to a maximum of Z unused PUTs).

tfa · May 19, 2019, 9:18pm

This is certainly doable but this is not what is currently implemented and I want to use Maidsafe code as-is, so that users can directly download and use vault binaries provided by Maidsafe.

drirmbda · May 19, 2019, 9:24pm

It would be great for testing if we could spin up a private safenet using VMs at home. Even if that required 20 or whatever number of separate servers. Is anything preventing this?

neo · May 20, 2019, 1:12am

I was running a at home vault, but after needing to restart it I cannot join until a couple more nodes are up and running. It seems there is an issue with the resource test that requires a good number of nodes to be in the network if you don’t have Gb/s internet.

drirmbda · May 20, 2019, 1:27am

Thanks, I see, so the resource test is on workstation (hardware) level and each need to have a GbE port, connected to a GbE switch?

neo · May 20, 2019, 1:33am

Considering that a RPi satisfies the hardware tests, its really down to the internet connection and for some reason the resource test has very slow overall upload speeds compared to the link speed. But it seems that with enough nodes in the network the overheads are more a constant delay and thus with less data per connection the slower internet link speeds (eg 40Mb/s up) are able to connect. For me it is about 15 or 16 nodes needed to exist with a 35+ Mb/s upload (tested) speed.

drirmbda · May 20, 2019, 1:35am

I was thinking of starting my on-premise safenet without internet connection. Is that even possible?

neo · May 20, 2019, 1:41am

If you can have routing with different subnets for each vault (virtualbox can do this between VMs I believe) then it is easy enough by using current code.

If you cannot then there needs to be one change w/recompiling code, and that is to remove the limitation of one node per subnet.

@mav has done this with his performance tests

drirmbda · May 20, 2019, 1:47am

Thanks, then there is hope. A router on a stick config with a large managed switch and VLANs may be easier than re-building the software. It promises to be a major project though. If this materializes ever, I will post, but no promises.

mav · May 20, 2019, 2:49am

You don’t need to recompile, there’s a safe_vault.routing.config parameter “allow_multiple_lan_nodes”

github.com

maidsafe/sn_routing/blob/ba977bd494b69269481e47d61372b394de8a0885/sample_config/sample.routing.config

{
  "dev": {
    "allow_multiple_lan_nodes": true,
    "disable_client_rate_limiter": true,
    "disable_resource_proof": true,
    "min_section_size": 5
  }
}

Also you don’t need VMs you can just change the port number to run multiple vaults per machine or vlan.

You might want to adjust “chunk_store_root” in the vault config safe_vault.vault.config so each vault has their own unique storage directory.

https://github.com/maidsafe/safe_vault/blob/551d153c406eee09438a1634c8a3e14bf656aa4d/src/config_handler.rs#L24

Main difficulty is managing configs for lots of vaults.

Would be interested to hear your experience if you do end up doing this.

drirmbda · May 20, 2019, 3:02am

Thank you very much @mav, that would save a lot of time.
I will try to script all setup, and use containers probably (I like Sylabs.io’s Singularity a lot). I have noted down your ID, and I will start a new topic on this either after succeeding, or more likely after running into the first major issue.

drirmbda · May 21, 2019, 4:01am

Today I managed to build and run two vaults on a single Ubuntu 16.04 workstation. I had up to 5 running for a shorter while, but I need to figure out how to set different ports for each instance; I suppose ending up with only two makes sense.
Is the code the documentation or is there separate documentation?
(I wanted to create a how to but by the end of the day something had broken the build process already.)

drirmbda · May 21, 2019, 4:03am

By the way, if you want me to join the community network, I could try. Do I need an access key, and if so how do I request one?

mav · May 21, 2019, 4:09am

Depending on the version of the vault (which in turn implies which version of crust). Alpha-2 is vault v0.17.2 (git checkout 0.17.2)

The way to set the port is in with tcp_acceptor_port in the safe_vault.crust.config file:

github.com

maidsafe-archive/crust/blob/master/src/main/config_handler.rs#L26


      
          
          #[cfg(test)]
          use std::path::PathBuf;
          
          /// Crust configuration settings
          #[derive(PartialEq, Eq, Debug, Serialize, Deserialize, Clone)]
          pub struct Config {
              /// Direct contacts one should connect to
              pub hard_coded_contacts: Vec<PeerInfo>,
              /// Port for TCP acceptor
              pub tcp_acceptor_port: Option<u16>,
              /// Force usage of `tcp_acceptor_port` as our router mapped port. Normally if there is a port
              /// forwarding, crust will find out what the external world sees our local tcp acceptor
              /// endpoint as and include this information in our connection info that we share with others.
              /// However there are routers/firewalls in the wild which behave differently when a port is
              /// forwarded. They allow inbound connection through the forwarded port, but all outbound
              /// connections through the forwarded port get remapped to some ephemeral port. This prevents
              /// crust from knowing what the world sees our `tcp_acceptor_port` as because outbound
              /// connections get remapped although the port had been forwarded. In such scenarios, the user
              /// can specify this value as true, which will force crust to add the above `tcp_acceptor_port`
              /// to one of our externally reachable endpoint.

Then you can also set hard_coded_contacts to include any existing vaults as a list of "ip:port" strings.

drirmbda · May 21, 2019, 4:27am

Ah thanks, I will try it out next time. Wish I had more time. By the way, using the same default directory for storing vault data seemed to work ok.

tfa · May 21, 2019, 8:37pm

You don’t even need to do that. I guess the port the number is ignored on a local machine because using the same port number just works. Demonstration of several vaults on the same machine here in asciinema.

(this video was a demonstration that a network with “min_section_size”=5 doesn’t work properly, which is the reason why I had to take “min_section_size”=8 for community network)

Exactly. With this in addition to the unique port number, you can reuse the same directory to launch all the vaults, like I did in the video.

drirmbda · May 21, 2019, 11:32pm

@tfa Thanks. I will try to reproduce what you show on asciinema. My vaults did not behave as well, I guess because I used the same port number for each. I saw a lot of bootstrapping failures “Bootstrapper has no active children left”.

folaht · May 24, 2019, 8:37pm

I’m in

tfa · May 24, 2019, 9:45pm

Congratulations! You have succeeded in spite of your IPv6 restriction.

Topic		Replies	Views
"community1" Test Network is Alive, Join Us! Development test_network	152	8915	June 11, 2016
Dev Update & 2nd Test Network Release :safe: 8th March 2016 Updates	103	10475	March 20, 2016
MaidSafe Dev Update - December 6, 2016 Updates	40	6542	January 4, 2017
Launch of community network 2 Community	109	2744	June 23, 2020
"community1" Test Network Has Restarted, Join Us! Development test_network	143	5804	June 29, 2016

Launch of a community safe network

Related Topics