Vault monitor app

tobbetj · December 3, 2019, 9:03am

I have been learning some new abilities in python and got an idea that it would be great for probably a lot of vault users if the experience was as much “fire and forget” as possible. So I have been thinking of making an app which reports by choosen interval on, vault status, delivered data, earned safecoins and so on. I would also like the app to message the user, if the vault loses connection.

Some questions I think of are if there is like a JSON file which shows for example a boolean value if the vault is running/not running and if there are similar information of JSON data for example delivered data, earned safecoins and so on.

I would like to try and build an app that monitor and send email on a regular set interval, or if something goes wrong, so that the user who runs a vault don’t have to look every day on how the vault perform. I tried crypto mining a few months and the most annoying thing, after the fans of the gpu, was the feeling of the need to check several times every day how the mining performed.

@bochaco, @joshuef, @nbaksalyar, @lionel.faber

joshuef · December 3, 2019, 9:43am

nice one @tobbetj.

I’m not sure of the vault interface/options at the mo. It’s still being designed out AFAIK. Perhaps @nbaksalyar or @lionel.faber may have more details on what sort of info we can expect.

JimCollinson · December 3, 2019, 9:52am

It’s a bit fluid at the moment, but here are the event logs I was using as a starting point for the Vault UI designs, if it helps…

So together with stopped/connected you’d have:

tobbetj · December 3, 2019, 10:04am

Looks good, Jim, lots of good info there.

Can you DM me a log, if you have one, so I can take a look at it.
I was supposed to start a vault on Windows a few weeks back but then I thought that I should set up a raspberry pi instead, but then I got stuck in some coding. Long story short, I don’t have a vault running right now.

Thanks, @joshuef

JimCollinson · December 3, 2019, 10:05am

I don’t have a log I’m afraid, this is just a list derived from discussions. Perhaps someone else can furnish you with one.

bochaco · December 3, 2019, 12:18pm

I think the vault should be exposing an interface with methods for subscribing and receiving some type of notifications, plus methods to retrieve all kind of information available in the vault. If there is some type of event not supported by the vault notifications mechanism any app like yours could do a polling and send the notifications back to the user. It could be a json-rpc interface which is easy to implement in any language, but I also think that we need a vault client API so apps don’t deal with the communication protocol but just with an API.

lionel.faber · December 5, 2019, 6:01am

Thanks for your interest @tobbetj
This is a good idea and it’s certainly one that’s on our plate. Right now the vault binary is purely CLI. There are some logs you can generate by passing -vvvvv when starting the vault. While these logs can be parsed to get some kind of UI, I imagine down the line that there might me some kind of API that can be used to build a UI. Writing a vault UI is Rust is also a possibility That way we can avoid the API / FFI requirements.

However for MVP, the vault might be just a CLI. I’m not too sure though.

dask · December 5, 2019, 6:32am

Also very interested in this topic. A python interface to a vault api would be a great addition to a pysafe suite. Being able to run simple cron jobs or polls against the network in python would open up a ton of ways to participate for third parties.

tobbetj · December 5, 2019, 9:28am

Thanks so far for the participation in this topic.

@bochaco
My knowlege of different communication methods between apps and similar is very limited, at current time.

My original thought was to have a standalone executable app that reads a JSON file, if the user wants stats every day/week or when something goes wrong, then the app would use for example google gmail API and send a mail to the user.

The idea comes from reading about how server administrators operate, they don’t always wan’t to actively monitor servers and there for make apps that message them, stats about the servers and/or if something is out of order

This discussion have broaden my thinking to include building app on the SAFE-network that monitor vault and also maybe using something like future SAFE-mail API to message the user, or if it can be combined with a cellphone app then it might also be possible to make something like push notifications direct to the phone.

@lionel.faber
It is good that you guys will make UI in the future. By exposing an API or something you guys might also get the option to let the community build a UI, which maybe could simplify for you guys as well. I get the feeling that it would be good to have important parameters going as soon as possible when Fleming is released, not only for building UI or similar but also so that users can evaluate performance to see how well a vault works, if there are room for improvements or finding bugs or situations when a vault don’t perform as expected.

davidpbrown · December 5, 2019, 1:18pm

On my to do list is trying to figure out how to resolve the best balance of bandwidth; cpu; ram; storage… I don’t know if the vault stats will suggest its perception of the hardware available… and what is the limiting element.

I wonder that bandwidth will be a limiting factor… low cpu and ram perhaps and just a balance, where storage is cheap and easier to get than bandwidth??

For example, could easily have 98TB on one box 16GB ram; 4cpu; on 4G … or less of those and more bandwidth balancing, cf wherever the pinch is.

Seems like a solvable problem in time and good opportunity for finding the sweet spot.

dask · December 5, 2019, 1:52pm

Some good ideas in there… I was thinking a bit more like a real time (JSON-RPC would work) interface so it could be stuffed with some simple scripts in a docker container or something and simplify deploying more cloud safe nodes. It would be really cool if the vault could use safe-email to email within the safe network itself (although that wouldn’t help necessarily for connection issues).

bochaco · December 6, 2019, 1:14pm

Just to clarify @tobbetj that what I was describing with aligned to your original idea, perhaps the oly difference is that instead of your standalone app to read the vault info from a json file it obtains it using a JSON-RPC channel with the vault (similar to how you’d do it with a webservice/webserver). And because many devs would want to create apps like the one you describe, then instead of having them all implement the same code to communicate through JSON-RPC with the vault, we can have an API which provides you with that, so from your app you can simply do thinks like let stats_json = vault_client.get_stats();, or vault_client.subscribe(my_callback_function); then your my_callback_function function would be invoked every time there is new stats/info reported from the vault and you can then send the info by google email API, SAFE email, or show it in a big screen in Piccadilly Circus

tobbetj · December 6, 2019, 1:42pm

That sounds cool. Thanks for explaining, now I understand more what JSON-RPC do, I picture it like some back-end server call.

I will put JSON-RPC functionality on the christmas wishlist asap.

bochaco · December 6, 2019, 1:53pm

Ok, then I can go one step further to clarify a bit more , JSON-RPC is just how to format the messages you send back and forth with the vault, but you could send these messages using HTTP, UDP, traditional post , etc., so what I was proposing is we use QUIC for this (this is exactly what we’ve done for the Authenticator daemon). Now, we already started putting all JSON-RPC over QUIC functionality into its own crate when developing authd, and we’ll continue doing so that crate is fully functional and self-contained, which means we will be able to reuse this for the vault communication channels to retrieve stats and so forth, leaving us with a reduced amount of effort to create such a vault API. I hope it helps to, even if it doesn’t make it to your Christmas wishlist

mav · December 8, 2019, 10:56pm

This is a log from vault v0.20.1 (ie the current latest) with some logins uploads downloads happening. Might be good enough to get started with a parser. The vault was run with ververyveryverbose flag -vvvv

I’ve been wanting to do something like this for a while (lacking time) so am happy to collaborate or provide pointers if you want.

safe_vault/target/release ((0.20.1)) $ ./safe_vault -vvvv
INFO 2019-12-08T21:51:39Z [src/bin/safe_vault.rs:109] Checking for updates...
INFO 2019-12-08T21:51:40Z [src/bin/safe_vault.rs:138] Current version is 0.20.1
INFO 2019-12-08T21:51:40Z [src/bin/safe_vault.rs:139] No releases are available for updates
INFO 2019-12-08T21:51:40Z [src/bin/safe_vault.rs:88] 

Running safe-vault v0.20.1
==========================
TRACE 2019-12-08T21:51:40Z [src/utils.rs:37] Loading database at /home/user/.local/share/safe_vault/root_dir/auth_keys.db
TRACE 2019-12-08T21:51:40Z [src/utils.rs:37] Loading database at /home/user/.local/share/safe_vault/root_dir/balances.db
INFO 2019-12-08T21:51:40Z [src/client_handler.rs:106] QuicP2p started on 127.0.0.1:57800
with certificate [48, 130, 1, 80, 48, 129, 246, 160, 3, 2, 1, 2, 2, 1, 42, 48, 10, 6, 8, 42, 134, 72, 206, 61, 4, 3, 2, 48, 33, 49, 31, 48, 29, 6, 3, 85, 4, 3, 12, 22, 114, 99, 103, 101, 110, 32, 115, 101, 108, 102, 32, 115, 105, 103, 110, 101, 100, 32, 99, 101, 114, 116, 48, 34, 24, 15, 49, 57, 55, 53, 48, 49, 48, 49, 48, 48, 48, 48, 48, 48, 90, 24, 15, 52, 48, 57, 54, 48, 49, 48, 49, 48, 48, 48, 48, 48, 48, 90, 48, 33, 49, 31, 48, 29, 6, 3, 85, 4, 3, 12, 22, 114, 99, 103, 101, 110, 32, 115, 101, 108, 102, 32, 115, 105, 103, 110, 101, 100, 32, 99, 101, 114, 116, 48, 89, 48, 19, 6, 7, 42, 134, 72, 206, 61, 2, 1, 6, 8, 42, 134, 72, 206, 61, 3, 1, 7, 3, 66, 0, 4, 97, 136, 66, 181, 17, 7, 102, 78, 24, 16, 159, 85, 164, 38, 169, 44, 172, 20, 18, 49, 31, 143, 229, 239, 251, 201, 65, 79, 39, 237, 226, 105, 0, 173, 94, 148, 65, 91, 79, 201, 94, 59, 254, 178, 172, 171, 28, 251, 166, 119, 104, 100, 139, 54, 6, 39, 47, 203, 117, 83, 188, 25, 172, 92, 163, 27, 48, 25, 48, 23, 6, 3, 85, 29, 17, 4, 16, 48, 14, 130, 12, 77, 97, 105, 100, 83, 65, 70, 69, 46, 110, 101, 116, 48, 10, 6, 8, 42, 134, 72, 206, 61, 4, 3, 2, 3, 73, 0, 48, 70, 2, 33, 0, 140, 58, 249, 22, 202, 122, 56, 106, 5, 133, 238, 76, 130, 239, 181, 31, 240, 173, 185, 94, 208, 12, 3, 200, 144, 170, 155, 102, 82, 204, 154, 175, 2, 33, 0, 149, 208, 235, 85, 255, 105, 144, 92, 50, 58, 247, 247, 120, 207, 150, 221, 189, 18, 174, 252, 65, 141, 60, 121, 186, 212, 255, 141, 82, 51, 15, 237]
Our connection info:
{"peer_addr":"127.0.0.1:57800","peer_cert_der":[48,130,1,80,48,129,246,160,3,2,1,2,2,1,42,48,10,6,8,42,134,72,206,61,4,3,2,48,33,49,31,48,29,6,3,85,4,3,12,22,114,99,103,101,110,32,115,101,108,102,32,115,105,103,110,101,100,32,99,101,114,116,48,34,24,15,49,57,55,53,48,49,48,49,48,48,48,48,48,48,90,24,15,52,48,57,54,48,49,48,49,48,48,48,48,48,48,90,48,33,49,31,48,29,6,3,85,4,3,12,22,114,99,103,101,110,32,115,101,108,102,32,115,105,103,110,101,100,32,99,101,114,116,48,89,48,19,6,7,42,134,72,206,61,2,1,6,8,42,134,72,206,61,3,1,7,3,66,0,4,97,136,66,181,17,7,102,78,24,16,159,85,164,38,169,44,172,20,18,49,31,143,229,239,251,201,65,79,39,237,226,105,0,173,94,148,65,91,79,201,94,59,254,178,172,171,28,251,166,119,104,100,139,54,6,39,47,203,117,83,188,25,172,92,163,27,48,25,48,23,6,3,85,29,17,4,16,48,14,130,12,77,97,105,100,83,65,70,69,46,110,101,116,48,10,6,8,42,134,72,206,61,4,3,2,3,73,0,48,70,2,33,0,140,58,249,22,202,122,56,106,5,133,238,76,130,239,181,31,240,173,185,94,208,12,3,200,144,170,155,102,82,204,154,175,2,33,0,149,208,235,85,255,105,144,92,50,58,247,247,120,207,150,221,189,18,174,252,65,141,60,121,186,212,255,141,82,51,15,237]}

Writing connection info to: /home/user/.config/safe_vault/vault_connection_info.config
Waiting for connections ...
TRACE 2019-12-08T21:51:40Z [src/chunk_store.rs:79] Loading ChunkStore at /home/user/.local/share/safe_vault/root_dir/chunks/login_packets
TRACE 2019-12-08T21:51:40Z [src/utils.rs:37] Loading database at /home/user/.local/share/safe_vault/root_dir/immutable_data.db
TRACE 2019-12-08T21:51:40Z [src/utils.rs:37] Loading database at /home/user/.local/share/safe_vault/root_dir/full_adults.db
TRACE 2019-12-08T21:51:40Z [src/chunk_store.rs:79] Loading ChunkStore at /home/user/.local/share/safe_vault/root_dir/chunks/immutable
TRACE 2019-12-08T21:51:40Z [src/chunk_store.rs:79] Loading ChunkStore at /home/user/.local/share/safe_vault/root_dir/chunks/mutable
TRACE 2019-12-08T21:51:40Z [src/chunk_store.rs:79] Loading ChunkStore at /home/user/.local/share/safe_vault/root_dir/chunks/append_only
TRACE 2019-12-08T21:51:40Z [src/utils.rs:37] Loading database at /home/user/.local/share/safe_vault/root_dir/coins.db
INFO 2019-12-08T21:53:49Z [src/client_handler.rs:157] 37a7fd..: Connected to new client on 127.0.0.1:59748
TRACE 2019-12-08T21:53:49Z [src/vault.rs:190] Node(37a7fd..): Succesfully sent message to: 127.0.0.1:59748
INFO 2019-12-08T21:53:49Z [src/client_handler.rs:693] 37a7fd..: Accepted Client(Bls(ac900f..)) on 127.0.0.1:59748.
TRACE 2019-12-08T21:53:49Z [src/client_handler.rs:242] 37a7fd..: Received (Request::GetLoginPacket MessageId(4637ce..)) from Client(Bls(ac900f..))
TRACE 2019-12-08T21:53:49Z [src/vault.rs:190] Node(37a7fd..): Succesfully sent message to: 127.0.0.1:59748
INFO 2019-12-08T21:53:49Z [src/client_handler.rs:161] 37a7fd..: Networking error: Connection was actively cancelled
INFO 2019-12-08T21:53:49Z [src/client_handler.rs:163] 37a7fd..: Disconnected from Client(Bls(ac900f..)) on 127.0.0.1:59748
INFO 2019-12-08T21:53:49Z [src/client_handler.rs:157] 37a7fd..: Connected to new client on 127.0.0.1:34555
TRACE 2019-12-08T21:53:49Z [src/vault.rs:190] Node(37a7fd..): Succesfully sent message to: 127.0.0.1:34555
INFO 2019-12-08T21:53:49Z [src/client_handler.rs:693] 37a7fd..: Accepted Client(Bls(b3673e..)) on 127.0.0.1:34555.
INFO 2019-12-08T21:57:18Z [src/client_handler.rs:157] 37a7fd..: Connected to new client on 127.0.0.1:37632
TRACE 2019-12-08T21:57:18Z [src/vault.rs:190] Node(37a7fd..): Succesfully sent message to: 127.0.0.1:37632
INFO 2019-12-08T21:57:18Z [src/client_handler.rs:693] 37a7fd..: Accepted App(Bls(b35ce8..), owner: Bls(b3673e..)) on 127.0.0.1:37632.
TRACE 2019-12-08T21:57:18Z [src/client_handler.rs:242] 37a7fd..: Received (Request::PutIData MessageId(4f8445..)) from App(Bls(b35ce8..), owner: Bls(b3673e..))
TRACE 2019-12-08T21:57:18Z [src/client_handler.rs:1202] 37a7fd..: App(Bls(b35ce8..), owner: Bls(b3673e..)) is paying 0.000000001 coins
TRACE 2019-12-08T21:57:18Z [src/client_handler.rs:1177] 37a7fd..: Setting balance to 1000.109999980 for Bls(b3673e..)
TRACE 2019-12-08T21:57:18Z [src/data_handler.rs:82] 37a7fd..: Received (Request::PutIData MessageId(4f8445..)) from src b3673e.. (client App(Bls(b35ce8..), owner: Bls(b3673e..)))
TRACE 2019-12-08T21:57:18Z [src/data_handler.rs:82] 37a7fd..: Received (Request::PutIData MessageId(4f8445..)) from src 0c0597.. (client App(Bls(b35ce8..), owner: Bls(b3673e..)))
TRACE 2019-12-08T21:57:18Z [src/data_handler.rs:300] 37a7fd..: Received (Response::Mutation(Success) MessageId(4f8445..)) from 37a7fd..
TRACE 2019-12-08T21:57:18Z [src/client_handler.rs:844] 37a7fd..: Received (Response::Mutation(Success) MessageId(4f8445..)) to App(Bls(b35ce8..), owner: Bls(b3673e..)) from 0c0597..
TRACE 2019-12-08T21:57:18Z [src/client_handler.rs:242] 37a7fd..: Received (Request::PutIData MessageId(130f41..)) from App(Bls(b35ce8..), owner: Bls(b3673e..))
TRACE 2019-12-08T21:57:18Z [src/client_handler.rs:1202] 37a7fd..: App(Bls(b35ce8..), owner: Bls(b3673e..)) is paying 0.000000001 coins
TRACE 2019-12-08T21:57:18Z [src/client_handler.rs:1177] 37a7fd..: Setting balance to 1000.109999979 for Bls(b3673e..)
TRACE 2019-12-08T21:57:18Z [src/data_handler.rs:82] 37a7fd..: Received (Request::PutIData MessageId(130f41..)) from src b3673e.. (client App(Bls(b35ce8..), owner: Bls(b3673e..)))
TRACE 2019-12-08T21:57:18Z [src/data_handler.rs:82] 37a7fd..: Received (Request::PutIData MessageId(130f41..)) from src 5bd67c.. (client App(Bls(b35ce8..), owner: Bls(b3673e..)))
TRACE 2019-12-08T21:57:18Z [src/data_handler.rs:300] 37a7fd..: Received (Response::Mutation(Success) MessageId(130f41..)) from 37a7fd..
TRACE 2019-12-08T21:57:18Z [src/client_handler.rs:844] 37a7fd..: Received (Response::Mutation(Success) MessageId(130f41..)) to App(Bls(b35ce8..), owner: Bls(b3673e..)) from 5bd67c..
TRACE 2019-12-08T21:57:18Z [src/client_handler.rs:242] 37a7fd..: Received (Request::PutIData MessageId(457fe6..)) from App(Bls(b35ce8..), owner: Bls(b3673e..))
TRACE 2019-12-08T21:57:18Z [src/client_handler.rs:1202] 37a7fd..: App(Bls(b35ce8..), owner: Bls(b3673e..)) is paying 0.000000001 coins
TRACE 2019-12-08T21:57:18Z [src/client_handler.rs:1177] 37a7fd..: Setting balance to 1000.109999978 for Bls(b3673e..)
TRACE 2019-12-08T21:57:18Z [src/data_handler.rs:82] 37a7fd..: Received (Request::PutIData MessageId(457fe6..)) from src b3673e.. (client App(Bls(b35ce8..), owner: Bls(b3673e..)))
TRACE 2019-12-08T21:57:18Z [src/data_handler.rs:82] 37a7fd..: Received (Request::PutIData MessageId(457fe6..)) from src b803d1.. (client App(Bls(b35ce8..), owner: Bls(b3673e..)))
TRACE 2019-12-08T21:57:18Z [src/data_handler.rs:300] 37a7fd..: Received (Response::Mutation(Success) MessageId(457fe6..)) from 37a7fd..
TRACE 2019-12-08T21:57:18Z [src/client_handler.rs:844] 37a7fd..: Received (Response::Mutation(Success) MessageId(457fe6..)) to App(Bls(b35ce8..), owner: Bls(b3673e..)) from b803d1..
TRACE 2019-12-08T21:57:18Z [src/vault.rs:190] Node(37a7fd..): Succesfully sent message to: 127.0.0.1:37632
TRACE 2019-12-08T21:57:18Z [src/vault.rs:190] Node(37a7fd..): Succesfully sent message to: 127.0.0.1:37632
TRACE 2019-12-08T21:57:18Z [src/vault.rs:190] Node(37a7fd..): Succesfully sent message to: 127.0.0.1:37632
TRACE 2019-12-08T21:57:18Z [src/client_handler.rs:242] 37a7fd..: Received (Request::PutIData MessageId(587dac..)) from App(Bls(b35ce8..), owner: Bls(b3673e..))
TRACE 2019-12-08T21:57:18Z [src/client_handler.rs:1202] 37a7fd..: App(Bls(b35ce8..), owner: Bls(b3673e..)) is paying 0.000000001 coins
TRACE 2019-12-08T21:57:18Z [src/client_handler.rs:1177] 37a7fd..: Setting balance to 1000.109999977 for Bls(b3673e..)
TRACE 2019-12-08T21:57:18Z [src/data_handler.rs:82] 37a7fd..: Received (Request::PutIData MessageId(587dac..)) from src b3673e.. (client App(Bls(b35ce8..), owner: Bls(b3673e..)))
TRACE 2019-12-08T21:57:18Z [src/data_handler.rs:82] 37a7fd..: Received (Request::PutIData MessageId(587dac..)) from src 8f00e3.. (client App(Bls(b35ce8..), owner: Bls(b3673e..)))
TRACE 2019-12-08T21:57:18Z [src/data_handler.rs:300] 37a7fd..: Received (Response::Mutation(Success) MessageId(587dac..)) from 37a7fd..
TRACE 2019-12-08T21:57:18Z [src/client_handler.rs:844] 37a7fd..: Received (Response::Mutation(Success) MessageId(587dac..)) to App(Bls(b35ce8..), owner: Bls(b3673e..)) from 8f00e3..
TRACE 2019-12-08T21:57:18Z [src/vault.rs:190] Node(37a7fd..): Succesfully sent message to: 127.0.0.1:37632
TRACE 2019-12-08T21:57:18Z [src/client_handler.rs:242] 37a7fd..: Received (Request::PutAData MessageId(53940f..)) from App(Bls(b35ce8..), owner: Bls(b3673e..))
TRACE 2019-12-08T21:57:18Z [src/client_handler.rs:1202] 37a7fd..: App(Bls(b35ce8..), owner: Bls(b3673e..)) is paying 0.000000001 coins
TRACE 2019-12-08T21:57:18Z [src/client_handler.rs:1177] 37a7fd..: Setting balance to 1000.109999976 for Bls(b3673e..)
TRACE 2019-12-08T21:57:18Z [src/data_handler.rs:82] 37a7fd..: Received (Request::PutAData MessageId(53940f..)) from src b3673e.. (client App(Bls(b35ce8..), owner: Bls(b3673e..)))
TRACE 2019-12-08T21:57:18Z [src/client_handler.rs:844] 37a7fd..: Received (Response::Mutation(Success) MessageId(53940f..)) to App(Bls(b35ce8..), owner: Bls(b3673e..)) from 9b7862..
TRACE 2019-12-08T21:57:18Z [src/vault.rs:190] Node(37a7fd..): Succesfully sent message to: 127.0.0.1:37632
TRACE 2019-12-08T21:57:18Z [src/client_handler.rs:242] 37a7fd..: Received (Request::AppendSeq MessageId(12d5a9..)) from App(Bls(b35ce8..), owner: Bls(b3673e..))
TRACE 2019-12-08T21:57:18Z [src/client_handler.rs:1202] 37a7fd..: App(Bls(b35ce8..), owner: Bls(b3673e..)) is paying 0.000000001 coins
TRACE 2019-12-08T21:57:18Z [src/client_handler.rs:1177] 37a7fd..: Setting balance to 1000.109999975 for Bls(b3673e..)
TRACE 2019-12-08T21:57:18Z [src/data_handler.rs:82] 37a7fd..: Received (Request::AppendSeq MessageId(12d5a9..)) from src b3673e.. (client App(Bls(b35ce8..), owner: Bls(b3673e..)))
TRACE 2019-12-08T21:57:18Z [src/client_handler.rs:844] 37a7fd..: Received (Response::Mutation(Success) MessageId(12d5a9..)) to App(Bls(b35ce8..), owner: Bls(b3673e..)) from 9b7862..
TRACE 2019-12-08T21:57:18Z [src/vault.rs:190] Node(37a7fd..): Succesfully sent message to: 127.0.0.1:37632
INFO 2019-12-08T21:57:44Z [src/client_handler.rs:157] 37a7fd..: Connected to new client on 127.0.0.1:46992
TRACE 2019-12-08T21:57:44Z [src/vault.rs:190] Node(37a7fd..): Succesfully sent message to: 127.0.0.1:46992
INFO 2019-12-08T21:57:44Z [src/client_handler.rs:693] 37a7fd..: Accepted App(Bls(b35ce8..), owner: Bls(b3673e..)) on 127.0.0.1:46992.
TRACE 2019-12-08T21:57:44Z [src/client_handler.rs:242] 37a7fd..: Received (Request::GetADataIndices MessageId(619244..)) from App(Bls(b35ce8..), owner: Bls(b3673e..))
TRACE 2019-12-08T21:57:44Z [src/data_handler.rs:82] 37a7fd..: Received (Request::GetADataIndices MessageId(619244..)) from src b3673e.. (client App(Bls(b35ce8..), owner: Bls(b3673e..)))
TRACE 2019-12-08T21:57:44Z [src/client_handler.rs:844] 37a7fd..: Received (Response::GetADataIndices(Success) MessageId(619244..)) to App(Bls(b35ce8..), owner: Bls(b3673e..)) from 9b7862..
TRACE 2019-12-08T21:57:44Z [src/vault.rs:190] Node(37a7fd..): Succesfully sent message to: 127.0.0.1:46992
TRACE 2019-12-08T21:57:44Z [src/client_handler.rs:242] 37a7fd..: Received (Request::GetADataLastEntry MessageId(13b54f..)) from App(Bls(b35ce8..), owner: Bls(b3673e..))
TRACE 2019-12-08T21:57:44Z [src/data_handler.rs:82] 37a7fd..: Received (Request::GetADataLastEntry MessageId(13b54f..)) from src b3673e.. (client App(Bls(b35ce8..), owner: Bls(b3673e..)))
TRACE 2019-12-08T21:57:44Z [src/client_handler.rs:844] 37a7fd..: Received (Response::GetADataLastEntry(Success) MessageId(13b54f..)) to App(Bls(b35ce8..), owner: Bls(b3673e..)) from 9b7862..
TRACE 2019-12-08T21:57:44Z [src/vault.rs:190] Node(37a7fd..): Succesfully sent message to: 127.0.0.1:46992
INFO 2019-12-08T21:57:57Z [src/client_handler.rs:157] 37a7fd..: Connected to new client on 127.0.0.1:43969
TRACE 2019-12-08T21:57:57Z [src/vault.rs:190] Node(37a7fd..): Succesfully sent message to: 127.0.0.1:43969
INFO 2019-12-08T21:57:57Z [src/client_handler.rs:693] 37a7fd..: Accepted App(Bls(b35ce8..), owner: Bls(b3673e..)) on 127.0.0.1:43969.
TRACE 2019-12-08T21:57:57Z [src/client_handler.rs:242] 37a7fd..: Received (Request::GetIData MessageId(75a94c..)) from App(Bls(b35ce8..), owner: Bls(b3673e..))
TRACE 2019-12-08T21:57:57Z [src/data_handler.rs:82] 37a7fd..: Received (Request::GetIData MessageId(75a94c..)) from src b3673e.. (client App(Bls(b35ce8..), owner: Bls(b3673e..)))
TRACE 2019-12-08T21:57:57Z [src/data_handler.rs:82] 37a7fd..: Received (Request::GetIData MessageId(75a94c..)) from src 8f00e3.. (client App(Bls(b35ce8..), owner: Bls(b3673e..)))
TRACE 2019-12-08T21:57:57Z [src/data_handler.rs:300] 37a7fd..: Received (Response::GetIData(Success) MessageId(75a94c..)) from 37a7fd..
TRACE 2019-12-08T21:57:57Z [src/client_handler.rs:844] 37a7fd..: Received (Response::GetIData(Success) MessageId(75a94c..)) to App(Bls(b35ce8..), owner: Bls(b3673e..)) from 8f00e3..
TRACE 2019-12-08T21:57:57Z [src/vault.rs:190] Node(37a7fd..): Succesfully sent message to: 127.0.0.1:43969
TRACE 2019-12-08T21:57:57Z [src/client_handler.rs:242] 37a7fd..: Received (Request::GetIData MessageId(94e232..)) from App(Bls(b35ce8..), owner: Bls(b3673e..))
TRACE 2019-12-08T21:57:57Z [src/data_handler.rs:82] 37a7fd..: Received (Request::GetIData MessageId(94e232..)) from src b3673e.. (client App(Bls(b35ce8..), owner: Bls(b3673e..)))
TRACE 2019-12-08T21:57:57Z [src/data_handler.rs:82] 37a7fd..: Received (Request::GetIData MessageId(94e232..)) from src 0c0597.. (client App(Bls(b35ce8..), owner: Bls(b3673e..)))
TRACE 2019-12-08T21:57:57Z [src/data_handler.rs:300] 37a7fd..: Received (Response::GetIData(Success) MessageId(94e232..)) from 37a7fd..
TRACE 2019-12-08T21:57:57Z [src/client_handler.rs:844] 37a7fd..: Received (Response::GetIData(Success) MessageId(94e232..)) to App(Bls(b35ce8..), owner: Bls(b3673e..)) from 0c0597..
TRACE 2019-12-08T21:57:57Z [src/vault.rs:190] Node(37a7fd..): Succesfully sent message to: 127.0.0.1:43969
TRACE 2019-12-08T21:57:57Z [src/client_handler.rs:242] 37a7fd..: Received (Request::GetIData MessageId(01e682..)) from App(Bls(b35ce8..), owner: Bls(b3673e..))
TRACE 2019-12-08T21:57:57Z [src/data_handler.rs:82] 37a7fd..: Received (Request::GetIData MessageId(01e682..)) from src b3673e.. (client App(Bls(b35ce8..), owner: Bls(b3673e..)))
TRACE 2019-12-08T21:57:57Z [src/data_handler.rs:82] 37a7fd..: Received (Request::GetIData MessageId(01e682..)) from src 5bd67c.. (client App(Bls(b35ce8..), owner: Bls(b3673e..)))
TRACE 2019-12-08T21:57:57Z [src/data_handler.rs:300] 37a7fd..: Received (Response::GetIData(Success) MessageId(01e682..)) from 37a7fd..
TRACE 2019-12-08T21:57:57Z [src/client_handler.rs:844] 37a7fd..: Received (Response::GetIData(Success) MessageId(01e682..)) to App(Bls(b35ce8..), owner: Bls(b3673e..)) from 5bd67c..
TRACE 2019-12-08T21:57:57Z [src/vault.rs:190] Node(37a7fd..): Succesfully sent message to: 127.0.0.1:43969
TRACE 2019-12-08T21:57:57Z [src/client_handler.rs:242] 37a7fd..: Received (Request::GetIData MessageId(ebac13..)) from App(Bls(b35ce8..), owner: Bls(b3673e..))
TRACE 2019-12-08T21:57:57Z [src/data_handler.rs:82] 37a7fd..: Received (Request::GetIData MessageId(ebac13..)) from src b3673e.. (client App(Bls(b35ce8..), owner: Bls(b3673e..)))
TRACE 2019-12-08T21:57:57Z [src/data_handler.rs:82] 37a7fd..: Received (Request::GetIData MessageId(ebac13..)) from src b803d1.. (client App(Bls(b35ce8..), owner: Bls(b3673e..)))
TRACE 2019-12-08T21:57:57Z [src/data_handler.rs:300] 37a7fd..: Received (Response::GetIData(Success) MessageId(ebac13..)) from 37a7fd..
TRACE 2019-12-08T21:57:57Z [src/client_handler.rs:844] 37a7fd..: Received (Response::GetIData(Success) MessageId(ebac13..)) to App(Bls(b35ce8..), owner: Bls(b3673e..)) from b803d1..
TRACE 2019-12-08T21:57:57Z [src/vault.rs:190] Node(37a7fd..): Succesfully sent message to: 127.0.0.1:43969
INFO 2019-12-08T21:57:58Z [src/client_handler.rs:161] 37a7fd..: Networking error: Connection was actively cancelled
INFO 2019-12-08T21:57:58Z [src/client_handler.rs:163] 37a7fd..: Disconnected from App(Bls(b35ce8..), owner: Bls(b3673e..)) on 127.0.0.1:37632
INFO 2019-12-08T21:58:24Z [src/client_handler.rs:161] 37a7fd..: Networking error: Connection was actively cancelled
INFO 2019-12-08T21:58:24Z [src/client_handler.rs:163] 37a7fd..: Disconnected from App(Bls(b35ce8..), owner: Bls(b3673e..)) on 127.0.0.1:46992
INFO 2019-12-08T21:58:37Z [src/client_handler.rs:161] 37a7fd..: Networking error: Connection was actively cancelled
INFO 2019-12-08T21:58:37Z [src/client_handler.rs:163] 37a7fd..: Disconnected from App(Bls(b35ce8..), owner: Bls(b3673e..)) on 127.0.0.1:43969
INFO 2019-12-08T21:59:09Z [src/client_handler.rs:157] 37a7fd..: Connected to new client on 127.0.0.1:60254
TRACE 2019-12-08T21:59:09Z [src/vault.rs:190] Node(37a7fd..): Succesfully sent message to: 127.0.0.1:60254
INFO 2019-12-08T21:59:09Z [src/client_handler.rs:693] 37a7fd..: Accepted App(Bls(b35ce8..), owner: Bls(b3673e..)) on 127.0.0.1:60254.
TRACE 2019-12-08T21:59:09Z [src/client_handler.rs:242] 37a7fd..: Received (Request::GetADataIndices MessageId(ef32c9..)) from App(Bls(b35ce8..), owner: Bls(b3673e..))
TRACE 2019-12-08T21:59:09Z [src/data_handler.rs:82] 37a7fd..: Received (Request::GetADataIndices MessageId(ef32c9..)) from src b3673e.. (client App(Bls(b35ce8..), owner: Bls(b3673e..)))
TRACE 2019-12-08T21:59:09Z [src/client_handler.rs:844] 37a7fd..: Received (Response::GetADataIndices(Success) MessageId(ef32c9..)) to App(Bls(b35ce8..), owner: Bls(b3673e..)) from 9b7862..
TRACE 2019-12-08T21:59:09Z [src/vault.rs:190] Node(37a7fd..): Succesfully sent message to: 127.0.0.1:60254
TRACE 2019-12-08T21:59:09Z [src/client_handler.rs:242] 37a7fd..: Received (Request::GetADataLastEntry MessageId(88b818..)) from App(Bls(b35ce8..), owner: Bls(b3673e..))
TRACE 2019-12-08T21:59:09Z [src/data_handler.rs:82] 37a7fd..: Received (Request::GetADataLastEntry MessageId(88b818..)) from src b3673e.. (client App(Bls(b35ce8..), owner: Bls(b3673e..)))
TRACE 2019-12-08T21:59:09Z [src/client_handler.rs:844] 37a7fd..: Received (Response::GetADataLastEntry(Success) MessageId(88b818..)) to App(Bls(b35ce8..), owner: Bls(b3673e..)) from 9b7862..
TRACE 2019-12-08T21:59:09Z [src/vault.rs:190] Node(37a7fd..): Succesfully sent message to: 127.0.0.1:60254
INFO 2019-12-08T21:59:49Z [src/client_handler.rs:161] 37a7fd..: Networking error: Connection was actively cancelled
INFO 2019-12-08T21:59:49Z [src/client_handler.rs:163] 37a7fd..: Disconnected from App(Bls(b35ce8..), owner: Bls(b3673e..)) on 127.0.0.1:60254

tobbetj · December 10, 2019, 12:16pm

@bochaco

Thank for the explanation, it helps me a lot to understand how things work, I use coding as screwdrivers, trying to find the right one who fits for solving each problem, I learn as I go.

How do you connect between apps, I only know right now how to connect through python request lib with http. Do you think a vault UI with QUIC can be done with python, it took me some time to learn how to get a UI working with multiprocess threads doing different algorithms at the same time. Or would it be easy enough to make a Rust app with UI?

Thanks @mav for the log, it seems a little to much work to parse that log into a UI at the moment, I only know how to work with JSON as seen in the picture, and if the structure will change then it needs to be re-done in the future. I would also like to collaborate if it is possible. Right now it only feels possible to decide which programming language to use and to think about design. @bochaco It would be nice if there could be authenticator QUIC example that we in the community can practice on, like a JSON-RPC message with QUIC, something like “Hello world” or similar.

Maybe it could be possible to make an UI right now that helps users open the terminal and start vault from home and similar, just a thought that got to me yesterday.

Namnlös

https://poloniex.com/public?command=return24hVolume

happybeing · December 10, 2019, 12:29pm

Parsing the log as a temporary API is not that hard, especially with a well supported language like Python.

If I were tackling this I’d start by searching for something (apps, modules) that does parsing of log files and see how it does it.

I’d also look for text parsing apps/modules, and more specifically something that implements ‘awk’ syntax to take a log and generate something more manageable like JSON, although you might not need an intermediate step. You could start just with Un*x text processing pipeline - there are excellent commands for this. And good to learn for future projects.

Some languages are very well suited to parsing simple formatted text like that and while I don’t know much Python, I suspect it might be one of them. Again, looking for similar examples will show ways to tackle this.

tobbetj · December 10, 2019, 12:43pm

Thanks @happybeing, I will look into it and research some. I’am often to intimidated by looking and understanding other peoples code, so it often ends up that I learn the technique and then make my custom code. I should try to look for things that already exist more and do print() and similar to see what things does.

happybeing · December 10, 2019, 1:09pm

We each have our own style, I’m only suggesting how I’d approach it in case it helps. Other people’s code can be unintelligible, or it can be clean and clear.

Thing is that these days it’s so easy to find not just relevant apps, libraries and code, but also tutorials and articles that have useful tips and pointers, so I find search very helpful. I use communities with chat and forums a lot too, when I get stuck. It all makes my progress much faster.

bochaco · December 10, 2019, 2:24pm

Yes a vault UI should be able to connect to such a vault QUIC interface using any programming language, and this is why I think we should work on a vault client API, that’s what you do basically when you say you know how to send a request using http lib, well I say we need a vault-client lib/api.

Once we have such an api in the Rust safe-api it’s easy to create the bindings for each language, as an example the latest version of SNAPP uses the JS bindings we created for the Rust authd-client API (this is a test not fully working yet in the JS binding for authd client api: https://github.com/maidsafe/safe-nodejs/blob/master/test/auth.test.js#L16). Those JS functions are the ones used by SNAPP to communicate with authd, you see all the QUIC and JSON-RPC is managed by the APIs already, no need to know or worry about it, you just ask for specific information/action to authd. The same could be for vault/vaultd, you call an api asking for information/action and obtain a json back.

Unless you meant you wanted to know how to create the vault client api you won’t need to understand QUIC and/or JSON-RPC, just use the api as I mentioned above. Otherwise, there are some example messages for the authd communication in the readme here: https://github.com/maidsafe/safe-api/tree/master/safe-authd#description

Topic		Replies	Views
Vault Dashboard ideas please! Apps	33	2047	July 1, 2021
SAFE Network Dev Update - November 28, 2019 Updates	40	2655	December 16, 2019
Vault questions and concerns Support	1	666	November 8, 2016
Testnet Vault Specs / Performance? Development	2	1000	March 3, 2016
Log viewer for SAFE nodes Development	14	1059	February 19, 2021

Vault monitor app

Related Topics