SAFE Network Dev Update - March 19, 2020

If you hammer the keyboard in all the ways, then you end up stumbling on an answer… at least that’s my method.

So, you want one line :thinking:

dd if=/dev/urandom of=./to-upload/file.dat bs=1k count=1500 2>/dev/null; COUNTER=0; while [ $COUNTER -lt 2 ]; do let COUNTER=COUNTER+1; echo $(ls -hs ./to-upload/file.dat | sed 's/^\([^ ]*\).*/\1/')"," $COUNTER; done

works… but that as example only because I can’t get time to work directly.

So, this works by logging to a file ./logfile:

echo -n > ./logfile; dd if=/dev/urandom of=./to-upload/file.dat bs=1k count=1500 2>/dev/null; COUNTER=0; while [ $COUNTER -lt 2 ]; do let COUNTER=COUNTER+1; echo -n $(ls -hs ./to-upload/file.dat | sed 's/^\([^ ]*\).*/\1/')"," >> ./logfile; (/usr/bin/time -f "\t%E " echo >/dev/null ) &>> ./logfile; done; cat ./logfile

replace the echo >/dev/null with something more useful
and if you want the output from that, obviously cat ./logfile > somefilecapturingthis

1 Like

This has worked for me in the past…

OK I’ll go away and play with this - thank you

Hard to say what’s most useful to be honest. Seeing the limitations is one very useful thing.

Right now, eg, I was basing benchmarks off the the tests you were working on. So uploading x kb y times. Having this data is good for reference when things are going well, and debugging if we’re seeing something off. So collating that is good, I think.

The spread of data in vaults is an interesting one. At least I’m not sure why that would be happening unevenly. So if you can get data of filesize vs vault spread that’d be good to chew on.

Otherwise, I’d say just keep pushing things. Keep what you can in case you come across something interesting. Otherwise we have the start of some benchmarking regression checks if nothing else! (Which is in itself quite valuable :+1:)

8 Likes

Really :clap: you kids giving feedback. Unfortunately cli is too much for me at the moment :sob: would have really loved to upload, send testnet SAFE. Keep going you super ants… at least if some of us are busy it’s like we all are

4 Likes

CLI is not scary, just a little less pretty. It lets you get to the guts of whats going on, not wondering if its the logic that is wrong or some pretty window not quite doing what it should that is causing a problem.
Go on push yourself and have a play about with a CLI :slight_smile:

10 Likes

Still getting random Access denied errors

The commands below were issued within a few seconds of each other
willie@gagarin:~/projects/maidsafe/nappies$ ls -al ./to-upload/file.dat -rw-r--r-- 1 willie willie 20480 Mar 25 18:30 ./to-upload/file.dat willie@gagarin:~/projects/maidsafe/nappies$ safe files put ./to-upload/file.dat FilesContainer created at: "safe://hnyynyzjamsr7qst48dz1ddyzfkix73cfynwspjwxdgjcd73ceqreukrkhbnc" E ./to-upload/file.dat <[Error] NetDataError - Failed to PUT Published ImmutableData: Error(CoreError(Self-encryption error: Storage error: Data error -> Access denied - CoreError::SelfEncryption -> Storage(SEStorageError(Data error -> Access denied - CoreError::DataError -> AccessDenied))))> willie@gagarin:~/projects/maidsafe/nappies$ safe files put ./to-upload/file.dat FilesContainer created at: "safe://hnyynyz4uk8xsxx34z6qnexh5q5yj1371gj4bkqa6kayw9a6pjgbznu1csbnc" + ./to-upload/file.dat safe://hbkyyodmk14c4x1xna71qjdhtoo5jormoi1sbyd3g5ok5asxuqw7k4yfcx willie@gagarin:~/projects/maidsafe/nappies$ safe files put ./to-upload/file.dat FilesContainer created at: "safe://hnyynyzsq3neapsmbiie6zch6b14zcfsjbe49fbk4cc9thtsje7ojnwd8kbnc" E ./to-upload/file.dat <[Error] NetDataError - Failed to PUT Published ImmutableData: Error(CoreError(Self-encryption error: Storage error: Data error -> Access denied - CoreError::SelfEncryption -> Storage(SEStorageError(Data error -> Access denied - CoreError::DataError -> AccessDenied))))>

All the vault logs show the same thing – ie not much

willie@gagarin:~/.safe/vault/baby-fleming-vaults$ tail -n20 safe-vault-6/safe_vault.log 
INFO 2020-03-25T21:57:53.455079601+00:00 [src/client_handler.rs:714] e24dcc..: Accepted App(PublicKey::Bls(9814c3..), owner: PublicKey::Bls(99dbbe..)) on 127.0.0.1:56325.
INFO 2020-03-25T21:57:55.549458900+00:00 [src/client_handler.rs:103] e24dcc..: Connected to new client on 127.0.0.1:55198
INFO 2020-03-25T21:57:55.598913455+00:00 [src/client_handler.rs:714] e24dcc..: Accepted App(PublicKey::Bls(9814c3..), owner: PublicKey::Bls(99dbbe..)) on 127.0.0.1:55198.
INFO 2020-03-25T21:57:56.241639974+00:00 [src/client_handler.rs:108] e24dcc..: Disconnected from App(PublicKey::Bls(9814c3..), owner: PublicKey::Bls(99dbbe..)) on 127.0.0.1:38110
INFO 2020-03-25T21:57:57.978324051+00:00 [src/client_handler.rs:108] e24dcc..: Disconnected from App(PublicKey::Bls(9814c3..), owner: PublicKey::Bls(99dbbe..)) on 127.0.0.1:55660
INFO 2020-03-25T21:57:59.623060351+00:00 [src/client_handler.rs:108] e24dcc..: Disconnected from App(PublicKey::Bls(9814c3..), owner: PublicKey::Bls(99dbbe..)) on 127.0.0.1:40723
INFO 2020-03-25T21:58:01.018459579+00:00 [src/client_handler.rs:108] e24dcc..: Disconnected from App(PublicKey::Bls(9814c3..), owner: PublicKey::Bls(99dbbe..)) on 127.0.0.1:40305
INFO 2020-03-25T21:58:02.442134300+00:00 [src/client_handler.rs:108] e24dcc..: Disconnected from App(PublicKey::Bls(9814c3..), owner: PublicKey::Bls(99dbbe..)) on 127.0.0.1:52044
INFO 2020-03-25T21:58:03.721687149+00:00 [src/client_handler.rs:108] e24dcc..: Disconnected from App(PublicKey::Bls(9814c3..), owner: PublicKey::Bls(99dbbe..)) on 127.0.0.1:42213
INFO 2020-03-25T21:58:16.529571760+00:00 [src/client_handler.rs:108] e24dcc..: Disconnected from App(PublicKey::Bls(9814c3..), owner: PublicKey::Bls(99dbbe..)) on 127.0.0.1:52162
INFO 2020-03-25T21:58:19.970782091+00:00 [src/client_handler.rs:108] e24dcc..: Disconnected from App(PublicKey::Bls(9814c3..), owner: PublicKey::Bls(99dbbe..)) on 127.0.0.1:57115
INFO 2020-03-25T21:58:21.579105642+00:00 [src/client_handler.rs:108] e24dcc..: Disconnected from App(PublicKey::Bls(9814c3..), owner: PublicKey::Bls(99dbbe..)) on 127.0.0.1:47594
INFO 2020-03-25T21:58:23.351591258+00:00 [src/client_handler.rs:108] e24dcc..: Disconnected from App(PublicKey::Bls(9814c3..), owner: PublicKey::Bls(99dbbe..)) on 127.0.0.1:33253
INFO 2020-03-25T21:58:24.784957632+00:00 [src/client_handler.rs:108] e24dcc..: Disconnected from App(PublicKey::Bls(9814c3..), owner: PublicKey::Bls(99dbbe..)) on 127.0.0.1:41688
INFO 2020-03-25T21:58:26.351904911+00:00 [src/client_handler.rs:108] e24dcc..: Disconnected from App(PublicKey::Bls(9814c3..), owner: PublicKey::Bls(99dbbe..)) on 127.0.0.1:59921
INFO 2020-03-25T21:58:27.768318003+00:00 [src/client_handler.rs:108] e24dcc..: Disconnected from App(PublicKey::Bls(9814c3..), owner: PublicKey::Bls(99dbbe..)) on 127.0.0.1:47575
INFO 2020-03-25T21:58:29.218056911+00:00 [src/client_handler.rs:108] e24dcc..: Disconnected from App(PublicKey::Bls(9814c3..), owner: PublicKey::Bls(99dbbe..)) on 127.0.0.1:38579
INFO 2020-03-25T21:58:31.705314953+00:00 [src/client_handler.rs:108] e24dcc..: Disconnected from App(PublicKey::Bls(9814c3..), owner: PublicKey::Bls(99dbbe..)) on 127.0.0.1:41424
INFO 2020-03-25T21:58:33.663670994+00:00 [src/client_handler.rs:108] e24dcc..: Disconnected from App(PublicKey::Bls(9814c3..), owner: PublicKey::Bls(99dbbe..)) on 127.0.0.1:56325
INFO 2020-03-25T21:58:35.813463124+00:00 [src/client_handler.rs:108] e24dcc..: Disconnected from App(PublicKey::Bls(9814c3..), owner: PublicKey::Bls(99dbbe..)) on 127.0.0.1:55198

tagging @bochaco @joshuef @StephenC

3 Likes

What does [safe auth status] suggest… too obvious to not be logged in?.. but done that myself.

1 Like

Oh I was logged in OK - it would work - then not work then work again
and what I showed there was fail-success - fail -fail -fail -fail

I built the latest safe_vault but my last few attempts to run my baby-fleming setup script are failing at the account creation stage.

Actually I gave up in disgust and am failing at the pentatonic scale in Aminor on the geetar.

I feel it is always important to have a variety of failures throughout the day. Its just started raining so it looks like I’m going to fail to go for a walk too.
On a positive note I failed to accept the wifes offer of a baked potato for lunch so I’m not failing at the low-carb diet :slight_smile:

1 Like

I wondered if there was ever occasion user gets logged out.

Baked potato; baked beans; and cheese :potato: … hmm

1 Like

well if I ever get logged on again successfully I’ll stick a safe auth status in the loop and see if that shows anything

roasted broccoli and cauliflower with olive oil garlic and chilli – el yummo

Though to be honest I’d prefer a bridie and beans with a pint

If for some strange reason you got logged out, that won’t affect CLI app and its credentials. Remember that once an app (like CLI in this case) is authorised by the user through authd, it doesn’t need authd and can connect to the network directly for all operations, as long as it’s not revoked ofc. CLI stores its credentials at ~/.local/share/safe-cli/credentials, therefore as long as the credential stored in that file are valid and not revoked, it can operate without user being logged in, or even without authd running or present at all.

4 Likes

I’m building the latest safe-api and safe_vault 0.23.0 so we’ll see how that goes.

BTW is there ANY convention for the use of safe-(hyhpen) package and safe_(underscore) package? - cos I always seem to type the wrong one :slight_smile:

3 Likes

safe-auth status still returns daemon version 0.0.5 but I just updated to safe-authd-0.0.7

willie@gagarin:~/.safe/authd$ safe-authd update
Checking target-arch... x86_64-unknown-linux-gnu
Checking current version... v0.0.6
Checking latest released version... v0.0.7
New release found! v0.0.6 --> v0.0.7
New release is compatible

safe-authd release status:
  * Current exe: "/home/willie/projects/maidsafe/safe-authd"
  * New exe release: "safe-authd-0.0.7-x86_64-unknown-linux-gnu.zip"
  * New exe download url: "https://safe-api.s3.eu-west-2.amazonaws.com/safe-authd-0.0.7-x86_64-unknown-linux-gnu.zip"

The new release will be downloaded/extracted and the existing binary will be replaced.
Do you want to continue? [Y/n] y
Downloading...
[00:00:01] [========================================] 6.74MB/6.74MB (0s) Done
Extracting archive... Done
Replacing binary file... Done
Setting execution permissions to installed binary '/home/willie/projects/maidsafe/safe-authd'...
Update status: `0.0.7`!
willie@gagarin:~/.safe/authd$ safe auth stop
Stopping SAFE Authenticator daemon (safe-authd)...
Success, safe-authd (PID: 11868) stopped!
willie@gagarin:~/.safe/authd$ safe auth start
Starting SAFE Authenticator daemon (safe-authd)...
willie@gagarin:~/.safe/authd$ safe auth status
Sending request to authd to obtain a status report...
+------------------------------------------+-------+
| SAFE Authenticator status                |       |
+------------------------------------------+-------+
| Authenticator daemon version             | 0.0.5 |
+------------------------------------------+-------+
| Logged in to a SAFE account?             | No    |
+------------------------------------------+-------+
| Number of pending authorisation requests | 0     |
+------------------------------------------+-------+
| Number of notifications subscribers      | 0     |
+------------------------------------------+-------+
2 Likes

Thanks for pointing this out, not sure yet why, it seems both versions are running, I just did killall safe-authd and then start it the new one and CLI connects to the new one, please try that, I’ll investigate the reason, very likely due to the upgrade in quinn we’ve introduced in new version

4 Likes

I think the problem is at this end… sorry

willie@gagarin:~/.safe/authd$ ./safe-authd -V
safe-authd 0.0.5
willie@gagarin:~/.safe/authd$ safe-authd -V
safe-authd 0.0.7
willie@gagarin:~/.safe/authd$ which safe-authd
/home/willie/projects/maidsafe/safe-authd
3 Likes

@bochaco thats better - sorry for the hassle

willie@gagarin:~/projects/maidsafe/nappies$ safe auth status
Sending request to authd to obtain a status report...
+------------------------------------------+-------+
| SAFE Authenticator status                |       |
+------------------------------------------+-------+
| Authenticator daemon version             | 0.0.7 |
+------------------------------------------+-------+
| Logged in to a SAFE account?             | Yes   |
+------------------------------------------+-------+
| Number of pending authorisation requests | 0     |
+------------------------------------------+-------+
| Number of notifications subscribers      | 0     |
+------------------------------------------+-------+
3 Likes

I wonder some uninstall option might be useful to ensure a clean state.

off the top of head something like:

  • safe vault killall
  • safe auth stop
  • remove all ~/.safe/ content
  • and remove also /home/safe/.config related folders

I’ve pretty much got that in run-new-network.sh
lines 30 -60 ish

#/bin/bash
#set -xv
clear
DATETIME="`date '+%Y%m%d%H%M%S'`"
DEV_LOGS=/tmp/baby-fleming-logs-for-devs
TEST_RUN_LOG=$DEV_LOGS-$DATETIME
WD=/home/$USER/.safe/vault
SAFE_AUTH_PASSWORD="aaaaa"
SAFE_AUTH_PASSPHRASE="aaaaa"
VAULT_RUN_CMD="safe vault run-baby-fleming"
#VAULT_RUN_CMD="~/projects/maidsafe/safe-network-launch-tool/target/release/safe-nlt -yyy -v -n16 -d ~/.safe/vault/baby-fleming-vaults/"




echo "======================================================================================="
echo "=                                                                                     ="
echo "=  This will set up a new Baby Fleming network on your computer.                      ="
echo "=                                                                                     ="
echo "=  Existing vault logs will be timestamped and archived and the old vaults DELETED    ="
echo "=                                                                                     ="
read -p "=  Are you entirely sure about this? (y/N)                                            =" -n 1 -r
echo "=                                                                                     ="
echo "======================================================================================="


if [[ $REPLY =~ ^[Yy]$ ]]
then
    
    #  Save existing logs, if any

    if [[ ! -e $DEV_LOGS ]]; then
        mkdir -p $DEV_LOGS
    elif [[ ! -d $DEV_LOGS ]]; then
        echo "$DEV_LOGS already exists but is not a directory" 1>&2
    fi

    # set up the destination for this test run logs
    mkdir -p $TEST_RUN_LOG

    # copy the individual vault logs
    for  i in genesis 2 3 4 5 6 7 
    do
            cp  /home/$USER/.safe/vault/baby-fleming-vaults/safe-vault-$i/safe_vault.log  $TEST_RUN_LOG/safe-vault${DATETIME}-$i.log
    done

    # copy the auth log and store it beside the vault logs
    #cp  /home/$USER/.safe/authd/logs/safe-authd.*  $TEST_RUN_LOG/

    echo " Existing logs (if any) Have been backed up to /tmp "

    # create login creds
    cd $WD
    echo "{ \"passphrase\":\"$SAFE_AUTH_PASSPHRASE\", \"password\":\"$SAFE_AUTH_PASSWORD\" }" > $WD/myconfig.json
    
    # Clean up
    safe vault killall
    rm -rf baby-fleming-vaults
    #rm -v ../authd/logs/safe-authd.out
    #rm -v ../authd/logs/safe-authd.err

    # OK    Do stuff .......
    # -y &&
    $VAULT_RUN_CMD &&
    safe networks add my-network &&
    safe networks switch my-network &&
    safe auth restart &&
    #safe auth status
    safe auth create-acc --test-coins -c $WD/myconfig.json &&
    safe auth login --self-auth -c $WD/myconfig.json &&
    safe auth status

   

    # TODO: error checking here
     # if we got this far, it's worked - report success  <-- this wrong, just wrong

    echo "==================================================================================="
    echo "=                                                                                 ="
    echo "=      Your new Baby Fleming test network is running                              ="
    echo "=                                                                                 ="
    echo "=      Logs from the previous run are saved in a timestamped directory in /tmp    ="
    echo "=                                                                                 ="
    echo "=      Have fun !!                                                                ="
    echo "=                                                                                 ="
    echo "==================================================================================="

fi

But there is much room for improvement here - to put it mildly

Feel free to fork https://github.com/willief/change-nappies

2 Likes

Thank you for the heavy work team MaidSafe! I add the translation into Bulgarian in the first post :dragon:

7 Likes