Thank you David. I just came back and re-read this. I get how this works for private use, you effectively put a part of your file system on safenet and it remembers aspects of how you want to interact with it and becomes agnostic to the physical machine you use when you connect. This is good.
I’m trying to understand how this is for public-facing data, eg, I want to mount a page containing a player and I want any public member to be able to view a nominated video through the player. In the model I just tested with an audio file, the resources get downloaded and assembled on the client browser, including the media file, and then, independently of the network, the user can play the media. Others here are suggesting a streaming style of delivery, where bits of the media file get delivered as chunks as required, which is the model we predominantly use through CDNs today.
Currently, my file set for my audio demo / test (http://testaudio.peteratcryptogoss.safenet/) weighs in at 940K on disk. After clearing my cache, re-load is around 4 seconds.
Well, please make sure you drop in and see us at Address: 1/89-91 City Rd, Southbank VIC 3006 whenever you’re over this way. Get in touch directly or via Meetup Blockchain Melbourne | Meetup. And yeah, Go the Crows! (I’m from SA myself, 30 years ago).
I’m not sure that http is a protocol that exists on the SAFEnetwork.
True, but there is a REST API which supports the ability to read file data from a particular offset. It’ll be a great experiment to do, so I hope @cretz will have a go and show us something new like this.
This is similar to what David just described (accessing the virtual file system) but he was giving a simpler “out of the box” approach which means any App would be able to pseudo-stream data, whereas @cretz is describing how this can be done explicitly using the API.
It will be useful to compare the two approaches for implementation difficulty and performance.
EDIT for @PeterRobinson: The REST API allows write to an offset too, so I don’t see any reason why you can’t stream a file that is being stored (appended to live). It just depends on whether the API implementation supports this kind of use - maybe @dirvine can give, a yes/no/depends?
Sorry to derail the thread here with media streaming stuff…I’ll respond to some things with my opinions and if more discussion is warranted a new thread may be appropriate.
Yes, basically. It is post processing done after the safe network. I will expand on adaptive streaming below.
Nope. I can request a range of a file and serve it to, say, a video element. Here is a crude gist showing on a flat file from node: Node.js HTML5 video streamer · GitHub. Imagine that instead calling safe_launcher’s getFile with the offsets/lengths. Granted this is the simple ranged approach, see below…
EDIT: I just reread this and you said “exist” not “downloaded”. Live streaming is a bit tougher but could be done, but would have to be stored as a bunch of separate sets of self-encrypted data. I am unsure the safecoin expense of this approach. I highly doubt messaging will have the throughput required to support live streaming not to mention I am unfamiliar w/ the broadcast capabilities of it.
So, the segment map (I made that term up) would be stored separately from the file IMO. (In fact, in my approach I was thinking about storing it with other metadata about the media to help with searching inside of structured data). So you could just obtain this “segment map” separately. The segment map in my opinion would be a tightly packed binary format just saying the byte offsets for every interval (5 or 10 seconds or something) of a video. You could then take this and build the metadata file needed for HLS (.m3u8 file) or DASH (.mpd file). These files then basically say what to ask the web server for each segment at which point we could live download that.
You didn’t really ask about it but I’ll touch on it here… The good thing about HLS/DASH is that they allow different encodings of the media. The way I figure it the app would store structured data about the media (title, description, etc) which would include its encoding (mime type, bitrate, aspect, etc) and also a list of “forks” for other formats that poeple may have encoded. You can then build the m3u8/mpd file to point to all of those and let the device (html5 video, chromecast, smart tv, etc) automatically adapt based on the perceived speed of downloading, device, etc.
The former. You run this app on your system just like the safe demo app. It registers with the safe launcher as an app. It then has a web server that listens on localhost and a port which your browser or device or the app itself or whatever can communicate with (there are other ways such as chromecast that do things a bit differently). Imagine the awesome Playback Video Player sourcing from safe.
Please do, the time has come. David said the following…
It’s a mix really,
In browser mode then it will download and play however the browser is configured to do that.
Nfs or in a virtual drive then it will be “downloaded” and played, but as many players do it will only need the parts of the stream at any one time. So here the download is a few chunks at a time to a memory based disk (which clears copies). So in many cases here it won’t actually touch the local hard drive. The code reads from the network through the VFS to the application (which thinks it is on disk).
These are two of the methods really. If we had portable browsers (firefox has a portable copy AFAIK) then these will also load from the network to RAM and again think they are looking at a local hard drive when playing.
So a couple of mechanisms in play here that can be toyed with over the tests. Using local apps will probably mean download and play, but if they use the vfs to find data then it can be secured and limit what touches the drive (we need to consider app caching locally as browsers do, but if the cache is pointed to the vfs then it’s much better and you take all your settings and cache with you from machine to machine.
[EDIT - I should add that a while back I mocked up what I think is a nice way for this to work. on login (I use linux) have the PAM modules use SAFE login which then gets the root dir of your SAFE drive and mounts that as /home/hash-user and logs you into the computer. Then your whole home drive is in SAFE and apps you install there (including libs etc.) will move around with you. This can be extended to select the proper app per OS, and with musl compiled apps then dependency free). That experiment will start up again when we get teh vfs back up and running as a stand alone). ]
To which you replied:
@dirvine: Using local apps will probably mean download and play
Nah, we can use the offsets in the request to get pieces and stream them right over HTTP to where we want. The real thing to fix here is to stop having the safe_launcher base64 the contents after they all have been received. Instead, have the launcher API send (and receive on the modify for content stuff) HTTP data like any other web server might when handling a file (you could go one further here and have some awesome webdav support, but that may be a bit much). If we need to support encrypting that API responses data, we could look at some local TLS or something…but we really need to stream it.
SAFEnetwork is described as occupying layers 4-6 in an OSI model, so any discussion of streaming video should be confined to those layers…agreed?
I’m eager to unsterstand how streaming can be achieved point to point on SAFE, without CDN;s, POP’s, Servers,
Seriously, lets nut this out…I was under the impression that no http streaming would be required to deliver great video over SAFE,make me understand why I was wrong.
YEs, also with public data it’s not just web sites as we have seen. Instead of http delivered content, it’s SAFE to use safe delivered content. This includes the apps to play the content. Think of it as a huge hard drive (your public “site”) that can contain anything, so html pages (actually safe pages using html), music … etc. any digital content.
A missing component right now (not hard) is autodetect (or be told) your current OS the client is running on (they know :D) and select the appropriate content for that OS, especially for apps.
Quick problem, I made an identity with the public client. There was no 4 digit PIN cap and my pin is much longer than anything acceptable in command line, is this intended?
At the core level the pin is a string of any length. It doesn’t even need to be made of digits and can contain any UTF-8 characters.
Client tools limit what the user is allowed to enter and these limits can differ among the tools, which creates the kind of compatibility problems you are facing.
