In most cases.
But imagine a orderly table/list of data points and you fix an error in 4 bytes of the table. Then only that chunk and the next 2 would need changing if I am not mistaken from past discussions.
Of course if you add/remove to/from the file then that chunk and all the following will change and need to be uploaded.
And the beauty is that you can keep the old datamap as the previous version and the new datamap is the current version. Both versions of the file exist forever.
Not thought this through, but can’t you just access the data map to see if the file is uploaded?
IIRC the map holds the addresses of all the chunks. You would risk that somebody uploaded a fake data map, but the lolz per PUT cost seem a poor incentive to do that and I assume somebody discovering that can fix it by uploading the file.
Yes, but the point is you have to check each chunk as you generate them.
The data map is generated as you do the self-encryption, so only relevant to your current upload and does not matter what others have. You generate the chunk (self-encrypted) then ask the network for that chunk using the address generated thru self-encryption. If network responds with the chunk existing then no need to upload that chunk.
The issue is that you cannot rely on chunk 1 === all of the file is uploaded. Nor can you rely on the first x chunks existing meaning the rest are there either.
What I’m thinking is that the chunk addresses are in the data map, so you only need to get the map and check the chunks against it - not GET each one, because they won’t be in the map unless: they are uploaded, or somebody deliberately faked the data map.
And where would you get that for private files. If its public then you only need to look up the file to check.
This is about files you **don’t know if they exist** so there is no known datamap to check against.
Oh and there is the idiots who for a giggle can make up datamaps from the file, but never upload the file and just store the datamap as if the file exists.
If its for a youtube app the immutable data are public, so to find the map you generate the chunks offline, and from that the map.
So this does appear to be a way to check if a datamap already exists for a file without having to GET every chunk.
I already mentioned that somebody could do this and upload just the map.
In this case yes. I was considering all files private or not.
Also you assume the file the datamap is for has actually been uploaded and some idiot for “giggles” didn’t upload the datamap without uploading the file. So you may match the datamap and not do upload. But then find the file doesn’t actually exist on the system.
But also one might want to check if the video being uploaded has not also been uploaded previously to someone’s private files. It is quite possible that the video you want to upload to safevid site was already uploaded to someone else’s private collection.
Actually this would also work for private files, because immutable data are public.
Only Mutable Data have permissions - you control access to the container of pointers to datamaps.
So if you have, or can calculate the address of the datamap for an immutable file you can access it regardless of the permissive on the container MD to which it was uploaded.
This means that it is possible to check if a file, public or private has been uploaded, without getting all the chunks unless:
I find the problem of this topic quite interesting. The problem might be a much more general one than just about video sharing platforms.
I came across VideoCoin, which Michael Novogratz is convinced of will ‘revolutionize the video streaming industry’. Their idea is a blockchain with nodes that play different roles like requestng, distributing and transcoding the videos. Their whitepaper goes in some detail about the ‘proof of transcoding’ and such:
VideoCoin addresses this problem by implementing a Distributed Video Processing platform, which splits a video encoding task into several sub tasks and processes them in parallel, thus effectively producing much higher frame rates on commodity hardware.
Compute miners execute video functions like transcoding. While any node can participate in the network as storage miners, only nodes that have staked tokens greater than barrier to entry, Ψ, are eligible to perform compute mining operations.
Both clients and verifiers can confirm work by a compute miner for a very small fraction of the compute cost by just checking transcode on a randomly selected GOP from the video.
If I have understood correctly: These ‘compute miners’ complete jobs that are requested by clients. The client takes some samples for himself (GOP; ‘Group Of Pictures in a video’) to verify whether the result of the miner is correct.
Ideally for as many formats to be available as possible, an existing standard transcoding method with proven efficiency such as ffmpeg would be used. And that isn’t optimised for distributed computing as far as I know. But maybe distributed computing could be the way forward even with whole video transcodings — the viewer who wants an unavailable format pays the app to commission other users of the app at random, who then transcode the video independently and compare results. If the results don’t match, they don’t get their payment, the user loses money without getting a result, and a bug report gets filed pretty quickly.
One key requirement would be that the transcodings are bit reproducible.
Hmhmm decentralized live streaming built on top of the ethereum blockchain
Interesting that they really want to do this kind of thing on a blockchain 
Interesting. It looks like a peer to peer app, which uses Ethereum only as a means to publish where the stream is located. A sort of public index of streams.
Blockquote Livepeer is a decentralized video broadcasting platform powered by a crypto token on the Ethereum blockchain. Livepeer is for:
Blockquote Developers who want to build applications that include live video.
Users who want to stream video, gaming, coding, entertainment, educational courses, and other types of content…
Broadcasters who currently have large audiences and high streaming bills or infrastructure costs can use the Livepeer network to potentially reduce costs or infrastructure overhead.
I was going to suggest that it was a good candidate for porting to SAFENetwork, but I actually think Maidsafe themselves are pretty close to this already. They already use SAFENetwork to publish user streams and are close to getting WebRTC over Crust. The latter sounds like a big security win here too, as no insecure proxy or custom app is needed to relay.
I think this is exactly the sort of ‘easy’ win we will start to expect as the SAFENetwork moves forward. The foundations are being built good and strong, to form a great platform to build rich applications on top.
Edit: The transcoding side is interesting. I suppose any SAFENetwork node could relay at a different quality to others. It could be an overlay with coins used to fund it or some such.
Another thought. It will probably make most sense for the app to split any video into many 5-second videos to allow streaming. The app could then upload those short videos, and then put their URIs into map file.
Thanks For This!! This is good info!