How does file access update work?


#1

Ok, I admit it, months have passed and I’ve still not read the docs fully, or hardly any of the code.

I did though watch the Intro to NoSQL by Martin Fowler video @dirvine posted and understand a lot more than I did about databases (not saying much), or at least the design patterns and terminology of the field. BTW On reflection I think I spent about ten years working on products centred on an something that probably should now be called a graph database, and some of that time helping it integrate with SQL and even unstructured text databases (ever heard of the Scottish Co Memex @dirvine?) - yet I managed to end my career without ever thinking I did much with databases :-). Anyway, the video and simmering questions about applications set me thinking about how one would design SAFE to work with databases, and I’m wondering if I’ve misunderstood something. I also realised that a filesystem is a key value store!

My impression was, that if you modified byte X in a file, it would change the chunk containing byte X and all chunks after byte X. I think this because I think there’s some kind of hash from one chunk that seeds the encryption of the next chunk. Am I wrong?

Because it seems daft if this could be avoided, and would make all sorts of file operations way more efficient, if modifying byte X only changed the chunk containing byte X. For an SQL database, or a filesystem, I imagine this would dramatically reduce the number of PUTS when updating records for example. Hence the issue becomes one not of updating a large amount of data for a relatively small change, but how frequently locally cached chunks get written back to SAFE.


#3

That’s about it for us imagine change something in chunk X is a change in a window of three chunks. So the thing is the size of the chunks (1Mb) in our case. It should be dynamic based on data population and will be eventually. So imagine as you do change chunk X and its only that (forget the window), The 3 chunks only complicates things but is important for security ( as humans are the only species that will attack itself, unfortunately). Easy to imagine chunk X is a certain size defined by the net, ignore it for now (imagine a byte was 4 bits or 16 bits etc. it makes little difference if the hardware supports it). Hope that makes sense. Paige made me drink a load of beer there :smiley:


#4

Well done @ioptio, I didn’t understand a word of @dirvine’s response and it wasn’t anything to do with accents.

Think I better read the docs… was that in fact a subtle way to get me to do that David?

Enjoy!


#5

But since MaidSafe is open source…

Then as soon as someone finds a better way that allows for having your cake and eating it too (only changing chunk with byte X while also keeping the whole file encrypted)…

Couldn’t it just be added to the SAFE network’s open source code?

Isn’t that the beauty of open source?

Like even though it’s not totally perfect, it can and will always be updated by everybody who can help?