Distributed databases... MongoDB? or?


#1

Think IoT and NodeBots… instead of storing usage data “locally” on the device, we’d like said data to be securely distributed, just like its cpu, fs, bandwidth, hash table, etc. Does anyone have points on which db we should choose? I wanna make sure we don’t waste manhours coding this device that won’t be secure or scalable. It has to be free open source from bare metal to gui. Please and thank you for any advice on this part :slight_smile:


#2

You will be able to use the SAFE Network itself as a distributed database. No need to use MongoDB or store data locally.

I’ll just refer you to this recent quote by @dirvine where he explains that everything will be clearer in testnet2 where we will see lots of example applications :smiley:


#3

[quote=“frabrunelle, post:2, topic:1804”]
No need to use MongoDB or store data locally.
[/quote] That’s been the hardest part for me to digest. Say i was running my website on my raspberry pi with nodejs and mongodb. Someone logs on to Maidsafe, pulls up my website via my pi and from my website extranet page, runs a query on all the labor orders they’ve placed with me. If i don’t securely house that data for my customer, I’m in deep doodoo. Yes, i take crypto but most of my Clients still pay with a cc, for many years to come I’m sure. Sorry if that sounds defeatist, i am extremely optimistic about decentralization, it’s evolution. I’m still thinkin that nodejs and a db app such as mongodb will be required to serve up websites, customer records, video streams, etc. Sooo, everything just becomes “localhost” in a sense? Sorry, jus thinkin out loud again hmmmf


#4

Obviously you wouldn’t serve such data that way.

You’d serve less active (usually data-at-rest) data and those that aren’t highly proprietary and confidential. At least initially.

It is possible to serve any kind of data (just encrypt and protect parts of DB that your user shouldn’t access). But there’s no need to bother with that when there are so many other use cases that are better.


#5

Uugh, ok now I’m even more confused. We write client/server apps, based on javascript, bare metal to gui. If the server is finally going away, I need to be able to plan how our apps are being built right now. With Forbes and the like giving mentions to Maidsafe, it’s only months from now where decentralized meshed computing is finally a reality. Small numbers, but a reality nonetheless. We have 2 little iot prototypes right now that need this, but we’re hung up on how to best proceed with the decentralization part of it. The hardware is Open, but the schemas will have to be chosen beforehand so we can show the proof of concept. This stuff is so cutting edge, it’s tough to see the clear path to profit without all of it actually running on our mesh here. Sorry janitor, I’m just fishing for as many ideas as possible 8) ken


#6

Ken, I’m just a future user, I’m not speaking for the Project so I may be wrong (and I may be misunderstanding your requirements as well - assume the worst).

If the DB is on the SAFE network, then it’s not cached locally and it’d have to be fetched form the nearest cache (or node that has the info, if that info is new so cache won’t help you). This can work, but if you have a huge DB or human users, they’ll wait at least 100% longer than they would with a normal site because of the expected (your node has to pull data from elsewhere before it can process it and pass it to your Web site users).

If your clients are native MaidSAFE clients, then they can talk to the network directly.

However not every app lends itself to either of these approaches. If you have a huge DB or if you need to allow dozens or hundreds of users to nearly concurrently (e.g. workflows, ERP, CRM) update the same content I don’t see how you can run that on a distributed architecture.

It’s not a “problem” of MaidSafe, it’s a problem of solution design. If you have a PB sized project you’d have to partition data appropriately. You could also perform intelligent caching and do other things to potentially execute your project on MaidSafe.


#7

I’m getting the impression from David Irvine’s comments that these kind of apps will be implemented directly onto SAFE’s Key/Value store (providing a native database API as part of the virtual drive API). Again, based on David’s comments, I predict that there will be a demo app showing this in action as part of testnet2. Prepare to be blown away folks! … and to a change of T-shirts :wink:

@kenCode I’ve moved this from Off-topic to Development


#8

Hey wait on, I haven’t got mine yet…looking forward wearing it to the local Open Mic night :slight_smile:


#9

[quote=“janitor, post:6, topic:1804”]
update the same content I don’t see how you can run that on a distributed architecture.
[/quote] Me too and that’s where I’m hoping that the masterminds behind the MaidSafe protocol (and others like Hyperboria/cjdns, projectmeshnet) can help. Hashing it out on this forum is awesome tho, as a Developer you realize that you can’t think of everything, this is a team effort.
My thoughts are that absolutely everything will be distributed/shared in the future. Think Jacque Fresco of The Venus Project.
200 years from now, either google won’t exist, or they will be the Buy n Large like in the movie Wall-E. The utopian side of me screams for decentralization, but reality says that the next 50 years can only give us so much. Some centralization must exist as long as there are businesses, and ones that must track their customers data, locally. ie: Amazon shopping.
So, it looks like business databases won’t go away in the next 50 years, hence no total decentralization. MaidSafe will be best at serving static data. Live, real-time data will still have to be served from one node. Even a live video feed must be broadcast from just one camera lens.
If I owned a super popular art gallery, only one person could buy this painting or that one, so caching will never work for one of a kind, competing sessions like this.
As long as there are businesses, there will be centralization, even if it’s just customer records.
Thank you @janitor @frabrunelle and @happybeing for all of your insights into this :slight_smile:


#10

[quote=“kenCode, post:9, topic:1804”]
Some centralization must exist as long as there are businesses, and ones that must track their customers data, locally. ie: Amazon shopping
[/quote] Ok, so I just read about OpenBazaar.org and looks as if the customer or the seller will maintain their own records in their cryptocurrency wallet.
So, please belay my last post RE centralization being needed because of the way businesses work. I would FAR prefer p2p private business, where my digital wallet is my extranet to transactions placed with this merchant or that. In my digital wallet, I can run reports and see who I paid for what and when. A central authority like ebay storing my personal details just so that they can get their cut is stupid.


#11

Correct me please if I’m missing something, but the only “databases” left will be your cryptowallet and aboutme api. My about me info like tagline, headline, bio, background image, work history, avatar/gravatar etc has to be maintained across many different websites. G+, FB, Twit, kencode.de, etc. If companies like Yext were free that would be great, but you still pay them to centralize your bio data. Not cool. I’m thinkin everybody needs to have a decentralized CryptoWallet AND CryptoMe. This about me “database” is broken up and stored across the entire MaidSafe mesh, just like the blockchain. If absolutely everything, including business, becomes decentralized, then a db like MongoDB will never be needed again, right?