"DynamicData" 😻

I recognize this may go against the philosophy of the network, but hey.

I’m talking about bloody S*RVERS … there, I said it :smirk_cat:

Server: A program running on a node, listening for requests, returning dynamically generated data. (HOW? Can we / will we have streams between nodes? But other forms of messaging could also be used, obviously.)

StructuredData blocks can act as proxies for the services: they holds the addresses of the the currently running instances, and they get updated by the owner(s) of the corresponding service (the group of servers, as a whole) whenever an instance goes online or offline. It’s dynamic DNS for the SAFE network.

This is an application layer thing, so anybody can implement it in their application without breaking anything :smiley_cat:

(well, if we have network streams or messaging)

4 Likes

You’re proposing a new data type of DynamicData be implemented to keep track of a piece of routinely updated piece of data and/or location on the network?

Sorry just having a bit of trouble understanding your post.

DynamicData is just a teaser headline thing. I’m not proposing creating anything new, only using what we already have (StructuredData) in a simple and obvious way to implement something many are looking for.

2 Likes

What would the “dynamically generated data” be based on? Can you give a RL example for the use of this?

EDIT:
How would you pay for the server’s services in the network? Couldn’t this be achieved with a website polling the desired data and computing/printing the output?
/EDIT

I can only see servers being utilized in a meaningful way after the network implements distributed computing and servers can be “outsourced” to the network providing decentralized apps.

Okay yes I think I understand :slightly_smiling:

I believe your recommended use-case is currently the popular choice on the forum of how we’ll be able to build dynamic web applications. Once the vault becomes part of the launcher, I think we’ll start seeing a lot of this type of dynamic data idea.

Think of it like this. I upload a file to SAFE, and it’s network hash (which means it’s location on the network) is A1B2C3. If anyone wants to download my file, it’s at A1B2C3. But what if I wat to make constant changes to that file? Or what if I want to update it? That would change its hash (location) on the network (the hash is generated from the file contents itself and so changes with any updates). Then EVERYONE would have to make sure they’re downloading from the most current file hash ALL THE TIME.

Instead, as @Tim87 mentioned, we can instead have “DNS” entries. So instead of sending out the most current file hash every time I update it, and hoping everyone is updating their URLs. I can instead give them the location of a piece of “StructedData” (a file I can read/write to) and have that always point to the most updated file.

This can be extended hopefully to build dynamic websites that store/retrieve information at different StructuredData locations. And can potentially replace the current PHP/SQL infrastructure that 99% of websites use.

4 Likes

Anything that can’t, or is unfeasible to, be pre-rendered and stored as static data before it’s requested. Typically when somebody has a bunch of information that they don’t want public, but they want to provide views on it to certain others on request. Granted, this can be addressed by simple messaging. I’m sure developers will have to find the best approach for all individual use case; it’ll take time to fully explore this new paradigm that the SAFE network represents.

Immutable data can’t be modified though, and structured data keeps its address, so I don’t believe we have this problem here.

If I understand correctly, you are basically running a server to serve dynamic content but use a fixed structured data to find it’s address in XOR space. So once you know the address of the server you can interact with it using Safe messaging / routing directly?

That’s neat, this gives all the feature of running a server but hidden somewhere on safe.

1 Like

Exact. And it could be a group of servers, not just one. They could be owned by one person, a business, a group of friends. A client would pick the one closest node in XOR space to cut down on routing hops (in fact, servers could advertise themselves by more than one addresses just for this purpose.) The servers could be connected to each other via plain IP tunnels because anonymity isn’t a concern within the group and this would decrease sync delay (if we need eventual, or stricter, consistency.)

(apologies for editing this to death after i already posted it)

3 Likes

Actually, “that’s neat” is quite the understatement. With this there isn’t much of a reason for someone that runs a service on a server to not have it listen on Safe. They would get all the fun feature of Safe and would be able to also keep their business running on the clearnet.

1 Like

I haven’t thought the slightest about payment related things. I’m not familiar with how all that works in the SAFE network to be honest, so I’ll leave it that for others to figure out.

I suspect this functionality will be built in:

  1. public shares (ie your pointer to a file) will not invalidate the file pointer even the file content is changed
  2. I know MaidSafe have it in mind to provide a method for people to be notified when data changes. This was under discussion, and I’m not sure if they had any implementation proposals, so it is not certain.
1 Like

I’m not sure we’re talking about the same thing :joy_cat: I’m talking about a way to implement “classic” client/server architecture within the confines of the SAFE network through existing (or planned? talking about messaging) functionality.

About your first point, isn’t StructuredData already “changing content identified by static address”? But I’m sure you know that, so I suspect you’re talking about something else.

I would love your second point be implemented. It raises some questions, because both keeping track of subscribers and sending notifications cost resources.

Messaging can provide this functionality. An App talks to a “Server” at a given address and the server can respond.

You could package upto 100K of data minus SD overheads, in SDs bought for this purpose. Maybe string a few together to transfer more dynamic data. Obviously you would have the user pay for those via the APP with shared ownership, and reusable by the user whenever they run the APP.

Larger quantities of data would be transferred using immutable chunks and transfer the data maps. Eg they want statistical data that others may want so a file is written to SAFE and the user gets the datamap to it.

Your server can also keep a “backend database” using a combination of SDs and immutable files, so the server is also an APP that you can run on another machine if you wish. Otherwise just use the machine’s local disk for backend data. I.E. session data can be SDs and so on.

Is this what you were meaning?

1 Like

Yea, you could say so, but I was more focused on the service discovery part. A service could be published as a SD block, and the operators of the service would keep it up-to-date to always point to the currently available instances (because addresses are valid only until restart, right?) Then you can send them messages, then get responses. There’s already a messaging thingy in the oven, right? (Yea, the response could point to large chunks of statistical data, put together by the service just for me.)

I’m still curious if there will be something like a direct stream between nodes, and by “direct” I mean not one without the usual routing hops, but without intermediary blocks (immutable or SD): just a “pass it on and forget about it” kind of transient stream of bytes, like TCP. We could call it TCP/SAFE, because it would use the SAFE routing layer and not the Internet Protocol.

That is for vault addresses.

Messaging uses account public/private IDs which do not change. Unless of course you scrape them and generate new ones

There has been some “noise” about this and at the moment the idea is to use SAFE to setup the connection between 2 computers securely and then use traditional programs to do the traffic/streams

1 Like

Accounts are about identity, vaults (or other worker-type nodes) are about … work. When I run a service with 23 nodes that all do the same thing, I want the requests balanced between them, so I don’t want to use one mailbox for the requests. One of course could create separate accounts for each, and share their public keys in a bucket…

So you could make the APPs randomly select the address to use from the list you keep updated in the SD. Is that what you were saying?

The updating is to just reflect which are currently accepting the requests.

It’s hard to understand how the safenetwork doing the database base server-client application services, maybe it would clear when the MVP release?

This is one of many ways that a connection between two computers can be established over IP. Nothing more.

That’s not the real problem that we’re trying to solve with the direct computer-to-computer connections. The real problem (with VoIP, MMO gaming, whatever “services” you were talking about, etc.) is how to obfuscate the IP addresses of the participants.

It most certainly is important, as this functionality is the core of the premise of the Network.

This is paramount to any computer-to-computer connection that is made on the SAFE Network. There are more than enough ways to accomplish service discovery - this one included. But a low-latency connection that utilizes only XOR addresses is the real solution here that is yet to be found.


Reading through this thread I can’t help but to name a seemingly perfect example of the “service” that you outline - that being an MMO game server. Is that a fair example?