How bad it would be a Safe Search Engine based on a centralized server?

I know it’s possible to build a Search Engine on a traditional way and use safe messages for the API communication instead of http. I would use Apache Solr + some scraper in whatever language to achieve this (and run it on a DigitalOcean machine). It would be easy and quick to set this up, but how bad it would be? Would be this considered a “true” safe way to develop a Search Engine or it would be considered just a workaround?

When I think in SAFE apps, I imagine them working in a total decentralized way, with no servers. And so, an application built with central servers would be against this idea. Plus, it would be vulnerable to censorship. The server location would be hide, but still would be working on a centralized way. At other hand, I can’t imagine an application like this working as the Seneca’s Decorum protocol - with no servers.

It would be acceptable to implement a Search Engine in that way?

So are you saying the crawler/scraper would be run on a DigitalOcean machine

Then are you saying the user has the search APP running that communicates with the server.

Or are you also suggesting the browser which interfaces to the launcher via the proxy does the communications with the server? If this is the case then just http to the server as well

I think this could be farmed the same way safecoin is. People could be paid for using compute cycles to index content on the safe network

Yes and it would be saving the results (safe pages) in the Solr database. Also, in this DigitalOcean machine, it would be an app built in, let’s say, python listening for MPID messages (user queries) and answer them (this app it’s our “server”).

The App would be a safe website. When the user asks for a query (eg.: “cat videos”), the JS would send a MPID message to our server (its public ID is hardcoded on JS), which would process it and answer with the results.

That sounds feasible.

Do we want to go that way? Not sure, it does remove the decentralised nature people are likely to expect.

Anonymity can be kept if the JS can be forced to supply to the “server” an ID to send the response back to that is chosen by the user. This way an ID can be created anytime by the user which prevents grouping of search details by the server. As you say the JS has hard coded the server’s ID which is not what I am talking of here.


Perhaps from a user point of view I’d rather see a “server” doing the crawling, which creates a database that is stored using a combination of immutable data and SD objects that an APP the user runs then accesses to do the search.

That way the server never knows what the user searched for and then total anonymity without any copy of searches kept.

To ensure this happens the APPs code would have to be opensourced for verification. The crawler software can be kept private if desired.

If its desired that a user keeps a list of searches, to improve future searches, then this can be kept by the APP in a user’s data area, never returned to the server or anyone else.

3 Likes

It’s funny because as I was just waking up, I cooked up an idea about search. I’m writing a feature post about it right now… I’ll link it here when I’m done.

[EDIT] Here: https://safenetforum.org/t/semantic-search/7710; not centralized, but requires changes to the core, but altogether a pretty sweet idea :smirk_cat: