Inspired by the SAFE Network, I started trying to learn a bit of coding a while ago, and as a project for that purpose I’ve made a little basic search app on Alpha 2. It’s just a basic keyword search and has almost zero security, but you can add or edit your own site, and search what’s already there.
More interestingly though, I was keen to start a discussion about search from a slightly different point of view to the way it has been discussed before.
Generally, we tend to think of search as everything that Google does, but obviously that’s actually a set of different but interlinked functions, and I’m curious as to the possibilities of separating out those functions, in order to make a search system that is more transparent, resilient and decentralised.
The following points are some of the steps I’ve noticed that are necessary for a search system, and although I have only a very cursory knowledge of the field, sometimes I think it’s interesting to look at things in this way, if for no other reason than to keep in mind that ideal of transparency, so that idiots like myself don’t suddenly get to a point where they feel gratuitously controlled by algorithms!
Basic indexing is obviously a pre-requisite of search, and at its most basic level the information for this is held (though not necessarily in an accessible format) in the NRS system.
A database of all the relevant information about sites needs to be built on top of this. The format of this data, how it is sourced, how it is stored, and who owns it, are themselves all hugely complicated and technical questions, and could easily be broken down further.
The search function itself, which is the bit that people first think of, is vaguely attributable to algorithms, and returns the results that whoever has written the algorithm thinks that we most want to see. These can be very useful, but are in themselves a huge driver of centralisation on the web.
Security, and deprecation of malicious sites, which may be achieved using many of the same objectives of the search algorithms.
Interested to hear anybody’s thought on how any of these elements could be de-centralised, and whether some of them could or should be taken care of on a network level.