We have grown accustomed to the mediocrity of web search engine's results. This proposal, really a Request For Comment (RFC) outlines a better way to do search - an open, fair, balanced approach.
We have the massive powerhouse of Google, Microsoft's Bing maintains some market share. International search engines sometimes dominate their markets and DuckDuckGo offers search with privacy included. Do we really need an overhaul of web search? Is there room for another competitor? Won't Google simply appropriate any true innovations negating the need for nextsearch?
We'll discuss each of these questions - for now lets focus on the significant reasons web search needs an overhaul.
While several engines are now offering privacy-centric services the majority of web traffic is still funneling through search engines that provide minimal protections.
In an endeavor to give us the most relevant results search engines no longer give us the best results. We find ourselves in an echo chamber that reinforces our preexisting beliefs and biases.
We all know that search is dominated by one major player and several smaller players - this is a problem...but there is another problem, perhaps just as serious - the monopolization of search results by large corporations and content farms. Try searching for product reviews and you'll find the results littered with sites of questionable trustworthiness offering seemingly in-depth reviews. The opportunity for smaller players to become well-known is limited without significant capital expenditures / luck.
Yes, search engines do index more frequently and offer relevant results for current events - but for non-event oriented information (such as software documentation, educational articles...) it does much more poorly. This demonstrates itself in two paradoxical extremes:
- New content is overvalued and older but high quality content is buried.
- Older, lower quality content is overvalued and high quality, newer content is buried.
While software largely trends towards open source, data, especially that used by search engines to generate their results is proprietary. It cannot be reused or improved upon.
Analysts spend significant amounts of time dissecting how the algorithms the search engines utilize for ranking results operate but the level of insight is limited. Are the algorithms fair? Are there biases or errors? We have no idea.
Search Engines lack the necessary tools to allow individuals and organizations to organize and optimize their research. Nor do search engines generally provide particularly advanced query parameters or the ability to save such parameters.
There is a great deal to be learned from the proper use of machines to understand data. But one of our main tenets will be that search engines fail to utilize the massive computing power of their aggregate user bases - that is, the human mind.