We worry about algorithms ruling our world
Some of these concerns are genuine
Others unfounded
But I believe that as of now, in the absence of AGI, any algorithm changes are specific and not exhaustive/global
That gives some comfort
But is there an algorithm modification that would affect all / most of the world today?
The name of the algorithm is obvious – its PageRank
But could it change drastically?
Last week, Google announced planned privacy changes similar to Apple’s, which wiped $230 billion off Facebook’s market cap
The new changes involve privacy-focused replacements for its advertising ID, a unique string of characters that identifies the user’s device
This affects media and agencies
From a privacy standpoint, it actually benefits consumers
But is this change the first step to something bigger?
What if in the future, PageRank itself changes dramatically?
Such an algorithm change would affect everyone
Last year, Google submitted a paper
Rethinking Search: Making Domain Experts out of Dilettantes∗
The abstract is
When experiencing an information need, users want to engage with a domain expert, but often turn to an information retrieval system, such as a search engine, instead. Classical information retrieval systems do not answer information needs directly, but instead provide references to (hopefully authoritative) answers. Successful question answering systems offer a limited corpus created on-demand by human experts, which is neither timely nor scalable. Pre-trained language models, by contrast, are capable of directly generating prose that may be responsive to an information need, but at present they are dilettantes rather than domain experts – they do not have a true understanding of the world, they are prone to hallucinating, and crucially they are incapable of justifying their utterances by referring to supporting documents in the corpus they were trained over. This paper examines how ideas from classical information retrieval and pre-trained language models can be synthesized and evolved into systems that truly deliver on the promise of domain expert advice.
Since then we have heard nothing more directly about this idea
But
If suddenly Google has become privacy-aware .. then we could see it as the first step to a more sweeping change in the future
So, what exactly are the implications of this paper?
1) The logic of PageRank is replaced by a future version of BERT (or similar large language model)
2) Instead of returning a web page as a response to a query, the algorithm returns a response to a question
3) Existing large language models have issues (ex bias)
4) pagerank is based on the index-retrieve-then-rank structure. That means, crawlers first crawl the web, responses to the users query are retrieved from the results of the crawled database and these are ranked. while the model works, it actually does not return the information asked for – but rather the document supposedly containing the information. A large language model based search engine could, in theory, respond like an expert. Despite its limitations, could large language based models do better?
I think currently not but It’s hard to tell in the future
But if it does, its a game changer and this algorithm change could actually impact the whole world … for better or for worse
References
Rethinking Search: Making Domain Experts out of Dilettantes∗
Language models like GPT-3 could herald a new type of search engine
image source:
