Perplexity Is a Bullshit Machine | WIRED

"it’s surprisingly unclear what the AI search startup actually is"- according to its CEO it is, depending on when he's asked:

• "an answer engine...
• almost like Wikipedia and ChatGPT had a kid,,,
• a mere “aggregator of information”"

All three cannot be true. The chatobot itself "is more specific... “Perplexity AI is an AI-powered search engine that combines features of traditional search engines and chatbots. It provides concise, real-time answers to user queries by pulling information from recent articles and indexing the web daily.”"

And it does this, according to Wired's analysis, by "surreptitiously scrape areas of websites that operators do not want accessed by bots, despite claiming that it won’t", and - despite its claims of reliability - it "is also prone to bullshitting, in the technical sense of the word."

The full article then sets out how they came to this conclusion, discovering Perplexity in the act of "closely paraphrasing WIRED stories, and at times summarizing stories inaccurately and with minimal attribution... falsely claimed that WIRED had reported that a specific police officer in California had committed a crime".

And all of this without Wired's server logs seeing any trace of Perplexity: it's "accessing and scraping websites from which coders have attempted to block its crawler... Perplexity is scraping websites without permission". They literally caught it in the act, despite "Perplexity claims to respect the robots.txt standard". Forbes and Knight report similar cases.

The article then delves deeper into what it is, and isn't: it isn't selling an LLM - subscribers can choose from a number of underlying models - but a wrapper with “real-time access to the web": it's not an “AI startup” but "as a sort of remora attached to existing AI systems."

Moreover, in at least some cases it isn't visiting and summarizing anything: "WIRED created a test website containing a single sentence... asked Perplexity to summarize ... found no evidence that Perplexity attempted to visit ... it invented a story about a young girl named Amelia who follows a trail of glowing mushrooms in a magical forest called Whisper Woods." Best guess: Perplexity is "approximating ... from related material found elsewhere ... [like] URLs and bits of digital detritus gathered by and submitted to search engine ... like describing a meal by tasting scraps ... [from] a garbage can.