Kagi and search.marginalia.nu are both proof that it is possible to deliver very good results - if you don't stack the incentives against it.
1. The point of search was no longer to provide requested data, but to generate clicks for Google's ad service.
2. Generating clicks for Google's ad service required that exact text search, boolean searches, & everything else useful had to be excised because giving what was asked for reduced engagement.
3. Ads had to be stuck on the top half of the results page, & the second half of the search results, for more clicks to be earned, had to be filled with garbage sites that did not provide what was sought. This encouraged the proliferation of scrapers & bot-generated text sites. Hand-in-hand with this was the elimination of long-tail results, as digging into results might give useful results.
4. It appears that a decision was made at some point to curate & direct answers toward particular results. While much has been made about certain political leanings being almost disappeared by this move, it appears to be much more likely that this was a result of returning results which generated more ad revenue & clicks (which may say more about the sorts of sites Google runs ads on than anything else).
5. In parallel with the dominance of ad-revenue mining, data mining became a major purpose of receiving search requests. Thus the requests for location information on every search, tied in with the drive to personalize results not for the purpose of giving good results, but to give identified users results they were more likely to interact with to both interact with Google ads & give Google more data to suck down & use.
If we could get them to revert back to the 2006-era search engine, where more than just major sites & bot farms are indexed, we would have something useful. But that's not going to happen.
Meanwhile a single Swede with a single desktop class machine in his living room created a search engine so good that I would often switch to it when Google failed.
These days I use Kagi, which has prioritization and block lists (which I don't use because the results are good out of the box).
Wanna know what is really interesting about the Kagi story?
While Kagi is building its own index, for a long time they were kind of reselling a wrapped version of Google + Bing results, but still were extremely much better IMO.
I have two theories:
- either Kagi has some seriously smart systems that read in the first tens of results and reshuffle them
- or more likely in my opinion the reason why results have been so good is because kagi has api access which bypass the "query expander and stupidifier"[1] on the way in to Google and the personalization thing on the way out. That way they just interact with the core of Google search which somehow still works.
[1]: "stupidifier" the thing in the Google pipeline that rewrites
- "obscure-js-lib" (think one that a previous dev used, that I now need to debug
- to "well-knowm-js-lib-with-kind-of-similar-name".
Or decide that when I search for Angular "mat-table" I probably want some tables with mats on even if they don't have anything to do with Angular.
Me: “exactly-this-thing.py”
Google: You misspelled “sorta-related.js”. Here you go.
Me: Did I stutter?!
Tried searching for quotes from the Matrix because of all these AI issues and asked:
"Quote of Agent Smith to Mr. Neo 'How will you speak with no mouth?'"
and got back:
"You're wrong. Agent Smith never refers to Neo as anything other than Mr. Anderson." Completely did not even try to answer the question.
Course, these days it's more Harlan Ellison "I Have No Mouth, and I Must Scream" [1]. Kind of went beyond the Matrix to total thought control and constant machine torture. "How will you search if all results are false?"
[1] https://en.wikipedia.org/wiki/I_Have_No_Mouth,_and_I_Must_Sc...
20 years ago in the "AI" class, the proffesor said, that the biggest problem with (machine) learning, is that sometines, it will learn wrong, and you have to teach it again.
It’s especially serious when searching for UN related documents, which have a special document symbol structure.
e.g. 99.9%+ of the documents under the classification A/C.1/… simply don’t show up with a “A/C.1/“ search, except for a handful. And it’s not like irrelevant results are clogging it up either, as Google only returns 40ish results for the whole internet. When there’s thousands of publicly accessible UN documents in that category…
Now it’s just a way to make the number perpetually go up, sucking every last drop of value out of the system.
Plus the complete lack of vision or strategy from Google’s senior leadership.
I've always built as many as I wanted from the fish I get by accident while building other stuff.
So how does Adtech generate more and more revenue and sells more and more ads year on year?
Simple answer - Fraud.
It's a lot like the credit card issuing banks. Two notable big names, Wells Fargo and Bank of America both "illegally used or obtained consumers’ credit reports, and then applied for and enrolled consumers in credit card accounts without consumers’ knowledge or authorization." [1][2]
Banks had each employee need to sell 50 credit cards a month. Employees sold 50 a month.
Banks needed "line goes up" for every quarter. Banks had each employee sell 100 a month. Employee's tried to sell 100 a month.
Banks needed "line goes up." Eventually market was saturated, yet banks said sell 1000 credit cards a month. Employees replied, "we cannot, market is saturated." Bank said "sell 1000 a month." Employee's responded with "make shit up, open accounts without consumers knowledge." Fraud.
[1] (Wells Fargo, millions of accounts, $3B civil settlement, $3.7B CFPB judgement, 2020), https://www.justice.gov/opa/pr/wells-fargo-agrees-pay-3-bill..., https://www.forbes.com/sites/anafaguy/2023/07/11/bank-of-ame...
[2] (Bank of America, unspecified # of accounts, 2023) https://www.consumerfinance.gov/about-us/newsroom/bank-of-am...
Yes it does. It scales with population, which last time I checked is still going up.
You can click `chart` for an easier visualization
Shifting spend from legacy media, and extracting more value from the sellers' margin.
I genuinely can't think of one impressive thing Google has done in the last 15 years.
Search peaked in like 2009
Maps has only become slower and less informative (I remember when it use to actually display everything that was in a location and not just the popular/paid for stuff) since 2009
Google Docs was incredibly impressive... in 2006 and now almost 20 years later there's been a few QoL improvements, but nothing wow worthy.
And it seems everything else they've done has been shuttered and/or wasn't all that innovative in the first place and usually just trying to copy someone else work but in an uninspired way.
Android is getting genuinely better. Also AndroidAuto.
And the average person won't even notice most of the new features/improvements; perhaps the biggest one is the camera, and that's all done in hardware or AI these days, which is not Android per-se than a photography app (which, again, is usually a Pixel exclusive)
Deep level impressive but maybe just by a company owned by Google is the Deepmind stuff like AlphaFold which recently got a nobel prize and AlphaGo and MuZero. Also you may have heard of the chatgpt/llm stuff that's trendy now, all based on https://en.wikipedia.org/wiki/Transformer_(deep_learning_arc...
Unless you're thinking of Picasa, which is a whole different conversation lol
Sunsetting blog from 2016 for good measure: http://googlephotos.blogspot.com/2016/02/moving-on-from-pica...
I think the last straw was when I was forced to replace the stock launcher just to avoid the built in unblockable google search ads. That made me consider why I bothered with Google at all anymore if all I do is hacks to work around their crappy ad filled UI.
that’s really something you don’t read often
At what ? Stealing user data ?
They are still not able to arrange some widgets on the screen and to underestand that, if an app does not have microphone permission, you don't need a bloody microphone widget on the screen.
Also the search is very lackluster
0: https://workspaceupdates.googleblog.com/2021/05/Google-Docs-...
Only the visible portion of the document is rendered (previously in HTML, now in Canvas). Everything before/after is just the document data.
Maybe you started working on longer docs coincidentally around the same time Docs switched to canvas?
You then have to wait for docs glacial performance to catch up and re-render things again.
It’s genuinely garbage and the “live collaboration” features go unused most of the time.
The fact Google did the initial work on transformers but only OpenAI was able to productize is an indictment of their stagnation more than an achievement.
If websites can't earn a living through ads because LLM's don't send them any traffic anymore, where do the LLM's ingest new information from?
The web continues to deteriorate because of this and nobody has solved this problem.
AI is messy
LLMs have to have something ‘true’ (or at least statistically probable) to validate against for training.
Taking the output from it and feeding it back into itself is, essentially, an Ouroboros. Or like locking a human in solitary.
The more/longer it happens, the more deranged it’s going to get.
It's culmination of decades of NLP researches, but it's also just East Asian predictive text percussively adapted for variable word length language. It must've been clear to whoever core people that it's fundamentally a sluggish demo or at most a cheap outright-sold commodity product.
IMO they would be right unless someone finds a reason that LLMs can't ever be self hosted in the way Google Search can't be. They must have just saw by the way it is that LLM is at best a regularization engine for that magic box.
What do you mean? Anthropic and Google both have widely used products based on transformers.
All of these companies are in the red, but OpenAI has the most revenue.
https://trends.google.com/trends/explore?date=today%205-y&ge...
2017, seven Google employees invent the transformer architecture and publish a paper. Google's investing heavily into ML, with their own custom 'TPU' chips and their 'Tensorflow' ML framework.
2019ish, Google has an internal chatbot they decide to do absolutely nothing with. Some idiot tells the press it's sentient, and they fire him.
2022, ChatGPT launches. It proves really powerful, a product loads of individuals and businesses are ready to pay for, and the value of the company skyrockets.
2023, none of the seven Transformer paper authors are at Google any more. Google rushes out Bard. Turns out they don't have a sentient super-intelligence after all. In fact it's badly received enough they end up needing to rebrand it a few months later.
Classic tortoise-and-hare situation - Google spent 5 years napping, then had to sprint flat out just to take third place.
At worst, Google is in a tie for first.
In the market, I would say both Anthropic and openAI have been able to do that much better than traditional big tech including Google.
E.g. when I start a search session to solve a programming problem (before llms), I will continually search different terms to get to my solution webpage. Then stop. This session metadata and the path I took is highly significant data that can be used to help llms recognise what research itself looks like.
Gemini is Google saying "look at me, I can jump on the AI bandwagon too."
Android has had a bunch of facelifts, but from a user perspective isn't much different to what it was a decade ago. Same for Google Photos.
Google Fiber is dead and all but buried.
Google Fi is just another MVNO.
I find that hard to fathom. I think what you meant to say is "an impressive right that actually got made into a b2c product".
Otherwise you'd have to ignore that they kinda pioneered llms, until OpenAI poached their tech, polished into a (for a consumer) breathtakingly functional "AI"
They also kept researching self driving via weymo etc
On the business side they've also made a significant mark on the programming world via k8s, golang and angular2 among other things
But I'd completely agree with the sentiment that they completely dropped the ball wrt their original target demographic. Beyond the improvements to android, I can't really think of anything since 2010 either that really improved things.
A lot of other companies do this. Nothing special about Google.
"Ground Truth" is truly dead... we've been to 25 states in the last year and the speed limits displayed in Maps were correct about 10% of the time.
It doesnt have all the features in the world but it has some technically impressive ones, least of all the “I can tell you’re all in a meeting room, so I am going to selectively increase the audio where people are speaking and prevent echo from all the speakers”.
I’d love more love for screen sharing, but meet is the only product I see that is getting materially better over time.
Meet team ported advanced audio processing, environmental noise cancelling and camera effects to Firefox, and they work really well.
It's not impressive of course, but interesting enough to notice.
When you hire an ex McKinsey CEO and get the guy who destroyed Yahoo Search to run your search business, this is what happens.
Give the product owner a raise then. Any time Microsoft tries to radically change Office, everyone just gets annoyed, and searches like "where is the print button in excel?" will suddenly skyrocket for a month or two.
They invented the technology behind LLM's. So whether you use Gemini or ChatGPT, that was pretty impressive.
Also, Waymo is crazy impressive. If you're considering Alphabet as a whole (which there's no reason not to, it's all under the same stock ticker).
Those are two industry-changing things, so I think Google's doing OK.
The difference to me between whether a company is able to do something amazing vs. a person working at that company does, is whether they're able to productize it/do something with it or not.
Google is famous for hiring the smartest people in the world, and has been for 20 years. That's good. It hires those people and gives them permission to do whatever they want. That's good too! (maybe not for the sharesholders, as we come to see, but for the world of technology and for humanity)
But if all of the authors of the Transformer paper end up having to leave Google and each start competing startups, that shows that the only impressive thing Google has done is managed to hire those people to do the research. That's it.
They had a goldmine and they wasted it.
What are you talking about? They launched Gemini. They didn't waste anything. And you can't force employees to stay. Employees leaving to compete is as old as Silicon Valley.
Google was just being more conservative about launching it until their hand was forced by OpenAI. And for good reason too, considering all the hallucinations. But then they launched. They didn't "waste" anything.
Depending on how you measure you could claim Facebook is 1st if you grade them on a different scale for being the most "open-source". But quality wise, they're pretty much in the "everyone else category.
Anthropic is new to a definitive 2nd position, and it remains to be seen if it's durable (probably not).
But regardless of how you measure, Google/Gemini is squarely in the "everyone else" bucket.
When they should have been the definitive first - leading the entire industry with every launch, the way OpenAI is.
If you look into 2005-2009, there were products like Chrome, Waymo, Youtube, GCP, Google docs and lot more that were very ambitious. After 2009 I can't name any one product on that realm.
And no, Waymo didn't "come out in 2009". It still hasn't come out except in three limited areas, but it's shown astonishing success where it has. So again, if you're only going to count the year a project was started, and not the decades of work it takes to build a world-changing business... I don't know what to tell you.
Except just... talk about moving the goalposts. Geez. I'm pretty sure there isn't a single company on earth that you're going to find impressive enough.
What was argued was not whether Google created impressive things. I was clear what I meant was Google hasn't started anything to their product suite. Adding to the product suite is more challenging than research project for the company, and it was where Google thrived and which led to success of Google.
? Microprocessors ? Neural networks ?
When google required hosting blogspot content on their servers instead of self hosting that was the end of the freedom there. 2009.
The Transformer.
>I genuinely can't think of one impressive thing Google has done in the last 15 years.
It is strange to see how the "narrative" of Google unfolds. If parents and grand parents said this in 2015 or 2005 it would perhaps be an extremely unpopular opinion. Or I doubt both would have that opinion in 2015 or 2005.
But >complete lack of vision or strategy from Google’s senior leadership > has been a thing since 2003 - 2004 during their IPO. None of their problems happening now is really new. Their "Dont be Evil" BS, their WiFi privacy issues in 2006 - 2007 before Steve Jobs take a jab on stage in 2008. That was before most people thought about privacy. Not developing a browser against Firefox and then Chrome. It took some Firefox developers some 10+ years before they realise may be Chrome and Google isn't what they thought. They basically earn more money than they know what to do with it and had zero discipline on what, how and where to use it. They continue to pay $10 - $20B a year to Apple as default search engine. Apple is very good at extracting value out of Google. While it seems no one at Google cares about it.
Needless to say I have been ringing the alarm bell on google for 20 years. I wished Mozilla take notes earlier. But they have their own sets of problems.
If there is one thing other than search that Google has achieved was they managed to lift up the salary of the whole Tech industry. They single handedly pulled the average salary of programmers up 10% YoY for many years since IPO. To the point in ~2018 - 2022 many really thought Google's starting salary for a new Junior Dev is $200K.
Ironically I think the dearth of innovation is related to consumers having been rendered too passive since 2010.
Nobody has "needs" for Google to address anymore; now we are told what we should want (clear indication that advertising execs now run the show), usually under protest as they forcibly reshape industry standards and kill beloved products.
The infantilization of AI and the subsequent hilarity of Gemini blackwashing white historical figures says all it needs to about the culture shift.
Search is also censored to an embarrassing degree. Queer theory is treated as irrefutable fact and you'll find infinite dubious content promoting it, while conspiracy theory and anything critical of Israel is downranked or delisted altogether.
Early Google gave us lightsabers [for "free"]. Now they sell lefthanded safety scissors and insist nothing has changed. All stars burn out eventually.
By the time I left, almost nine years later, the culture was dominated by fear and conflicting top-down directives — and the autonomy was gone.
The open sourcing and open patenting of VP8, and the work done on VP9 and AV1 were genuine public goods from Google.
What is advertising now.
Is it possible that the "technology" being funded is "the delivery of advertising over a computer network".
Is that "cool technology". If not, then is the "cool technology" serving as bait to lure in ad targets, i.e., is it merely a component of the advertising services technology.
Why not sell or license the "cool technology" for fees instead of hiring "advertising guys". Why can't this unspecified "cool technology" exist on its own. The parent comment implies there is "value" in "the system", presumably independent of advertising.
How would that explain Google search results getting worse though?
Ads only get viewed as long as Search is high quality and people don't switch to a competitor.
Ads fill up the top with sponsored results, but they don't affect the organic results. If by "the advertising guys won" you mean they got more sponsored slots, all that means is they got more sponsored slots. It doesn't affect the quality of organic results.
So I don't understand what your theory is here.
This is what someone thinking long term would conclude. On the contrary, lowering the quality of your search causes people to spend more time in the search engine since they have to try more searches. This in turn increases the probability of clicking an ad.
I didn't like working at AWS for the most part, but I have never seen Google-level dysfunction there. There were a lot of times I disagreed with decision, but I could always understand the reasoning behind it. On the contrary, I can't explain most of the decision being made at Google. The enshittification from the very top has been amazing to watch, even for someone like me who joined only 3.5 years ago. Both senior and mid-level leadership lack a clear vision and the execution has obviously been horrible. Google needs a hard reset if they want to be successful again. I'm not buying the "too-big-to-fail" bullshit.
when the search engine guys won over the diverse team of web surfers.
search engine was the automation of cheap labour. but early search relief heavily on the ranks early sites won by directory era curation.
search engine tech was unknowingly piggy backing on work from the surfers.
it would crumble for sure since there were no more surfers. AI just made it extremely obviously that Its done.
— The Anatomy of a Large-Scale Hypertextual Web Search Engine, Sergey Brin and Lawrence Page, 1998
If only they'd down-rank Yelp, etc.
It was actually later than I expected it to happen but it seems like distinct enough event that it's had reverberation all the way to the present.
What is nice, is that it is small enough user base that it is not worth it for sites to spend a lot of effort on SEO for it.
Anything that requires a tiny bit of creativity of a new change requirement, it is going to be an issue.
It seems kind of unreasonable to expect Google to never experiment with their algorithm; and unfortunately at its core it is a zero sum game. You might be a winner today but a loser tomorrow.
if your concern is about revenue, sharing or referrals or ad placements or ??? then I would point out that it’s very unwise to build a business whose success is based entirely on the whims of another business.
I think search in general is becoming a very poor way to discover content as it is slowly getting planted by LLMs and also for years has been gamified by SEO.
I think that the right model for content discovery is either crowd sourced by a like-minded community, like hacker news or curated; if the curator or community drifts away from your interests, then you have to find a new one, but oddly enough, this can actually be done within the same framework.
Put another way, Google launched 1000 experiments that got +0.2% CTR and seemed innocuous, and now they have a system that only wants to give out Reddit and Quora links.
I've seen this story on Google's private Blind section multiple times. Usually, coupled to discussion about a more cautionary approach taken by the pre-2020 head of search, who was worried about this outcome.
IMHO they're feeling out the same outcome without the benefit of the above knowledge. I don't see this theory well-understood outside Googlers, modulo a pair of popular blog posts, whose name I can't recall, that hit the nail on the head.
(disclaimer: xoogler as of oct 2023, didn't work on search)
I don't just mean filtering out the spam; that's a hard problem on its own. But the good-click metrics... if serving up reddit and quora links at #1 makes more people satisfied with the first result, defining satisfied as 'reads the #1 page and doesn't go back for more', then...
What's wrong with that metric?
That isn't a rhetorical question. It's tempting to claim the users are wrong to prefer it, but not very nice. Why do they do so?
Bravely flattening a root cause analysis to one agreeable-sounding villain, I'd do Goodhart's Law. Click-through rate not being a great proxy for user satisfaction.
i.e. I always clicked reddit first, I used to append reddit to my searches. But I get a very uneasy feeling now that everything has reddit 3 times in top ten. Impossible. It's a big world. It'd be like if Encyclopedia Britannica started just including the snippet-sized version of Wikipedia articles. Not what I'm looking for from that product.
It does not, what google is optimizing for is ad click-through not organic results click-through. The whole model is you will search for X see a couple of ads and a sea of bad results and opt to click on the ad.
It's the same reason that if you search for "graphics card" on amazon and sort by price you will get 20 pages of "Graphics card Holders" and "Graphics Card Elbows". They want you to not actually use the search just click on their ad.
The problem appears to be it's hard for them to realize that people are getting fed up with the antics and are seeing through the manipulations. It's like an abuser or bully, that thinks they will keep getting away with the poor treatment of others, who they think are "below" them. At some point, enough people can take a stand, and not put up with the lack of transparency and black-box manipulations. They are going to keep demanding change or find alternatives.
Show me a business free of dependencies and I'll eat my hat.
And yes, like you alluded to, search engines aren’t the only way to acquire your readership or regular customers. Some media outlets basically don’t even use search engines to drive traffic, like so many social media based businesses out there. There are hundreds of thousands of businesses that barely have a linktree page.
This is how all the subreddits that are remotely related to politics or news got so biased.
Of course I don't know what the alternative is. If I did I would be making that ideal site instead of scrolling HN.
I'm curious how you'd scale that. Google gets 8.5 billion searches per day. 8.5 billion "Ask HN: Where's the best resource for X" isn't going to scale to that.
Google is now moving towards the position AltaVista used to have: everybody is using this search engine because of its breadth of coverage, the users are often unhappy with having to search for a long time and adjust the query many times, and complain about the inefficiency of it.
I hope Kagi will grow and attain a large enough market share to afford indexing more of the web. Maybe we'll have a really useful search engine again, like it's 2001. For a few more years at least :-/
Reddit is a pretty credible alternative for that.
Edit: for example, I was searching for woodworking design software this morning and got a lot of helpful-looking results from reddit.
When that other business is a monopoly, what choice do you have? The rise of google has effectively killed all other sources of traffic. Web sites used to get their traffic from things like webrings, directories and a variety of smaller search engines. Now? Google, or one of its properties, and to a smaller degree Meta are basically it. The curator and community discovery model is a victim of google, not a solution to it.
Also de-ranking SEO spammers is a net positive IMO.
Add blogs, News sites, and forums - to that as well.
My impression is that if you aren’t listed on google, your site is pretty close to dead. In any case, if you somehow managed to drive traffic via other social media, you are one underpaid moderator’s hasty decision from ruin.
The point isn’t about deranking spammers it is about concentrating a huge majority of traffic (and therefore power) in the hands of a single entity which is unaccountable to anyone.
Ideally, we need a fix as well for more regular discory of problems people have. If I want "Norwegian cooking blogs" google will probably return me mostly spam, and waiting for someone on reddit to maybe post something in that category obviously isn't an option.
Typing w w w . b i n g . c o m on a mobile device is are a huge barrier to entry vs just using the google search box Google pays the phone manufactures to bake into the the OS.
If anything I'd say mobile demonstrates that Google doesn't have a monopoly... TikTok literally occupies a majority of gen Z's app usage... 4 of the most used mobile apps (Instagram, WhatsApp, Messenger and Facebook) all belong to Meta. My wife doesn't even know which browser is on her phone (Samsung Internet)...
Despite how awful Google has become in recent years...Bing is still consistently worse.
I think that is basically what they are complaining about, from a content creator's perspective. They are upset that their quality content (according to them at least) is losing to ML generated garbage, and that google engineers don't even seem to be able to understand why that happens.
Perhaps the ranking was terrible before and now is better. Or is it the other way around?
A manually curated system can only truly work with a constrained domain of content/subject matter and tolerance for high lag regarding new information. Wikipedia is an example of what that kind of system would look like.
Arguably, the problem is black-box algorithms and how they are manipulated. The same problem can be faced by any curated community, where there is little to no transparency. Curated can still mean manipulated. People can jump from place to place, but that may not be advantageous, because where they want to be or the most likely places for discovery are not available.
Did you mean supplanted here?
Maybe the future is something like publishers supplying datasets to train models on? Or like how search was built on top of the web, it will be something more organic and ad hoc.
People don't want to read articles, they want to watch short form videos about what's interesting to them.
An interesting side effect is that I think this will lead to more direct-to-consumer sales. Customers will be led straight to your website instead of an Amazon order page.
> An interesting side effect is that I think this will lead to more direct-to-consumer sales. Customers will be led straight to your website instead of an Amazon order page.
That's an excellent outcome if it happens. Businesses should produce quality information and content related to their product category, instead of purchasing ads on other websites. There's a lot of good examples of this.
After switching to DuckDuckGo years ago, and Kagi last year, it's obvious every time I go back to Google how much they have lost the plot.
It'll take another decade before they lose dominance, but the writing is on the wall. Inertia and market position are the only reason they're still on top. Meanwhile, the younger generation barely uses web search, and the tech savvy are starting to drain away more and more quickly.
Startups should be excited. Rather than being the 800-lb gorilla that is going to come take your lunch, Google is the walking dead behemoth waiting to be harvested for conceptual parts.
I might be a little out of touch, but what do they do instead?
Perhaps they are easily distracted and forget what they were searching for in the first place, because they're presented with a shiny object?
Recently I realized that i only !g to google from kagi just a handful of times this year and every time to instant regret .
A marked contrast to couple of years back with DDG !g would instinctive and probably half the time and results felt better in google . Sticking with DDG felt idealistic and the quality second class, not so with Kagi, it feels the $10 pays off every month in much improved productivity.
It is to be noted that Kagi uses google search index as a source, so it is not like Google cannot improve the results or UX technically, just not possible institutionally.
Perhaps there’s domestic Russian things that are censored but that’s far outside my use-case.
Is there anything preventing new search engines? Except scale and servers. But what most of us want is just plain old ordinary search as it existed in 2006, so that is probably reproducible.
I know there are some other engines. Like DuckDuckGo, but just found out they are really just Bing.
Which I guess begs the question, if Google Sucks so bad, why doesn't Bing take over??? It isn't as bad, even if not great.
Edit: After reading more posts. Appears Kagi is doing this.
Is it? How many more pages get published each day? If you include each YouTube video and social media post, it's easily several orders of magnitude. Organizing this kind of firehose isn't easy, neither is prioritizing/ranking it.
2006 was child's play in comparison.
If you don’t believe me, search “watch frozen 2” (or anything else) on google, then on yandex. Both knew what you wanted, but only one is actually going to return the relevant results.
Another factor is that since 2006, social media has displaced blogs. Blogs were a rich source of "authentic" cross-domain links, and the movement of online discussions to consolidated and closed platforms has dried up that well. Some even go so far as to pin the downturn of blogging on the demise of Google Reader.
OTOH Yandex still gives your direct image links and is better at showing you the largest versions of an image (although too often that's just a blurry upscale).
> I barely getting by, I’m eating at the food bank now, I had grossed $250,000 last year
It’s too bad he’s in such difficulty that he has to eat at a food bank, but where did all that money go? And, presumably, decent money earned in previous years?
In a cyclical business, or a business dependent on the vagaries of a giant monopolistic corporation that can change the rules seemingly arbitrarily, it’s prudent to save for a rainy day.
It seems like the idea of building a lean company has totally fallen by the wayside.
I spent a few minutes looking at my search history (filtering chrome history by "google search"), and the vast of my queries are quite simple (e.g. people's names) that google does well on (in fact I find google search for people better than linkedin sometimes).
I also tried a few complex queries and compared them to Kagi:
"How much bitcoin does microstrategy own" -> Google returns the correct snippet from here[0] while Kagi only linked to articles about how much it acquired in the last few days.
"how to pronounce stratchery" -> Google returns the correct snippet from the Stratechery website[1] while Kagi's first result is a spam entry[2] with the wrong pronunciation (the second result is a tweet with the correct pronunciation).
I'd be curious to see more comparisons!
Edit: I just remembered Dan Luu's post (https://danluu.com/seo-spam/) but after looking through my search history, the queries he uses are not at all representative of my day to day searches.
[0]https://www.forbes.com/sites/digital-assets/2024/11/29/micro...
[1]https://stratechery.com/category/about/#:~:text=UPDATE%3A%20....
Some example of something that's hard to find with Google but easy with something else?
I mean some recent global use stats are
Google 89.33% bing 4.15% YANDEX 2.8% Yahoo! 1.33% Baidu 0.83% DuckDuckGo 0.69% (oct 2024, https://gs.statcounter.com/search-engine-market-share)
If Google is so bad why don't people, myself included click on one of the other ones?
I sympathise with Giant Freakin Robot not getting clicks - I'd never heard of them. But that's different from Google being bad from a user point of view.
I just tried clicking on them all - they all work. Baidu is kind of funny as it's all in Chinese and searching The Sound of Music came back with Chinese which Google translated to "The Nun and the Seven Naughty Children!"
I believe because stickyness is a strong force. Google gets worse, but you know it and you know how it fails and what to do to make it better (i.e. how to word your query differently, what parameters to add etc when you get terrible results).
You don't know this for any other search engine, so you need to make a) an active decision to try another search engine, and b) learn how to use it so it delivers what you want.
And I assume that, for most people, Google isn't bad enough to make that kind of investment.
Earlier today I tried to find out what's currently happening in Sudan, is there any shift in the civil war etc. Google was pretty useless. News articles from 6 weeks ago, the best results were probably Wikipedia, but they ranked the Timeline higher than the actual article. I tried with "sudan civil war", "sudan civil war maps" etc.
I tried just now and Yandex actuall provided much better results for what I wanted. They had sudan.liveuamap.com, they showed polgeonow.com, which looks very interesting (can't say if accurate or trustworthy, but definitely topically relevant), sudanwarmonitor.org etc. Compared to Google, they show much less "top 1000 media sites" and more of what look to be topic-experts.
Bing found sudanshahid.org (again, don't know how accurate), multiple arcgis.com-hosted articles, and also included sudanwarmonitor.org. I'd say more Big News Corps than Yandex, but less than Google.
On this search, I'd say Yandex was best, Bing second, Google last. Yet still I use Google as my daily driver, I think mostly because I don't know how far I can trust Yandex, and I have a general bias against MS products. It's certainly not because they don't deliver better results -- but I know Google and it's "the devil you know", until Google becomes too hard to extract results from, at which point I'll be forced to switch.
Any product reviews will be SEO garbage (blogspam too 10 lists). Anything travel will be a page full of ads before organic results, if any. You just know to not even bother so you’re left with the queries that still work.
To me it feels that leadership matters more than we give them credit for. It's just that most leadership sucks.
I agree with you that Nadella has done a spectacular job, and that leadership matters more than most people think it does. I'd call his performance an outlier.
- Top of the page is slow loading AI regurgitation of deeply buried real web results, but often wildly inaccurate. No thanks.
- Next are a bunch of YouTube videos you don't want to watch where you have to wade through dozens of ads and hours of content to get the 3 seconds of information you're looking for. No thanks.
- "Related search" nonsense that nobody ever wants to see but if you click on those you will get more of the above. No thanks.
- Some useless unrelated shopping links you almost certainly don't want. No thanks.
- Way down at the bottom, 2-3 real web search results, non-keyword matched, and only from major mainstream outlets that are part of the Trusted News Initiative (Orwell?!) that have turned into glorified content farms which spit out non-expert written content on every conceivable subject for Googlebot (and now most of this content is AI written or from the cheapest possible third world contractors). No thanks.
Those "real web search results" used to be independent publishers that are referenced in this article, which were often topical experts with deep knowledge on their respective subjects, and those people and businesses have been destroyed over the last few years by Google updates that clearly prioritize their own slop and their "trusted" corporate ally content farms over the independent web.
They also disappears tons of content, and anything critical or outside of the mainstream acceptable narrative is nowhere to be found, sort of like searching for "tank man" from inside China, something everyone in the west used to poke fun of and point to as an example of digital totalitarianism.
If I were running Google search I would immediately roll back all of their search changes to somewhere around 2014-2016, which was roughly the last time you could find true keyword matched web results from a hugely diverse array of expert sources, and then very cautiously reassess. Obviously they would never do that, so I am not sure they can recover from their own demise.
BTW I don't find DuckDuckGo or Bing to be much better, they seem to just mimic Google results. Search is in real trouble.
Direct loss of knowledgeable people is real but it is not the main reason for these systems becoming black boxes.
For every knowledgeable person laid off there's twenty who stay and adjust to the new reality where their future at the company is much less certain. These adjustments vary person to person but literally nobody goes to say "Whoa, I better improve the documentation and share my knowledge so that I'm easier to fire!"
Where people in a large organisations work to make an open source project more resistant to organizational changes (and priorities).
But yes, layoffs certainly has negative side effects.
A month after I left, they contacted me and offered contract work, which i declined. For the following year, employees reached out to me directly for help. It didn't matter that everything I helped with was already documented.
My take is there isn't a solution for proper knowledge transfer in companies. That's why I find it fascinating when people get fired on a whim. You lose so much more than an employee.
I don't disagree that people react as you say and create knowledge silos with the misguided idea that it protects their job, but I do want to point out the consequences of it for any here who may be considering doing so on purpose. Knowledge silos do not secure your job. They in fact limit your ability to advance, limit your ability to collaborate across an organization, limit your ability to adapt, limit your ability to take up other tasks, limit your ability to delegate, and reduce collaboration and project success thereby reducing your perceived performance within an org. There are a bunch more reasons why knowledge silos are bad not just for an organization but for individual engineers living in the silos, but you don't need to take my word for it, there are multiple studies published on this you can find.
People who are uncertain that they're still around in a year are much less willing to put this additional effort which pays off in the long term.
I've been using AI tools to search now. I barely ever use a raw search bar anymore.
I've basically moved all my search to perplexity but would like to do something on prem now that I can fit the 70B models on my machine, I still have to choose a search index (maybe I can self host searXNG) but that still relies in part on what Google classifies as worthy of ranking, no?
I hope webrings and reputation networks can make a comeback somehow so my LLM only has to comb through high quality material instead of searching for a needle of useful text in a mountain of spam.
Search spam is not new, but the use of LLMs simplifies the ability to make pages that look like legit content (increasing the likelihood they’ll show up in search results).
a. I know what website has this information and I go directly there (e.g. Wikipedia or Github or Google Maps)
b. I need a real human opinion or feedback (usually a Google search by Reddit domain only)
c. I need some well-known information that is easy to verify and this is the problem LLMs are very good at
This is >95% of my "surfing" activity. I think I would barely notice major search engines going down one day.
There is value in anyone being able to publish their content without a gate keeper. All content shouldn't be locked within a few walled gardens.
What is really needed is a search that lets me filter to results where the author doesn't profit from me seeing the result. Filtering for websites without advertising might be good enough. Not something Google will ever provide due to conflicting interests.
They have started breaking down search in Gmail as well, I've noticed that for a month or so. Just as it started on Google search, exact, verbatim queries, are not being returned in the result set.
Google's too big to fail already, as evidenced by the complete inept leadership it has had for the past decade, while the stock keeps going up.
Midwit's take would be : "But that's the job of a CEO, to make the stock go up". Nope, what I'm saying is that in spite of the CEO, it is a natural monopoly that is prone to grow. Them missing out the AI wave, which they pretty much created, will go down in history as top idiocracy in companies.
With Chrome's forced departure and ChatGPT actually giving you accurate results their hegemony could be threatened. I would have expected these things to be a wake up call for them, but nope, whoever runs the thing recently decided to start enshittifying Gmail now. Great work, pal, destroy whatever's left!
Sometimes I search the same business type I see under a listing, and it won’t return any results. Not even the business I was just looking at.
Queries like “cafe laptop” returned cafes and highlighted reviews about laptop-friendliness. Now it returns random cafes.
The search experience across Google products feels like this now.
I find that blogs and small sites do not have a chance when looking for a commercial products or when trying to find a review for a product. There is too much SEO spam and fighting for the top positions.
But if you are doing something that cannot be commercialised easily or is very niche your blog will have easy time on Google (programming is not a niche anymore).
Unable to compete with us on a level playing field, the old print media industries resorted to lawyers. They saw independent publishers getting a lot of traffic from Facebook, so they began threatening Facebook with lawsuits if they didn’t show their content more often, and our content less.
Can anyone explain this? How do you sue Facebook for giving bad recommendations?