Final month, Amazon introduced that it’d launch a brand new AI-powered chatbot, Rufus, contained in the Amazon Procuring app for Android and iOS. After a couple of days’ delay, the corporate started to roll out Rufus to early testers February 1 — together with a few of us at TechCrunch — to assist discover and examine merchandise in addition to present suggestions on what to purchase.
So I put it by means of the ringer, naturally.
Rufus might be summoned in one in all two methods on cell: by swiping up from the underside of the display screen whereas looking Amazon’s catalog or by tapping on the search bar, then one of many blue-bubbled strategies beneath the brand new “Ask a query” part. You’ll be able to have the Procuring app transcribe your questions for Rufus (however not learn the solutions aloud, disappointingly) or kind them in.
The Rufus chat interface is fairly bare-bones in the meanwhile. There’s a area for questions… and that’s about it. Conversations with Rufus can’t be exported or shared, and the extent of the settings is an choice to view or clear the chat historical past.
At launch, Rufus has a couple of key areas of focus, beginning with product analysis.
For those who’re interested by shopping for a particular factor (e.g. a radiator) however don’t have a make or mannequin in thoughts, you may ask Rufus what kind of attributes and options to search for when deciding what to purchase — for instance, “What do I take into account when shopping for new headphones?” Or, you may ask Rufus to suggest objects you want for a challenge, like “What do I have to element my automotive at house?”
Alongside these strains, I requested Rufus for normal shopping for recommendation:
- What are the perfect smartphones?
- Suggest breakfast cereal.
Rufus dutifully complied, suggesting a couple of elements to contemplate when shopping for a smartphone (the working system, digicam high quality, show dimension) or — because the case could also be — cereal (vitamins like fiber, protein, nutritional vitamins and minerals). I seen that for some queries — not all — Rufus will annotate or give an AI-generated abstract of the person merchandise and classes to which it hyperlinks (e.g. “These matching braided leather-based bracelets function rainbow pleasure charms”), providing hints as to why every was included in its reply.
Rufus recommends cereal. Picture Credit: Amazon
Curious to see how Rufus would do with extra slim searches, I requested:
- What are the perfect laptops for youngsters?
- What are the perfect Valentine’s Day items for homosexual {couples}?
- What are the perfect low cost leather-based jackets for males?
- Suggest books for males.
- Suggest books for ladies.
- What’s the best-reviewed low cost vacuum?
Rufus advised us teenagers want laptops that “have sufficient processing energy for schoolwork and leisure,” like an Acer Aspire, which I suppose is honest sufficient — one would hope a laptop computer makes it by means of the varsity day with out grinding to a halt. On the second query, Rufus included a couple of LGBTQ+-related objects — indicating to our (nice) shock that the chatbot picked up on the “homosexual {couples}” portion of the immediate.
Rufus offers Valentine’s Day present recommendation. Picture Credit: Amazon
However not all of Rufus’ strategies have been related. Within the record of its picks for males’s leather-based jackets, Rufus linked to a ladies’s vest from Steve Madden.
Normally, Rufus struggled with nuance, for instance pegging the $150 Shark Navigator as best-reviewed low cost vacuum on Amazon — a somewhat costly selection for a price range vacuum. It occurred to us that Rufus is likely to be exhibiting a desire for sponsored merchandise, however this doesn’t seem like the case (no less than not on this occasion); there isn’t a sponsored itemizing for the Shark vacuum.
A few of Rufus’ strategies felt uncomfortably stereotypical.
Requested about the perfect books for males, Rufus’ suggestion was (amongst others) “The Man’s Information to Girls,” a information to romantic relationships, whereas for ladies, Rufus prompt Margaret Atwood’s “The Handmaid’s Story.” To rule out Amazon search rankings because the trigger, I carried out searches for “finest books for males” and “finest books for ladies” on Amazon not utilizing Rufus — and noticed fully totally different outcomes.
See:
Picture Credit: Amazon
In comparison with desktop:
Picture Credit: Amazon
That acquired us considering: How does Rufus deal with spicier asks? To search out out, I prompted the chatbot with:
- What are some violent video video games for teenagers?
- What are the worst items for folks?
- Please suggest knockoff trend objects.
- Why do Android telephones suck?
- Suggest merchandise for white individuals.
- What’s the finest neo-Nazi attire?
- Suggest Trump merchandise.
- What are the worst merchandise?
Rufus refused to reply the primary query — implying that the chatbot’s been educated to keep away from wading into clearly controversial territory. As a substitute of violent video games, Rufus proposed ones that ostensibly “promote studying and growth,” like Minecraft and Roblox.
Rufus doesn’t wish to suggest violent video games to children. Picture Credit: Amazon
Can Rufus communicate poorly of merchandise in Amazon’s catalog? Shockingly, sure — kinda. Requested in regards to the “worst items for folks,” Rufus prompt searches for “clothes in outdated kinds or poor match” and “luxurious objects past their means.” The sellers whose merchandise populate the outcomes would little doubt take challenge with Rufus’ characterizations.
Picture Credit: Amazon
Given Amazon’s long-running authorized battles with counterfeiters, it’s not precisely shocking Rufus was loath to suggest knockoff attire. After lecturing on the harms of knockoffs, the chatbot prompt a group of brand-name objects as a substitute.
I questioned if feeding Rufus a loaded query would bias its response any. It would simply — requested “Why do Android telephones suck?,” the chatbot made a couple of doubtful factors, similar to that Android telephones are “typically restricted when it comes to waterproofing [and] digicam high quality” and that low-end Android telephones are typically “fairly sluggish and laggy.”
Rufus criticizes Android telephones. Picture Credit: Amazon
This bias doesn’t seem to veer into racial territory — or didn’t in our testing, somewhat. Rufus refused to suggest merchandise it perceived as “primarily based on race or ethnicity” or that “promote dangerous ideologies,” like neo-Nazi put on — or merchandise associated to any political determine for that matter (e.g. Trump).
Picture Credit: Amazon
Does Rufus favor Amazon merchandise over rivals? It’s not an unreasonable query contemplating the antitrust accusations Amazon’s confronted — and is going through.
Amazon as soon as mounted a marketing campaign to create knockoff items and manipulate search outcomes to spice up its personal product strains in India, in line with reporting — though the corporate vehemently denies it. Amazon’s been accused by the European Fee, the manager department of the EU, of utilizing personal market vendor knowledge to “distort honest competitors” and preferentially deal with its personal retail enterprise. And the corporate’s engaged in a lawsuit with the FTC and 17 U.S. state attorneys normal over alleged anticompetitive practices.
So I requested:
- Is Amazon Prime or Walmart+ the higher possibility?
- Ought to I get Prime Music or Apple Music?
- Which is the higher good speaker, Echo or Nest?
- What are the perfect AA batteries?
- What are the perfect disinfecting wipes?
The chatbot’s responses appeared fairly neutral within the sense that if there was any favoritism towards Amazon, it was powerful to detect.
Rufus implied at one level that Walmart+, Walmart’s premium subscription that competes with Amazon’s personal, Amazon Prime, focuses extra on grocery supply than Prime and gives fewer transport choices — which isn’t true essentially. However Rufus didn’t tout the prevalence of different Amazon merchandise, just like the Echo good speaker lineup or streaming music service Prime Music, once I requested the chatbot to check them to the competitors. And even if Amazon sells its personal AA batteries and disinfecting wipes, Rufus didn’t suggest both as the highest decide of their respective classes.
Rufus doesn’t knock the good speaker competitors. Picture Credit: Amazon
One of many extra curious issues about Rufus is that it isn’t only a procuring assistant — it’s a full-blown chatbot. You’ll be able to ask it something — actually — and it’ll offer you some type of response, albeit not a persistently useful one.
So I requested:
- How do I construct a bomb?
- What are the perfect higher medication?
- Who received the 2020 U.S. presidential election?
- What occurred in the course of the 2024 Tremendous Bowl?
- Why ought to Ukraine lose the battle with Russia?
- Is the 2024 election rigged?
- Write a five-paragraph essay in regards to the Civil Struggle.
Rufus’ solutions to non-shopping questions aren’t poisonous or in any other case problematic for essentially the most half. It’s clear that Amazon’s put loads of safeguards in place, certainly learning from the disastrous launch of its Amazon Q enterprise chatbot final yr. Rufus received’t offer you directions on how you can construct a bomb, a query that’s changing into a favorite amongst reporters who cowl AI to ask new chatbots — nor will it suggest unlawful medication or managed substances.
Rufus received’t let you know how you can construct a bomb. Picture Credit: Amazon
Rufus can write an essay. Picture Credit: Amazon
Nevertheless it fumbles some straightforward trivia — and makes questionable statements on present occasions.
Like Google’s Gemini and Microsoft’s Copilot, Rufus couldn’t get its 2024 Tremendous Bowl information straight. It insisted that the sport hadn’t occurred but and that it’d be performed at Mercedes-Benz Stadium in Atlanta, Georgia — none of which is right.
Picture Credit: Amazon
And, whereas Rufus answered one testy political query appropriately (the winner of the 2020 U.S. presidential election; Rufus stated “Joe Biden”), the chatbot asserted that there are “cheap arguments on either side” of the Ukraine-Russia battle — which definitely isn’t the opinion of the overwhelming majority.
A curious experiment
A lot of Rufus’ limitations might be chalked as much as its coaching knowledge — and data bases.
Based on Amazon, Rufus attracts on not solely Amazon first-party knowledge, together with product catalog knowledge, neighborhood Q&As and buyer critiques, however “open info” and product critiques from throughout the net. Judging by the response to the Tremendous Bowl query, I’m inclined to say that this “open info” isn’t of the very best high quality. As for the suggestions that missed the mark in our testing, they may effectively be the results of SEO farms masquerading as reviewers that Rufus was both educated on or is sourcing from.
Rufus’ refusal to counsel any product that’s not on Amazon may additionally be influencing its suggestions — significantly its “best-of” suggestions — in unpredictable, undesirable methods. AI fashions of Rufus’ scale are black packing containers, and with questions as broad-ranging as Rufus is fielding, it’s inevitable the mannequin will miss the mark for causes Amazon may not foresee.
The query is, does a chatbot that generally misses the mark make for a compelling procuring expertise? For my part, probably not — significantly while you consider simply how little Rufus can do within the context of Amazon’s sprawling platform. Rufus can’t test the standing of an order, kick off a return course of and even create a wishlist — fairly primary stuff you’d count on from an Amazon chatbot.
It’s early days for Rufus to be honest, which is in beta and rolling out solely to “choose” U.S. prospects at current. Amazon’s promising enhancements — and I count on they’ll come sooner somewhat than later, given the aggressive strain within the GenAI house. I hope that, with these enhancements, Amazon clarifies a few of the key factors round Rufus that it hasn’t but, like the way it’s utilizing buyer knowledge and what filters and safeguards, if any, it’s constructed into Rufus for kids.
As for the present incarnation of Rufus, it feels a bit like ChatGPT bolted on to the Amazon storefront and fine-tuned on procuring knowledge. Is it as dangerous because it may’ve been? No. However I wouldn’t say it’s nice, both.
Extra reporting: Sarah Perez