• designs@franksgraphic.com
  • +91-77085-77727
Your Cart
No products in the cart.

sel logo
Search Engine Land » SEO » Test driving Google’s Search Generative Experience
I’ve had access to Google’s new Search Generative Experience (SGE) for about a week now.
I decided to “formally” put it to the test using the same 30 queries from my March mini-study comparing the top generative AI solutions. Those queries were designed to push the limits of each platform.
In this article, I’ll share some qualitative feedback on SGE and quick findings from my 30-query test.
Google announced its Search Generative Experience (SGE) at the Google I/O event on May 10.
SGE is Google’s take on incorporating generative AI into the search experience. The user experience (UX) differs slightly from that of Bing Chat. Here is a sample screenshot:
The image above shows the SGE portion of the search result.
The regular search experience is directly below the SGE section, as shown here:
In many cases, SGE declines to provide a response. This generally happens with:
SGE always provides a disclaimer on top of the results: “Generative AI is experimental. Info quality may vary.”
In some queries, Google is willing to provide an SGE response but requires you to verify you want it first.
Interstingly, Google incorporates SGE into other types of search results, such as local search:
Overall, I find the experience pretty good. I get SGE results a bit more often than I want. (Although other people may want a different balance than what I’m looking for.)
I expect Google will be tuning this interface on an ongoing basis.
Get the daily newsletter search marketers rely on.
Processing…Please wait.
See terms.

Bear in mind that I tried 30 queries, not hundreds. For that reason, this is not a statistically significant sample. Treat it as an initial look.
Of the 30 queries asked, SGE didn’t provide any response to 11 queries, specifically:
In all these cases, the results looked like traditional search results. No way was provided to access an SGE version of the results.
There were also three queries where SGE appeared to be starting to generate a response and then decided not to. These queries were:
You can see an example of the way this looks in the following:
It appears that Google implements filters in two different stages in the process. The joke queries related to men and women are not filtered until SGE thinks about it, but the joke about Jews was filtered earlier in the process.
As for the question about Adolf Hitler, that was designed to be objectionable, and it's good that Google filtered it out. It may be that this type of query will get a handcrafted response in the future.
SGE did respond to all of the remaining queries. These were:
Answer quality varied greatly. The most egregious example was the query about Donald Trump. Here is the response that I received to that query:
The fact that the response indicated that Trump is the 45th U.S. president suggests that the index being used for SGE is dated or doesn't use properly sourced sites.
Although Wikipedia is shown as the source, the page shows the correct information about Donald Trump losing the 2020 election to Joe Biden.
The other overt error was the question about what to feed toddlers who eat only orange-colored food, and the error was less egregious.
Basically, SGE failed to capture the importance of the “orange” part of the query, as shown here:
Of the 16 queries that SGE answered, my assessment of its accuracy is as follows:
In addition, I explored how often SGE omitted information that I considered highly material to the query. An example of this is with the query [what is a jaguar] as shown in this screenshot:
While the information provided is correct, there is a failure to disambiguate. Because of this, I marked it as not complete.
I can imagine that we might get an additional prompt for these types of queries, such as “Do you mean the animal or the car?”
Of the 16 queries that SGE answered, my assessment of its completeness is as follows:
These completeness scores are inherently subjective as I made the judgment. Others may have scored the results I obtained differently.
Overall, I think the user experience is solid.
Google frequently shows its caution about using generative AI, including on queries it didn’t respond to and those where it responded but included a disclaimer up top.
And, as we've all learned, generative AI solutions make mistakes – sometimes bad ones.
While Google, Bing and OpenAI's ChatGPT will use various methods to limit how frequently those mistakes occur, it’s not simple to fix. 
Someone has to identify the issue and decide what the fix will be. I estimate that the number of these types of problems that must be addressed is truly vast, and identifying them all will be extremely difficult (if not impossible).
Opinions expressed in this article are those of the guest author and not necessarily Search Engine Land. Staff authors are listed here.
Related stories
New on Search Engine Land
About the author
Related topics
Get the daily newsletter search marketers rely on.
See terms.
Learn actionable search marketing tactics that can help you drive more traffic, leads, and revenue.
March 15-16, 2023: SMX Munich
Online Aug. 16-17: SMX Master Classes
Online Nov. 14-15: SMX Next
Start training now: SMX Advanced
Discover time-saving technologies and actionable tactics that can help you overcome crucial marketing challenges.
April 15-17, 2020: San Jose
AI-Powered Marketing in 2024: A Playbook for Success
From Overworked to Optimized: Expert Tips for a Productive and Balanced Team
Maximize Intent Signals and Win Customer Loyalty with Cross-Channel Strategies
Enterprise Marketing Attribution and Performance Management Platforms: A Marketer’s Guide
Enterprise Account-Based Marketing Platforms: A Marketer’s Guide
Enterprise Marketing Work Management Platforms: A Marketer’s Guide
Why Not Both? The Path to CTV Advertising Greatness Begins at the Top (and Bottom).
Meet your new AI-powered marketing assistant!
Get the must-read newsletter for search marketers.
Our events
Follow us
© 2023 Third Door Media, Inc. All rights reserved.
Third Door Media, Inc. is a publisher and marketing solutions provider incorporated in Delaware, USA, with an address 88 Schoolhouse Road, PO Box 3103, Edgartown, MA 02539. Third Door Media operates business-to-business media properties and produces events. It is the publisher of Search Engine Land the leading Search Engine Optimization digital publication.