Google Lens, the AI search tool in Gemini’s collaborative environment, has seen an impressive evolution in 2024, especially with the introduction of artificial intelligence-based capabilities that allow searching both images and videos.
These new features not only improve the accuracy of search results, but also open up a range of possibilities to solve day-to-day problems and optimize the online shopping experience.
Below, we explain the main new features of Google Lens and how they will impact the way we interact with technology.
Searches with video
Until recently, Google Lens only allowed searches based on static images, which already made it easier for users to identify objects, plants, animals and monuments, among other things, simply with a photo.
However, it is now possible to search using videos, which is a significant advance in the tool’s ability to understand context and provide more detailed answers.
This new functionality, which was presented at Google’s I/O conference in May 2024 and launched in October, allows users to record videos directly from the application and ask questions about what appears on the screen.
For example, if you are recording a video of your plants and you notice that they don’t look good, you can ask Google Lens “why are my plants like this?” and Google’s AI will process both the video and audio to give you a response in real time.
Multimodality in search
The ability to search with videos is especially useful for situations where describing an issue with just words or a still image could be difficult.
The most innovative aspect is that Google Lens is not only limited to identifying objects in videos, but also uses Google’s Gemini artificial intelligence model.
Gemini to analyze the content and provide answers based on a summary of information available on the web, known as AI Overviews. This facilitates a deeper and more dynamic understanding of what you are looking for.
Using video adds a multimodal layer to search, combining moving images, text, and audio.
This not only allows Google Lens to provide more complete results, but also reduces the need for users to ask additional questions or clarify their queries.
For example, instead of trying to describe an error on your car dashboard, you can simply record a video and let Google’s AI interpret the whole thing to offer you a solution.
Improved shopping integration in Google Lens
Another of the great new features of Google Lens in 2024 is its improved integration with e-commerce platforms.
Google has recognized the potential of Lens as a tool for product identification and, as a result, has optimized its functionality to make online shopping easier.
Now, when Lens identifies a product in an image or video, it not only displays basic information but also provides additional details such as price, user reviews, and offers available in different stores.
This feature, known as Shopping Graph, has been designed to work across key categories such as clothing, electronics, toys and beauty products, allowing users to compare prices and make more informed purchasing decisions.
For example, if you see a backpack you like, just take a photo with Lens to receive all the necessary information, including the best offers to buy it online.
In addition, Lens allows you to perform specific searches related to products. If you see a sofa in a store that you like, but you need it in another color, just take a photo and type “red color” in the search box.
Google’s AI will find available options that match your personalized search.
Voice and text: Combining images with spoken questions
In addition to video-based and shopping searches, Google Lens has improved its still image search capabilities by adding the ability to ask spoken questions.
This functionality is available in English and allows users to perform searches that combine an image with an oral question.
This is particularly useful for cases where visual information alone is not sufficient and more context needs to be provided.
So, you could take a photo of a plant and ask “how much water does this plant need?” to get a specific answer.
What can we expect from the future?
The new Google Lens features are just the beginning of what Google has planned for visual search and artificial intelligence.
The ability to search through video opens the door to future applications in areas such as augmented reality and real-time assistance
In fact, other companies such as Meta and OpenAI are already working on similar tools, which suggests that we are at the beginning of a race to develop technologies capable of understanding the world through moving images.
With the ability to process videos and offer answers using artificial intelligence, the tool is positioned as a powerful visual assistant that helps us solve everyday problems faster and more efficiently.
In addition, its integration with e-commerce makes it easier than ever to find and buy products directly from our mobile.
This post is also available in: