Point your camera at a plant, a shop window or a page of text and Google Lens will try to tell you what it sees — translate the words, identify the species, find the product online, or help with a homework question. It’s not magic; it’s the practical combination of machine vision, search data and a big knowledge graph working together to turn pictures into useful results.
What Lens can do
- Read and act on text: Lens can recognise printed words and numbers in photos or the live camera view. That lets you copy text to your phone clipboard, call a phone number, open a URL, or translate a menu or sign into another language. - Identify things: point it at plants, animals or everyday objects and Lens will try to match what it sees to similar items in Google’s index and say what they are. - Recognise landmarks and places: for buildings and famous sights it can surface basic facts, and often images and web results related to the location. - Shopping help: point Lens at a product or label and it will look for that item or similar ones online so you can compare prices or retailers. - Homework help: Lens can take a look at textbook problems or handwritten equations and link to explanations or worked examples that can help you understand the steps.
These features work whether you use the live camera or analyse an existing photo from your gallery, so you can check something you’ve already snapped as well as things you point your camera at in the moment.
How it works, in simple terms
Google Lens combines two broad technologies: computer vision and Google’s search knowledge. On the computer vision side, machine-learning models trained on millions of images learn to spot shapes, patterns and text. When Lens sees a photo it first tries to detect the important parts of the image (text blocks, faces, objects) and then classifies those parts — for example, ‘‘this looks like a Labrador’’ or ‘‘this is a paragraph of English text’’. That process is driven by neural networks similar to those used in other image-recognition systems.
Once Lens has a visual guess, it links that guess to web information. This is where Google’s search index and Knowledge Graph come into play: recognized items are matched to web pages, product listings, encyclopaedia-style facts and translation services to give you context and actions (buy, translate, call, read). For text it uses optical character recognition (OCR) before passing the result to translation or search.
Some of this analysis can happen on the device, while more complex lookups or the retrieval of web results involve communicating with Google servers — that’s how Lens pulls in product results or background information.
Good to know
Lens lives in a few places: the Google Assistant and Google Photos on many Android phones, and in Google’s apps on iOS. Some camera apps on newer phones also include Lens features. Results are suggestions and aren’t perfect — framing the subject clearly and using a good-quality image makes a big difference. Also bear in mind that images you ask Lens to analyse may be sent to Google’s services to produce the result.
This article has been restored to the What's New On The Net archive as part of the site's relaunch.
