Google Photos is not shy about asking for your help to improve its algorithms. More than a year ago, the app started asking users simple “yes/no/not sure” questions to refine its facial recognition chops and better group the same face under the same profile. More recently, the app added a similar survey for things, so when you open the sunset (for example) category under Photos’ Search > Things, you get a little box on top where you can confirm whether certain pics have a sunset in them or not. The app is now pushing things further by requesting your assistance in labeling your images from scratch, not just confirming whether or not its assumption is accurate. Basically, you’ll be doing some work for free, if you feel like it, and the end result is everyone gets better image and object recognition.
One of the core principles of Machine Learning is having a properly-labeled training data set so the engineers can build their algorithms from it. In Google Photos’ case, and in simpler terms, that means Google wants a batch of images of a cactus that are labeled as “cactus” so it can teach its system or recognize any other cactus image as a cactus. It also ideally wants lots of cactus images because a cactus can be photographed from different angles and distances, in varied lighting conditions, with/without other things nearby, not to mention the thousands of species of cactus plants and their colors and shapes. The image alone isn’t enough data to train an algorithm, the label is required as well, and that’s where this latest Photos feature/survey comes in.
The option is still rolling out server-side and lives as a little card at the bottom of the Search tab. Once you tap it and agree to do the work, Photos will surface images to you and a text box, and ask you to type what’s important in each particular pic. Think of it as tagging images: day, night, tree, cactus, store, person, balloon, car, building, sunset, flower, etc… You can write as many of these as you want and submit them. The first batch contains ten photos, but you can do more if you feel like it, skip an image if you don’t know how to describe it, or stop the process altogether.
When you’re done, Photos asks if you’d like to try another kind of question, and that’s where you see three more training exercises. One checks if certain pics are print-worthy (likely to aid the monthly print subscription), the other sees which kinds of animations and collages and other automated photo creations you’re interested in, and the third one wants to know whether pics taken on a specific day are related to that day’s known holiday or festivity.
Since these surveys and questions are rolling out via a server-side change, you can’t force them to show up for you. All you have to do is wait and you’d probably soon be able to work for Google for free, in a way.