Researchers from Stanford University created AI models that can identify where a photo was taken with remarkable accuracy. There are good use cases for their solution but important ethical concerns will need to be addressed too. If your holiday...
Researchers from Stanford University created AI models that can identify where a photo was taken with remarkable accuracy. There are good use cases for their solution but important ethical concerns will need to be addressed too.
If your holiday snap has the Eiffel Tower or Mount Rushmore behind you then it’s pretty easy to tell where the photo was taken. If the photo was taken in a field in Illinois with a random tree behind you then it’s a lot more challenging to identify the location.
That’s just the kind of challenge that more than 50 million players of the GeoGuessr game enjoy.
Three researchers from Stanford University were avid players of the game and wanted to see how AI models would fare against the best human players.
They started with a neural network called CLIP, developed by OpenAI, as a base for two different image geolocalization models.
The first model, Predicting Image Geolocations (PIGEON), was trained using a dataset of around 100,000 locations from the GeoGuessr game. For each location, PIGEON was provided with a 360-degree panorama made from a set of four Google Street View images.
The second model, Predicting Image Geolocations with Omni-Terrain Training Optimizations (PIGEOTTO), was trained exclusively on more than 4 million geo-tagged Flickr images from all over the world.
When presented with a Google Street View image of anywhere in the world, PIGEON was able to guess the country correctly 95% of the time and could pick the location within about 16 miles just over 40% of the time.
The researchers then pitted PIGEON against the best GeoGuessr players where it beat them by a significant margin.
PIGEON’s geolocalization distance error vs human GeoGuessr players of various in-game skill levels across 458 multi-round matches. The Champion Division consists of the top 0.01% of players. Source: arXiv
PIGEOTTO, which is a more generalized model, beat the current state-of-the-art models on geolocalization benchmark datasets by a large margin.
When PIGEON went up against Trevor Rainbolt, considered one of the best geoguessers around, it beat him several times. It’s a little reminiscent of Deep Blue beating Gary Kasparov at chess back in 1997.
We built an AI to compete against the world’s best GeoGuessr player and won! 
In @geoguessr, >50 million players compete on guessing where they are based on a random locations on Google Street View. 
Thanks @georainbolt for the fun challenge!https://t.co/g7tIeUELz5
— Lukas Haas (@lkshaas) May 13, 2023
Ethical considerations
The usefulness of these models goes beyond the potential for cheating at GeoGuessr. The paper noted that the models could be used in “autonomous driving, navigation, geography education, open-source intelligence, and visual investigations in journalism.”
As these models inevitably improve in accuracy, privacy issues become more of a concern. Besides identifying your location, other privacy issues include the ability of some models to infer local income, race, education, and voting patterns from image geolocalization data.
There is also intense interest from the defense industry in what these models can do. The Stanford University researchers said, “To our knowledge, this is the first state-of-the-art image geolocalization paper in the last five years that is not funded by military contracts.”
As with a lot of AI tech, the dual-use risks associated with these models raise ethical concerns that will need to be managed somehow.
With the likes of Meta training their models on your Instagram photos, it’s worth remembering that your selfies reveal more to AI than you think.
The post PIGEON AI model knows where you took that photo appeared first on DailyAI.