Bench Talk

A prototype smart contact with 8x8 LED display. Source: University of Washington

Pokémon Go is a new smartphone game by Niantic which uses augmented reality (AR) to allow players to find and “catch” the popular fictional species Pokémon. Using the phone’s camera and gyroscope, Pokémon appear as if they are in the real world. The concept of augmented reality is far from new, but Pokémon Go is introducing many people around the world to the possibilities of AR. Pokémon Go, with its smartphone integration and franchise appeal, has found the intersection between ease-of-access and mass interest, bringing millions of people into contact with AR for the first time. In a few years, people might be augmenting their reality without having to look down at their phone.

AR’s cousin, Virtual Reality (VR), is constantly in technology news and is changing the way we digest entertainment. VR gaming systems such as Oculus Rift and HTC Vive are completely immersive, putting the wearer in a virtual world visually removed from our own. Augmented reality, on the other hand, involves displaying virtual images on top of our real world.

Augmented reality and Google Maps. Imagine this sans iPhone. Source: Wikipedia.org

AR uses computer vision to analyze the geometry of the image that the camera sees, then uses that information to integrate the augmentation, whether that be an information pop-up, a down-line on a football field during a live game, or a Pokémon who wants to be caught. Google Glass was in the news when the prototype was released a while back, spurring interest in AR, but the cost of the hardware meant that many people didn’t get to try it for themselves. As a free app, Pokémon Go provides little barrier for entry into Niantic’s Pokémon-infested, digitally augmented world. But what if we could have reality augmented without having the hassle of pulling out our phones every time we want to catch a Pokémon? Soon we might. Babak Parviz, the leader of the Google X team that developed Google Glass, started his research in the field of smart contact lenses. Seven years ago, he and his students at The University of Washington were wrestling with developing a safe and reliable way to project images into the wearer’s eye. In a 2009 article published in IEEE Spectrum, Parviz proposes a number of uses for these technologically-enhanced contact lenses, including medical and entertainment applications. The team also experimented with powering the contact using radio frequency (RF) technology. His later work with Google X focuses on medical applications, notably a solution for diabetics who need their glucose levels checked often. His smart contact collects tears from diabetics and analyzes their glucose levels, transmitting the data to a smartphone, Google Glass, or other device that the wearer can check on. After securing a patent in 2015, the Google X team working on the lens was spun off as Verily, an Alphabet company.

Google’s work and Parviz’s research have inspired other companies to look into smart contacts. Samsung has their own patent on a smart contact (apparently dubbed “Gear Blink”) with a camera at its center which will allow the user to take pictures by blinking deliberately. Any data gathered will be transmitted via a tiny antenna to be stored on the user’s phone. Samsung’s contact will include a display, which would be the first step into being able to catch Pokémon without holding a phone. Sony has the most recent patent, which describes a lens similar to Samsung’s except that this one will have on-board storage in addition to wireless capability. The Swiss Federal Institute of Technology (SFIT) has their own version of the smart contact which uses special glasses to allow the wearer to magnify their vision 2.8 times, like a zoom lens on a camera.

Sony's patented smart lens design. Source: U.S. Patent Office 

While the current advances in contact technology are certainly eye-popping, it will be a while before you can start catching Pokémon by blinking at them. Verity and the SFIT are working on medical applications while Samsung and Sony are focusing on eye cameras (“iCameras,” if anyone is looking for a name suggestion…) but neither seems to be making significant advances in the display features of their products. A quality display and a way to make it visible to the wearer at such a close distance (see the Parviz article for details on that issue) are problems that need solutions if AR is to be integrated into the smart lens concept. Safety is a key concern, and Google has announced plans to start testing their device on humans this year, but no results have surfaced as of yet. Sony and Samsung, the most likely companies to put out an AR-capable device, have not given much information on the state of development.

Experience historic sites with augmented imagery to imagine what it looked like when it was new. Source: Screen capture of YouTube video of AR-media’s augmented reality application.

When these issues are tackled, the applications could be endless. We could navigate using Google Maps routes superimposed onto the road or read a Yelp review by looking at a restaurant. If you are struggling fixing your car, a smart lens would let you load up an instructional YouTube video to watch in one eye while you fix the engine problem with the other eye. Smart lenses would shake the entertainment world as much as VR is now. Imagine watching a film and toggling subtitles or 3D features without disturbing the experience of the people watching with you. Historic sites would let you experience the world as it was in different time periods. All of these ideas are currently possible with today’s AR technology, but in the future we won’t have to fiddle with our phone or video screens. Our eyes will automatically connect, likely via Bluetooth LE due to the energy restrictions on an eyeball-sized device, to our phones, which will be the hub that organizes our augmented reality. Pokémon Go may not keep its popularity past this summer, but if people are still playing it in a few years, just remember: blink twice to catch that Pidgey.