…you could interact with pretty much anything your mind can dream up.
Disney Research developed a somewhat lo-fi solution for mixed reality that requires no special glasses or singularity type of stuff. Its “Magic Bench” allows people to interact with things that aren’t there, watching the action in 3rd person view, on a screen broadcasting them. It even provides haptic feedback to make it feel like the imaginary character or object truly is on the bench with you.
Why It’s Hot:
1) It’s a great example of technology enabling a physical experience without getting in the way. Historically, augmented/mixed reality required some type of personal technology like glasses/headset, or a phone. This requires nothing from the user but their presence.
2) It shows how Disney is using technology to create experiences that extend its “magical” brand into the digital age.
Thanks to the recent release of Apple’s ARKit, front end developer Frances Ng has created a point-and-translate app. That’s right, simply point your phone at the item you want to translate and if the item is recognized, associated language options will display.
Whether you’re looking to learn a new language in the comfort of your own home or in a foreign land looking for a helping hand, the app is a great example of often-too-rare AR utility.
The UX magic is possible through a combination of Apple’s ARKit and an existing database of about 1,000 learned objects that Ng ported into the app. While Ng says her app is just a demo and she has no immediate plans to take it to market, what’s so remarkable is that while companies like Microsoft spent many years and dollars on mastering object recognition, Ng was able to build her app in a weekend, simply because it’s building off so much past work that’s now freely available and baked into platforms like Apple’s.
Six months ago, Omega Ophthalmics did a small trial of seven patients outside of the US. Their goal was to test for adverse effects of a surgery similar to lens replacements that often accompany cataract removals. The difference? Rather than replacing the cloudy lens with a normal artificial lens, surgeons instead implanted a lens that could be used for augmented reality, interactive sensors, or drug delivery.
Why it’s hot
Although widespread adoption of this technology is unlikely in the near future, scientists, entrepreneurs, and venture capitalists hope that there is a market for such implants in an aging population that wants to be independent for longer. Whether this small trial is successful may pave the way for larger trials to test additional possibilities and risk.
Apple released ARKit, a mobile AR platform that uses Visual Inertial Odometry (VIO) to accurately track the world around it. VIO fuses camera sensor data with CoreMotion data, allowing the device to sense how it moves within a room with a high degree of accuracy, and without any additional calibration. It’s open to all developers who want to come create and play…and it’s also probably luring them away from FB/Snapchat/Googs.
By having the experience live in an app on millions of iOS devices, it’s giving creators (developers) a much wider audience (and incentive) to test and socialize.
Already, ARKit’s sharing platform is seeing some really awesome crowdsourced examples of the tech in action.
See more examples of what developers have already been toying with, go here.
Why It’s Hot:
Democratizing AR to both users and developers (HoloLens who?)
Positioning it’s self the Go-To AR platform
Low-Overhead cutting edge technology for brands to explore and own
A developer named Abhishek Singh recreated the first level of Super Mario Bros. as an augmented reality game on Microsoft HoloLens, giving himself a first-person perspective into a unique 3D look inside the classic setting.
This demo was done in NYC’s own Central Park.
Why It’s Hot
Seeing the game as Mario sees it is totally hot. Watching a guy in Central Park acting like he’s slowly losing his mind: not so hot.
Nike built an augmented reality application called SNKRS for users to gain access to limited-edition sneakers available for purchase. The first sneaker to debut through the app was the Nike SB Dunk High Pro Momofuku, a collaboration with David Chang, creator and owner of the Momofuku restaurant group.
For a user to gain access to the shoe, they have to open the app and point their camera at the menu at Fuki East Village Momofuku in New York. People can still gain access to the shoes elsewhere, as an online menu works as well. Users need to look for a special ‘SNKRS’ label for the app to work properly. Once scanned, the shoes are unlocked and users have an opportunity to purchase a pair, as long as they’re in stock.
Right now, the SNRKS application only works on iOS phones, but Nike plans to release a version for Android soon.
How do brands use augmented reality in a way that engages their core audience? What’s interesting about this is (1) the audience understanding — sneaker freaks DO care about insider, unique, unlocking-type tasks and (2) the localization factor + partnership factor. However, I have to wonder what the reach is on something like this — is it a lot of effort for a little engagement?
Air New Zealand has partnered with Microsoft to begin beta testing HoloLens augmented reality headsets on flights to help their crews better serve their passengers.
Flight attendants using headsets on their faces might look really strange and scare little children, but the practical applications are pretty cool. Being able to know, for example, which passengers have dietary restrictions or are in a certain mood can enhance the customer experience.
It’s hot because while this might not be a solution that gets mass adoption with every airline, it is nice to know that there is an airline out there that is trying to improve the travel experience (*cough* unlike United *cough*).
UC Berkeley Art Museum and Pacific Film Archive—or BAMPFA—wanted to give their new exhibition titled “Hippie Modernism: The Struggle for Utopia” a little bump.
But Goodby Silverstein & Partners, their agency, does not know how to give “bumps”. Instead, they shot this out of a bloody cannon by developing a geolocation-based augmented-reality mobile app that lets people pepper the virtual space over the Bay Area with digital balloon hearts containing hippy-like messages such as, “Love Is in the Air” or “Free sex”. Okay, okay, to be transparent, I TRIED to upload this last message but the app wouldn’t let me. Apparently, you have to be in the Bay Area. Bummer.
Anyway, think of it as a Pokémon, except instead of catching the little trolls, you are making and popping virtual hearts that release lovey-dovey messages.
In addition to allowing you to spread the virtual love, the app also plays ’60s and ’70s songs like John Lennon’s “Imagine” and features a virtual tour of Bay Area locations significant to the hippie movement’s artistic and political history.
Why It’s Hot:
NAILS a very unique and cool use of AR
Provides a gamified AR experience ala Pokémon
App is fun and practical – i.e. virtual tours tied to the thematic of the exhibition
Subtly hints at a larger cultural sentiment of acceptance and hippy rebellion
Earlier this week, Facebook’s Oculus Rift, the clunky VR headset, shipped to the market. But Facebook has hopes for the tech and form factor to be reduced to the size of a normal pair of glasses.
During a recent developer conference, CEO Mark Zuckerberg demonstrated a concept pair of smart glasses that he envisions being able to view both virtual reality AND augmented reality. It would be similar to what Google Glass has tried to establish.
To distinguish, AR provides you overlays of data and information while you view the real world through the glasses vs VR, which is a virtual rendering of worlds.
There may be a race to market heating up as Microsoft just shipped its HoloLens headset to developers and Snapchat is apparently working on its own augmented reality glasses.
Why It’s Hot
AR and VR show true integration of the real world with information and data that can enrich our experiences. Particularly with augmented reality, overlaying information into your field of vision that is contextually relevant to what you are experiencing, can augment and enrich the experience.
Perhaps the only thing more confusing than the buttons, displays and lights on a new car is its manual — that 100+ page booklet of details, photos, and instructions that are enough to make you dizzy and vow never to read it, unless you need to know what that strange glow on your rearview mirror is.
Here comes Hyundai to the rescue. In the first of its kind, the car manufacturer is introducing virtual manuals that rely on augmented reality to explain your car. AR is first being introduced on the 2015 Hyundai Sonata, but will soon be compatible for other models.
Using a smartphone or tablet, car owners will be able to recognize “more than 45 major features of the Sonata and contains 82 how-to videos, six 3D overlay images that appear once users scan different areas of their vehicle, and more than 50 informational guides,” according to the post in psfk.com. Users position their device over the part they want to learn about and AR will provide information about warning lights, Bluetooth phone pairing, brakes, fuse box, etc.
Why it’s hot
As cars become more complicated, augmented reality can take the mystery out of key features of your car and educate the owner on conditions, DIY repairs and an overall understanding of how their car works. Eventually, AR technology will be used in other electronics manuals, such as household appliances and electrical devices, helping to make wordy, poorly written manuals a thing of the past.
Alien landscapes with hot breezes and falling rain, or an Aztec Temple where you feel the heat of your torch. This is the Star Trek holodeck come to life, a true mapping of virtual world over physical world and its being developed by a small group of enthusiasts– setting the bar for companies like Oculus to beat.
Why It’s Hot
The most advanced of the true VR experiences and its moving rapidly toward deployment worldwide. But its also setting the bar for the category in timing and sophistication.
The SmartSpecs device is simple to put on, sits comfortably on the eyes and is very portable. Despite its size, the device has three camera sensors, a processor and a display.
For people with difficulty in seeing colors and contrast, the glasses can convert scenes into simple, high-contrast image fields. This can help legally-blind people see object locations in relevance to other items in the scene.
The depth camera, the one which can record the distance of an object from the wearer, also functions in the dark. With the use of the Smart Specs, a legally blind person may easily navigate independently both during day and night.
The SmartSpecs run on Android and microprojectors display the processed images onto the transparent lenses. The wearable also has zoom and pause functions for viewing scenes in greater detail. Customizable to user needs, the SmartSpecs can display various amounts of detail.
At SXSW, I focused on wearables, and there was a lot of discussion as to the breakthrough medical uses they may have. This is a great example of that type of application — and makes you wonder what advances may be in store in the near future
There is now a new and exciting way to search and interact with the world around us. The Blippar app, which was first created as an image recognition platform, has been reinvented. The re-launched app focuses on non-brand visual search, allowing users to delve more deeply into an array of both products and real world objects. Blippar uses the camera on your smartphone to look at an object. This image activates Blippar’s digital search and serves up relevant information directly from the local area. Blippar’s CEO and founder Ambarish Mitra seeks to expand the app’s focus into an all-encompassing and universal visual search engine to complement text- and link-based engines like Google and Bing.
Why It’s Hot
The service is being launched in waves and for now it is only available for use on English language album covers, movie posters, fiction books, and DVD covers. Eventually, any object in the physical world will be fully searchable. Our ability to find out more about the world around us will be limitless – all from your smartphone screen. As the app is developed, it will incorporate artificial intelligence and location-based technology to personalize and refine results for the user. It will be faster than the traditional web search, so instead of scrolling through pages of search results, you can receive key information in an instant.
Blippar brings out the curiosity in all of us. For example “when you blip a puppy” said Mitra, “you know what puppy it is, its history and genealogy, how to look after it, if there’s a vet nearby, the dog sitters and dog walkers — all the connecting points of information. It’s almost like that puppy becomes its own portal.”
Blippar is available for iOS, Android and Windows Phone.
Modiface’s Mirror combines augmented reality and facial recognition so shoppers can see what they’ll look like with different shades of makeup before making a purchase decision.
The Mirror combines a 3D live-video feed with a Photoshop-style sampler of more than 2,000 existing products and that maps the shades and textures to a movable, rotatable image of the user’s face.
Mirror’s core format is a sales kiosk, a robust and stand-alone version that automatically grabs a browser’s image and attracts their attention with recommended shades and brands. Once interested, the shopper can experiment with various products until the shopper becomes a buyer.
The concept is also available as a less robust, but considerably less expensive, app that uses a smart phone’s camera to capture the image. The apps are available for iPhones, Android and most tablets.
Mirror also includes an anti-aging mirror that shows the long-term benefits of using various moisturizing and age-reversing products. The featurette shows these effects as a slow progression, demonstrating the gradual effects.
ModiFace began in 1999 with initial research into facial analysis at Stanford University. The work continued through 2006 when ModiFace, Inc was born. Modiface technology has gone on to hold the leading patent portfolio on skin and facial analysis, powering over 150 web and mobile apps with a total of over 50 million downloads as of March 2015.
I’ve been paying close attention to retail trends that (1) enable creative sampling in a digital world and (2) connect the digital & physical in useful ways that can encourage purchase. This technology is a great example of both of these trends, and could open the door for even more impactful usage — think pre-surgical previews, impacts of UV exposure and more.
The technology of Google Glass combined with the fun of the ski slopes is a recipe for success. RideOn, new Israeli crowdfunding campaign, claims to be the first augmented reality ski goggles.
The ski goggles enable wearers to see projected ski paths via AR on their goggles, as if the paths are being projected on the snow 15 feet in front of them. Wearers can also play games and challenges with friends, and make phone calls and video messages.
The technology uses sensors, a video camera, and an Internet connection (all built into the goggles) to make the experience happen. But the goggles also still include the anti-scratch and anti-fog features of most ski goggles. In addition, there will be a free mobile app which wearers can use to track their progress, interact with other nearby users and friends, and instantly upload videos of their skiing.
Why It’s Hot | Augmented reality is increasingly in popularity, but many brands are struggling with ways to use it in a sensible way, beyond being a novelty. This usage enhances a sport that many already enjoy, by increasing its competitiveness (via the built-in games), safety (via the projected ski paths), and shareability (via the video creation and upload feature). It will be interesting to see if, in the future, AR is more accepted in targeted, niche environments, as opposed to broad everyday uses.
First launched softly on Twitter last August, the viral campaign for Universal’s Jurassic World movie went into high gear after the Super Bowl. Two incredibly immersive websites launched– one for the fictional park which will have you believing you could book a trip there tomorrow (take a tour through the resort rooms, check the restaurant menu, look at activities, live webcams of the park, even buy merchandise), and the other for the equally fictional company that supposedly owns and built the park, Masrani Global.
Neither site acknowledges they are fictional. Rich in video, imagery and detail; updates are now being released that have fans poring through the work for clues about the film.
Why It’s Hot
Viral marketing is often a “leap of faith” as we cant predict or forecast success, it is completely in the hands of the user to determine whether the content is shareable. It is impressive how much detail and depth went into these websites, and is continuously being released without a shred of paid promotion. That’s the definition of faith in your idea, your instincts and the resulting content is so rich and compelling i am waiting to see when this gets real traction.
USPS’ new mobile app features an AR experience that enables users to enhance the real world via their smartphones. By using the app to scan any of the 156,000 blue collection boxes throughout the country, users receive a “magical mailbox” holiday message.
The app is meant to compel more business from direct-mail marketers by showcasing innovations they can use in their own direct mail pieces via the USPS.
Why It’s Hot | What sets augmented reality apart from other mobile ad units is its ability to compel users to engage in a more meaningful way. It can be used to provide information about a user’s surroundings, or provide video extensions to print ads. As the world is becoming more mobile, innovation that integrates direct mail with the digital world is helping to bridge the gap between traditional channels and new audiences.
even with the same or similar product available in different places, we will gravitate toward the superior retail experience– as brands are now shaped by experience as much as ideas, the ubiquity brought by technology continues to push the importance of brands in a new direction.
This is a pair of articles that together drive home the need to understand people as we build multi-channel, multi-device, multi-format experiences.
First, a new study that confirms mobile commerce has not upset or displaced brick-and-mortar shopping– in fact, it drives in-store sales by extending the experience. The way we use smartphones is so fundamentally different from desktop or tablet; we use smartphones for couponing, reviews, location-finding but not for actual commerce. This harkens back to a “scholarly disagreement” I had with a Harvard Economics professor 12 years ago, who predicted that ecommerce would be frictionless and drive product sales on price alone, that brands would be irrelevant.
Use of Force is a VR simulation, that places the viewer as a witness to a real-life incident where border guards beat a non-resisting man to death. It uses actual cell-phone audio, 3D imaging of the location and witness accounts as the basis. It was created by Nonny de la Pena, a former Newsweek journalist to change the way we perceive events being broadcast to us. It mirrors the way television changed our perceptions of the Vietnam War.
While most applications now talked about are gaming or virtual meetings (social and/or business), this kind of VR experience has the potential to radically change our perceptions– good or bad. Studies are showing our perceptions of VR experiences are fundamentally different than other immersive even 3D experiences.
British Airways ‘Magic’ Billboards has recently won some awards, and it’s clear why.It’s informative, intriguing, and well executed! According to Ad Age, these digital billboards are located throughout London, and serve to highlight the company’s various destinations by encouraging passers to look up into the sky, to spot the airplane passing over.
Why it’s hot | The idea behind the campaign is quite simple, but because of the technological complexity and accuracy, it’s pretty brilliant. It involves mounting an antennae on the roof of each board, which picks up data from British Airways aircraft transponders. This information is then sent to a server which detects the aircrafts origin or destination. A trigger zone was also included, providing accuracy of the aircraft passing, based on cloud altitude which showed whether the plane could actually be seen.
For a campaign which serves the purpose of drawing potential customers to their website, the agency behind the idea (Ogilvy) did a pretty good job of making it enjoyable. The only call to action in the advertisement includes the hashtag #LookUp, as well as the html, ‘ba.com/lookup’ which drives the user to a website listing of the companies cheapest flights. Well done!
Lowe’s is looking to the future of home improvement, and it’s using physical lab spaces, along with a team of science-fiction writers, to help it envision what that might look like.
“You take all of your market research, all of your trend data and hire professional science-fiction writers. And they write real stories with conflict and resolution and characters. We turned it into a comic book and created possible stories or visions of the future.” (Kyle Neil, Executive Director)
One of those visions involved giving homeowners the ability to envision remodeling projects with augmented reality.
The first project to come out the Labs and science fiction-prototyping is the “Holoroom.” The 20-foot- by-20-foot room allows customers to simulate renovation projects. Customers can create realistic rooms on an iPad — stocked with anything Lowe’s sells, right down to Valspar paint colors — and then enter the Holoroom to experience a 3D version of the room. An app, paired with a printout of the room, allows customers to view and adjust their 3D creation at home, as well as share it with friends.
The Holoroom will be installed later this year in two Toronto-area stores and will focus on bathroom remodeling. The plan is to add new categories and rooms over the next 12-to-18 months. There is no firm plan for rollout in the U.S.
They always say a picture is worth a thousand words… well, how about a thousand songs? That’s where the fun new iOS app, Moodsnap, comes in!
Moodsnap is the world’s first image-based music streaming app, where deciding what to hear is as easy as knowing how you feel. Simply launch the app, click on a photograph that matches your mood, and presto!
That’s right, in Moodsnap, music stations are crowd-sourced collections of songs that users have collectively associated with emotive photographs. It is the community’s judgement of what songs feel right for each photograph, that fuels the listening experience for everyone.
Moodsnap is uniquely powered by Spotify and is free for Spotify Premium subscribers on iOS. Check out the app here!
Why It’s Hot
Taste-algorithms for music are nothing new, but usually are based off of components like artists, songs, or even activities. Photography has become a huge aspect of social media and the way the mass is expressing themselves. Music has always been a way to express and show emotion. Music and photography share that they are a way people identify and find common interests, so it makes sense they should be integrated. There may be potential for brands to use this as well. Such as having consumers upload pictures that create a brand playlist…. interesting.
SemaConnect, which makes electric vehicle charging stations, has launched an application on Google Glass to make it easier for drivers to navigate to the closest charging stations at a nearby Walgreens or Dunkin’ Donuts.
The app leverages augmented reality to make navigation faster and easier, with users able to locate the closest charging stations within a 20-mile radius. Users can also enable turn-by-turn navigation to station locations and initiate a charging session.
When a driver gets to the station, then the user says “Control my car” and the station begins charging the vehicle. If there is a fee applicable, it is automatically billed to the user’s credit card.
Why It’s Hot
While the Google Glass is still in its early days, and people are just getting started in getting and using this device and figuring out its capabilities, electric vehicle owners are most likely early adopters anyways.
The big advantage to using the Google Glass is that the user need not take her hands off the wheel or her eyes off the road. And the app is also driven largely by voice commands.
The World Cup is almost upon us, and official sponsor McDonald’s is launching a huge global effort that features new fry boxes that allow you to play and an augmented-reality app called McDonald’s Gol!
Beginning May 26, the chain will — for the first time — change its medium and large fry boxes globally for the promotion, offering 12 different World-Cup-themed designs featuring work from artists commissioned from around the world. The fry boxes will also serve as the entry point for an augmented-reality game on the app.
The app essentially turns the packaging into a virtual reality soccer field, where players try to flick a ball into the fry box, which serves as a goal. The game also uses other real-world objects in view as a way to bounce the ball off them to avoid increasingly difficult obstacles. Points are scored by bouncing off or moving the ball around the obstacles. Points are tallied by country, and players can share their scores via social media.
First promo commercial:
Why it’s HOT:
Seeing one of the biggest and most recognizable brands embrace the World Cup by updating their packaging to include an interactive augmented reality game is VERY HOT. Especially knowing that this is the first time in McDonald’s history that they’ve changed their french fries packaging makes it even more special. As a consumer, I’m going to go to McDonald’s and try out this game on my own smartphone! As far as McDonald’s is concerned, this is exactly what they wanted.
At the MIT Media Lab, the Tangible Media Group believes the future of computing is tactile. Unveiled today, the inFORM is MIT’s new scrying pool for imagining the interfaces of tomorrow. Almost like a table of living clay, the inFORM is a surface that three-dimensionally changes shape, allowing users to not only interact with digital content in meatspace, but even hold hands with a person hundreds of miles away.
Put it in the simplest terms, the inFORM is a self-aware computer monitor that doesn’t just display light, but shape as well. Remotely, two people Skyping could physically interact by playing catch, for example, or manipulating an object together, or even slapping high five from across the planetat’s only the beginning
The virtual world and the real world are bound to connect, in the near term. What this means for brands and marketers–including companies like eHarmony and Facebook–is the ability to now enable sensory relationships between their prospects and products–to “feel” the shape of a new phone, for instance, or to enable people living apart to touch each other.
According to a CNN report, Google Class is being targeted as a symbol of the tech elite–something that has even lead to physical altercations in the Bay Area. The rally against Google Glass appears to be part of a larger movement by an anti-tech crowd that is concerned about what the tech sector in Silicon Valley is doing to the city, particularly pushing people out as costs skyrocket.
If Google Glass falls flat or creates animosity in San Francisco, tech-haven of the US if not the World, then how will it fair in other areas?! Granted, the issue doesn’t appear to be so much against Google or Glass itself, but the concerns over lack of privacy (given someone can record you without your permission and not have to hold a camera phone to do it) and over how the tech sector is driving up costs of living in San Fran. That last part is nothing new though–remember the Bubble?
While AR or augmented reality apps seem like a thing of the future, there are quite a few that are already helping travelers acclimate to a new city.
According to this CNtraveler.com article, “The most common idea with these, for the uninitiated, is that you point your smartphone camera at something (an object, a street, an attraction) and instantly, pop-up cards of information about that thing will appear onscreen—particularly useful for travelers.”
To me, the neatest example is Google Googles. If you find yourself in a new city and want to learn more about a tourist site, label or business, simply point your camera, shoot and wait for the search results.
Why It’s Hot
Now more than ever information is right at our fingertips, especially in situations where one is traveling and the most in need of information in a quick and easy way. Google Googles was just one example of AR technology at work, other apps included:Yelp Monocle, Metro AR pro, and Wikitude.
This video from Google was used to create an incredibly powerful first-person account regarding the horrors of domestic violence, in this case against women (and it should be watched just for that reason alone). It depicts a day-in-the-life record of her experience, and while used as an ad, it spurs the idea of using Glass for capture of first-person journeys.
Used in real life, this is could open new doors to see just what really happens in the most emotional/private of situations, as people really live them. That has implications for brands to get closer than ever before to lives and experiences, including more private areas like medical issues and their barriers, how people interact with their families, their personal regimens, shopping behavior, product usage behavior, etc. The idea is to outfit people with Google Glass and use it to capture these Journeys as never before and give us a much more effective means to understand and portray Journeys for our clients.
The Skully Helmet uses augmented reality tech to enable the wearer to see everything that is happening around them–including behind them! The AR-1 model is being dubbed as the world’s first fully-integrated smart heads-up display helmet for the consumer market. The helmet links “advanced optics to an intelligent network of cameras, sensors, and microprocessors.”
Why It’s Hot
Skully Helmets was awarded the Accelerator Award in the Wearable Technology category at last week’s SXSW. This award is considered the “marquee” award of the SXSW Interactive Festival’s Startup Village. Over 500 companies submitted to present at SXSW Accelerator. Skully Helmets was selected out of 48 finalists in six different categories including Enterprise and Big Data Technologies, Entertainment and Content Technologies, Health Technologies, Innovative World Technologies, Social Technologies, and Wearable Technologies. While wearable tech is an exploding area, the Skully Helmet is the first that combines helmet safety and AR. It includes a rear-view camera, Bluetooth connectivity, GPS navigation and voice control. The AR-1 helmet is currently in beta testing.