Augmented Reality: Expanding the User Experience

Augmented reality and mobile apps have met before, but some developers contend that the next encounters will produce an even more sophisticated class of technology.

The field of augmented reality places a digital overlay on the real-world view through a mobile device’s camera. Over the past couple of years, developers have taken advantage of a mobile platform’s camera and GPS to provide apps that help users find particular stores, restaurants or other points of interest. Games such as ARDefender also employ augmented reality.

But app creators have begun to engage more of a mobile device’s sensors -- accelerometers and gyroscopes, for example. Augmented reality apps that use detailed animations are also in the works. The objective: inject augmented reality technology in a wider range of apps to boost the user experience.

Expanding Augmented Reality’s Scope
“It’s been kind of a novelty item, but we are working with different companies that are putting augmented reality into everyday apps,” says Terry Hoy, senior vice president and director of sales at Gravity Jack, a company in Liberty Lake, Wash., that specializes in augmented reality software development.

Hoy cites one augmented reality app that adds an animation showing how a particular item may be taken apart and reassembled. He says that type of app is something the company can do today, noting that it should be available in the market in about six months. Another app is designed to pinpoint the location of a heating and cooling system in a building and then help technicians learn about the HVAC unit if they are not familiar with that particular model. Hoy says that app is a work in progress.

“Augmented reality is such a new technology that I think it is wide open,” says Hoy.

Gravity Jack worked for two years on an augmented reality SDK, which Hoy says will help spark the delivery of apps that go beyond the current offerings.

Trak Lord, head of U.S. marketing for metaio Inc., a software company that focuses on augmented reality, says metaio anticipates the technology moving toward the visualization of complex real-world objects, such as engine parts. He says store-finder apps still have their uses, but he adds that customers -- for the most part -- are no longer solely interested in location-driven augmented reality apps.

“No one says, ‘I only want to do GPS,’” he notes. Instead, customers who are interested in GPS overlay it into an existing, full-service application instead of building an application around that single function.

In another nod toward greater sophistication, software developers’ R&D shops are harnessing sensor data to further improve the user experience.

According to Lord, metaio “can now use sensor data to make phones aware of gravity.” A mobile device’s camera has no sense of up versus down, but metaio’s Gravity-Aligned Feature Description technology provides the camera with an up and down framework. Gravity awareness lets apps render “virtual content that behaves like real objects,” according to metaio. The company cites virtual accessories -- such as a pair of earrings that move according to how users turn their heads -- as an example of improved object rendering.

“As mobile devices and the underlying hardware become more sophisticated, we’re starting to see more developers combining internal sensor data to create more robust augmented reality experiences,” says Lord.

Limits of Augmented Reality Development
Augmented reality’s growth track bumps into a couple of obstacles, namely computing power and programming skill.

The mobile device’s processor places restrictions on what developers can currently accomplish. Certain types of animation, for example, can run into some limitations, even on the more powerful tablets, says Hoy.

“If an augmented reality application has rich graphics, this will require higher performance of a smart device’s processor,” adds Eugene Filipkov, senior game developer and augmented reality expert at Eligraphics Studio, an Elinext Group member company based in Minsk, Belarus.

More graphics and functionality require more resources, says Filipkov. He notes that some new smartphones have dual-core processors in which one of the second core’s functions is graphics processing.

Developments in the mobile processor space, however, seek to boost to augmented reality apps. Intel, for example, is taking an integrated approach. The company’s Z2460 processor includes an integrated 2D/3D graphics engine. [Disclosure: Intel is the sponsor of this content.]

OpenCL, a standard maintained by the Khronos Group, also has implications for augmented reality. The specification lets developers code applications that harness GPUs for general-purpose, non-graphical processing. On a mobile device, the additional processing capability could help power augmented reality applications.

How App Developers Can Use Augmented Reality
On the skills side, a novice developer might find it “quite challenging to create a

high-quality augmented reality application with rich functionality and cool graphics,” says Filipkov. In his opinion, the prerequisites for developing a successful augmented reality app include a very good command of mathematics, knowledge of matrices, a good sense of space, and knowledge of the C language. The latter, he notes, is especially helpful with OpenCV, a programming library that targets computer vision.

Some developments, however, aim to broaden the augmented reality base for app developers. In May, metaio launched software -- metaio Creator -- that lets non-developers take on augmented reality. Lord describes metaio Creator as “a more intuitive and straightforward way of making augmented reality more accessible.” The product, which reduces the process of adding augmented reality to a three-step drag-and-drop workflow, does not require knowledge of programming. Lord says the company has been presenting Creator at universities in the San Francisco area, where metaio’s U.S. office is located.

Creator users deploy augmented reality through metaio’s junaio mobile AR browser, so they need a free junaio developer account in addition to Creator. A future version of Creator will be compatible with all of metaio’s development tools, including the mobile SDK for creating custom mobile apps.

Photo: Flickr.com/jason.mcdermott

By John Moore

John Moore has written about business and technology for more than 20 years. His articles have appeared in Baseline, CIO Insight, Federal Computer Week, Government Health IT and Tech Target. Areas of focus include cloud computing, health information technology, systems integration and virtualization. He is a frequent contributor to Digital Innovation Gazette.