This story was written in partnership with NBC Boston, which produced video for this feature.
September 19, 2017, was a big day for Wayfair, but it was an even more significant day for one team in particular at the Boston-based ecommerce company.
For more than two years, the team and its members had been hard at work on an augmented reality app that would aim to solve a significant challenge for the online seller of home goods. By using a smartphone or tablet’s camera, the app could help customers visualize furniture and decor inside their homes with a 3D representation that would appear in front of them on the floor, as if it were really there.
The occasion for this particular Wayfair team was that the AR app would finally become available to the company’s largest audience yet: anyone who had an iPhone or iPad with Apple’s latest operating system, iOS 11. Before that, the AR app was only available commercially through two Android phones that had special hardware capabilities.
“It’s another rung in the ladder of how you get to where people say, ‘oh my gosh, there’s an overwhelming amount of benefits to shopping online,'” Steve Conine, Wayfair co-founder and co-chairman, told BostInno at a recent office visit.
A bet that began two years ago
Local
In-depth news coverage of the Greater Boston Area.
The team behind the new AR app is called Wayfair Next, and it was established two years ago, as first reported by BostInno, as a new research and development initiative that would focus on AR, virtual reality and other cutting-edge technologies.
When Mike Festa, the director of Wayfair Next, joined the ecommerce company in 2013, he had taken a job as a software engineer working on its warehouse management system. But one of the reasons he joined the company in the first place was because he knew VR could serve as a potential solution to Wayfair’s visualization problem. Festa had been an early adopter of VR, having been one of the first 20 people to order an early prototype of the Oculus Rift headset from its inventor, Palmer Luckey, before it became a hit on Kickstarter.
“I think consumer adoption of those devices will take a lot longer.”
“I see VR as a tool of the future,” Festa said. “Right now it’s still a novelty and it’s primarily used in gaming, but I think it has a lot of potential to give people this kind of inspirational conceptualization of space.”
When Wayfair Next first started in November 2015, it felt like VR and AR were far out from mass adoption. Oculus, which Facebook acquired in 2014, was still a few months from away releasing its first Rift headset. Pokemon GO, the location-based AR game for smartphones, wouldn’t come out until the next summer and introduce the world to the concept of AR.
Since then, Facebook, Sony and smartphone manufacturer HTC have all released their own VR headsets. However, while VR headset sales fell below expectations for 2016 and companies have been making course corrections in the space, AR is starting to show more potential. A large part of that is because of Apple’s new iOS11 operating system and Google’s plan to enable AR in millions of Android devices with its own software solution next year. Research firm Digi-Capital estimates that mobile AR could reach 900 million active users by the end of next year while VR’s market potential has diminished.
For Wayfair, that means its big bet on AR is finally starting to pay off while the company’s VR applications won’t see a broader impact for some time. Last year, the company released a free app for the Oculus Rift called Patio Playground, where users could arrange furniture on a virtual patio, but it felt more like a demonstration than something that customers would often use.
“I think consumer adoption of those devices will take a lot longer,” Conine, the company’s co-founder, said of VR. “Every holiday season you’ll have people get them for gaming, for entertainment, and some amount of users will try our experiences and use those tools.”
Conine said he could imagine a scenario where VR stations are set up at physical locations to help people visualize different home designs. Wayfair’s VR applications could also help interior designers provide new ways for their clients to imagine a space.
AR and VR lead to more opportunities
The foundation for Wayfair Next was based on VR and AR prototypes Festa built during company hackathons over the summers of 2014 and 2015, respectively. When Festa was working on his AR app in 2015, Conine said, the co-founder and co-chairman was experimenting with room scanning equipment and creating photorealistic renderings of Wayfair’s products — which would serve as an important component of the company’s VR and AR applications. They would also open pathways to new uses.
“Now that we’ve created this 3D content pipeline we’re able to take advantage of it throughout the organization.”
Wayfair Next has its own dedicated room for taking high-resolution photos of products, which then get turned into photorealistic renderings using the same kind of software tools used in the gaming and entertainment industries. The company also has similar studios at its Westborough, Mass., office and at one of its warehouses in Hebron, Ky., with a separate 3D visualization team that is now close to 100 employees.
Using a process called photogrammetry, Wayfair creates 3D representations of its products by placing an item on a turntable and taking hundreds of photos of the item from all angles using a 3D scanning rig that is armed with multiple DSLR cameras. From those photos, the company employs software to reconstruct the geometry of each item and add photorealistic textures. The result is that these 3D models can be almost indistinguishable from their real-life counterparts.
“Now that we’ve created this 3D content pipeline we’re able to take advantage of it throughout the organization,” Festa said.
This includes using the models to create life-like renderings for Wayfair’s website and catalogs that could be mistaken for actual photos. Festa said this gives the company more flexibility than traditional photography would in swapping out products or changing the style of the photos depending on the customer’s interest.
The company provides third-party developers with open access to 10,000 of its 3D models through an API released last year. One example of a company using the API is TrueScale, a VR interior design tool. The company is also using its models for a custom upholstery project and a room planning tool in development.
“If you’re an early enabler, you will benefit down the road from whatever direction the industry goes,” Conine said of the company’s 3D model strategy.
The future of Wayfair Next
When the Wayfair Next team was first established two years ago, it only had three members. It has since grown to 11, but the total number of Wayfair employees touching the team’s work has ballooned dramatically. To bring AR into Wayfair’s iOS11 app, for instance, it took employees across 10 teams, including search tech, customer experience and product media.
“If you got a space and you want to know what’s going to fit there, we can enable that so you can search by volume.”
While Wayfair’s AR feature is only a couple months old on Apple devices, the company is already receiving positive reception for how it can help customers. The company will measure its success based on sales and engagement.
“It’s still the early days for ongoing metrics we’re trying to collect, but we definitely see high engagement with the feature,” Festa said, “and from in-person demos and surveys, the feedback is always really enthusiastic.”
As for what’s next for the team, Festa said the company wants to build off its visual search feature, which also came out of a company hackathon and lets people search for products by taking photos, and put an AR spin on it. Instead of taking photos to find visually similar products, Festa said this new AR feature would let customers search for products based on the size of open space available in the home.
“If you got a space and you want to know what’s going to fit there, we can enable that so you can search by volume,” he said. He added that the company is also looking into how it can use visual tools to understand a customer’s taste in design.
On a broader level, Conine said the company is making large investments in transportation, shipping, logistics and warehousing. The goal is to make shopping on Wayfair so convenient and low-risk that customers don’t go anywhere else.
“We’re finally at the scale where we can afford to do it, it makes sense to do and we can do it nationally,” Conine said. “We think it will be a real game changer over the next several years in terms of how our shoppers think about us and why they love to shop with us time and again.”
