Escher Reality is building the backend for cross-platform mobile AR

The potential of mobile augmented reality is clear. Last summer Pokemon Go gave a glimpse of just how big this craze could be, as thousands of excited humans converged on parks, bus stops and other locations around the world to chase virtual monsters through the lens of their smartphones.

Apple was also watching. And this summer the company signaled its own conviction in the technology by announcing ARKit: a developer toolkit to support iOS developers to build augmented reality apps. CEO Tim Cook said iOS will become the worlds biggest augmented reality platform once iOS 11 hits consumers devices in fall underlining Cupertinos expectation that big things are coming down the mobile AR pipe.

Y Combinator-backed, MIT spin-out Escher Realitys belief in the social power of mobile AR predates both these trigger points. Its building a cross-platform toolkit and custom backend for mobile AR developers, aiming to lower the barrier to entry to building compelling experiences, as the co-founders put it.

Keep in mind this was before Pokemon Go, says CEO Ross Finman, discussing how he and CTO Diana Hu founded the company about a year and a half ago, initially as a bit of a side project before going all in full time last November. Everyone thought we were crazy at that time, and now this summer its the summer for mobile augmented reality ARKit has been the best thing ever for us.

But if Apple has ARKit, and you can bet Google will be coming out with an Android equivalent in the not-too-distant future, where exactly does Escher Reality come in?

Think of us more as the backend for augmented reality, says Finman. What we offer is the cross-platform, multiuser and persistent experiences so those are three things that Apple and ARKit doesnt do. So if you want to do any type of shared AR experience you need to connect the two different devices together so then thats what we offer Theres a lot of computer vision problems associated with that.

Think about the problem of what ARKit doesnt provide you, adds Hu. If youve seen a lot of the current demos outside, theyre okay-ish, you can see 3D models there, but when you start thinking longer term what does it take to create compelling AR experiences? And part of that is a lot of the tooling and a lot of the SDK are not there to provide that functionality. Because as game developers or app developers they dont want to think about all that low level stuff and theres a lot of really complex techs going on that we have built.

If you think about in the future, as AR becomes a bigger movement, as the next computing platform, it will need a backend to support a lot of the networking, it will need a lot of the tools that were building in order to build compelling AR experiences.

We will be offering Android support for now, but then we imagine Google will probably come out with something like that in the future, adds Finman, couching that part of the business as the free bit in freemium and one theyre therefore more than happy to hand off to Google when the time comes.

The team has put together a demo to illustrate the sorts of mobile AR gaming experiences theyre aiming to support in which two people play the same mobile AR game, each using their own device as a paddle

What youre looking at here is very low latency, custom computer vision network protocols enabling two players to share augmented reality at the same time, as Hu explains it.

Sketching another scenario the tech could enable, Finman says it could support a version of Pokemon Go in which friends could battle each other at the same time and see their Pokemons fight in real time. Or allow players to locate a Gym at a very specific location that makes sense in the real-world.

In essence, the teams bet is that mobile AR especially mobile AR gaming gets a whole lot more interesting with support for richly interactive and multiplayer apps that work cross-platform and cross device. So theyre building tools and a backend to support developers wanting to build apps that can connect Android users and iPhone owners in the same augmented play space.

After all, Apple especially isnt incentivized to help support AR collaboration on Android. Which leaves room for a neutral third party to help bridge platform and hardware gaps and smooth AR play for every mobile gamer.

The core tech is essentially knitting different SLAM maps and network connections together in an efficient way, says Finman, i.e. without the latency that would make a game unplayable, so that it runs in real-time and is a consistent experience. So tuning everything up for mobile processors.

We go down to, not just even the network layer, but even to the assembly level so that we can run some of the execution instructions very efficiently and some of the image processing on the GPU for phones, says Hu. So on a high level it is a SLAM system, but the exact method and how we engineered it is novel for efficient mobile devices.

Consider ARKit as step one, were steps two and three, adds Finman. You can do multi-user experiences, but then you can also do persistent experiences once you turn off the app, once you start it up again, all the objects that you left will be in the same location.

Consider ARKit as step one, were steps two and three.

People can collaborate in AR experiences at the same time, adds Hu. Thats one main thing that we can really provide, that Google or Apple wouldnt provide.

Hardware wise, their system supports premium smartphones from the last three years. Although, looking ahead, they say they see no reason why they wouldnt expand to support additional types of hardware such as headsets when/if those start gaining traction too.

In mobile theres a billion devices out there that can run augmented reality right now, notes Finman. Apple has one part of the market, Android has a larger part. Thats where youre going to see the most adoption by developers in the short term.

Escher Reality was founded about a year and a half ago, spun out of MIT and initially bootstrapped in Finmans dorm room first as a bit of a side project, before they went all in full time in November. The co-founders go back a decade or so as friends, and say they had often kicked around startup ideas and been interested in augmented reality.

Finman describes the business theyve ended up co-founding as really just a nice blend of both of our backgrounds. For me I was working on my PhD at MIT in 3D perception its the same type of technology underneath, he tells TechCrunch.

Ive been in industry running a lot of different teams in computer vision and data science, adds Hu. So a lot of experience bringing research into production and building large scale data systems with low latency.

They now have five people working full time on the startup, and two part time. At this point the SDK is being used by a limited number of developers, with a wait-list for new sign ups. Theyre aiming to open up to all comers in fall.

Were targeting games studios to begin with, says Finman. The technology can be used across many different industries but were going after gaming first because they are usually at the cutting edge of new technology and adoption, and then theres a whole bunch of really smart developers that are going after interesting new projects.

One of the reasons why augmented reality is considered so much bigger, the shared experiences in the real world really opens up a whole lot of new capabilities and interactions and experiences that are going to improve the current thoughts of augmented reality. But really it opens up the door for so many different possibilities, he adds.

Discussing some of the compelling experiences the team see coming down the mobile AR pipe, he points to three areas he reckons the technology can especially support namely: instruction, visualization and entertainment.

When you have to look at a piece of paper and imagine whats in the real world for building anything, getting direction, having distance professions, thats all going to need shared augmented reality experiences, he suggests.

Although, in the nearer term, consumer entertainment (and specifically gaming) is the teams first bet for traction.

In the entertainment space in the consumer side, youre going to see short films so beyond just Snapchat, its kind of real time special effects, that you can video and set up your own kind of movie scene, he suggests.

Designing games in AR does also present developers with new conceptual and design challenges, of course, which in turn bring additional development challenges and the toolkit is being designed to help with those challenges.

If you think about augmented reality theres two new mechanics that you can work with; one is the position of the phone now matters, notes Finman. The second thing is the real world become content. So like the map data, the real world, can be integrated into the game. So those are two mechanics that didnt exist in any other medium before.

From a developer standpoint, one added constraint with augmented reality is because it depends on the real world its difficult to debug so weve developed tools so that you can play back logs. So then you can actually go through videos that were in the real world and interact with it in a simulated environment.

Discussing some of the ideas and clever mechanics theyre seeing early developers playing with, he suggests color as one interesting area. Thinking about the real world as content is really fascinating, he says. Think about color as a resource. So then you can mine color from the real world. So if you want more gold, put up more Post-It notes.

The business model for Escher Realitys SDK is usage based, meaning they will charge developers for usage on a sliding scale that reflects the success of their applications. Its also offered as a Unity plug-in so the target developers can easily integrate into current dev environments.

Its a very similar model to Unity, which encourages a very healthy indie developer ecosystem where theyre not charging any money until you actually start making money, says Hu. So developers can start working on it and during development time they dont get charged anything, even when they launch it, if they dont have that many users they dont get charged, its only when they start making money we also start making money so in that sense a lot of the incentives align pretty well.

The startup, which is graduating YC in the summer 2017 batch and now headed towards demo day, will be looking to raise funding so they can amp up their bandwidth to support more developers. Once theyve got additional outside investment secured the plan is to sign on and work with as many gaming studios as possible, says Finman, as well as be head down on building the product.

The AR space is just exploding at the moment so we need to make sure we can move fast enough to keep up with it, he adds.

More From this publisher : HERE

 


RELATED PRODUCTS
  • Get a guided tour through the concepts of augmented reality.
  • Discover the secret of how and found the perfect formula for building muscle and strength quickly
  • Unleash Your Ad Daily To Thousands Of Responsive Mobile Phone Users
  • Build Incredible Wealth Online Right Now From Your Very Own 7 Figure List..
  • A Simple Training Blueprint To Help Cops, Soldiers, And Prepared Citizens Gain Tactical Muscle
  • This reveals a Divine nature and purpose for true meaning of life to enjoy a life of peace.
  • This is an advanced 12 week muscle building protocol to build muscles.
  • This system can be used by anyone... even entrepreneurs who started businesses
  • Clickbank Ads
     

    If you like our content help us with a share… or two …  

    Also -  I have some free stuff for you .......

    Nr 1. In This FREE Report you can Discover:

    Secrets to acquiring the freshest, hottest leads via YouTube In-Stream ads quickly and easily! Best secret tips for the most effective targeting for your ads! Totally legal, but little-known method for literally stealing traffic from your competitors! Get more traffic to your videos and landing pages without spending a fortune! And much more - all within this special FREE report!

     

    Nr 2. Did you know you can Absolutely EXPLODE YOUR EMAIL LIST FOR FREE with UNTOUCHED offline sources?

     

    Nr 3. The #1 secret to becoming an affiliate superstar - Earn your first $100 online – without a dime to invest.

    If you’re still looking to “make it online – I have just the thing for you. It’s a video series that shows how to make your first $100 online – even if you don’t have a dime to invest. Yes – nothing.  Free.  Nada. Is this possible? You bet. It’s the most powerful system used by those “in the know”. Good things don’t have to be difficult. And they don’t only come to those who wait. Have A Look

     

    Nr 4. Top 10 Email Marketing Mistakes. Which of these mistakes do you make?

     

    Nr 5.  You can learn about : The hottest social networking site that will send an unlimited stampede of traffic to your site - for FREE! How to quickly locate a swarm of hungry buyers using "information portals"! The fast and easy way to use simple images to siphon targeted traffic, on command! Learn how to get the most possible traffic from Instagram, and make sure that traffic converts! And much more ...

     

    Nr 6. It’s no secret… Facebook is an incredible place to get tons of free, viral traffic. But… most people are going about it all wrong.  How many of these mistakes are you making? Download the free report to find out.

       

    Nr 7.  Special free report .  It's called "Operation Midas Touch" and you can download it here...

    The report features a surefire method for generating at least $1000 per week online,without a website or product of your own!

    Nr 8.  Instagram is still one of the hottest ways to get a LOT of traffic fast. All by sharing images. Sounds easy enough, yeah? Well, yes and no. If you want to really start driving a lot of  traffic and making sales using Instagram then you should download this report.

    Nr 9.  When it comes to creating a product online there are so many ways to approach it. Wouldn’t it be great to just get the meat of it all so you can get started faster? Well, the good news is, today you can download a guide called the Product Creation Formula Quickstart Guide. It’s as the title suggests - a Quickstart guide. You’ll get a list of action steps to take right away. Download your copy today !

    Nr 10. Would you like to finally USE all of that dusty PLR you have sitting around on your harddrive? Or maybe find the resources for the best PLR available - at the best prices? What about how to use that PLR to make you money with just a few minutes of work. Well, you’re not going to want to miss this!

      Nr 11. Buzzfeed’s top post has more than 21 million views. Think about that for a sec… ONE of their articles has had over 21 million people who have read it! And they have thousands of articles... what would you do with that kind of traffic? A friend of mine hacked their method and was able to use it to get 11,592 targeted new subscribers in 2 weeks on their very first attempt!

    Nr 12. Push notifications are one of today's hottest trends in website traffic and conversions, and for good reason. Studies have shown that as many as 40% of people will opt-in for push notifications, which is about TEN TIMES the rate of people who opt-in for emails! That’s huge! Not only that, but they're also highly effective at pulling clicks, as well. Some studies have shown CTR at an average of around 40%, with highs of as much as 80%! Again, that’s huge!

    Nr 13. Would you like to know how to get more money from the exact same number of website visitors you’re currently getting? Even if you’re website is getting as few as 1-2 visitors a day, this report will show you how to increase your profits using these few handy hacks.

    Nr 14. People have now realised that it's not the size of the list which matters, it's the quality. ... and the only way to create a quality list is by launching your own products. Let it be $7 ebooks, $17 info products, $27 video training product or software's... all of these work. My good friend Kevin Fahey has is revealing how he's launched over 20 products in the past 4 years, many of them top sellers. Needless to say he's banked a handsome amount in this time.

    Nr 15. Do you ever feel that the man you like or even love, just doesn't see you or desire you in way you want him to? Or... maybe you're able to get a man's attention though not the right kind of attention? Or... maybe you're still single because you wait and wait for men to approach you first? What you may not know is that the way we flirt will either attract the "wrong" man for us or the "right" man.

    Nr 16. When it comes to getting sales and leads online, there is ONE thing that 95% of marketers use... it's called a LIST. My friend Jimmy just released a free book that explains WHY a list is important and how YOU can build your own list. It's available right here (and it won't cost you a penny)

    Nr 17. If you have tried everything, and still aren't able to get your Ex back, then you need to pay attention.

    Nr 18. What if I told you that you could make ANY woman WANT to do ANYTHING for you? Would you think I'm crazy? Well... what If I told you that you could also make her LOVE every minute of it and come back for more?

       

    Nr 19. I have a special free plugin for you today. It's called "Covert Hover Mini"

    This plugin will triple your blog traffic by compelling your visitors to share your images on social media. This is some really clever stuff and it works like a charm. After you grab your free plugin, don't  forget to read the free bonus report as it will show exactly how and why this strategy works like crazy.

    Nr 20. The days of being able to get away with a basic WP theme and a pic or two are gone, your site needs to look good. Even Google are looking for you to have engaging graphics on your site. But getting good quality graphics made for your site is a pain, AND it will cost you both time and money (if you outsource it). But I have a special free download for you today. A virtual treasure chest of over 21K profit pulling graphics you can start sing today to generate more traffic & sales!

    Nr 21. 20 professional full HD background videos. These background videos are perfect for giving your videos that extra kick they need to stand out from the crowd Oh yeah they are in full 1080p HD too, I hope you'll enjoy them.

    Nr 22. If you've ever failed to create a WordPress site of your own or typically waste hours & even weeks trying to figure it all out on your own....

    Nr 23. No doubt, you've seen all of the video launches lately! There is a reason for it... Video marketing is a GREAT way to make money, and with the recent creation tools making it super accessable for anyone to get started... It's no surprise that more people are making videos. However... there is a missing element to most video marketing products...

    Nr 24. For the first time ever... You can get the straight scoop on how someone is selling over $1,000,000/year of t-shirts on Facebook!

    Use them  .... and have fun !