Augmented Reality using AFrame: Tips & Tricks

AFrame is a very cool javascript library which allows you to create Virtual Reality environments completely within the browser. Aframe also combines with AR.js to allow you to create Augmented Reality environments through the use of “markers” or “anchors”. Anchors are very similar to simple QR codes which allow AR.js to project shapes onto the marker, for example:

taken from https://aframe.io/blog/arjs/

We recently updated our business cards to include a combined QRCode and marker. The back of each business card would look like so:

A recipient of the business card would then:

• scan the QRCode with a run-of the mill QRCode scanner
• get redirected to a site which hosts our AFrame + AR.js code
• keeping the camera pointed at the combined QRCode, get shown some augmented reality scene:

• once the “press” icon is pressed, users would be taken to content of our choice.

The main challenge while doing this was controlling and responding to the “touch” events when a user taps on the virtual hand. A sample of problems we ran into:

• No tap or click events were being registered at all
• On page load, the click event would fire immediately as soon as the camera feed started
• Once the page was loaded, if a user did not point their camera towards the marker, and tapped anywhere on the camera feed, a click event was fired, even though the user didn’t actually tap on the virtual hand. If the camera was pointed towards the marker and the virtual scene was displayed, this issue didn’t happen

Here we outline how we resolved each of the above problems. The trick lies completely within the <a-scene> element. This is the code we used:

<a-scene id="scene" cursor="fuse: false; rayOrigin: mouse;" embedded arjs="debugUIEnabled: false; trackingMethod: best; sourceType: webcam; detectionMode: mono_and_matrix; matrixCodeType: 3x3;">
   <a-marker type="barcode" value="17">
       <a-entity scale="1 1 1" rotation="0 0 0"> 
          <a-image id="logo" geometry="primitive: plane" src="./logo.png" rotation="-90 0 0" width="3" height="3" position='0 0 -2.5'></a-image>
          <a-image id="press-here" geometry="primitive: plane" src="./press.png" rotation="-90 0 0" width="2" height="2" position='0 0 0'></a-image>
       </a-entity>
   </a-marker>
   <a-entity camera mouse-cursor position="0 0 0"></a-entity>
</a-scene>

No tap or click events at all

This required the addition of the cursor=”rayOrigin: mouse;” prop to the a-scene element. The cursor properties are explained here:

https://aframe.io/docs/0.9.0/components/cursor.html#properties

We set the rayOrigin to “mouse” since we want the “click” to fire when an object intersects with our “mouse” (in this case the tap event). This started registering clicks… but too many of them!

Click event fired immediately on load

By default, the “cursor” attribute has it’s fuse property set to “true” on mobile. According to the documentation, a fuse-based cursor:

Also known as gaze-based cursor. If we set the cursor to be fuse-based, the cursor will trigger a click if the user gazes at an entity for a set amount of time.

https://aframe.io/docs/0.9.0/components/cursor.html#fuse-based-cursor

Our theory is that the cursor is triggering a click event because for some reason the timeout passes. Setting the cursor property to the below did the trick

cursor="fuse: false; rayOrigin: mouse;"

A corner case… the user gets a click event regardless of where they tap

AFrame is exactly like HTML in the sense that if you’d like to listen on click events the process is exactly the same. Referring to the code above, note that we have two <a-image> tags, each with their own id. If we want to respond to click (or tap) events on the “press” icon (which has an ID of ‘press-here’, then we simply do:

document.querySelector("#press-here")
        .addEventListener('click', (evt) => {              
                  window.location="https://cybersift.io"                        
        }
})   

This would in theory redirect the user to our site when they tap on the image. However, if when the website was loading the user pointed their camera away from the business card (i.e. away from the marker), they would get a camera feed and no matter where they tapped, a click even would be fired. In our simple demo this isn’t a big deal, but if we add more objects and elements it will become a problem.

We noticed that the above doesn’t happen if the camera is pointed towards the marker and AR.js detects the marker and displays the AR scene. So our workaround was to attach ‘markerFound’ and ‘markerLost’ event listeners to the scene. As the names imply, the former is fired when AR.js detects a marker in the camera view, and the latter fires when a marker disappears.

This allows us to set a flag, which allows or disallows clicks:

let allow_clicks = false;
document.querySelector("#scene")
      .addEventListener('markerFound', (evt) => {
               allow_clicks = true
      })

document.querySelector("#scene")
      .addEventListener('markerLost', (evt) => {
                allow_clicks = false
      }) 

document.querySelector("#press-here")
      .addEventListener('click', (evt) => {
           if (allow_clicks){
               window.location="https://cybersift.io"
           }            
      })   

The markerFound and markerLost events control the “allowClicks” flag, which is checked everytime the click event bound to #press-here is fired