VR Controllers: The Way Of Interacting With The Virtual Worlds
In 2021, when the extended reality technology is already in the next level and professionals and enthusiasts can choose from a variety of headsets there’s one component that is of equal importance as the visual element — the way we interact and control ourselves and digital game objects in the virtual space. For years, virtual reality (VR) enthusiasts have concerned themselves with the question: “What is the next best VR controller?”
No wonder the question is as relevant today as ever: VR controllers are still an integral part of almost every modern virtual reality system and its experience.
However, comparing various controllers becomes increasingly difficult. For starters, the number of available controllers multiplied over years as did the number of new VR head-mounted displays (headsets).
Moreover, there are so many qualities to compare. From ergonomics and tracking precision to design and longevity, there are winners and flops in each category. A new VR headset doesn’t necessarily come with better controllers as well. Each virtual reality system (headset, controllers and software that comes with it) should be evaluated as a whole and not as a sum of its parts.
Having extensive experience with a variety of VR headsets and controllers, in this article we tried to explain which controller to choose based on your unique requirements. We are also going to check the compatibility level for each VR controller.
What Are VR Controllers?
If a VR headset allows you to see the virtual world, controllers allow you to interact with it.
Controllers help you register your hand and finger movements in a virtual environment. They turn your physical, kind of mechanic movements of your hands (and/or body) into a digital movements inside your chosen virtual world.
Although all VR controllers may look somewhat similar, they are different based on a number of qualities, and even somewhat minor differences matter when you start using them for long.
Here are the main qualities that you should look at when comparing VR controllers with each other:
- Tracking precision. Detecting your hand and finger movements in VR requires a great deal of accuracy, whether you are playing VR games where one mistake can lead to a lost round or implement complex business VR simulations where the precision of actions is often a training objective.Different controllers utilize different tracking systems with varying degrees of accuracy and we’ll be covering these in detail later.
- Build quality. VR simulations can last from several minutes to hours and this is where the build quality of a controller starts to make a difference. A cheap plastic cover leads to low durability. Worse yet, poorly built slippery controllers can fall out of your hands, especially during intensive VR experiences such as gaming or fitness.
- Ergonomics. As you’ll see some controllers are simply more pleasant to use. From the location of action buttons and form factor to the weight distribution, every small detail defines whether your VR experience will be decent or exceptional.
- Battery life. The capacity of controllers’ batteries defines how long the controllers can be used without you having to interrupt your VR experience and recharge them. Some controllers require recharging after a few hours of usage where others can provide tens of hours of gameplay before you have to switch batteries.
- Degree of Movement. Wired headset that are connected with a desktop often allows better performance but restricts your movement around the room. The accidents when playing games and damaging VR controllers are known to most VR gamers. Some virtual reality systems such as Sony’s first Playstation VR have wireless orb shaped controllers that allow for a high degree of freedom.
Before we dive into comparing the most popular/modern controllers based on these qualities, let’s quickly figure out how controllers are actually able to register your movements and whether you can interact with a virtual reality environment without controllers altogether.
How Do VR Controllers Work?
Most VR controllers come with a set of buttons, triggers, and typically a thumb stick that allow you to grab, push, throw, and move around virtual objects. For example, pressing a trigger button on a controller translates into pulling a trigger of a virtual gun whereas using a thumb stick can help you walk in a virtual world.
But the real question is how controllers are able to track your hand movement so precisely that it feels like you’re actually using your hands in VR, waving them, or even making gestures.
To achieve that your virtual reality headset needs to know the position of controllers related to its own position.
The two most common tracking systems for controllers right now are lighthouse tracking and inside-out tracking.
Lighthouse Tracking System
Lighthouse tracking is used by Valve Index and HTC Vive Pro headsets and it requires at least two base stations installed in your play area.
These base stations serve as a reference point for a headset and controllers. Each base station contains an IR beacon and two laser emitters, whereas each controller has multiple IR photosensors. The base station’s beacon emits a synchronization pulse 60 times per second, while one of its lasers sends a sweeping beam across the entire room. As soon as the controllers detect the pulse, they start counting until one of their sensors is hit by a laser beam. Based on the laser’s time of flight the system detects where the controller is.
Inside-out Tracking System
Utilized by Oculus Quest 2 (now called Meta Quest 2), inside-out tracking system works quite differently. Every controller has a set of infrared LEDs located on the controller’s rings. The cameras located on the headset detect said LEDs and continuously take images of them. Based on these images the so-called Constellation tracking system then triangulates the position of controllers in space.
While the benefits of the lighthouse system are enhanced tracking accuracy and lightweight data transfer, the Constellation tracking system is cheaper to implement and doesn’t require purchasing and setting up any external base stations.
VR Without Controllers: Is It Possible?
It’s worth mentioning that several modern virtual reality systems support the ability to interact with a VR world without controllers altogether.
For example, Oculus Quest 2 comes with native hand tracking support. The built-in headset cameras detect hand’s natural gestures that you can use to interact with objects and interfaces in VR. Despite the recent tracking rate update from 30 Hz to 60 Hz, the tracking latency is yet far from perfect while not many VR applications fully support hand controls.
Another option for controller-less experience is a Leap Motion Controller. With a cost of around $100, the device maps the front space using two cameras and three infrared LEDs. Leap motion can be used to enable hand tracking for headsets such as Oculus Rift, HTC Vive, and even Valve Index, as well as traditional PC desktop systems.
The current downside of hand-tracking VR is the lack of haptic feedback, lower latency, and the limited number of VR applications that fully support the technology, although these issues can be resolved as the VR innovation spreads out.
Modern VR Controllers: Ultimate Comparison
Valve Index “Knuckles” Overview
Price: $279 for a pair
Battery type: lithium-ion battery
Battery time: 6-8 hours (rechargeable via USB 2.0)
Weight: 196 grams
Compatible headsets: Valve Index, Pimax, HP Reverb G2, HTC Vive Cosmos Elite
From the outside, Valve Index “knuckles” seem similar to Oculus Touch and HTC Vive controllers, featuring a wide handle, a front trigger, a circular control surface with two face buttons on top, and an analog stick with a system button.
But there’s a drastic difference under the hood. Index controllers are laced with sensors — there are 87 proximity, optical, force, touch, grip, and motion sensors that allow accurate and comprehensive tracking of your hand and finger movements.
Using this rich sensory data Valve Index controllers can, for example, detect when your fingers are loosely curled around the strap hilt without touching it and provide realistic grabbing mechanics instead of relying on abstract trigger buttons.
Another quality that sets Valve Index controllers apart is that you don’t actually have to hold them — a fabric strap between the handle and the plastic arc curves over your knuckles and keeps controllers in your hand even when you let loose all your fingers.
Being a part of a well-known Valve’s Lighthouse 2.0 tracking system, Index controllers boast a high degree of tracking accuracy. With at least three base stations positioned in your play area, controller motions can be easily detected even when your hands are behind the headset with no occlusion issues that inside-out tracking systems may have.
Oculus Quest 2 Controllers Overview
Price: $69 per one
Battery type: AA
Battery time: 60+ hours
Weight: 147 grams
Compatible headsets: Oculus Quest 2
It would be unfair to compare Oculus Quest 2 controllers with other high-end options as the controllers were designed with pretty much the same mindset as the whole Oculus Quest 2 headset: cheap, tetherless, and casual experience.
Understanding this mindset leads us to understand all their best and bitter bits. As their hallmark, Oculus controllers offer unparalleled battery life. Whereas Valve Index knuckles have to be recharged every 7 hours or so, Oculus controllers offer 60+ hours of gameplay before you have to replace or recharge two AA batteries they operate with.
In terms of design, controllers are a slight variation of the original Oculus touch controllers with rounded handles, two triggers for index and middle fingers, and constellation tracking rings on top. As a result of the new design, the controller feels thicker in the hand and is easier to hold, while the battery door is less likely to fall off during intense gameplay sessions.
Oculus Quest 2 utilizes inside-out tracking which means that in order to track controller movements the frontal headset cameras have to always “see” controllers or at least detect part of them in their line of sight. Such an approach leads to certain limitations: there can be a tracking collision when one controller is in front of the other. Additionally, Oculus controllers can’t be effectively tracked with your hands behind your back.
On a side note, Oculus Quest 2 supports hand tracking so you can use the headset without controllers altogether. However, due to the high latency and less than ideal accuracy, hand tracking is mostly used with system apps.
HP Reverb G2 Controllers Overview
Price: $179 for a pair
Battery type: 1.5V AA batteries
Battery time: 5 hours
Weight: 196 grams
Compatible headsets: HP Reverb G2, WMR Headsets (e.g. Samsung Odyssey, Reverb G1)
HP Reverb G2 is praised for the astonishing quality of its visuals, but not for its controllers. Surely they are a step up compared to the previous generation of Windows Mixed Reality controllers, but they lack in both ergonomics and accuracy when compared to both Index and Oculus Touch controls.
The gamepad is not particularly responsive and the build quality seems cheaper than you would expect for a price. The controllers sport inside-out tracking which means you don’t need to buy external base stations to use them.
One particular highlight of the built-in tracking system is motion gyro sensors (IMU) that help predict fluid controller movements when your hand leaves an immediate line of the headset’s side. However, the issues of colliding controllers and the blind zone when you move your hands an inch close to the headset are still present.
HTC Vive “Wand” Controllers Overview
Price: $129.99 per one
Battery type: 960 mAh (rechargeable via USB cable)
Battery time: 6-9 hours
Weight: 210 grams
Compatible headsets: HTC VIVE, VIVE Pro Series, VIVE Cosmos Elite
At the time of their release in 2016, HTC Vive controllers embodied the pinnacle of ergonomics and build quality and set somewhat of an industry standard before the Valve Index came out.
Made out of durable high-quality plastic, Vive controllers are called “wands” and feature 24 sensors, a dual-stage trigger, a multi-functional trackpad. Although Wands provide pleasant haptic feedback, the experience is less rich compared to Index knuckles which are obvious when you compare the number of sensors in each. Another minor issue users report is a small options button.
Frankly, the iconic design of the controllers have outlived themselves and their price no longer reflects the level of quality you’d expect. What makes matters worse is that Vive is not rushing to release the newer version and even the latest Vive Pro 2 headset comes out with a pair of old controllers. Vive controllers work with Lighthouse tracking, but only with its 1.0 iteration which makes them incompatible with newer Valve Index Base 2.0 stations.
HTC Vive Cosmos Controllers Overview
Price: $99.00 per one
Battery type: 2 AA batteries for each controller
Battery time: 2-3 hours
Weight: 211 grams
Compatible headsets: HTC Vive Cosmos only
When the Vive Cosmos headset first came out, a huge portion of criticism was aimed at its sub-par tracking accuracy. Being part of the tracking rig, controllers shared the criticism. You would be immediately familiarized with the bow (similar to Oculus Quest) around the controllers.
The inside tracking that the Cosmos system employed ended up being worse than that of Oculus Quest 2 and far inferior to the lighthouse tracking that both Valve Index and original Vive Pro sport.
HTC Vive actually released the Elite Cosmos version with a traditional Lighthouse tracking to partially address the issues of inaccurate tracking, but the Cosmo controllers don’t work with Lighthouse tracking stations, so Cosmo Elite users also have to buy original wand controllers or Index knuckles in order to use track their movements.
PlayStation VR Controllers Overview
Coming out: 2022 or later
Compatible headsets: Playstation 5
Although the newest Playstation 2 VR (PSVR 2) controllers are rumored to be released in 2022 or later, we already have a wealth of information thanks to the official Sony PlayStation blog.
Among the prominent features are immersive haptic feedback, finger touch detection similar to Valve’s touchless experience, and an adaptive trigger button that adds palpable tension similar to that in DualSense controllers.
Judging by the company's demos the PSVR 2 is a substantial improvement from the first generation move controllers introduced in 2016 together with the release of Playstation 4. The finger tracking depth in the PS5 VR new controller might actually surpass that of Valve Index knuckles as tests showed that prototypes can track when fingers move apart -- something that Valve Index knuckles can’t accurately detect.
Since Playstation 5 is a gaming console where players need highly responsive controllers especially for melee combat games the tracking and time lag would have to be number one priority. Imagine trading blows on the triple A ps5 game with an annoying time delay?
The controllers will be tracked by the headset using a tracking ring across the lower section of the controllers.
So far there are no Pimax VR controllers available on the market, although Pimax has been promising to deliver its own set of Sword and Sword Sense controllers since 2017 after a successful $4M Kickstarter campaign.
There is currently no specific release date for the project that has undergone several redesigns and iterations since 2017during the last four years. Among the promised features is the ability to switch controllers batteries in the middle of the gameplay.
According to the company’s forum posting, Pimax Sword controllers entered the production testing phase in June 2021.
Varjo company currently doesn't have its own controllers. Varjo headset, however, supports Steam VR tracking and thus can work with both Valve Index knuckles and HTC Vive wands depending on what version of Lighthouse tracking base stations you use.
The latest Varjo VR3 headset features a built-in Ultraleap hand tracking platform to accurately capture hand movements. Given this extensive focus of a company on a hand-controlled VR, it’s safe to speculate we won’t be seeing any Varjo-designed VR controllers soon.
Overview of The Most Useful VR Controller Accessories
Here are the most common accessories to improve comfort and immersion when using your VR controllers.
Note: you might find variations of these for different controllers’ models or at least their 3D-printed replicas.
Controller Grips for Oculus Quest 2
If you dropped your Quest controllers once or twice, this grip solves the problem of slippery or ever missing Quest 2 controllers for good as now they’ll be comfortably strapped to your hands knuckles’-style.
Anker Charging Dock for Oculus Quest 2
Although Quest 2 controllers already boast exceptional battery life, this Anker Charging will solve the problem of dead batteries until you buy the next headset.
SINWEVR VR Controller Gun for Quest 2
If you’re an avid fan of VR shooters, check out this case that will turn your Quest 2 controllers into a gun.
Magni Stock Carbon Fiber for Valve Index
This product is for rifle fans who also happen to be Valve Index owners. Made out of light weather materials, this adjustable stock vividly improves immersion and aim in VR shooters. An accessory of choice for upcoming Doom VR game?
And lastly, did you know that you can turn your Android phone into a VR controller via bluetooth?
The Future Of VR Controllers
The future of controlling your movements in VR is quite certain: our movements will be tracked more accurately while VR experience itself will become fluid and natural. But the exact path we’ll take there is unknown as many companies offer different solutions when it comes to the future of VR controls.
Obviously, we could take the path of optimizing existing controllers, making them more lightweight and accurate, but more innovative solutions are already popping up.
With all the detrimental features of controllers, something seems quite obvious: the best controllers are our hands. There's no left or right buttons and our fleshy tools can grab throw and squeeze objects efficiently and we have our lifetime to use them. But we might have to wait for a while for proper tracking.
Just a couple of years ago Facebook acquired CTRL-labs, a startup that develops a brain-controlled interface. A wrist-wrapped gadget decodes electrical signals in hand muscles, allowing a VR headset or other computing devices to understand exactly what your hand is trying to do.
Although using this device you can interact with a virtual environment without any physical controllers, the lack of haptic feedback from actual physical sensation (imagine trying to squeeze a virtual ball, but instead just squeezing the air) somewhat delays VR immersion.
To address the lack of haptic feedback in a VR environment, several companies offer haptic gloves. A haptic glove from Dexmo is a mechanical system that connects with your fingers and palms. It can, for example, mimic the physical feedback by preventing your fingers from connecting when squeezing virtual objects.
Ultraleap, on the other hand, uses an array of small ultrasound speakers that emit non-perceivable sound waves, creating a tiny dent in the skin that feels like a physical touch.
Lastly, Valve registered brain interface patents, aiming to create a system where you can control VR environments solely with your brain impulses, skipping hand controls altogether. Such a system might be able to detect player emotional responses and adjust the virtual environment accordingly.
Although direct brain interfaces are years away in terms of consumer-ready production, this scenario of VR controlled environment using direct brain impulses seems like the most promising and effortless as they come.
In terms of accuracy, interactivity, and ergonomics, Valve Index controllers are still way ahead of the competition. With the arrival and promise of Valve Index 2 being wireless, most of VR enthusiast is looking at them.
However, to realize the full potential of Valve knuckles you have to weigh-in the overall costs of buying at least two base stations the controllers require to operate.
In this aspect, nothing comes close to Oculus Quest 2 controllers in terms of cost-efficiency as currently Oculus Quest 2 offers the cheapest tetherless VR experience on the market.
But let’s wait and see just how good the upcoming PlayStation VR 5 and Pimax controllers will turn out. Perhaps the status quo of the VR controllers scene will change the very next year.
Note: It goes without saying, but take special care of your controllers. Most of them are made from plastic of variable quality. Since you won't pay attention to the physical world while playing, you can smash your controllers with ease. Although you can buy VR controllers separately on the official manufacturer website or through Amazon or eBay, those purchases are going to be quite expensive.
Interested in XR Unity development or learning about AR and VR?
Take a look at our course content and syllabus.
Download XR Development with Unity Course Syllabus