Tutorial written by Amanda End and Circuit Stream.

Welcome to Section Six of the HTC Vive Virtual Reality Programming Tutorial. Previously, we created a Raycast input script that allow us to look and point at stuff and make it do stuff. Great! But what do we do with it? Well, learning by example is fun, so let’s try a few more examples so you can see how these new inputs can be used.

We’re going to cover the following examples:

Most of these aren’t as practical as the interaction examples, but I hope they’ll give you an idea of how you can use Raycast input in your games just the same.

Watch

 

 
To give an example of a “screen” in world space, let’s make a simple watch script. In order to utilize our new Raycasting methods, let’s also made it only show when the user is looking at their hand.

The Model

The model is a pretty simple construction. It is a parent gameobject named Watch (which will hold our script) that has a collider. The collider won’t automatically generate, so you’ll have to add a box collider and adjust it manually. I made mine just a little bit bigger than the canvas, and quite a bit thicker. Parented to the watch gameobject is a World Space Canvas that has been scaled way down (280×150 and a scale of 0.0005). If you don’t know how to set your canvas to World Space, check the “Render Mode” dropdown in the Canvas Component. Within the canvas there is one Text UI element that is stretched to fill the canvas.

The Script

As with our complex interactables, the watch script is going to inherit from our VRInteractableObject class. The methods we’ll want to override for this situation are the RayEnter and RayExit. We won’t need RayStay.

using UnityEngine;
using UnityEngine.UI;

public class VRIO_WatchExample : VRInteractableObject
{
	void Update ()
	{

	}

	public override void RayEnter(RaycastHit hit, VRControllerInput controller = null)
	{

	}

	public override void RayExit(VRControllerInput controller = null)
	{

	}
}

Variables

We’re going to want a couple public variables here – one for storing a reference to our Canvas (watchCanvas) and another for storing the Text component for our watch display (watchText). Once those are defined, assign them in the inspector.

public GameObject watchCanvas;
public Text watchText;

Showing and Hiding the Watch

Now we can utilize RayEnter and RayExit to hide the watch. But, we need to make sure it only happens when we look at the watch, not point at it with our hands. So we’re going to check our controller argument, and if it is null (which means it was triggered by the head, and not one of the hands) we’re showing or hiding our watchCanvas.

public override void RayEnter(RaycastHit hit, VRControllerInput controller = null)
{
	//If controller is null, then it was the head that triggered the method
	if (controller == null)
	{
		//Show Watch
		watchCanvas.SetActive(true);
	}
}

public override void RayExit(VRControllerInput controller = null)
{
	//If controller is null, then it was the head that triggered the method
	if (controller == null)
	{
		//Hide Watch
		watchCanvas.SetActive(false);
	}
}

Making the watch work

Now that the watch shows and hides when we look at and away from it, we need it to actually tick. So, on Update we’ll set the our Time.text based on DateTime.Now. We can also throw in a line saying to only update the time if the watch is actually active.

void Update()
{
	//If watch is showing, update text on watch to display time
	if (watchCanvas.activeInHierarchy)
		watchText.text = System.DateTime.Now.ToString("HH:mm:ss");
}

And that completes our watch script! Play the scene and take a look at your fancy new watch hand. There’s a 50% chance you’ll realize a flaw in our example – it’s real easy for the watch to be on the wrong hand and end up upside down (just like real watches?). But never fear! We’re going to go over something in one of our later examples that will let you detect which hand your controller reference is (left, or right). Test your skills by seeing if you can utilize that to make the watch work on either hand and be correctly oriented for either.

Raycast Cube

 

 
Now let’s go over something that differentiates between your head and your hands, but reacts to both. So I’m going to make a cube that changes color to pink when you look at it, and blue when you point at it. You might be thinking that this isn’t really an applicable example in a game, and you’re probably right but I’m sure you’ll think of something.

Setup

The setup for this cube is pretty simple. It’s just a cube. For the script, create a new script that inherits from VRInteractableObject, and put it on the cube.

The Script

As with the watch, we’re concerned with the RayEnter and RayExit scripts. We want to do something different depending on whether the hand is triggering the object, or the head. So we’ll set up an if else checking to see if the controller reference is null.

using UnityEngine;

public class VRIO_RaycastExample : VRInteractableObject
{
	public override void RayEnter(RaycastHit hit, VRControllerInput controller = null)
	{
		if (controller == null)
		{
			//Triggered by Head
		}
		else
		{
			//Triggered by Hands
		}
	}

	public override void RayExit(VRControllerInput controller = null)
	{
		if (controller == null)
		{
			//Triggered by Head
		}
		else
		{
			//Triggered by Hands
		}
	}
}

If you remember, the controller being null means that the raycast hit is coming from the head, and if it isn’t, it’s coming from the hands.

Setting up Material and Colors

Since we want the cube want to change the color when we look or point at it, we need a reference to the cube’s material. So let’s set up a few public variables (desired colors and a reference to the renderer) so that we can set them in the inspector.

public Color headRayColor;
public Color controllerRayColor;
public Renderer objectRenderer;

 

 
We’ll also want a couple protected class variables to hold the original color and a reference to the material, which we can assign on Awake.

protected Color originalColor;
protected Material objectMaterial;

void Awake()
{
	//Get object's material from the renderer, and the original color
	objectMaterial = objectRenderer.material;
	originalColor = objectMaterial.color;
}

Changing the Colors

Now that we have our variables all set up, We can change the material’s color on RayEnter and RayExit to the colors we set in the inspector.

public override void RayEnter(RaycastHit hit, VRControllerInput controller = null)
{
	Color c;
	if (controller == null)
	{
		//Use color designated for head
		c = headRayColor;
	}
	else
	{
		//Triggered by Hands
		c = controllerRayColor;
	}
	objectMaterial.color = c;
}

public override void RayExit(VRControllerInput controller = null)
{
	objectMaterial.color = originalColor;
}

Once I got to this point I realized that I didn’t actually have to differentiate between head and hands for RayExit, and if you caught that, good for you! That was totally a test (not really a test)! So you can delete most of what we had in there. Also, if you know about ternary operators, you know that I can cut down RayEnter to take up a little less space as well.

public override void RayEnter(RaycastHit hit, VRControllerInput controller = null)
{
	//Get desired color and apply to the object's material
	Color c = controller == null ? headRayColor : controllerRayColor;
	objectMaterial.color = c;
}

And that’s it! Easy peasy little color cube that probably won’t ever have a real application. But hopefully now you’re getting how we can utilize Raycasting to interact with objects in the scene!

Raycast Canvas

 

 
One last helpful thing when it comes to Raycasting is the ability to depict where you’re looking or pointing on a Canvas. This is also a good opportunity for us to go over how to differentiate between the two controllers! Our goal here will be to create the canvas pictured above, and make the pink dot follow wherever on the canvas the player is looking, the dark blue dot will follow wherever their hand is pointing, and the light dot where their right hand is pointing.

The Setup

The object we’re going to be working with is similar to our watch example. We have a World Space Canvas that isn’t parented to anything, and placed wherever desired in the world. As with the watch, you’re going to add a box collider to it. This time, make the collider the exact same size height and width as the canvas (curious why? When we’re done, make it bigger and see what happens!), but still make it a bit thicker. The canvas itself will also hold the script we’re going to write. On the canvas is a background Panel object, a Text object holding my instructions, and three different Image objects, one for each of the hand and the head.

 
Make sure the images are all anchored to middle center and have a pivot of 0.5,0.5 (should be the default) otherwise you’ll have to adjust your code.

The Script

As usual, the script is going to inherit from the VRInteractableObject class. This time, we’re only going to be overriding the RayStay method.

Identifying Controllers

 
The first thing we want to set up is our ability to tell which controller is the “leftmost”. The SteamVRController class has a static method called GetDeviceIndex that, given an argument describing a relative location (such as leftmost or rightmost), returns the device index of the desired controller. Since this can change at any time, We need to ask SteamVRController every single frame that we want this information. So let’s set up a property that, as it’s get, calls the method in SteamVRController and returns the result. Since we only have two controllers, I only need to keep track of which is the leftmost, and the will be the right.

using UnityEngine;

public class VRIO_RaycastCanvasExample : VRInteractableObject
{
	protected int leftControllerIndex
	{
		get
		{
			//Get index of leftmost controller
			return SteamVR_Controller.GetDeviceIndex(SteamVR_Controller.DeviceRelation.Leftmost);
		}
	}

	public override void RayStay(RaycastHit hit, VRControllerInput controller = null)
	{

	}
}

Other Variables

 
We will also need a reference to the RectTransforms of the objects on the canvas that we want to move, as well as a reference to the CanvasRectTransform. So let’s add those public variables and then assign them in the inspector.

public RectTransform headIndicatorRect;
public RectTransform[] handIndicatorRects;
public RectTransform canvasRect;

The handIndicatorRects should be an array of two – one UI element for each hand.

 

Get the Position of the Hit

 
Before we can move a UI element to the place that our ray hits, we need to figure out where that is relative to the canvas. The Raycast object will tell us where in world coordinates that it hit, but that doesn’t help us know where on the canvas that it hit. So we’re going to use – can you guess? – the InverseTransformPoint method again on our canvas to convert the position in world space to a local position relative to the canvas.

public override void RayStay(RaycastHit hit, VRControllerInput controller = null)
{
	//Get position of the raycast hit, relative to the Canvas
	Vector2 position = canvasRect.InverseTransformPoint(hit.point);
}

Apply position to the proper UI Element

We’ll start with the controllers. We know that to check if the sender was a hand, we just check to see if the VRControllerInput was null. If it wasn’t, then the sender was a hand.

public override void RayStay(RaycastHit hit, VRControllerInput controller = null)
{
	//Get position of the raycast hit, relative to the Canvas
	Vector2 position = canvasRect.InverseTransformPoint(hit.point);

	//If there is a controller script, which meands that a controller is triggering
	if (controller)
	{
		//Get a controller index of either 0 or 1, 0 if the controller is leftmost, 1 otherwise
		int controllerIndex = controller.device.index == leftControllerIndex ? 0 : 1;

		//Make sure that enough objects on the canvas are assigned
		if (controllerIndex < handIndicatorRects.Length)
		{
			//Move canvas object that matches the hands
			handIndicatorRects[controllerIndex].anchoredPosition = position;
		}
	}
}

Next We Get the Index

Then we get the index of the controller using controller.device.index, and compare it to the leftControllerIndex, the property that we set up earlier to determine the index of our leftmost hand, to see if it is the left controller. Using a ternary operator, we’re going to set our index variable to 0 if it is the left hand, and 1 if it is the right. This index will be used as an index on our handIndicatorRects array so that the one in the 0 index is always controlled by the leftmost hand, and the 1 index the right. Once we’ve figured out which RectTransform we’re updating, we set it’s anchoredPosition to the position we got in the last step.

public override void RayStay(RaycastHit hit, VRControllerInput controller = null)
{
	//Get position of the raycast hit, relative to the Canvas
	Vector2 position = canvasRect.InverseTransformPoint(hit.point);

	//If there is a controller script, which meands that a controller is triggering
	if (controller)
	{
		//Get a controller index of either 0 or 1, 0 if the controller is leftmost, 1 otherwise
		int controllerIndex = controller.device.index == leftControllerIndex ? 0 : 1;

		//Make sure that enough objects on the canvas are assigned
		if (controllerIndex < handIndicatorRects.Length)
		{
			//Move canvas object that matches the hands
			handIndicatorRects[controllerIndex].anchoredPosition = position;
		}
	}
}
else	
{
	//Move head canvas object to match position
	headIndicatorRect.anchoredPosition = position;
}

At the Very End

 
After the hands are sorted, we just need to add an ‘else’ to cover the case for the head (if the controller was null) and do the same, but for the headIndicator, and we’re all set!

Screens and displays are very familiar things to us, especially when you work in technology. Chances are you’re going to have a flat UI of some sort in your game, and one of the easiest ways to interact with that UI will be pointing and looking. Hopefully you find the above example useful to your endeavors!

See you next time, on Intro to VR Development for Unity and HTC Vive!
Previously: Input Via Raycasting.
Next Up: Some Useful Tools.

Learn how to code for virtual reality with our Unity Course. For 1 on 1 help, the course is one of the best ways to learn unity and how to make VR games and applications for any virtual reality platform. Visit the courses page for more info.