Items with no label
3335 Discussions

Implement MouseButtonDown like mouse using RSUnityToolkit

CCH
Beginner
3,312 Views

Hello, I'm new at using Unity and camera Realsense.

 

I'm making a game and and it can be only controlled using left click mouse.

 

I want it can be played using gesture Realsense and has same function like click mouse.

Anyone can help me to solve my problem ?

0 Kudos
1 Solution
MartyG
Honored Contributor III
917 Views

I have written a large range of detailed step by step guides in the past on numerous aspects of using RealSense with Unity. You may find some of these useful as you learn.

https://software.intel.com/en-us/forums/realsense/topic/676139 Index of Marty G's RealSense Unity How-To Guides

Also, this short guide to using the RealSense Unity Toolkit's 'SendMessageAction' script to activate a function with a gesture may be helpful.

In the guide I activate the function with a thumbs-up gesture, but you could equally use the Hand closed gesture instead as a substitute for a mouse-click.

View solution in original post

0 Kudos
34 Replies
MartyG
Honored Contributor III
918 Views

I have written a large range of detailed step by step guides in the past on numerous aspects of using RealSense with Unity. You may find some of these useful as you learn.

https://software.intel.com/en-us/forums/realsense/topic/676139 Index of Marty G's RealSense Unity How-To Guides

Also, this short guide to using the RealSense Unity Toolkit's 'SendMessageAction' script to activate a function with a gesture may be helpful.

In the guide I activate the function with a thumbs-up gesture, but you could equally use the Hand closed gesture instead as a substitute for a mouse-click.

0 Kudos
CCH
Beginner
572 Views

Can you explain more step 6 , 7 and 8 ?

Is in the step 6 are we need create a new C# script in same GameObject with SendMessageAction script ?

0 Kudos
MartyG
Honored Contributor III
572 Views

Yes, in Step 6, create a new C# script in the same object that the SendMessageAction component is in (you can call your script any name you want to).

The important thing is the name of the function that the code that you want the SendMessageAction to activate is placed inside. If you put it inside a LateUpdate() function, then you must put the word LateUpdate into the 'Function Name' section of the SendMessageAction.

0 Kudos
CCH
Beginner
572 Views

I already try and success implement it to my game, but the problem now is like my gesture is detected very fast (cause like get clicked itself).

Any option to make delay it or making it only detect my gesture one by one ?

0 Kudos
MartyG
Honored Contributor III
572 Views

If you expand open the Hand Closed gesture in SendMessageAction then you will see an Openness Factor option. By default it is set to '50', meaning that the hand only needs to be closed halfway before it can be recognized as a hand-closed gesture.

If the Openness Factor is set high then it is easy for the camera to think you've made a closed-fist even when your hand is open. At one point during my own project's development, my game character used open and closed hand to toggle between crouch and standing positions. It was bouncing up and down on the spot like a kangaroo!

If you change it to a lower value such as '20' though then it will be more difficult to trigger.

0 Kudos
CCH
Beginner
572 Views

I already try it and success delay, but i cant found "Openless Factor" in Event Source "Gesture detected".

Is that possible apply gesture like "V sign" or "Grab" as toggle but with "openless factor" ?

0 Kudos
MartyG
Honored Contributor III
572 Views

I believe you should choose Hand Closed as your event source, not Gesture Detected.

0 Kudos
CCH
Beginner
572 Views

Thank you for your suggestion, After i think again, I will choose Hand Closed as my toggle because i need that "Openless Factor".

Thank you very much.

0 Kudos
CCH
Beginner
572 Views

Is that possible if we implement MouseButtonDown and Gesture "Hand Closed at the same time ?

0 Kudos
MartyG
Honored Contributor III
572 Views

Yes, RealSense controls and traditional controls such as mouse, keyboard and joypad can exist simultaneously. They can even trigger the same function name in the same script. So you can can either trigger with a Hand Closed gesture, a mouse click, a keyboard button or a joypad button. When using traditional controls, usually the RealSense controls will not be able to be triggered because your hands are outside of the camera's view or not in the hand-closed position when you are using these types of control device.

0 Kudos
CCH
Beginner
572 Views

Can you give me an example how to use traditional control and realsense control ?

i want implement it to button.

0 Kudos
MartyG
Honored Contributor III
572 Views

Keyboard keys in Unity are programmed with the GetKey command, whilst joypad buttons use GetAxis. Here are some examples:

Press the space key to make something happen

if (Input.GetKey ("space") == true) {

Debug.Log("Space key is pressed");

}

Click the red 'B' joypad button to make something happen

if (Input.GetAxis (Fire2) == 1) {

Debug.Log("Red 'B' joypad button is pressed");

}

When defining key-presses, you can put in the quotation marks "a" to "z", and "1" to "0" to use the numeric keys on the top row of the keyboard. There is also "esc" (Escape), "enter" (Enter), "del" (Del), "return" (Return), "tab" (Tab), "left ctrl" and "right ctrl" (left control and right control), "left shift" and "right shift" (left and right shift). There are other codes as well for the special symbol keys. I recommend looking these up on the internet.

Number-pad keys at the side are placed in square brackets. For example, numpad 1 is written as "[1]"

Function keys at the top of the keyboard can be written with the KeyCode instruction as:

if (Input.GetKey (KeyCode.F1) {

all the way up to

if (Input.GetKey (KeyCode.F12)

You can program your own custom keypress and joypad controls in Unity's 'Input Manager', accessed by going to the 'Edit' menu and selecting 'Project > Input Manager, and give each definition a custom name.

For example, you could program a definition called Back Button, assign the joypad's Back button to it and then use an instruction such as:

if (Input.GetAxis (Back Button) == 1) {

Debug.Log("'Back' joypad button is pressed");

}

It is important to remember that when you are using a custom control that is defined in the Input Manager, you do not put the name in quotation marks. So it is Back Button, not "Back Button".

With many buttons on a joypad, we say that it is activated when its value is "1" because "1" is the value it has when the button is pushed in. When the button is not being used, it has a value of "0".

For controls that have left- right, up and down directions, like the sticks and cross-shaped "D-pad" it is a bit more complex. Up is '1', down is '-1', left is '-1' and right is '1'. When these controls are not being pressed, their value is '0'.

I hope this helps!

0 Kudos
CCH
Beginner
572 Views

so we need to set our Keypress first at Input Manager ?

 

but, i still need some more explain how a button can be accessed both with traditional control and realsense control.

like jump action with key "space" , and can jump too with "hand clossed"

0 Kudos
MartyG
Honored Contributor III
572 Views

Most ordinary keys on the keyboard do not have to use the Input Manager. You can just put the key in quotations, like "a" for the a key. The advantage of using the Input Manager for keys is if you want to have a few different controls trigger the same event, or set how sensitive a key is when it is pressed. The more sensitive it is, the more times the action will trigger.

Here's a guide to keycodes.

https://docs.unity3d.com/ScriptReference/KeyCode.html Unity - Scripting API: KeyCode

My approach to making an object do a motion such as Jump is to make an Update() function script that causes the object to move a certain distance every second whilst the script is active. Then I un-tick the script so it is inactive, and program my controls to enable the script when the control = 1 (pressed) and deactivate it again when the control = 0 (not pressed). This ensures that the object only moves when the control is pressed and becomes '1'. As soon as you stop pressing and the control becomes '0', the Update() script is disabled again and the object's motion ceases.

At the end of the script, place:

this.enabled = false;

This tells the script to deactivate when it has completed processing it the first time. BUT because it is being sent constant activation signals whilst you are holding the control down, it activates - deactivates - reactivates so fast that you do not see any pause in the motion of the object.

So you use the same Update() script for both the camera control and traditional control. The only thing that is different is the way in which you enable the script.

With the camera, you use a script with SendMessageAction to send an activation signal to your movement script. When you stop making the hand gesture, SendMessageAction stops sending the activation signal to the motion script. This means that the next time that the script reaches the this.enabled = false line at the end of the script, the script turns off and object motion ceases, because it is no longer being told to make the script active again.

With the keyboard control, if you set it to enable the motion script when a space bar press is true, then the motion script will be forced to be active for as long as you are holding the space key down, and when you let go of the space bar the motion script will stop running when it reaches this.enabled = false the next time.

0 Kudos
CCH
Beginner
572 Views

please help me if i wrong, so

1. if we want use both traditional control and camera control we need put it in one script ?

2. here my example script for move an object :

public void LateUpdate()

{

if (Input.GetKeyDown(KeyCode.Space))

Debug.Log ("Jump");

this.enabled = false;

} // is this wrong ?

0 Kudos
MartyG
Honored Contributor III
572 Views

You almost got it right. You left the curly brackets out in the If statement.

public void LateUpdate()

{

if (Input.GetKeyDown(KeyCode.Space))

{

Debug.Log ("Jump");

}

this.enabled = false;

}

}

0 Kudos
CCH
Beginner
572 Views

i dont understand why there 3 close curly bracket ?

0 Kudos
MartyG
Honored Contributor III
572 Views

The first one closes the If statement. The second closes the LateUpdate function. The third closes the overall script.

0 Kudos
MartyG
Honored Contributor III
572 Views

* Copying a PM from Chris into the public thread with his permission*

Hello, I'm have a problem with function camera script.

First, (before using camera) i put my traditional control to control gameobject. and i put that code into my "private void Update()" with another function to move my gameobject. *note : my function to move gameobject play automatically.

Second, after i implement realsense function, I can control gameobject with camera like traditional control in function "public void LateUpdate". But the problem is my function to move gameobject didnt work. And only move my gameobject a little when camera is triggered (fell like only move a little when camera can read my gesture). I put my game script with camera function.

here my traditional function :

private void Update () {

if (Input.GetMouseButton(0))

{

if (PutObject())

{

SpawnObject();

count++;

}

else

{

EndGame();

}

}

MoveObject();

}

0 Kudos
MartyG
Honored Contributor III
516 Views

Reading through your code, it looks as though when the mouse button is pressed, you want to spawn an object and increase the value of a count by 1.

Now, as for the other part of your script ... it looks as though because MoveObject() is outside of the If statement's brackets, it should activate automatically when the script is run. So if the object is not moving, that suggests that the problem is inside the MoveObject() function and not in this code.

0 Kudos
Reply