1 2 3 Previous Next 34 Replies Latest reply on Feb 13, 2017 11:18 AM by MartyG

    Implement MouseButtonDown like mouse using RSUnityToolkit

    Chris_CH

      Hello, I'm new at using Unity and camera Realsense.
      I'm making a game and and it can be only controlled using left click mouse.
      I want it can be played using gesture Realsense and has same function like click mouse.

       

      Anyone can help me to solve my problem ?

        • 1. Re: Implement MouseButtonDown like mouse using RSUnityToolkit
          MartyG

          I have written a large range of detailed step by step guides in the past on numerous aspects of using RealSense with Unity.  You may find some of these useful as you learn.

           

          Index of Marty G's RealSense Unity How-To Guides

           

          Also, this short guide to using the RealSense Unity Toolkit's 'SendMessageAction' script to activate a function with a gesture may be helpful.

           

          Re: [SR300] Can't reconize the gestures

           

          In the guide I activate the function with a thumbs-up gesture, but you could equally use the Hand closed gesture instead as a substitute for a mouse-click.

          • 2. Re: Implement MouseButtonDown like mouse using RSUnityToolkit
            Chris_CH

            Can you explain more step 6 , 7 and 8 ?

             

            Is in the step 6 are we need create a new C#script in same GameObject with SendMessageAction script ?

            • 3. Re: Implement MouseButtonDown like mouse using RSUnityToolkit
              MartyG

              Yes, in Step 6, create a new C# script in the same object that the SendMessageAction component is in (you can call your script any name you want to).

               

              The important thing is the name of the function that the code that you want the SendMessageAction to activate is placed inside.  If you put it inside a LateUpdate() function, then you must put the word LateUpdate into the 'Function Name' section of the SendMessageAction.

              • 4. Re: Implement MouseButtonDown like mouse using RSUnityToolkit
                Chris_CH

                I already try and success implement it to my game, but the problem now is like my gesture is detected very fast (cause like get clicked itself).

                Any option to make delay it or making it only detect my gesture one by one ?

                • 5. Re: Implement MouseButtonDown like mouse using RSUnityToolkit
                  MartyG

                  If you expand open the Hand Closed gesture in SendMessageAction then you will see an Openness Factor option.  By default it is set to '50', meaning that the hand only needs to be closed halfway before it can be recognized as a hand-closed gesture.

                   

                  If the Openness Factor is set high then it is easy for the camera to think you've made a closed-fist even when your hand is open.  At one point during my own project's development, my game character used open and closed hand to toggle between crouch and standing positions.  It was bouncing up and down on the spot like a kangaroo!

                   

                  If you change it to a lower value such as '20' though then it will be more difficult to trigger.

                   

                  1.jpg

                  • 6. Re: Implement MouseButtonDown like mouse using RSUnityToolkit
                    Chris_CH

                    I already try it and success delay, but i cant found "Openless Factor" in Event Source "Gesture detected".

                     

                    Is that possible apply gesture like "V sign" or "Grab" as toggle but with "openless factor" ?

                    • 7. Re: Implement MouseButtonDown like mouse using RSUnityToolkit
                      MartyG

                      I believe you should choose Hand Closed as your event source, not Gesture Detected.

                      • 8. Re: Implement MouseButtonDown like mouse using RSUnityToolkit
                        Chris_CH

                        Thank you for your suggestion, After i think again, I will choose Hand Closed as my toggle because i need that "Openless Factor".

                         

                        Thank you very much.

                        • 9. Re: Implement MouseButtonDown like mouse using RSUnityToolkit
                          Chris_CH

                          Is that possible if we implement MouseButtonDown and Gesture "Hand Closed at the same time ?

                          • 10. Re: Implement MouseButtonDown like mouse using RSUnityToolkit
                            MartyG

                            Yes, RealSense controls and traditional controls such as mouse, keyboard and joypad can exist simultaneously.  They can even trigger the same function name in the same script.  So you can can either trigger with a Hand Closed gesture, a mouse click, a keyboard button or a joypad button.  When using traditional controls, usually the RealSense controls will not be able to be triggered because your hands are outside of the camera's view or not in the hand-closed position when you are using these types of control device.

                            • 11. Re: Implement MouseButtonDown like mouse using RSUnityToolkit
                              Chris_CH

                              Can you give me an example how to use traditional control and realsense control ?

                              i want implement it to button.

                              • 12. Re: Implement MouseButtonDown like mouse using RSUnityToolkit
                                MartyG

                                Keyboard keys in Unity are programmed with the GetKey command, whilst joypad buttons use GetAxis.  Here are some examples:

                                 

                                Press the space key to make something happen

                                 

                                if (Input.GetKey ("space") == true) {

                                 

                                Debug.Log("Space key is pressed");

                                 

                                }

                                 

                                Click the red 'B' joypad button to make something happen

                                 

                                if (Input.GetAxis (Fire2) == 1) {

                                 

                                Debug.Log("Red 'B' joypad button is pressed");

                                 

                                }

                                 

                                When defining key-presses, you can put in the quotation marks "a" to "z", and "1" to "0" to use the numeric keys on the top row of the keyboard.  There is also "esc" (Escape), "enter" (Enter), "del" (Del), "return" (Return), "tab" (Tab), "left ctrl" and "right ctrl" (left control and right control), "left shift" and "right shift" (left and right shift).  There are other codes as well for the special symbol keys.  I recommend looking these up on the internet.

                                 

                                Number-pad keys at the side are placed in square brackets.  For example, numpad 1 is written as "[1]"

                                 

                                Function keys at the top of the keyboard can be written with the KeyCode instruction as:

                                 

                                if (Input.GetKey (KeyCode.F1) {

                                 

                                all the way up to

                                 

                                if (Input.GetKey (KeyCode.F12)

                                 

                                You can program your own custom keypress and joypad controls in Unity's 'Input Manager', accessed by going to the 'Edit' menu and selecting 'Project > Input Manager, and give each definition a custom name.

                                 

                                For example, you could program a definition called Back Button, assign the joypad's Back button to it and then use an instruction such as:

                                 

                                if (Input.GetAxis (Back Button) == 1) {

                                 

                                Debug.Log("'Back' joypad button is pressed");

                                 

                                }

                                 

                                It is important to remember that when you are using a custom control that is defined in the Input Manager, you do not put the name in quotation marks.  So it is Back Button, not "Back Button".

                                 

                                With many buttons on a joypad, we say that it is activated when its value is "1" because "1" is the value it has when the button is pushed in.  When the button is not being used, it has a value of "0".

                                 

                                For controls that have left- right, up and down directions, like the sticks and cross-shaped "D-pad" it is a bit more complex.  Up is '1', down is '-1', left is '-1' and right is '1'.  When these controls are not being pressed, their value is '0'.

                                 

                                I hope this helps!

                                • 13. Re: Implement MouseButtonDown like mouse using RSUnityToolkit
                                  Chris_CH

                                  so we need to set our Keypress first at Input Manager ?
                                  but, i still need some more explain how a button can be accessed both with traditional control and realsense control.

                                   

                                  like jump action with key "space" , and can jump too with "hand clossed"

                                  • 14. Re: Implement MouseButtonDown like mouse using RSUnityToolkit
                                    MartyG

                                    Most ordinary keys on the keyboard do not have to use the Input Manager.  You can just put the key in quotations, like "a" for the a key.  The advantage of using the Input Manager for keys is if you want to have a few different controls trigger the same event, or set how sensitive a key is when it is pressed.  The more sensitive it is, the more times the action will trigger.

                                     

                                    Here's a guide to keycodes.

                                     

                                    Unity - Scripting API: KeyCode

                                     

                                    My approach to making an object do a motion such as Jump is to make an Update() function script that causes the object to move a certain distance every second whilst the script is active.  Then I un-tick the script so it is inactive, and program my controls to enable the script when the control = 1 (pressed) and deactivate it again when the control = 0 (not pressed).  This ensures that the object only moves when the control is pressed and becomes '1'.  As soon as you stop pressing and the control becomes '0', the Update() script is disabled again and the object's motion ceases.

                                     

                                    At the end of the script, place:

                                     

                                    this.enabled = false;

                                     

                                    This tells the script to deactivate when it has completed processing it the first time.  BUT because it is being sent constant activation signals whilst you are holding the control down, it activates - deactivates - reactivates so fast that you do not see any pause in the motion of the object.

                                     

                                    So you use the same Update() script for both the camera control and traditional control.  The only thing that is different is the way in which you enable the script. 

                                     

                                    With the camera, you use a script with SendMessageAction to send an activation signal to your movement script.  When you stop making the hand gesture, SendMessageAction stops sending the activation signal to the motion script.  This means that the next time that the script reaches the this.enabled = false line at the end of the script, the script turns off and object motion ceases, because it is no longer being told to make the script active again.

                                     

                                    With the keyboard control, if you set it to enable the motion script when a space bar press is true, then the motion script will be forced to be active for as long as you are holding the space key down, and when you let go of the space bar the motion script will stop running when it reaches this.enabled = false the next time.

                                    1 2 3 Previous Next