12 Replies Latest reply on Feb 16, 2017 1:40 AM by MartyG

    Touchless main menu buttons (SR300)

    supermomo90

      Hi,

       

      I have an application that has a main menu and would like to control the buttons using hand or face tracking. I referred to a post ( Unity TIp: Creating a RealSense-Powered Menu In Unity ) and was able to complete it, but I am unsure how to implement it on the buttons to go to another scene. May I know if anyone knows how I can do it?

       

      Thank you.

        • 1. Re: Touchless main menu buttons (SR300)
          MartyG

          I'm glad you found the Unity guide useful!

           

          Loading in a scene when a button is activated involves a simple one line instruction called Application.LoadLevel ("name of your scene file");

           

          Unity - Scripting API: Application.LoadLevel

           

          It is important to remember that you need to program Unity to wait for an input before activating the instruction, otherwise it will activate the scene load as soon as your program starts instead of waiting for an input from you.

           

          One way to do this is to create an OnMouseDown() type function in your scripting.  This tells Unity to wait until a mouse button has been clicked before activating the scene load.

           

          public void OnMouseDown() {

           

          Application.LoadLevel ("name of your scene file");

           

          }

           

          Since you are creating camera controls for your main menu though, it is preferable to use a SendMessageAction component to send the activation signal to a script containing your Application.LoadLevel instruction when a particular gesture is detected.  If I remember my menu guide correctly, I believe that I had set it to activate a highlighted option when the Hand Closed gesture is made.

           

          Please feel free to ask here if you have more questions.  Good luck!

          1 of 1 people found this helpful
          • 2. Re: Touchless main menu buttons (SR300)
            supermomo90

            Hi MartyG,

             

            I have managed to use hand tracking to make 1 button work but I cannot get the same Menu Selector to trigger more than 1 button. The script shown below allows only the "start game" button to work but not the "quit" button. The OnTriggerEnter() is used twice. Thank you.

             

            public void OnTriggerEnter(Collider other)

                {

                    Debug.Log("Hello");

                    SceneManager.LoadScene("game");

                }

             

                public void OnTriggerEnter(Collider other)

                {

                    Debug.Log("Quitting...");

                    SceneManager.LoadScene("game1");

                }

            • 3. Re: Touchless main menu buttons (SR300)
              MartyG

              I had a look through my own menu system, which uses the design in the guide, to refresh my memory about how to select multiple options with the same Hand Closed Gesture.

               

              STEP ONE

               

              Highlight the button that you want to add an activation mechanism to.  Go to 'GameObject > Create Empty Child' to child-attach an invisible Empty GameObject to the button.

               

              1.jpg

               

              STEP TWO

               

              Highlight the newly created Empty GameObject.  Go to 'Component > Add > Physics > Box Collider' to add a Box Collider to the Empty GameObject.  Then play with the collider's X, Y and Z size values until you have defined a collider that is a bit larger than your option selector object, so that the selector is enclosed totally by the collider when it enters the field.

               

              Remember to place a tick in the collider's Is Trigger box so that Unity treats it as a trigger field.

               

              STEP THREE

               

              Place a SendMessageAction component in the Empty GameObject and un-tick it so that it is dormant by default.  Also set it to activate when the Hand Closed gesture is detected.

               

              STEP FOUR

               

              Create a trigger script in the Empty GameObject.  It should include the following OnTriggerEnter and OnTriggerExit instructions.

               

              public void OnTriggerEnter(Collider other)

              {

               

              GetComponent<SendMessageAction>().enabled = true;

               

              }

               

              public void OnTriggerExit(Collider other)

              {

               

              GetComponent<SendMessageAction>().enabled = true;

               

              }

               

              When the selector object enters the Empty GameObject trigger field, it will automatically enable the SendMessageAction component.  When the selector exits the field, the SendMessageAction will be disabled again.  What this does is ensure that the action linked to this particular SendMessageAction can only be triggered when the selector is over that option.  When the selector is outside of the field whilst over another option, the SendMessageAction cannot respond to the Hand Closed gesture, and so the action linked to that SendMessageAction cannot accidentally trigger.

               

              STEP FIVE

               

              In your trigger script, create a fire-once function with a name of your own choosing, such as TitleExit_Handconfirm.  This function can only activate when an activation signal is specifically sent to it, and will be ignored otherwise.  For example:

               

              public void TitleExit_HandConfirm ()

              {

               

              TitleScreenGUI_Confirmation.GetComponent<Canvas> ().enabled = true;

               

              }

               

              What this example does is to activate a canvas GUI that has a "Are you sure you want to exit?" message with Yes and No buttons.  You can put your own instructions inside the function to state what you want to happen when the button is activated.

               

              STEP SIX

               

              Go to the SendMessageAction and put in the 'Function Name' text box the name of the function that contains your activation instructions ( e.g TitleExit_HandConfirm ).  Remember that you do NOT need to put the name of the script in this box.  You are instead stating the name of a function that is inside a script that is inside the same object as the Empty GameObject.  This is one of the most important things to remember - whatever function you are activating, its script must be in the same object as the SendMessageAction.

               

              STEP SEVEN

               

              Run your project, move your selector over the button that has your trigger, and close the hand to send an activation signal to the function that you have specified.

               

              STEP EIGHT

               

              Finally, repeat the above process for each of your buttons, giving each Empty GameObject its own box trigger field, SendMessageAction and a trigger script that will make something happen when the Hand Closed gesture is made whilst the selector is hovering over that button.

               

              2.jpg

              • 4. Re: Touchless main menu buttons (SR300)
                supermomo90

                Hi MartyG,

                 

                Thank you for this detailed explanation.

                • 5. Re: Touchless main menu buttons (SR300)
                  Chris_CH

                  I'm already follow until step 4.

                   

                  If i want make my "Play" button can load my another scene, what should i write in my trigger script ?

                  • 6. Re: Touchless main menu buttons (SR300)
                    MartyG

                    Put this in your trigger script.  Put the name of the scene file in the quotation marks.

                     

                    Application.LoadLevel ("name of your scene file");

                    • 7. Re: Touchless main menu buttons (SR300)
                      Chris_CH

                      so this will ok ?

                       

                      public void PlayButton()

                      {

                           Application.LoadLevel ("game");

                      }

                      • 8. Re: Touchless main menu buttons (SR300)
                        MartyG

                        Yes, if your scene file is called "game" then that will be fine.

                        • 9. Re: Touchless main menu buttons (SR300)
                          Chris_CH

                          i got this error when run project, suddenly my project paused after run

                           

                          "Null Reference Exception : Object Reference not set to an instance of an object"


                          can help to fix my problem ?

                          • 10. Re: Touchless main menu buttons (SR300)
                            MartyG

                            Can you post the complete error message please?

                            • 12. Re: Touchless main menu buttons (SR300)
                              MartyG

                              I suspect that the problem is that when you added an instruction to load another scene, you have not added that scene to the Build window of Unity and so Unity does not know how to find it.

                               

                              Open your 'game' scene file (the one that the LoadLevel instruction refers to).  Then go to File > Build Settings and click the 'Add Open Scene' button.  Your 'game' scene will be added to Unity's build list so it knows how to find the file when your click your scene loading button.

                               

                              1.jpg