Hi everyone,


In this article, I will showcase the design philosophy of the User Interface (UI) for the character selection screen of my company's RealSense-enabled game, 'My Father's Face'.  The game is targeted for release on PC and Intel Project Alloy merged-reality headset in 2017.


In the offline mode of 'My Father's Face', one or two players can play, with the second player using a joypad and able to seamlessly drop in and out of the game without interrupting Player 1's game. 




Because the game centers around a pair of characters, Player 1 is asked to select a second character to define a family, even if they will only be controlling their own character.  Any combination of male and female characters can be selected (male / female, female / male, male / male and female / female), enabling the player or players to form their ideal family unit.


Where possible, icons are used instead of text to broaden the appeal of the game to international audiences and make it economical to create a range of language localizations for the game.  The male and female gender of the characters is therefore conveyed through the universal symbols for gender, rather than stating 'male' and 'female' in text.


In the top corner of the screen, icons representing the controls for returning to the title screen or starting the game are prominently displayed, informing the player how to enter the game quickly and easily if they do not wish to take time making character selections on their first time with the game.  If Start is activated immediately then the default character pairing selected is Male for Player 1 and Female for Player 2.


The player selector is controlled with the arrow keys, the joypad left stick or the left hand of Player 1 with the RealSense camera controls.


When 'Player 2' is highlighted, the text below the selector changes to inform Player 2 that they are able to use their joypad to make their own character selection to override the choice of Player 2 character made by player 1, if they wish to do so.




Both characters are controllable within the character selection room before the game even begins, in order to provide a gentle practice environment where nothing can go wrong and they can try out different buttons, keys and – in the case of the camera controls – body movements.


Because the character section screen is the first time that the players are exposed to the game's controls, we do not wish to overwhelm them with control information and create confusion though.  So whilst most of the character controls are available at this time, the on-screen UI only informs the player of the existence of the most vital core controls for navigating the selection room – turn left, turn right and walk forwards.


These controls are overlaid on the camera as a Heads Up Display (HUD) so that even if the player characters walk across them, they are always visible at the front of the screen and the characters walk behind the UI elements.


In the above image, the keyboard and joypad controls for turning and walking are displayed, as the camera control icons have not been created at this stage in the project.  The player can use the RealSense camera to turn 360 degrees left or right with a tilt or a left-right turn of the head, whilst walking forwards is currently achieved by leaning forwards in the chair (this action will likely be changed in the final Project Alloy headset release).




As demonstrated in the image below, other character controls such as crouching and side-stepping are also accessible in the character control room if the player discovers them through exploration.  They are not a requirement to leave the selection room though, ensuring that the first-time player will never become frustratingly trapped in the starting room and unable to progress to the main game.




When the Start control is activated, the game begins, either in full-screen mode if only Player 1 is playing, or in multiplayer split-screen if the '2 Player' option is highlighted at the top of the screen when Start is activated.


Upon starting, a set of doors to the outside world opens up, revealing a much larger environment outside the cozy, closed-off confines of the selection room and beckoning the player or players to move forwards into this new world using the basic navigation controls.




At this point, the full range of controls becomes accessible.  'My Father's Face' uses an innovative control system in which all the control advantages of a camera are available through a joypad, with the button layout designed to maximize comfort whilst carrying out complex body and limb movements with the greatest of ease during walking / running.


On the pad, control of the arms is mapped to the left and right sticks, whilst turning is accomplished with the left and right analog triggers and walking with the left digital bumper button above the left analog stick.  This control layout enables the fingers to flow subconsciously over the pad, enabling the players to navigate, touch and explore the world with all the motion and tactile capabilities of their real-world body.




The characters can also be controlled with a keyboard and mouse combo, with the arrow keys the default for turning left-right and walking forwards back, and the arms and crouch action controlled one at a time with the mouse, its buttons and the scroll-wheel.  The left mouse button opens and closes the currently selected arm's hand, and the right mouse button toggles control of the left and right arm.


An auto-walk control toggle enables the player character to walk and run automatically whilst it is active, freeing up the player's hand to focus comfortably on other controls such as turning and arm movement.  This allows spectacular, complex Power Rangers style moves to be enacted intuitively with the player character.


When the RealSense camera controls are being used, the two arms mirror Player 1's real life arm movements almost 1:1, able to make every movement range of the upper and lower arms that the real arms can.  This opens up almost limitless possibilities for realistic interaction, from hugging Player 2 (or more than hugging!) to operating an in-game computer keyboard or using a handle.


This control capability is also available with the joypad or mouse, but using both of your real arms simultaneously with the RealSense camera  creates an incredible level of immersion in the virtual environment that makes you believe that you are really there.  This sensation is amplified when switching the camera mode into first-person 'through the eyes' view that lets you see the world like your real eyes do, see your arms moving in front of your line of vision and look down at your own virtual torso and feet.




You are not merely limited to interacting with a friend sitting next to you either.  Players can set up a private online session through match-making and meet in a shared instance of the game world, each player using the controls at their real-world location to independently control their character in the online environment.  This is made possible by the Unity engine's UNet networking system.


When this level of multiplayer virtual character control is made available online to people in any location with a good internet connection, the potential number of applications of the technology – from physically tactile personal long-distance relationships to professional team meetings and collaborative creativity – increases exponentially.


You can view the most recent “tech trailer” demonstration video for 'My Father's Face' (minus the latest character selection UI elements) here:



Link: 'My Father's Face' Tech Trailer 7 - YouTube


'My Father's Face' will be released by Sambiglyon (www.sambiglyon.org) for PC and Intel Project Alloy headset in 2017.