User Manual 4.1 - FROG Recognizer of Gestures
Contents
1. ADISPLAY2 me Figure 13 Allows the user to set the active monitor for touch input Note Does not always reflect the display numbering found on Control Panel All Control Panel Items Display Screen Resolution Optimal Distance Displays a progress bar that shows the optimal Kinect distance from Progress Analyzer the projection surface Note Only updates when the four points are selected Current Elevation Angle Displays the current elevation angle of the Kinect You may change the current elevation angle by dragging the slider up and down Note Clicking the slider up and down won t move the slider 4 Reset Clears the current calibration screen and allows the user to re calibrate 5 Saves the current screen calibration Texas Christian University 4 System Walkthrough EUR USER MANUAL Touchy 4 Upon successful calibration the application will minimize to the taskbar 2 01 PM A Ep Pre 123204 Figure 14 5 The user will now be able to interact with their system via the touch screen a The user now has the option to right click on the TouchCU icon in their taskbar to access some additional options 4 Calibration 2 Settings 3 Debug 4 Exit Figure 15 Opens the calibration wizard for re calibration Opens the TouchCU settings menu Enables a debugging overlay Exits the application b Ifthe user clicks on Settings they will see the following options Texas Christian U2. Texas Christian University April 23 2014 USER MANUAL 2013 2014 Computer Science Department Texas Christian University Version 4 1 USER MANUAL Touchy Table of Contents k RV SOMEN ES et E E II H REN VSG MIS COU aeran E E E AN I OO OR et acces 1 LPO a a an ec io oi io 1 TPO CT CV ION acouivcctcisuticteiniacredinieatnirtersenin ats erotica te to ie ie Au an pein 1 E NO IS 1 2D SUE CONDONE ca ee de ee 2 2 Bef SU IN OOS MS ance ccs a ve E di ee ie en te oo de ne 2 2 1 1 TouchCU Standalone Application cccccccsseccccssseccceesececeesececeenscceceusececeuseceeeeueecesauecesseneceeseges 2 2 1 2 Microsoft Kinect for WINGOWS sixcsstsissorecccicrssesiaversaccdsneebidvcrsennesserssudivsatevsasevalueecenneusscnnsiubtoesenmiacts 2 LC PO E A A 2 LL ENVI ONEI oea E E T nee geek eee se eeocaab eeeee eects 2 3 Set up Installation and Uninstallation esseeeseessesseserenerrrserrssrrnesrrnsrrnssrrrsrtrssrrnsrrrsstessrrnsrtesstrnseresreeseereseees 3 3 1 Installing TouchCU Application sssessesesseeserrenesrersssreressrrsssreressresssreressrersseeressrereseeresereresrereserereseeresse 3 3 2 Microsoft Kinect for Windows Placement ss 7 3 3 Uninstallation Procedure cccccssssssseccccccseeesececcccccsaeessseeeeceessaueuseeececesssauuesseeceeeesssugaaeeeseeessaagugsses 8 AY VE Vi ACU VE OS EAP ER E EA E PER NE EAE EE 9 Dy SRE OV SF VS WN eeen cr iate bas said acircarts gain ai co tn lo don 9 HP Cfo afl
3. This manual will cover an overview of the required system components instructions for set up installation and uninstallation and detailed walkthroughs of the system 1 2 Project Overview The massive growth of touch technology has created an increased demand for new and innovative ways for users to interact with their devices TouchCU a 2013 2014 capstone project consists of a desktop application that turns any flat surface into a multi touchscreen utilizing the Microsoft Kinect for Windows a standard projector and a Windows 8 PC Research teams at Intel Labs and Ubi Interactive have created similar products with one supported by Microsoft and available commercially TouchCU while similar in concept allows for a greater operating range and implements voice interaction 1 3 Overview This document includes the following sections Section 2 System Components Lists and describes the required components for running TouchCu Section 3 Set up and Installation Lists detailed instructions on setting up the Microsoft Kinect with Windows and installing the software on a computer Section 4 System Walkthrough Provides walkthroughs of the functions that are provided by the system Section 5 Glossary of Terms Includes a list of abbreviations and their technical terms and their associated definitions Texas Christian University 1 Introduction i USER MANUAL Touchy 2 System Components 2 1 System Components To
4. USER MANUAL Touchy 4 2 2 Supported Voice Commands TouchCU supports various voice commands for an additional way to interact with the computer These can be found in 6 2 Appendix B Voice Command Tables Voice 4 4 Start Menu Window My Documents Settings Debug Figure 19 Texas Christian University 4 System Walkthrough ES USER MANUAL Touchy 4 3 Debug Mode The Debug part of TouchCU can be started by right clicking the TouchCU system tray icon and selecting Debug It can also be started by using the appropriate voice command The Debug application allows you to test and analyze your touch interaction while using TouchCU Using this interaction feedback you can adjust TouchCU s settings to enhance the accuracy and performance of TouchCU The position of each hand is tracked in a 3D space using X Y and Z coordinates The positive x axis extends to the left the positive y axis extends upward and the positive z access extends outward in the direction that the Kinect is facing X and Y are pixel coordinates and Z is meters from the projection surface Debug mode is composed of two windows The debug output window located at the top of the screen and the interaction window The debug output window displays the details of TouchCU s injected pointer information right alongside the Windows Pointer message The list menu populates touch interaction events that the application has received through the Windows messa
5. ee Mahe 6 0 0 eee a en a eer en on 14 42 NM SUDO CG GOS EUR E D ne ae de a ee de er cn 14 24 22 Support d VOICE COMMANOS 22 ns tes ns das sent nee sean nent en 15 D DD MIO a a ee ee ne de ce 16 SHGIOSS ANY OF TENS senere ae den este ee den codr emo end tien do ces an deco nee ones 18 AO recent ee ne ener eee ere 19 6 1LAppendix A OBS CES CAD a cere ee cee 19 6 2 Appendix B Voice Command Tables ccccccsssecccesseccccenscccceesececeeececseesecesseecessegeceeseesecetsugecetsegas 20 Texas Christian University Revision Signatures ae USER MANUAL Touchy I Revision Signatures By signing this document the team member is acknowledging that he she has read through this document thoroughly and has certified that the information within this document is accurate and satisfies all requirements Name Signature Date Signed Blake LaFleur Thales Lessa Matthew Spector Texas Christian University Revision Signatures CS USER MANUAL Touchy ll Revision History v1 0 Initial Documentation Draft 04 February 2014 v2 0 Additional development information added 18 February 2014 v3 0 Updated based on current project state 28 February 2014 v4 1 Final Draft 23 April 2014 Texas Christian University Revision History US USER MANUAL Touchy 1 Introduction 1 1 Purpose The purpose of this document is to provide the user with a complete overview and explanation of all aspects of the TouchCU system
6. the following Navigate to C Users USER NAME GOES HERE Documents TouchCU Double click unis000 to begin the uninstallation You will be prompted with a box asking you if you would like to completely remove all TouchCU components Press Yes to continue components E Are you sure you want to completely remove TouchCU and all of its Figure 8 4 Aprogress bar will appear as the program begins to uninstall Uninstall Status Please wait while TouchCU is removed from your computer Uninstalling TouchCu Figure 9 5 Once the uninstallation is complete you may click Ok to close the uninstaller TouchCU was successfully removed from your computer Figure 10 Texas Christian University 3 Set up Installation and Uninstallation B USER MANUAL Touchy 4 System Walkthrough 4 1 System Overview The user will interface with the TouchCU desktop application by doing the following 1 Open the TouchCU application COLIC Loading Figure 11 2 Once the application has opened successfully the user will receive a welcome message instructing them to begin calibration Welcome to TouchCU ax After calibration you may use the Kinect as a touch input device for your PC Figure 12 Texas Christian University 4 System Walkthrough USER MANUAL 3 Follow on screen instructions to begin calibration TouchCU Calibration OE Kinect Elevatior Angle
7. Next If vou would like to select a different folder cick Browse Browse _ Don t create a Start Menu folder Figure 3 10 Now you are given the option to add a desktop shortcut If you would like this feature click the checkbox and click Next Texas Christian University 3 Set up Installation and Uninstallation Ua USER MANUAL Touchy Select Additional Tasks Which additonal tasks should be performed Select the additional tasks you would like Setup to perform while installing TouchCU then dick Next Figure 4 11 An overview of your options will now be displayed Click Install to continue Ready to Install Setup is now ready to begin installing TouchCU on your computer Click Install to continue with the installation or dick Back if you want to review or change any settings Destination location C WUsers TouchCU Documents TouchCU Start Menu folder TouchCU Figure 5 12 The installer will now extract all the necessary files to the directory you specified in step 8 and prompt you to Launch TouchCU If you do not wish to run TouchCU at this time please uncheck the checkbox 13 Click Finish to complete the installation of TouchCU Texas Christian University 3 Set up Installation and Uninstallation a USER MANUAL Touchy Completing the Tou
8. an body at a capture rate of 30 Hz e Kinect Studio Debugging tool for the Kinect sensor that allows the depth and color streams to be viewed in real time e NTASC North Texas Area Student Conference e Rich Interaction Supports multi touch e SDK Software Development Kit e Skeletal Stream Data stream from the Kinect that tracks up to 20 joints on the human body e SRS Student Research Symposium e TBD To Be Determined e TCU Texas Christian University e TouchCU The team and project name for the 2013 2014 senior design team e USB Universal Serial Bus Texas Christian University 5 Glossary of Terms ca USER MANUAL Touchy 6 Appendix 6 1 Appendix A Gesture Tables Single Hand Gestures Name of Gesture How it s Performed What it s Used For Tap GR 1 Tap an item on the screen once Simulates a left click from a mouse mouse ee hold Drag GR 4 Tap and hold the screen while moving in any direction screen Two Hand Gestures Name of Gesture How it s Performed What it s Used For Zoom GR 5 Both hands will be placed on the Simulates making an object larger or screen and move either farther smaller on the screen or closer apart Rotate GR 6 Both hands will be placed on the Simulate moving the object around a screen to emulate a clockwise or center point counter clockwise motion Air Gestures Swipe Left GR 7 One hand in mid air will move a_ Simulates using the l
9. an external power source 6 Now double click TouchCUInstaller and follow the on screen instructions A dialog will appear asking if you have installed NET Framework v4 5 and Kinect Runtime v1 8 Click Yes to continue WET Framework v4 5 and Kinect Runtime Environment v1 8 MUST be installed te continue Have you installed these from the INSTALL ME FIRST directory included in the TouchCU Installer zip file Figure 1 7 You will now be prompted with the welcome screen of the installer Please click Next to continue 8 Select an installation directory of your choice You may leave this defaulted to your Documents folder and then click Next Texas Christian University 3 Set up Installation and Uninstallation ay USER MANUAL Touchy Select Destination Location Where should TouchCU be installed Setup will install TouchCU into the following folder To continue dick Next If you would like to select a different folder dick Browse Wesers TouchCuU Documents TouchcCu At least 6 1 MB of free disk space is required Figure 2 9 This step allows you to change the name of the application folder in the Start Menu and can be left as default Please click Next to continue Select Start Menu Folder Where should Setup place the program s shortcuts CA Setup will create the program s shortcuts in the following Start Menu folder MEL To continue dick
10. chCU Setup Wizard Setup has finished installing TouchCU on your computer The application may be launched by selecting the installed icons Click Finish to exit Setup Figure 6 14 Make sure your projector is connected turned on and displaying your Windows OS successfully before calibrating the system Note While there may be newer drivers available for the Kinect for Windows TouchCU has only been tested with v1 8 0 Texas Christian University 3 Set up Installation and Uninstallation 6 USER MANUAL Touchy 3 2 Microsoft Kinect for Windows Placement While the Kinect can be placed anywhere in front of the screen Figure 7 shows the optimal physical placement of the Microsoft Kinect for Windows This placement utilizes a horizontal table top with the Kinect below the bottom of the projection surface We recommend placing the Kinect between 7 11 feet from the projection surface It is important to note that both the entire projected image and the users hands must be visible to the Kinect at all times 0 3 5ft 0in 42in Projected Image from center of projection from Projector 7 11 ft 84in 132in Figure 7 Texas Christian University 3 Set up Installation and Uninstallation USER MANUAL Touchy 3 3 Uninstallation Procedure TouchCU includes a built in uninstaller that will completely remove all traces of the standalone application from your system To completely uninstall please do
11. eft arrow on the PES Swipe Right GR 8 One hand in mid air will move a Simulates using the right arrow on the FPS SES Texas Christian University 6 Appendix ES USER MANUAL Touchy 6 2 Appendix B Voice Command Tables Trigger Word Name of Command How it s Performed What it s Used For Addie VCR 1 User will say Addie aloud ad ee followed by a command word action word Command Words Name of Command How it s Performed What it s Used For Open VCR 2 User will say Open aloud oh puh n followed by an action word Close VCR 3 User will say Close aloud klohz followed by an action word Action Words Name of Command Howit s Performed Start Menu VCR 4 User will say Open Close Start stahrt men yoo Menu aloud Window VCR 5 User will say Close Window win doh aloud My Documents VCR 6 User will say Open My mahy dok yuh muh nts Documents aloud Settings VCR 7 User will say Open Settings set ings aloud Debug VCR 8 User will say Open Close dee buhg Debug aloud Texas Christian University 6 Appendix 2 Initiates the voice recognition process action word action word What it s Used For Opens or closes the Windows Start Menu Closes the active window Opens the user s Documents Opens the TouchCU settings menu Opens or closes the TouchCU debugging overlay
12. ge loop These events include TouchDown TouchUp Hold amp Drag The debug output window displays manipulation data upon completion of the appropriate manipulation Point tracker Used to show current touch data and location and provides feedback on user touch interactions TouchCU currently supports up to 2 touch points PointerID x 8 382 Figure 20 Texas Christian University 4 System Walkthrough 160 USER MANUAL Touchy Manipulation Used to test single and two hand gestures while providing feedback on touch manipulations such as slide rotate and transform Figure 21 Texas Christian University 4 System Walkthrough a7 USER MANUAL Touchy 5 Glossary of Terms e Audio Stream Data stream from the Kinect that allows for high quality audio capture that can be used for voice commands e Color Stream Data stream from the Kinect that acts as an RGB web camera e Depth Stream Data stream from the Kinect that creates a depth map from the distance of the sensor to the object e EOD End of Day e HDMI High Definition Multimedia Interface e IDE Integrated Development Environment e Kinect A motion sensing input device by Microsoft for the Xbox 360 video game console and Windows PCs Based around a webcam style add on peripheral it enables users to control and interact with PC through a natural user interface using gestures and spoken commands The Kinect recognizes 20 joints on the hum
13. nitor as its current main display by going to Control Panel All Control Panel Items Display Screen Resolution then selecting the Make this my main display checkbox and then clicking OK Texas Christian University 2 System Components D USER MANUAL Touchy 3 Set up installation and Uninstallation 3 1 Installing TouchCU Application The TouchCU Install Files zip file on the release DVD contains an INSTALL ME FIRST folder and the installer executable named TouchCuUInstaller This is the installer for the TouchCU application and all of its components Inside of the INSTALL ME FIRST folder there are two executables one for NET 4 5 along with the Kinect Runtime v1 8 These MUST be installed before running TouchCU for the first time The following steps will provide successful installation of TouchCU Right click the TouchCU Install Files zip file and in the context menu click Extract All Select the location you wish to extract the files to and click Extract Once completed a folder will appear that contains the location of the extracted files 3 Verify that the Kinect is not plugged into the computer Within the INSTALL ME FIRST folder double click and install KinectRuntime v1 8 Setup and NETv45 Net v4 5 should already be included with Windows 8 5 After NET 4 5 and Kinect for Windows Runtime v1 8 have installed successfully plug the Kinect for Windows into a USB port on the computer and into
14. niversity 4 System Walkthrough FE USER MANUAL Touchy M Disable Voice Commands 3 M Enable Advanced Settings somoothing Correction e Prediction a Jitter Radius Max radius in meters that filtered positions are allows gt 0 in meters JHand Extension Figure 16 Texas Christian University 4 System Walkthrough ETES USER MANUAL Touchy Disable Air Gestures Disables all air gestures Disable Voice Commands Disables all voice commands Enable Advanced Settings Allows the user to change the advanced settings below 4 Smoothing Sets the amount of smoothing applied to the raw data A value of zero does not alter the raw skeleton data e Ahigher value gt more smoothing Increased latency e A lower value less smoothing Lower latency 5 Correction Sets the amount of correction applied to the raw data Ahigher value gt Corrects towards raw data quicker e A lower value Corrects slower and appears smoother 6 Prediction Sets the number of frames to be predicted 7 Jitter Radius Determines how aggressively to remove jitter from raw data If a joint is tracked outside the radius it is corrected and clamped to the radius Measured in meters Max Deviation Radius Sets maximum radius that filtered data can deviate from raw data Used in conjunction with Jitter Radius to set the outer bounds of the jitter radius Measured in meters Distance to consider a
15. touch Changes the depth of the touch interaction zone in reference to the touch screen surface Starting from O where 0 is at the projection surface this value increases the size or thickness of the users interaction zone Measured in meters Hand Extension This multiplier value is used to extend the touch point from the center of the palm outwards towards the fingertips Specifically the hand extension value scalar is multiplied against the wrist to palm distance to determine how far out to extend the touch point For example wrist to palm distance X hand extension distance to extend touch point from palm center eset Default Settings Resets all settings back to their default options Saves your current settings and closes the settings menu Texas Christian University 4 System Walkthrough 130 USER MANUAL Touchy 4 2 General Interaction Once the application has been properly configured the user will be able to interact with the projection display as if it were a touchscreen The touch point will be at the center of your hand due to the way the Kinect tracks your hands Figure 17 4 2 1 Supported Gestures TouchCU supports all built in Windows 8 gestures as well as a few additional gestures These can be found in 6 1 Appendix A Gesture Tables Gesture Icons can be seen below Touch VE wala gt Lae nn Figure 18 Texas Christian University 4 System Walkthrough
16. uchCU is comprised of four components the standalone application the Microsoft Kinect for Windows a standard projector and a Windows 8 PC 2 1 1 TouchCU Standalone Application Initially the TouchCU standalone application calibrates the system for use Then once the user has calibrated the system the application will be minimized to the System Tray Once the application is running the user will be able to open the settings menu enable the debug overlay or exit the application from the System Tray 2 1 2 Microsoft Kinect for Windows TouchCU uses the Kinect as its primary data input source Through the Kinect s proprietary USB cable TouchCU accesses the AudioStream ColorStream DepthStream and the SkeletonStream which are used to track user interaction 2 1 3 Standard Project TouchCU is compatible with any projector A resolution of 1920x1080 is recommended for best performance if 1920x1080 is not possible choose a resolution as close to this as possible The projector can use either analog or digital input as long as it supports the required resolution 2 2 Environment TouchCU will be a software application that can be installed on any computer running Windows 8 or later with a Microsoft Kinect for Windows installed and connected The minimum system requirements to use the Kinect and therefore TouchCU are e dual core processor 2 66 GHz e 2GBRAM e a dedicated USB 2 0 port Note It is recommended that the user set the mo
Download Pdf Manuals
Related Search