Home

PDF 3.4MB - SmartSketches

1. 58 7 5 4 Multiple UNDOPREDO ee eN dios 59 LS TATE TETTE Blin 59 7 6 ETP GE iis ODEEXOOUS ets 60 7 6 1 PICO EE EE E N N N N 61 7 6 2 UI eee RD E E PRN RD 61 7 6 3 ed de ee EO E EO a sa a 61 7 6 4 Eege 63 7 6 5 Mil oie RE aes Re ve torsos omen UU was Re ER ssn eaten mee ee 63 7 6 6 Ai PP o E EE 64 7 6 7 MI And dl AE ENE EE NE ds d Eni eI OE 65 7 6 8 Camera Zoom MOVE T OR N N EE N 65 7 6 9 Ortho VIEW EE EE E 65 ONE DL EER EE Ee Ge A 66 OM SD IAC or A EA 67 Page 5 of 90 SmartSkada s User Manual D18a 2004 10 07 P Santos A Stork 7 1 Network Collaboration SketchNET ccccsscccscccscccccccsccccecccsccccscccscccescceeeee 67 7 7 1 SESION E over GIE pe 67 Ga Connecting and Disconnecting to from SketchNET ss ees ee ee ee ee 70 8 ANNEX A REFERENECES issie id ee ee ee deus eo oe ed ee ee cais De asda 71 9 ANNEX B OPERATION TABLE ee ee ee eek eke nnnm nnn nnns 73 10 ANNEX C MENU REFERENCE ee ee eek ek ek eke snes 18 Page 6 of 90 SmartSkada s User Manual D18a 2004 10 07 P Santos A Stork Figures Fieure 1 Hardware Setup OVER ICN cata Dia se oe se eg ee se ee 11 Piha 2 Vies Setup deti veste Fe OS CE UT DO retiene ais 12 Figure 3 VR Setup Passive sterco SCETIIETO scere iiS rex ser Orr nO EE Ee EF cw ds Sae ON net NUES 13 Fisute EA RN A no II T M 14 Figure 5 Optical Tracking ART Tracking System with ARTtrackl cameras
2. 15 Fies O Interde Hof DEVICES icici AE tro EN EO Ee Se OO ER Oe ER Sta 17 Figure 7 SketchAR Architecture esses eene nnne nennen nennen nennt nns 18 Figure 8 Abstract execution flow of operations ooocccccccccnnnnnnnnnnnnnonnnnnnnnnnnnnnnononnncnnnananannnnns 19 NU Pert BAO se ong oe A esI Ba OE UMEN RUM UM ONE UON AD NUMMUS 2 Fisies O PPC alibi OO ae e E DIR reve nr rt CUT Ee Ee ee een freer 22 FS ECC SU Ee N lia 26 Fiore 12 Tracked Model T0 3 booa os Ee EE ER N ND EO EE Ge da ee GR 21 Field DIP PEL are E O On ea ii E E E ee Ge ee 27 Pap ursi O NR OO A 28 Figure 15 Pie sub menu SURF animated pie menu 29 Figure 16 C Ole Seis EEN 29 ere To Gesturedbase d Input a cesset unie EE ee end tas 3l Figure 18 Dragger Eer HEER oos ce sete anasto RD RIR ee UNE 22 Figure 20 SpaceMouse supporting the virtual paper metaphor sesse sesse 33 IS ob ide valdc duum 35 Figure 22 Package Model Constraints Wheels essen 35 Frente 25 Package Constante Hook see eN EE seii nto RENE EPIS ENNIUS 36 Figure ZAC Curve CL O cb Nm 27 Figure 20 Automatic C rve DEEN 39 s aor ue Wal TT DAN O NR MET 40 Figure 28 Virtual Tape Drawing Finger Tracking ooooooonnnnncccccnonnnnnonononnnnnnonanonncnnnnnnnnnos 40 TASA A e 41 AA A O E EE 42 Fire ol NICE prc P O eb 43 TUU o2 Primitive Eie Ed Ee EE ee S DE Oe Pe ed EE 46 Figure 33 Assembly Parts 49 rara O ESC DIS les A E E S E 50 gy
3. a 28 6 5 Speech Recognition Synthesis ssssssseeeececcccccccosssssssescecceccccoccosssssssssssssseceeeeeossoss 30 6 6 Implicit Gesture Recognition os 25 00 205 006 Gen oe Dee ea op Poss tee Gee oe ce EERS 30 Page 4 of 90 SmartSkada s User Manual D18a 2004 10 07 P Santos A Stork 6 7 bies sees 6 5 S EE ER AE eiie aiiud 31 6 8 3D Picking NIIT GEE Ee Ee Ee Ge ene ee ee N Ee ee 32 6 9 Virtual Paper NEE UL 33 f SYSTEMEUNGTIONALIFM os Essen ende ese RS Sie Oe eed 34 7 1 ole di ER nen 34 T Package Model ENE sessie eis Role ee Oe ee 34 Fr Package Model Parameters esses eene eene eene enne 35 72 2 Package Model Constraint Check sie eise de bua pte usd ebbe is 36 7 3 Geometry 1 m 36 7 3 1 Cue OPORTET 37 LEE Ge Lai sd A RR RR 38 7 3 3 rooie nb EE 38 7 3 4 bino ME JD rada 39 7 3 5 n gt Ne Pe C ICH Oia 4 7 3 6 PLE E DEAL Ree ne PREDETERMINED ME 44 dad Scene Modelling and Assembly ee 46 7 4 Geometry MOGHICALUOT ooi eiie oo ipo eno 29 ede ween en Ceo E open a oca Cep isa dor esa ENE 50 7 4 1 A ER ER EE OE ee OR ER RE ne eee ee 50 7 4 2 Constrained ON CRS CCI ta dde tan eDim dU MEI MM DE 51 7 4 3 E tino Control POS es UND III EMEND IEEE NM MEE 53 7 4 4 INGE Suria e OVerskete orita 54 7 4 5 History based EEN 55 7 4 6 Boolean Operations see ee ee ee ee Re Re ee ee ee RR ee ee ee 56 7 5 bruet 57 7 5 1 AE LUC CL PNE ER nn ee Ge Ee ee 57 1 5 2 MOV COP ano RR 58 7 5 3 p
4. Page 19 of 90 SmartSkada s User Manual D18a 2004 10 07 P Santos A Stork Merge lt StbSink gt lt StbSink station 7 gt A node like this transfers and eventually manipulates tracking information on a tracked artefact from a specific tracking system into a SketchAR data structure for this artefact It has the following attributes e Station integer starting from O giving the station number to use in Stb e Event on off whether this station should emit events or not default is off lt Merge gt A Merge node is an EventGenerator node that listens to several other EventGenerator nodes and merges data from these It has several inputs that are marked with different wrapper tags It stores an internal state and updates it with parts of the data depending on the type of input Then it generates an event of its own Timestamps are treated in a special way If no child node is connected to the MergeTime input then the timestamp of the new event equals the timestamp of the last received event Otherwise it behaves like other inputs MergeDefault any data that is not set by a child of another wrapper element is used lt ARTDataTrackerSource number 8 gt The ARTDataTrackerSource node is a simple EventGenerator that inputs the data from the ART Data Tracker The attribute number the body number of the specific source starting with o lt EventVirtualTransform rotation 0 1 0 1 57 translation 0 053 0 261 0 058 gt lt
5. line to start taping Going forward with the left forefinger will glue the tape to the wall going back with the left forefinger will detach the tape from the imaginary wall such as with a real tape To end the taped line the user says stop For each command the user receives synthesized speech feedback to know it has been recognized Page 40 of 90 SmartSkada s User Manual D18a 2004 10 07 P Santos A Stork 7 3 5 Surface Creation In SketchAR surfaces can be generated using two different paradigms The first is to explicitly select already drawn curves and perform surface creation operations on them The second is to implicitly create surfaces by sketching necessary curves during the surface creation operation Spline Extrusion Spline Surface Surface Pen Extrude Spline Spline Extrude Extrude NURBS Surface Surface Pen Surface gt NURBS NURBS Surface Surface SketchAR supports explicit and implicit spline extrusion In the first case a previously drawn curve is selected and extruded to build a surface In the latter case a curve is drawn and extruded in one single interactive step Figure 29 Spline Extrusion To use the Spline Extrude operation and to create a surface of an existing NURBS curve you have to press the Spline Extrude button in the Surface menu on the PIP sheet or to activate the pie menu using the third button go to the submenu Surface select the Spline Extrude
6. Plane Workplan Workplane e To allow the stylist to create a car silhouette line as a curve on a plane we introduced the work plane concept which constraints the input to a plane by projecting the pen tip to the nearest point of the work plane The plane can be positioned arbitrarily in space using a tracked artifact or a dedicated dragger Page 61 of 90 SmartSketebey E User Manual D18a 2004 10 07 P Santos A Stork Figure 45 Workplane positioned by dragger In addition the work plane can be linked to a tangible physical element and positioned arbitrarily in discrete angles or moved along a pre defined grid Figure 46 Context pie menu allowing the work plane to be positioned in different ways Use the Workplane button in the Tools menu on the PIP sheet to activate the Work Plane operation e First press free positioning mode e Second press Workplane in car mode e Third press Workplane off Alternatively use the pie context menu clicking the third button of the pen once you are close to the Workplane Page 62 of 90 SmartSkedehu E User Manual D18a 2004 10 07 P Santos A Stork 7 6 4 Workplane to table Workplane to Tools gt Table Table Projection Stylists are used to have physical feedback while drawing In free space this kind of feedback is not available unless you use special haptic input devices Today haptic devices that cover an area as big as a virtual table working space are hardl
7. Wheels Hd aa NEE EE EE A Adjust ment Overall Dies ore bill EE EE EE Ii Adjust ment Human Ergomomic Encum Package gt Package gt brance Tech Free Enc Adjust ment On Off On Off Collision Package gt mE gt Detection Collision Collision Detect Detect Supporting Operations Snap Tools gt Pen Snap Work PR Tools gt Tools gt LE I keg send Er plane Work plane LE I Work plane Tools gt Plane to Table Table Pen Projection id Id AM ll en omie Mirror plane E Clipping Tools gt Constraint Plane Plane Clipping gt Clip Pen Plane Y dt Model Toggle Head Toggle Camera Tools gt Pen Zoom Move Camera Sliders E Layer FourView FourView mouse Pao OrthoView OrthoView Page 76 of 90 SmartSkecha User Manual D18a 2004 10 07 P Santos A Stork Primitive Constr coincident Primitive Constr parallel Perpen Primitive dicular Constr perpen dicular Tangent Primitive Constr tangent Concentric Primitive Constr concentric Dimen Primitive sioning Constr dimension Primitive Constr un fix Primitive Constr show free ham DR le 1 Connect username Seen LL Amer O m Disconnect disconnect Page 77 of 90 SmartSketefe Z User Manual D18a 2004 10 07 P Santos A Stork 10 Annex C Menu Reference Functionality Main pie menu File Fil
8. EventTransform scale 0 001 0 001 0 001 gt With this tags you can calibrate the virtual objects to fit with there physical representation tracked artifacts In the following a calibration of the pen and the PIP is shown exemplarily All the other artifacts have to be calibrated in the same way Page 20 of 90 SmartSkeacha User Manual D18a 2004 10 07 P Santos A Stork 5 2 1 Calibrating the pen the zero point of the y object is in this marker Figure 9 Pen Calibration e The point of rotation of the pen artifact in SKetchAR is the tip the tip of the blue cone in the picture above e Ifthe orientation at calibration was different from the one suggested above in the initial Studierstube artifact arrangement picture let the rotational offset be p qr s Let the position offset between the zero marker and the tip of the pen be x y z We need to shift the coordinate system first from the zero marker to the tip using a translation We translate the coordinate system within the pen s LOCAL coordinate system EventVirtualTransformation translation then we rotate the pen lt EventVirtualTransformation rotation into the desired SketchAR initial orientation We need to perform the following sequence for the transformation e lt EventVirtualTransform translation x y z rotation p q r s gt e lt NetworkSource number 1 multicast address 224 100 200 101 port 12346 DEF peno gt e lt EventVirtua
9. Pair of glasses According to the setup the glasses are shutter glasses pole filter glasses or a head mounted display These devices are tracked so that the virtual picture can be calculated according to the head position user point of view e Pen A pen like device with 3 buttons is used as main input device Cyberstilo e PIP sheet Personal Interaction Panel A transparent Plexiglas panel on which the application menu is projected The menu on the sheet is operated with the pen e Navigator axis L shape cube shape The navigator axis is used to navigate the model in 3DOF You can choose between a device in the shape of an L and a cube Page 16 of 90 SmartSkdefe E User Manual D18a 2004 10 07 P Santos A Stork Cube Axis Tangible Plane ELI CyberStilo Q Figure 6 Interaction Devices e Tangible planes mirror plane work plane These tracked artifacts are used by the mirror and work plane function Both functions operate on a virtual plane which are moved according to the artifacts e Tape fingers These devices are only used for the virtual taping function That function operates with two input devices e SpaceMouse The SpaceMouse is used in four view mode to control the projection plane in analogy to a sheet of paper which can be panned rotated and zoomed Page 17 of 90 SmartSkadaus User Manual D18a 2004 10 07 P Santos A Stork 5 Software Setup This chapter shows how to configure t
10. Pen gt Shape1 gt Cylinder Cirular Primitive Cone gt Shape1 gt CircCone Cirular Primitive Prism gt Shape1 gt CircPrism Cirular Primitive Pyramide gt Shape1 gt CircPyr Primitive Cone gt Shape1 EllipCone Primitive Cone Shape2 EllipCone Ellipse Primitive Prism Shape2 Ellipse Primitive Pyramide gt Shape2 gt EllipPyr Circular Primitive Trunc Shape2 Cone CircTrunc Circular Primitive Trunc Shape2 Cone gt Cire Trunc Cone Circular Primitive Trunc Shape2 Pyramide gt Circ Page 45 of 90 SmartSkada s User Manual D18a 2004 10 07 P Santos A Stork Co Tm Ellipse Primitive Trunc Shape2 Cone gt Ellip Trunc Cone Ellipse Primitive Trunc gt Shape2 Pyramide gt Ellip Trunc Pyr To create primitives in SketchAR select one of the Primitive operations pressing the according button in the Primitive menu on the PIP sheet To start the Primitive operation press and hold the first button of the pen and draw the primitive moving the pen Release the first button to complete the Primitive operation Figure 32 Primitive Creation Primitives can be moved and copied In addition the following Boolean operations can be performed on primitives which can be found under the BOOL menu e Intersection e Subtraction e Addition 7 3 7 Scene Modelling and Assembly In SketchAR constraints can be applied to the entities edges faces o
11. Recognition SpeechRecoConfig language english gt lt CommandSet id TapeD gt Command id 1 name line weight 1 0 gt Command id 2 name stop weight 1 0 gt lt Command id 3 name red weight 1 0 gt lt Command id 4 name green weight 1 0 gt lt Command id 5 name blue weight 0 5 gt lt Command id 6 name black weight 1 0 gt Command id 7 name white weight 1 0 gt lt Command id 8 name yellow weight 1 0 gt lt Command id 9 name undo operation weight 1 0 gt lt Command id 10 name redo operation weight 1 0 gt lt Command id 11 name damn ull9 Peight 1 0 gt Command id 12 name open menu weight 1 0 lt Command id 13 name go weight 1 0 gt lt Command id 14 name Shaaesa weight 1 0 gt Command id 15 name speech activate weight 1 0 gt lt Command id 16 name speech deactivate weight 1 0 gt Command id 17 name connect joe weight 1 0 gt lt Command id 18 name connect jack weight 1 0 gt lt Command id 19 name network disconnect weight 1 0 gt Command id 20 name userlist weight 1 0 gt lt CommandSet gt lt SpeechRecoConfig gt Explanation of speech specific tags e SpeechRecoConfig select the speech of the SR component e lt CommandSet gt define a set of speech command e Command define a single speech command e lt SpeechRecoSource g
12. SketchAR from a technical point of view focussing on the specific details of the hardware used and the implementation of the software Page 8 of 90 SmartSkedehu E User Manual D18a 2004 10 07 P Santos A Stork 2 Hard and Software Requirements SketchAR the immersive design and modelling application has the following hard and software requirements to run properly Minimum Hardware Requirements gt Pentium 4 1 6 Ghz 512 MB RAM 300 MB Disk Memory Nvidia Quadro Graphics Board 1 or ATI FireGL active 120Hz output passive stereo 60Hz output 2 Opentracker Supported Trackingsystem such as AR Tracking Optical Tracking System 3 or others Spacemouse to use four view drawing 4 Tracked artifacts Model Cyberstilo 5 PIP Active Stereo CrystalEyes Active stereo shutter glasses 6 Passive Stereo Circular Pole filter glasses and filters for beamers Active Stereo Beamer 120Hz 7 or two regular beamers 60Hz for passive stereo 8 Desktop Autostereo Display 9 Augmented Reality Trivisio Optical See through glasses 10 In case of beamer usage Back Projection Plexiglas which keeps polarization Software Requirements Windows 2000 XP Operating System 11 TGS Open Inventor License gt 3 1 1 Scenegraph functionality 12 ACIS 8 Modeling Kernel License modeling kernel 13 TAO CORBA and ACE Distribution distributed communication 14 Magic Software distribution utils libs 15 Studierstube di
13. a two step calculation on marker recognition and position calculation The body to be tracked e g a human body or an object is equipped with markers that are often covered with retro reflective surfaces Tracking cameras scanning a certain volume recognize these markers The data of the tracking cameras are handed over to a central PC for final processing The result of each measurement are coordinates that describe the position of the markers and hence the position of the body carrying the markers Page 14 of 90 SmartSkadaus User Manual D18a 2004 10 07 P Santos A Stork The results are then broadcasted onto a local tracking network Ethernet to which a tracking PC is attached which will perform the final calculation of object positions merging data from all attached tracking cameras output 3dof 6dof etc lt 2d image coordinates Infrared flash 2d image coordinates gt bs Infrared flash Figure 5 Optical Tracking ART Tracking System with ARTtrackl cameras The system is highly stackable so tracking volumes only depend on the number of cameras used The final computed data by tracking PCs is broadcasted onto the network for further processing by one or more applications Technical Data Range Accurac Latency Max Cost Remarks y Update Bodies Rate ARTtracki 300cmx Position Latency Upto20 EUR The camera http ww 300cm 04mm no target 30000 CCD chip is w ar using 4 Orientati
14. allows for 1 N collaboration by implementing locking mechanisms on shape by shape basis In fact in its current version SketchNET is a shape exchange repository It can be accessed using the event subscriber paradigm Interested applications subscribe to certain events Applications communicate events like login logout of users or shape updates via notification to the Web service which in turn connects to an SQL database and executes the necessary changes in the particular tables Then should the event description specify this the Web service will broadcast the results to the event s subscribers 7 7 1 SketchNET overview SketchNET is split in a client side API that enables subscription to events and notification of events Each participating design and modelling application has to include the SketchNET API to be able to subscribe to events and send them to SketchNET Furthermore it is required that each application is extended to handle subscription to and arrival of events In addition SketchNET is based on the principle that each shape should be self describing and that an application connected to SKETCHNET will decode shapes it understands and ignore all shapes it does not understand Page 67 of 90 SmartSketchey E User Manual D18a 2004 10 07 P Santos A Stork SketchNET Web Server SketchNET Database SmartSketches WebService SketchAR User Joe SketchAR User Jack Collaboration 1 N Figure 50 Sk
15. details bodies includin 658x496 tracking d cameras ono 12 Update 10 in g2 pixel e rangeis degrees rate accurate cameras Cameras are only Standard Max mode and one available for limited deviation 60Hz DTrack different by the Position server Field of amount 0 06mm machine Views of Standard horizontal cameras Deviatio up to 60 used n degrees max Orientati vertical up to supporte on 0 03 45 degrees datthe degrees Camera moment weight is 2 5 256 KG Page 15 of 90 SmartSkada s User Manual D18a 2004 10 07 P Santos A Stork Range of a single camera up to 10 meters dependin gon marker size Pros e Un tethered tracking system e Manytracked bodies possible e Unlimited extension of tracked area using more cameras max 256 Cons e Line of sight occlusion problem e Expensive compared to other systems Conclusion ART offers a tracking system that is not susceptible to electro magnetic interferences However there is an inherent line of sight problem which can be solved by using more cameras Nevertheless certain scenarios require a user to go into a physical mockup and in such places this technology will not work as well as others Very good about optical tracking systems is the fact that interaction devices are un tethered and quite a large tracking area can in principle be achieved by adding more cameras 4 3 Interaction Devices Different kind of tracked objects can be used as input devices Figure 6 e
16. http www barco com VirtualReality Liesegang Optoelectronics GmbH Germany HomePage www liesegang de A C T Kern GmbH amp Co KG Germany HomePage http www actkern info eHome eProdukte eprodukte html Trivisio Prototyping GmbH Germany HomePage http www trivisio de Microsoft Corporation Redmond WA USA HomePage http www microsoft com windows default mspx Mercury Computer Systems USA HomePage www tgs com Spatial Corporation Dassault Systems France HomePage www spatial com TAO Corba Solution and ACE Abstract Socket Library U Washington USA HomePage http www cs wustl edu schmidt TAO html Magic Software Utility Libraries HomePage http www magic software com SourceCode html Studierstube Augmented Reality Project TU Wien Austria HomePage www studierstube org Page 71 of 90 SmartSkada s User Manual D18a 2004 10 07 P Santos A Stork 17 OpenTracker Abstract Tracking System Support TU Wien Austria HomePage http studierstube org opentracker 18 Apache Software Foundation USA HomePage http xml apache org xerces c 19 Microsoft Corporation Redmond WA USA HomePage http www microsoft com speech Page 72 of 90 SmartSkedehu E User Manual D18a 2004 10 07 P Santos A Stork 9 Annex B Operation Table Editing Move Edit Edit Move Move Copy Edit Edit gt Copy gt Copy Select Edit Edit un NEN Select Select shape ele All gt
17. in SketchAR has two different modes The first is a free positioning mode Position the Mirrorplane using the tracked Tangible Plane Artefact chapter 1 1 1 and 3 1 The second mode is the Mirrorplane in car mode In this mode the Mirrorplane is fixed in the centre of the car Use the Mirrorplane button in the Tools menu on the PIP sheet to activate the Mirrorplane operation e First press free positioning mode e Second press Mirrorplane in car mode e Third press Mirrorplane off 7 6 6 Clipping Plane Clipping Plane Tools gt Constraint Plane Clipping gt Clip Pen Plane To investigate the shape of a model we added a clipping plane to our immersive modeler The clipping plane clips away one side of a plane and presents a cut through Page 64 of 90 SmariSkdda E User Manual D18a 2004 10 07 P Santos A Stork the model The plane can be arbitrarily positioned in the scene using the tracked tangible plane 7 6 7 Freeze Scene and View Freeze Scene Freeze View One characteristic of an immersive environment is that the scene changes when you move your head This feature was very well accepted by stylists able to view the model while styling it However sometimes while drawing the continuous movement of the scene irritated them and prevented them to sketch with precision Therefore we developed the possibility to freeze the view and the scene so they can draw accurately without distraction To freeze unfree
18. model When reading such a file the ACIS tessellation functionality is used to generate a displayable version Please click on the 7 2 Package Model Editing Ree Ere op m Adjustment Wheels Wheels Se sl IP Adjustment Overall Dimensions Adjustment Human Ergomomic oo ieee FE Adjustment Tech Free Enc EN El III On Off On Off A package model defines a set of constraints which have to be left untouched by the stylists These constraints may be imposed by car parts such as the engine wheelbase or overall dimensions and generally also involve security constraints such as size of bumpers head clearance and visibility angles The constraints may vary according to the car model developed sports car sedan etc Page 34 of 90 SmartSkadaus User Manual D18a 2004 10 07 P Santos A Stork Usually the package model is given to the stylists by the costumer or by the planning department The package model constraints the freedom of the stylists and give hints which dimensions to respect Instead of importing package drawings into our system or in addition to this the system provides the possibility to configure a default aD package model Using draggers or sliders on the pip or alternatively in 3D space the package model parameters can be changed Figure 21 Package Model Base i Tel i We Figure 22 Package Model Constraints Wheels 7 2 1 Package Model Parameters In the following we show a list
19. of configurable package parameters which can be accessed through the package menu e Wheels e Dimension o Wheelbase o Front Overhang o Front Axis Width o Length o Front Wheel Size o Width o Rear Axis Width o Height o Rear Wheel Size e Ergonomic e Encumbrance o Dummy Position o Wheel envelope Size o Steering Wheel Position o Engine Encumbrance Page 35 of 90 SmartSketchey E User Manual D18a 2004 10 07 P Santos A Stork o Head Clearance o Heater Encumbrance o Front Visibility Angle o Rear Visibility Angle o Side Visibility Angle 7 2 2 Package Model Constraint Check Collision Package gt Package gt Pen Detection Collision Collision Detect Detect To support monitoring of whether package constraints are being respected by the stylist a check functionality has been implemented Figure 23 Package Constraint Check The constraint checking in SketchAR uses the ACIS CAD kernel Every shape that is created with ACIS is checked against the parameters of the package model in a post process function which can be triggered by the user after a geometric operation For this reason the geometries imposed by the package are represented in ACIS If a collision occurs SketchAR shows the involved parts in red to signal the collision To activate deactivate the Collision Detection use the Collision Detect button in the Package menu on the PIP sheet or in the Pie menu After activating the Collision Detection start creating ge
20. product information according to the actual modelling context which is defined by the current user operation The realised model contexts are a part view during the modelling of parts and an assembly view for assembling operations and kinematics validating of obtained mechanisms Semantic Context Filter Specific On Off Semantic Filter gt In the following an excerpt of an assembly sequence is presented N 5 Figure 33 Assembly Parts Page 49 of 90 SmartSkdda E User Manual D18a 2004 10 07 P Santos A Stork Figure 34 Assembly I Figure 35 Assembly II 7 4 Geometry Modification In SketchAR several ways for modifying and refining geometry exist This section will introduce them 7 4 1 3D Oversketching Operation PIP Pie Gesture Speech Inter Synthesis action Device sketching Overrein oo Im sketching Oversketch A common technique to create a desired curve is to repetitively draw over a curve this technique is called oversketching To mimic this curve modification method we developed a virtual oversketching technique With this function a curve can be changed with sketching a new curve over the existing one The user can control the influence of the new curve to the resulting curve with two sliders In Figure 45 two examples of oversketching are shown In a the influence parameter of the oversketched curve is 0 5 Therefore the resulting curve lies between the Page 50 of 90
21. surface and release the third button To create the surface pick the NURBS curve with the pen press the first button and extrude the curve along the path of the pen movement to get the surface Release the first button to complete the Spline Extrude operation To use the NURBS Surface operation and create a surface of an on the fly created NURBS curve press the NURBS Surface button in the Surface menu on the PIP sheet or activate the pie menu using the third button go to the submenu Surface select the NURBS Surface and release the third button Page 41 of 90 SmartSkadaus User Manual D18a 2004 10 07 P Santos A Stork To create the surface press and hold the first button to create a new curve Release the first button to complete the curve creation After creating the curve you can extrude the curve along the path of the pen movement without pressing a button to get the surface Press the first button one more time to complete the NURBS Surface operation Skinning Skin Surface gt Surface gt Surface Skin Skin Section Section Skin Surface gt Surface gt Select Skin Skin Surface Select Select If curves are created within the skinning operation a real time preview of the surface to be expected is shown during the operation Figure 30 For this purpose an algorithm has been developed which generates a triangle mesh between the curves Figure 30 Skinning Net Surface Net Surface Surface gt
22. the selected shapes 5 Clear the selection 6 UNDO the last operation can be called successively 7 REDO the last undone operation can be called successively 8 oversketching 9 material editor 10 edit control points Page 82 of 90 SmartSkechea User Manual D18a 2004 10 07 P Santos A Stork Edit pie menu Edit Material Editor E La LA Material editor Edit Material Editor Color Editor sees T Cane tes y P Sk E DNUS R iher be SE orem Green Blue Shine Alpha Color editor Page 83 of 90 SmartSketchea User Manual D18a 2004 10 07 P Santos A Stork Edit gt Oversketch Tools 5 Packagef ht Camera Light Snap PLANE Sliders Page 84 of 90 SmartSkechea User Manual D18a 2004 10 07 P Santos A Stork Tools pie menu X Y Zoom Camera Adjust Pick Radius Adjust Page 85 of 90 SmartSketchey E User Manual D18a 2004 10 07 P Santos A Stork KE Package pie menu Package gt Dimensions y Fr i m Le Y d Page 86 of 90 User Manual D18a 2004 10 07 SmartSketchey E P Santos A Stork Package Human WR scale X Scale Y es Page 87 of 90 User Manual D18a 2004 10 07 SmartSkadaus P Santos A Stork jena aM Cement specific y E Tr A BB E EN Seale X Seale Z Seale Y Seale Strale Y Serale p e Jac oo L L KEE ESE a f E iF 9g L E Ka f 4 ke E ur i Y
23. times it is nearer to the tip of the pen than the curve On the other hand most of the times the user does not want to pick the surface but the curve The problem is that a curve is more difficult to pick than a surface Therefore our algorithm prioritizes lower dimensional topological elements when Page 32 of 90 SmartSkadaus User Manual D18a 2004 10 07 P Santos A Stork picking and snapping according to the following order so the user picks the curve and not the surface e Vertices e Curves Edges e Surfaces Faces 6 9 Virtual Paper Metaphor We have implemented a virtual paper metaphor using the SpaceMouse In a four view the position size and orientation of the background image the model can be controlled with the SpaceMouse in the same manner as paper can be rotated on a table top rotate around plane normal pan in x Figure 20 SpaceMouse supporting the virtual paper metaphor Page 33 of 90 SmartSkada s User Manual D18a 2004 10 07 P Santos A Stork 7 System Functionality In the following the functionality of the SketchAR prototype is presented from a technical point of view 7 1 File Operations File File File Pen Load Load Save Save For file I O we are using the ACIS reader and writer The model created within SketchAR and represented in the ACIS modeling kernel can be exported into the following formats e ACIS sat e ProEngineer proE e CATIA V5
24. uie pov ao ho MOT M 50 Figure 36 Curve Oversketching esse ee ee nnnnnnn nnne nennen nnne n nnne nnne nns 5 Figure 37 Constraint oversketching principle ass qnn e eta i ta pa earl ida 32 Procure 36 Two possible Cury e ee ENEE 22 Figuro 39 Constrained WE E E 53 Figuro 40 Editing Control POINTS ses oe Ek N EP ie S 54 Essure 41 Net SUL aCe OV ls elc DB eoo cuin oso coche EK eN Re UP aci qu EE NM CIS SE 54 Figure 42 History based surface modification essen 55 Pigure A Doolcan Mi SCC LOT a see dopo Pares Ee A 56 Figure 44 Material Fdutor 60 Figure 45 Workplane positioned by dragger in seen 62 Figure 46 Context pie menu allowing the work plane to be positioned in different ways 62 Peur ds Di IIO ie OGE HOE a ee EE posue GE EE ee Leo sua beso ON odio Tree 64 Eier Ad Dou Vie ee EE ev IA 66 rabia 20 ele TE ss EE O E I EE EA GE Ge 68 Page 7 of 90 SmartSkada s User Manual D18a 2004 10 07 P Santos A Stork 1 Introduction The SketchAR User Manual D18a is intended as a guide for the user to set up a working SketchAR environment as well as getting to know how all available features work It is split up as follows e Hard and Software requirements This section specifies the minimum hardware requirements for SketchAR the target operating systems as well as supporting software e CD content This section describes the content of the Installation DVD D17 e Hardware Setup SketchAR relies on t
25. user can then augment the physical model with virtual elements for example drawing a new roof This scenario can for example be used for design reviews on physical mock ups In this way design changes are recorded in a digital way and the created CAD geometries can be merged with the virtual model the physical mock up was created from Page 13 of 90 SmartSketchey E User Manual D18a 2004 10 07 P Santos A Stork M gt Figure 4 AR Setup Figure 4 shows a fake image to render the effect obvious An outside observer sees both The user with the HMD plus the physical car with the virtual overlay 4 2 Tracking System Optical tracking systems use a variety of detectors from ordinary video cameras to LEDs to detect either ambient light or light emitted under control of the position tracker Infrared light is often used to prevent interferences with other activities Main advantages of optical tracking systems are their high availability large area tracking capacity their high accuracy and their immunity against magnetic interferences Their weaknesses are the need for a clear line of sight light source interferences and their need for processing power Also they are more expensive than other tracking solutions ART Advanced Real Time Tracking The ART 22 optical tracking system is composed of two or more theoretically extendable to N CCD infrared tracking cameras which contain Linux Embedded PC systems performing
26. 0 SmartSkada s User Manual D18a 2004 10 07 P Santos A Stork 6 5 Speech Recognition Synthesis The Microsoft Speech SDK 51 is freely distributable and represents an interesting choice because it enables applications to respond to spoken commands and includes a text to speech engine in its distribution In this way commands will not only be recognized but the user may be informed about application status and operation results by synthesized speech which helps him not to become distracted from its original task at hand by obtrusive status displays or similar information conveyers In addition the fact that Microsoft Speech SDK does not come with a dictionary and grammar but instead relies on the developer to specify a grammar for his specific purpose allows for not too much overhead in the application if the main goal is simply being able to input speech commands and returning synthesized speech output Speech is to be used to support the following functionality Speech Commands in Virtual Taping Implementation of multiple UNDO REDO Color selection Pie menu activation and operation choice Activate and deactivate speech For available speech commands see annex B 6 6 Implicit Gesture Recognition As direct consequence of the second round of usability tests we have observed that users would perform gestures much faster than stroke input Subsequently we implemented an implicit approach that analyses the user s movement for po
27. Augmented Reality usage More information is available in the README files in each directory Page 10 of 90 SmartSketchey E User Manual D18a 2004 10 07 P Santos A Stork 4 SketchAR Hardware Setup In this chapter we present the SketchAR hardware and software setup We describe the rendering systems tracking systems and input devices Figure 1 shows a schematic diagram of our sample setup with an active stereo display and optical tracking system Optical Tracking Local Tracking Subnet Cameras Camera Sync Shutter Sync Active Stereo RGBYC Signal Internet Tracking Server Graphics Server Figure 1 Hardware Setup Overview 4 1 Rendering Systems Rendering in SketchAR can be done in a number of scenarios as there are the back projection display the virtual table or the augmented reality setup allowing to design on a physical mock up of a car 4 1 1 Stereovision active passive stereo Stereovision means that scenes created with immersive SketchAR applications will have to be rendered simultaneously for the left and right eye of a user and then be displayed on a screen back projection system or an HMD This poses additional challenges to system hardware such as graphic accelerators projectors and HMDs which need to be able to support it Graphics boards like the 3D Labs Wildcat 4 or ATI Fire GL powered by Radeon 8800 GPU are able to provide the required output for active stereo vision by supporting
28. E Page 88 of 90 SmartSketchey E P Santos A Stork User Manual D18a 2004 10 07 Package sleni Clearance E GEES Jr T J Scale X d Y de L atiesiittabtatteen E EX PRG CIE tle A F me n E en cumb bra nc Ice Page 89 of 90 SmartSketches User Manual D18a 2004 10 07 P Santos A Stork SEA Ast T E Ce T idth EY View Scene Modeling and Assembly File Primitve Edit id lt jajajo gt e LJ A A IS Page 90 of 90
29. Erase de ML HMM EE Select Clear Select Delete Select operation operation Redo operation Curve Creation Free Sketch Curve Curve Pen Sketch Sketch Pen Closed Curve Sketch Closed Tae e Line ia Curve Curve Pen Curve gt NURBS gt NURBS Curve Freehand Curve gt Curve gt Spline Freehand Freehand Spline umm Line Curve gt p Tape Draw Curve E Tape Draw Surface Creation Spline Surface Surface Extrude Spline Spline Extrude Extrude NURBS Surface Surface Surface NURBS NURBS Surface Surface ESE Skin Section Skin on Skin Surface gt Surface gt Surface Select Select Net Surface Surface gt Surface gt Pen Net Surface Net Surface Net Surface gt Surface gt Pen Select Net Select Net Surface Select Page 73 of 90 SmartSkada s User Manual D18a 2004 10 07 P Santos A Stork Coon Patch Surface Surface Pen Coon Patch Coon Patch Select Coon Surface Surface gt Pen Patch Coon Coon Select Select Primitive Creation Sphere Primitive gt Shape1 gt Sphere Quad Primitive gt Shape1 gt Quad gt Shape1 gt Cylinder Cirular Cone Primitive Pen gt Shape1 gt CircCone Primitive gt Shape1 Cirular Pyramide CircPyr gt Shape1 gt EllipCone gt Shape2 gt EllipCone EE fo gt Shape2 EllipPrism Pyramide gt Shape2 gt EllipPyr Circular Primitive Trunc Cone gt Shape2 gt CircTrunc Cone Circula
30. INFORMATION SOCIETY TECHNOLOGIES IST PROGRAMME information Society technologies SmartSketches Z Project acronym SmartSketches Project full title SmartSketches A Multimodal Approach to Improve Usability in the Early States of Product Design Contract no IST 2000 28169 DELIVERABLE D 18a SketchAR User Manual SmartSketche User Manual D18a 2004 10 07 P Santos A Stork Deliverable DELIVERABLE N 18a NAME SketchAR User Manual WORKPACKAGE 4 LEAD PARTICIPANT IG TYPE Report DATE 2004 09 21 CIRCULATION Public Executive Summary SketchAR is one of the first immersive design and modeling systems for early stages of product design It combines precise optical tracking with interaction on an accurate CAD model which is not just a triangle or subdivision surface model This document represents the SketchAR User Manual Figure 0 A car body sketched with SketchAR in a few minutes Page 2 of 90 SmartSkada s User Manual The SketchAR User Manual is intended as a guide for the user to set up a working SketchAR environment as well as getting to know how all available features work It D18a 2004 10 07 is split up as follows O O O 0000 Please note that additional information can be found on the installation DVD D17 Hard and Software requirements CD content Hardware Setup Software Setup Multimodal Interaction Techniques Functionality File Operations Package Model Editin
31. O transparencyType SORTED OBJECT BLEND Page 24 of 90 SmartSkada s User Manual D18a 2004 10 07 P Santos A Stork The desktop configuration uses a Studierstube Viewer Widget to render the scene to the desktop Additionally a 2D PIP is offered which can be used with the mouse 5 3 2 2 Active Stereo The active stereo configiguration is meant for devices that support active stereo such as BARCO Virtual Reality Tables display DisplayKit 1 station 2 stereoCameraKit File name VTStereoKit iv display FieldSequentialDisplayMode 1stereoMode QUAD BUFFER cameraControl TrackedViewpointControlMode 1j headlight TRUE headlightIntensity 1 0 backgroundColor 0 2 0 3 0 4 xoffset O yoffset O Size of the viewer window in pixels width 1024 height 768 transparencyl ype SORTED OBJECT BLEND 5 3 2 3 Passive Stereo The passive stereo setup can be used for passive stereo back projection systems using pol filters for both beamers display DisplayKit 1 station 2 stereoCameraKit File name VTStereoKit iv display DualChannelDisplayMode splitMode VERTICAL cameraControl TrackedViewpointControlMode 1j headlight TRUE headlightIntensity 1 0 backgroundColor 0 2 0 3 0 4 xoffset O yoffset O Size of the viewer window in pixels width 2048 height 768 transparencyType SORTED OBJECT BLEND Page 25 of 90 SmartSkada s User Manual D18a 2004 10 07 P Santos A Stork 6 Multimodal Input Methods This chapter describes all ex
32. OpenInventor file reflecting the pie section s operation outcome Figure 14 Pie Menu Animated Pie Ring Menus To render pie menus even less obtrusive and more self explaining animated pie menus have been introduced This basically means that in the configuration file of a pie menu a user can specify an open inventor file representing the visual outcome of an operation Note that the shapes displayed in the pie menu were actually created with SketchAR When moving the 3D into such a piece of the pie the 3D iconic model starts to spin Page 28 of 90 SmartSketchey E User Manual D18a 2004 10 07 P Santos A Stork Figure 15 Pie sub menu SURF animated pie menu Context Sensitive Pie Ring Menus Extending our pie menu implementation context sensitive pie menus have been implemented When the 3D pen is close to an activated projection plane the context sensitive pie menu allows translation rotation or free positioning of the plane In SketchAR the pie menu is activated by pressing the third button of the Cyberstilo However if the projection plane is activated and the pen is close to it and the user presses the third button the user will see a context menu allowing him to perform a translation rotation or free movement of the projection plane along dragger axis or in space Support for this feature was implemented in a class named SoArtifactPlaneWidget Figure 16 Context sensitive pie menu Page 29 of 9
33. Pen Surface gt Net Net Surface Surface m eI T IF Select gt Net Page 42 of 90 SmartSketchey E User Manual D18a 2004 10 07 P Santos A Stork Surface Net Select Select Net surfaces are surfaces created from a net of four or more not necessarily intersecting curves ACIS supports net surfaces but requires the curves to be input in a certain order to process them correctly To relieve the user from fulfilling these ACIS requirements we have implemented an intelligent mapping algorithm that sorts the selected curves so ACIS can generate the corresponding surface Figure 31 Net Surface To use the Net Select Surface operation and to create a surface of existing NURBS curves press the Net Select button in the Surface menu on the PIP sheet or activate the pie menu using the third button go to the submenu Surface select the Net Select Surface and release the third button To create the surface pick the NURBS curves with the pen press the first button to select them one after another press the second button to create the surface and complete the Net Select Surface operation To use the Net Surface operation and create a surface of on the fly created NURBS curves press the Net Surface button in the Surface menu on the PIP sheet or activate the pie menu using the third button go to the submenu Surface select the Net Surface and release the third button To create the surface press and hold the first button to c
34. SmartSkada s User Manual D18a 2004 10 07 P Santos A Stork original and the oversketched one In b the influence parameter is 1 0 therefore the result follows the oversketched curve exactly a oversketch result EN ON oversketch result Figure 36 Curve Oversketching 7 4 2 Constrained Oversketching Over Edit gt sketching Oversketch See additional buttons for influence Vertical Horizontal The oversketching method described in the previous section allows oversketching the curve in all three dimensions directly when applied in an immersive environment using direct 3D input devices The following modification to the idea of oversketching adds the possibility to constrain the changes to two coordinates To perform a constrained oversketch the user has to choose a 2d orthographic view of the curve he intends to change For example in figure 37 the oversketch is done on the xy plane The modification of the curve is now constrained in a way that the original profile from the xz plane is not changed Page 51 of 90 SmartSketchey E User Manual D18a 2004 10 07 P Santos A Stork oversketch curve d Pd d 1 d 1 a 1 I I 1 H 1 1 1 1 1 i resulting original SES Curve M Figure 37 Constraint oversketching principle The implementation of the constrained oversketching makes use of the CAD library ACIS In contrast to the
35. ccording to the user s point of view In this way the virtual objects appear floating in the space above the table Figure 2 A second scenario uses an upright projection wall to display the picture In this case the tracked artefacts are placed on a table and the user operates with the pen in front of the wall As an advantage to the immersive table this setup is portable The disadvantage is that the tracked artefacts cannot be put somewhere conveniently Therefore a portable table like stereo display would be the best solution Figure 3 shows the setup at the review in Brussels 2003 Figure 2 VR Setup active stereo scenario Page 12 of 90 SmartSkadaus User Manual D18a 2004 10 07 P Santos A Stork Figure 3 VR etup passive stereo scenario 4 1 3 Augmented Reality Setup Besides sketching in virtual reality SketchAR also supports sketching in mixed reality For this purpose the user wears a video or optical see through head mounted display The video HMD is equipped with two cameras at the position of the users eyes They capture the real environment in front of the user The camera picture is then mixed with the virtual image so the virtual elements appear embedded in the real world The optical HMD has semi transparent glasses on which the augmented content is projected so the user sees both the virtual parts and the real scene As demonstration of this technology we use a small tracked physical model of a car The
36. cess restarts In this manner it is very simple to correct an error or to go back to follow a different path The advantage respectively the common use of a rubber tool is that the user can correct the curve instantaneously and the user may try and retry many times until the desired shape takes the form the user had mind 7 3 3 Automatic Curve Splitting Curves of high quality should have as few control points as possible But one problem with curves build from few control points is that it is not possible to have high curvature points HCP We therefore developed a function which analyses the drawn curve according to its curvature It detects points of high curvature and cuts the curve at these points The resulting curve consists of partial curves with few control points and possible sharp edges at the junctions Page 38 of 90 SmartSkada s User Manual D18a 2004 10 07 P Santos A Stork In Figure 26 three curves are shown 1 Input stroke 2 Approximation with one low degree spline tip rounded 3 Approximation with two low degree splines HCP maintained The upper curve is drawn with the conventional freehand spline function It can be clearly seen that the curve s tip is smoothened The lower curve is automatic tiled using our function therefore the tip is as sharp as you draw it Figure 26 Automatic Curve Splitting 7 3 4 Virtual Tape Drawing lt color gt Confirmat Fingers line to ion of s
37. controlpoint modification is known to be not very intuitive from the scientific literature the users in SmartSketches demanded this functionality We implemented two ways of control point modification a The user selects a curve and can freely manipulate each control point b The user can use a dragger to constrain the movement of a control point To edit the control points of a curve or a surface choose the Edit Control Points button in the Edit menu of the PIP sheet To start editing the control points pick a curve or surface with the pen and press the first button to select it The control points are displayed now You can manipulate the control points in two ways Figure 40 the first a is to pick them with the pen press and hold the first button and move them free in space The second way b is to pick a control point and press the second button The control point turns into a dragger Page 53 of 90 SmartSkada s User Manual D18a 2004 10 07 P Santos A Stork which can be moved either free in space or only in one axis and can be rotated Pick the centre of the dragger and press the second button again to make it disappear Figure 40 Editing Control Points 7 4 4 Net Surface Oversketching SS Bien O sketching Oversketch Control points modification might be very useful for an accurate modification in a small area of interest But imagine changing the character of a complete patch then fiddling around with l
38. d button Page 58 of 90 SmartSkeacha User Manual D18a 2004 10 07 P Santos A Stork To use the Delete Select operation pick a shape with the top of the pen and press the first button The shape is deleted To delete a shape using gesture recognition just make a rectangle gesture near the shape The synthetic speech output confirms the Delete operation with deleting shape 7 5 4 Multiple UNDO REDO gt Undo gt Undo Rectangle operation operation gt Redo Redo Triangle operation operation Multiple Undo Redo has been implemented which allows going back N operations and forward N operations Multiple Undo Redo also takes into account if an operation was performed while a mirror or projection plane was activated and stores the generated or manipulated shapes accordingly by storing the associated operations that lead to their creation or manipulation Multiple Undo Redo is a model centered global operation and not a user centred operation which means that each user can go back and forward again through all modifications made by all users according to the chronological sequence of a session If two users are working at the same model then the first user can undo the operation the second user just performed This is useful in collaborative scenarios where both discuss on the same model To use Undo or Redo press the Undo or Redo button in the Edit menu on the PIP sheet or activate the pie menu using the t
39. de top front or back view With this feature we support the stylist to draw e g characteristic curves from a certain point of view While working in the four side view the user is able to control the camera move and zoom using the SpaceMouse This feature lets him work as on a virtual piece of paper which he can pan rotate and zoom in any position to do his sketches in the most comfortable way r3 Pd E Al WcAWCATS Figure 49 Four View Use the FourView button in the View menu to switch between the four side views Page 66 of 90 SmartSkada s User Manual D18a 2004 10 07 P Santos A Stork e First press Left e Second press Front e Third press Back e Fourth press Top e Fifth press Fourview off In the FourView mode you can move and zoom the scene using the space mouse 7 6 11 3D Layers To hide and show parts of the model we integrated 4 drawing layers The current drawing is always made on the selected one The user can then choose which layer is visible and which is not These layers may contain parts of the 3D model to compare alternatives 7 7 Network Collaboration SketchNET SketchNET is a framework which allows design and modeling applications to share and collaboratively generate shapes The general term shape here means any objects which their CAD modelling kernels can compute The SketchNet framework uses a Web service approach with an SQL database back end serving as shared memory It
40. e geometric operations to select or draw next to existing objects 3D Picking To select and move objects in free space they can be picked In contrast to other VR CAD systems SketchAR did not simply use ray picking for selecting objects because it is an indirect distant interaction Instead a fast 3D picking on the precise topological element s geometry was implemented using a two step procedure which first checks for bounding boxes and then calculates the nearest distance of the topological elements geometry to the current pen position 3D Snapping When operating in free space it is difficult to sketch a new surface close to an existing one But obviously it is crucial for a car model that all parts match without any gaps To avoid those gaps we extended our 3D picking approach toward snapping Snapping is performed by successively doing 3D pick actions When snapping is active the virtual 3D pen snaps automatically to faces edges or vertices in its proximity The user can define its pick snap radius in 3D A corresponding visual feed back is generated on the fly Intelligent 3D Picking and Snapping We have extended our picking and snapping algorithm to a more intelligent behaviour that prioritizes lower dimensional topological elements when they are inside the picking and snapping radius of the tip of the pen Let a curve be on a surface A regular picking algorithm would in general return the surface object since most of the
41. e pie menu Page 78 of 90 SmartS etches User Manual D18a 2004 10 07 P Santos A Stork Curve ple menu Primitive File Curve Primitive Surf Edit Tools Peckas View See ee E FR S A 1 Page 79 of 90 SmartSketchey E P Santos A Stork User Manual D18a 2004 10 07 SE Een Edit Tools View oper DSi y d Ellip Tan Cone Ellip Pyr Page 80 of 90 SmartSketchey E User Manual D18a 2004 10 07 P Santos A Stork Primitive gt Bool The surf menu allows to create a Coons patch from one 3D stroke a skin surface by sketching several 3D curves a skin surface by selecting several existing 3D curves a net surface by sketching several 3D curves in arbitrary order and direction a net surface by sketching several existing 3D curves a NURBS surface by sketching a curve and directly extruding it in a singe Interactive step a NURBS surface by extruding an existing 3D curve a Coons patch by selecting 4 existing curves which have to match at its endpoints LE HE 90 Page 81 of 90 SmartSkadaus User Manual D18a 2004 10 07 P Santos A Stork For NURBS curve extrusion the eraser pen approach is active so that the operation can be partially undone by moving backwards These operations are also available form a corresponding pie menu see below The edit menu allows to initiate the following operations 1 select ashape 2 movea shape 3 copy a shape 4 erase delete
42. etchNet On the server side there is a Web Server which hosts a Web service The Web Service is made of three parts One is the business logic to process subscriptions and incoming events to an SQL data base which is used as shared memory The second part is a Data Access Module which handles connection to the database and executes table modifications The third part is a SketchNET remoting Module used to establish a connection back to each subscriber Page 68 of 90 SmartSkada s User Manual D18a 2004 10 07 P Santos A Stork Microsoft SQL DB Shape Repository Locking mechanisms D Conversion NET WebService And Database Subscriber Information DataAccess DataBase SQL Connection SQL Query Generation Trigger for Event Callbacks Remoting wal SmartSketches in Subscriber Application WebService using SOAP WebService Eventmanagement Core Subscriber management Event Parsing Generation SOAP SOAP Communication MS IIS Webserver Event forwarding to WebService GSoap Compiler produced Stub Files for VC 6 0 C to communicate with Net WebServices Event preprocessing parsing SketchNetAPI Triggers Callbacks in host application SketchAR Eventloop Generation Parsing of Events XML instrumentation of Shapes Self descriptive shapes Speech Commands for Network Connectivity Page 69 of 90 SmartSkdda E User Manual D18a 2004 10 07 P Santos A Stork 7 7 2 Con
43. eview is displayed while moving Release the first button to apply the operation 7 5 Editing Operations Editing operations let you move copy select or delete a curve surface or primitive Additional SketchAR provides multiple undo redo functionality 7 5 1 Select Multiple Select Select Edit Edit Ellipse gt Select gt Select selecting shape One or more shapes in the scene can be selected with the virtual 3D pen To activate the Select operation press the Select button in the Edit menu on the PIP sheet or activate the pie menu using the third button go to the submenu Edit select the Select button and release the third button To use the Select operation pick a shape with the top of the pen and press the first button To deselect the shape just do this once again However the easiest way to select deselect a Shape is to use gesture recognition Just make an ellipse gesture near the shape to select deselect it The synthetic speech output confirms the select deselect operation with selecting deselecting shape Page 57 of 90 SmartSkada s User Manual D18a 2004 10 07 P Santos A Stork 7 5 2 Move Copy m meme m gt Move gt Move Bee Be NN EE EE gt Copy gt Copy The selected shapes can be moved or copied in the scene A copy move operation allows to copy and move selected objects in one step thus reducing the number of pen interactions To activate the Move operation press
44. f two shapes In detail SketchAR provides the following constraints e Coincidence e Concentric center points axis for cylinders and spheres Page 46 of 90 SmartSkedehu E User Manual D18a 2004 10 07 Parallelism Tangential Perpendicular Fixation in world space Parallel Perpendicular Primitive gt Constr gt coincident Primitive gt Constr gt parallel Primitive gt Constr gt perpen dicular Primitive gt Constr gt tangent Primitive gt Constr gt concentric Primitive gt Constr gt dimension Primitive gt Constr gt un fix Primitive gt Constr gt show free Concentric Dimensioning Fix Unfix Show Free P Santos A Stork To use Constraint operations press one of the according buttons in the Primitive gt Constraints menu on the PIP sheet To apply a constraint to geometries first select the geometries by picking them with the pen and pressing the first button then press the second button to apply the constraint and complete the operation Page 47 of 90 SmartSkadau s User Manual D18a 2004 10 07 P Santos A Stork Conc Design Sliding Sliding Joint Joint Block Conc with Hole Design Block with Hole Pen m i Block Conc Pen with Bolt Design Block with Bolt Disc Conc Pen Design gt Disc Pivor Conc Lever Design gt Pivot Lever Upright Conc Pen Bar Des
45. g Editing Operations Geometry Creation Geometry Modification Supporting Operations Network Collaboration P Santos A Stork Page 3 of 90 SmartSkada s User Manual D18a 2004 10 07 P Santos A Stork Content 1 INTRODUCTION counts lll eee we nic ou oe N EE 8 2 HARD AND SOFTWARE REOUIREMENTS ees ees ee ske ske eke eine 9 3 DVD CONTENT WEN 10 4 SKETCHAR HARDWARE SETUP ccccccccsscesccnecnscenscnscessonscnseensonscnseenees 11 4 1 eege 11 4 1 1 DICECOVISION EE 11 4 1 2 K sse ss es ER ee ee ee 12 4 1 3 zxusmcnted Realty UD ais 13 4 2 Bra dE EE OE EE RE 14 d 3 Poterie aon Eege 16 S SOFIWARE SE EUR ace esc NE NEER GN RE RR We ONSE ES ENE RR ORE NEE EE ED one 18 5 1 SKetchAR Software Architecture iese sesde sesse ede GR N EN Ewe GE OE ode Ke ESE de 18 5 2 AMIL Conticuration dd EE 19 3 241 Calibra TS A RO Dee RR OER OE GR 21 S22 Cambra H EE 22 5 2 3 SS Cle FC O E RR UR PR RR 23 5 3 User Kif CONG WE ATO e 24 5 3 1 DEVICE steep RE OE EN 24 3 29 2 Bal RE N 24 AN Re POPE N EE ee 24 E AUVE DOTO EE Um 25 SEDEER BASE oa 25 6 MULTIMODAL INPUT METHODS kke ske Ek nennen nnne nnn nnn 26 6 1 Pen bi Se CU m M 26 6 2 Tracked POW ACG EE AE A 26 6 3 PIP Personal Interaction Panel zeessen geebeesskessuegee deeg Enge EENS 27 6 4 Pie Mena Rine Menu P M
46. he PIP Personal Interaction Panel sheet is a virtual menu from which the user is able to choose SketchAR functionality from It has its match in the real world in form of a tracked Plexiglas artifact The PIP sheet is operated by the Pen Cyberstilo As a result of the second round of usability tests D11 the PIP has consequently been supplemented by a pie menu methodology which is described further below However it remains active for users who prefer it Figure 13 PIP amp Pen Page 27 of 90 SmartSkadaus User Manual D18a 2004 10 07 P Santos A Stork 6 4 Pie Menu Ring Menu The ultimate aim of any good user interface design for immersive environments is to reduce the number of interaction devices to as few as necessary For this reason we have developed and implemented dynamically configurable pie ring menus to reduce interaction with the PIP and allow selection of operations with the pen only Pie ring menus operate as follows Each section of the menu may be an operation or another pie ring sub menu The user selects a specific section by implicitly moving from the center of the pie ring menu to that section If the section is another pie ring sub menu this menu will appear subsequently otherwise an operation is triggered Pie ring menus are implemented in two flavors e Static pie ring menus They display a texture reflecting the pie section s operation or sub menu e Animated pie ring menus They display an animated
47. he SketchAR software by showing how to edit the two SketchAR configuration files Three setups are discussed in this chapter the active stereo the passive stereo and the desktop configuration You find sample configuration files in Annex B 5 1 SketchAR Software Architecture SketchAR consists of three layers Figure 7 In the first layer the visualisation takes place Operations related to the visualization library OpenInventor are implemented there The main layer contains the actual intelligence of the functions Here the user input is processed to create and modify geometries The lower layer is the connection to the CAD library ACIS Modeling Kernel Figure 7 SketchAR Architecture Due to the strict separation of the layers the visualization and CAD library can be exchanged by other products with reasonable low effort Following an OO approach using UML diagrams we wanted to ensure that all functions fit into a comprehensive framework The system s conceptual key entities are e Users e Shapes e Operations Each instance of the user class represents one user currently using the system The system is designed to support multiple users working together at one place or spatially distributed Shape is a super class of all shapes being created with the system Different types of shapes such as curves and surfaces are represented as sub classes Page 18 of 90 SmartSkada s User Manual D18a 2004 10 07 P Santos A Stor
48. hird button go to the submenu Edit select the Undo or Redo button and release the third button The gestures used for multiple Undo Redo are respectively a bold triangle for Redo and a bold rectangle for Undo recognized by CALI The Synthetic Speech Output confirms the Undo or Redo operation with undoing operation or redoing operation 7 5 5 Material Editor Color Edit gt black black Selection Material white white Editor etc etc 7 colors 7 colors Stylists use different colors and materials to emphasize the character of the concept drawing We implemented a small material editor to assign arbitrary colors as well as material properties like transparency to shapes Page 59 of 90 SmartSketchey E User Manual D18a 2004 10 07 P Santos A Stork There are two ways of working with the Material Editor e Selection of a new material which entails that newly created shapes get the selected material e First select a shape and then change its material properties by selecting a new material To change the default material just use one of the buttons in the Material Editor submenu in the Edit menu on the PIP sheet To change the Material of one geometry select the geometry using the select operation use one of the buttons in the Material Editor submenu and deselect the geometry While drawing any shape you are able to choose one of the material properties of a shape namely the color by simp
49. ign gt Upright Bar Simple Conc Pen Block Design gt Simple Block Drilled Conc Lever Bar Design gt Drilled Pen Lever Bar Bolted Conc Lever Bar Design gt Bolted Lever Bar Pen To create a component in SketchAR select one of the scene modeling and assembly operations pressing the according button in the Conceptual Design menu on the PIP sheet To start the scene modeling operation press and hold the first button of the pen and position the component moving the pen Release the first button to complete the Scene Modeling operation Constraint operations may help you to create assemblies in VR or AR An additional operation is the resize operation for the components Just select the Resize operation in the Scene Modeling menu To use the Resize operation pick an existing component with the pen press and hold the first button and resize it moving the pen Release the first button to complete the Resize operation Page 48 of 90 SmartSkada s User Manual D18a 2004 10 07 P Santos A Stork Resize Design gt Resize With the feature based approach users are supported to define the parametric interdependencies between the parts of an assembly For this features and parameters are mapped into a net of geometric constraints that are solved whenever the user interacts with the model by either inserting or modifying parts or constraints The semantic filter is to restrict the view on the
50. isting interaction methods which range from pen input to draw in 3D as well as in 2D table top projection to short cut gestures for editing commands to Speech I O and two alternative menu types to choose functionality from In contrast to much work done in this area SketchAR not only features a few input methods but tries to combine different forms of interaction in a seamless and user friendly way 6 1 Pen Interaction The pen is the most important input device It is used for Sketching geometry creation in space Menu interaction Performing gestures Picking and Dragging Editing Figure 11 Cyberstilo The wireless CyberStilo has three buttons e The first button is the one closest to the tip of the pen It is for interacting with widgets on the PIP and starting line and surface creation operations for example e The second button in the middle is used for certain two state operations which require an explicit finishing action like implicit spline extrusion for example e The third button is used to activate the pie menu 6 2 Tracked Artifact Tracked artifacts are the real world representation of virtual objects One can move the virtual objects e g the mirror plane or the car by moving around the corresponding tracked artifacts in the tracked space Page 26 of 90 SmartSkadaus User Manual D18a 2004 10 07 P Santos A Stork Figure 12 Tracked Model Artifact 6 3 PIP Personal Interaction Panel T
51. k Operations represent the actions preformed to create or change a shape For instance the creation of surfaces or mirroring a curve with a mirror plane are operations Note that the operations are user specific so each user can have activated a different operation To create geometries the operations follow a strict pattern regarding the communication between the layers When the user starts an operation the operation is initialized Then while operating a preview of the operation result is displayed To achieve high speed preview the CAD kernel is not invoked at that time After the user finished the operation the operation data is processed and the CAD kernel is called to create the accurate geometries Finally the preview is exchanged by a visualization of the CAD kernel result Do preview Figure 8 Abstract execution flow of operations 5 2 XML Configuration File Here an XML Fragment from the SketchARActiveStereo xml is shown to explain the XML sections StbSink station 7 gt Merge lt MergeDefault gt lt Pen gt lt EventVirtualTransform rotation 0 1 O 1 57 translation 0 053 0 261 0 058 lt EventTransform scale 0 001 0 001 0 001 translation 0 055 0 515 0 01 ARTDataTrackerSource number 6 gt EventTransform lt EventVirtualTransform gt MergeDefault lt MergeButton gt lt NetworkSource number 2 multicast address 224 100 200 101 port 6667 gt lt MergeButton gt
52. lTransform gt e You can add lt EventTransform gt tags if necessary for the post fine tuning Page 21 of 90 SmartSketches User Manual D18a 2004 10 07 P Santos A Stork 5 2 2 Calibrating the PIP the zero point of the TY object is in this marker eem Figure 10 PIP Calibration e The point of rotation of the PIP is the centre point of the panel e Ifthe orientation at calibration was different from the one suggested above in the initial SketchAR artifact arrangement picture let the rotational offset be p q r S Let the position offset between the zero marker and the centre of the panel be x y z We need to shift the coordinate system first from the marker to the centre of rotation We translate the coordinate system within the panel s LOCAL coordinate system EventVirtualTransformation translation then we rotate the panel EventVirtualTransformation rotation into the desired SketchAR initial orientation We need the following script for the transformation EventVirtualTransform rotation p q r s translation x y z NetworkSource number 2 multicast address 224 100 200 101 port 12346 DEF pipo gt lt EventVirtualTransform gt e You can add EventTransform tags if necessary for the post fine tuning Page 22 of 90 SmartSkada s User Manual D18a 2004 10 07 P Santos A Stork 5 2 3 Speech Recognition An additional part of the XML configuration is the configuration of Speech
53. ly saying using speech recognition and receiving synthesized speech feed back for each command Figure 44 Material Editor If you would like to define your own material SketchAR defines the Color Editor You find it in the Material Editor menu pressing the ColorEdit button Use the sliders to define red blue green transparency and shininess of your material In the lower left corner of the PIP you find a preview of the material Use the Apply button to apply the material to your geometry 7 6 Supporting Operations In this section we present a number of useful tools and operations that help users to accurately draw and position shapes Page 60 of 90 SmartSkada s User Manual D18a 2004 10 07 P Santos A Stork 7 6 1 Picking Implicitly Pen supported feature As stated in the chapter on input techniques SketchAR uses an intelligent picking that privileges objects with lower topological priority first vertices curves faces To pick a geometry part in SketchAR just move the pen near the part How near you have to move it depends on the pick radius You can adjust the pick radius using the slider in the Tools Pick Radius submenu on the PIP sheet The Pick Radius is represented as a sphere which can be shown and hidden using the show hide button next to the slider 7 6 2 Snapping Activate Snapping using the Snap button in the Tools menu on the PIP sheet 7 6 3 Workplane Workplane Tools Tools Pen
54. n the previous implementation In addition SketchAR visualizes an approximate preview of the created surface during the sketching phase To use the Coon Patch Select operation and create a surface of existing NURBS curves press the Coon Select button in the Surface menu on the PIP sheet or activate the pie menu using the third button go to the submenu Surface select the Coon Patch Select Surface and release the third button To create the surface pick the NURBS curves with the pen press the first button to select them one after another press the second button to create the surface and complete the Coon Patch Select operation To use the Coon Patch operation press the Coon Patch button in the Surface menu on the PIP sheet or activate the pie menu using the third button go to the submenu Surface select the Coon Patch and release the third button To create the surface press and hold the first button to create the surface Release the first button to complete the Coon Patch operation 7 3 6 Primitive Creation SketchAR supports the creation of many different primitives To create primitives select the Primitive menu on the PIP sheet Then you can choose between two sub menus Shape1 and Shape2 Shape1 consists the following primitives Page 44 of 90 SmartSkadau s User Manual D18a 2004 10 07 P Santos A Stork Sphere Primitive Pen gt Shape1 gt Sphere Quad Primitive Pen gt Shape1 gt Quad Cylinder Primitive
55. necting and Disconnecting to from SketchNET SketchNET connect Speech Connect lt username gt ae network keet ae le SketchNET userlist aliu Userlist users online SketchNET connection is activated and deactivated by speech commands If a local user wants to connect to SketchNET he says connect lt username gt where username is his name and the name he will be known to SketchNET At the same time all events will be subscribed SketchNET connection is deactivated if a local user says network disconnect He will unsubscribe from all events and log off SketchNET SketchNET provides a userlist of users online it the user says userlist Page 70 of 90 SmartSkdda E User Manual D18a 2004 10 07 P Santos A Stork S Annex A References 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 NVidia Corporation USA HomePage http www nvidia com page quadrofx_family html ATI Corporation USA HomePage http www ati com products fireglz1 Advanced RealTime Tracking GmbH Herrsching Germany HomePage http www ar tracking de 3D Connexion Company USA HomePage http www 3dconnexion com Fraunhofer IGD A2 Industrial Applications Darmstadt Germany HomePage http www igd fhg de igd a2 index html StereoGraphics Corporation USA HomePage http www stereographics com Barco Corporation Belgium HomePage
56. olean Boolean Boolean Intersect Realtime Boolean Realtime Boolean Realtime Boolean Intersect Primitive gt Bool gt Bool Primitive gt Bool gt Bool Primitive gt Bool gt Bool int Primitive gt Bool gt Real Primitive gt Bool gt Real Primitive gt Bool gt Real int Boolean operations can be applied to all geometries in SketchAR Additional Real time Boolean operations are supported that let the user move one geometry into another one while seeing the boolean result in real time Figure 43 Boolean Intersection To activate a Boolean operation in SketchAR select one of the Bool operations pressing the according button in the Bool submenu in the Primitive menu on the PIP sheet Page 56 of 90 SmartSkada s User Manual D18a 2004 10 07 P Santos A Stork To apply a Boolean operation to geometries select two geometries by picking them with the pen one after another and pressing the first button After selecting the geometries press the second button to apply the operation To activate a Realtime Boolean operation in SketchAR select one of the Realtime Boolean operations pressing the according button in the Bool submenu in the Primitive menu on the PIP sheet To apply a Realtime Boolean operation to geometries select one of the geometries by picking them with the pen and pressing the first button Hold the first button and move the geometry into the other one A pr
57. ometry Every conflict of the created geometry with the package model will be displayed 7 3 Geometry Creation There are two ways to create geometry in SketchAR Page 36 of 90 SmartSketchey E User Manual D18a 2004 10 07 P Santos A Stork e Implicitly Create curves and surfaces in a single interaction process e Explicitly Select existing curves to create surfaces The following section explains geometry creation in SketchAR 7 3 1 Curve Creation Free Curve Curve Sketch Free Free Sketch Sketch Closed Curve Sketch Closed Sketch gt NURBS Curve Freehand Curve gt d spine Spline ME EE IR Polyline SketchAR supports the creation of free form curves lines and polylines Free form curves are easily created by pressing the pen button and move the pen in free space The created curve is internally converted into a spline curve The user can choose two kinds of free form input e Polyline The first draws a polyline in the preview and creates the spline curve after button release e NURBS curve The second calculates the spline in real time and also displays the control polygon while drawing Figure 24 Curve Creation To create curves in SketchAR select one of the Curve operations pressing the according button in the Curve menu on the PIP sheet or activate the pie menu using Page 37 of 90 SmartSkada s User Manual D18a 2004 10 07 P Santos A Stork the third button go to the submen
58. one axis by just picking the arrow at the tip of the axis The dragger handles 3D events from the pen and maps them to the according axis A rotation can be performed by picking the rotator of one axis and by using the pen just like a screwdriver rotating it around its main axis Only the rotation around the main axis is mapped The other rotations are not considered The rotation will be performed on the dragger and the related object Our slider to adjust parameters can be adjusted continuously or incrementally by using the arrows at both ends of the slider When using the slider to adjust the parameter the slider maps the 3D events of the pen to the axis The arrows at both ends of the slider body increment or decrement the current value by a predefined step Draggers and sliders are widgets which can be placed in free space They can be used to manipulate for example control points or parameters fast and intuitively They are especially useful in package creation where the user can control package parameter like overall car dimensions chassis length and width dimensions or encumbrances and other package constraints Page 31 of 90 SmartSkadaus User Manual D18a 2004 10 07 P Santos A Stork In addition they are also used to position mirror and work planes accurately in 3D space Figure 18 Dragger Figure 19 Slider 6 8 3D Picking amp Snapping Picking and Snapping is a feature used throughout most of the availabl
59. ots of control points becomes a difficult and maybe frustrating task The idea of oversketching is to change the shape of a surface according to a curve sketched on top of it In SketchAR a surface created by a Net Surface operation can be modified by sketching additional lines over and over the surface again Figure 41 Figure 41 Net Surface Oversketching Net surface oversketching is implemented by successively using our mapping and sorting algorithm for input strokes to feed the ACIS net surface functionality Page 54 of 90 SmartSketefe Z User Manual D18a 2004 10 07 P Santos A Stork 7 4 5 History based surface modification Implicitly supported functional Ity In SketchAR each shape object stores all operations that were applied to it In addition each shape object knows all its descendant shapes that were created by duplication etc Any modification applied to the original shapes will be propagated to their descendants which causes re computation of the operations that led to them Let a surface be generated explicitly from previously drawn curves If subsequently one of the sections is edited for example by oversketching or control point manipulation then this change will also be reflected in the surface and the surface changes accordingly Figure 42 History based surface modification Page 55 of 90 SmartSkeachea User Manual D18a 2004 10 07 P Santos A Stork 7 4 6 Boolean Operations Bo
60. r Primitive Trunc Cone gt Shape2 gt CircTrunc Cone Circular Primitive Trunc gt Shape2 Pyramide gt Cire Trunc Pyr Ellipse Primitive Trunc Cone gt Shape2 gt Ellip Trunc Cone Ellipse Trunc Primitive Pyramide gt Shape2 gt Ellip Trunc Pyr Page 74 of 90 SmartSkeachea User Manual D18a 2004 10 07 P Santos A Stork Sliding Joint Conc Design gt Sliding Block with Hole Design gt Block Block with Bolt Design gt Block with Disc Conc Pen Disc Pivor Lever Conc Pen Design gt Pivot Lever Upright Bar Conc Pen Design gt Upright Bar Simple Block Conc Pen Design gt Simple Block Drilled Lever Bar Design gt Drilled Lever Bolted Lever Bar Design gt Bolted Lever Resize Conc Pen Design gt Resize Semantic Context Filter On Off Specific Semantic Filter gt On E A o EN woa 1o o T Te Points EditCPs Sue IO IO IO sketching SE E Boolean Primitive gt Bool gt Bool Boolean Primitive Pen ee gt Bool Intersect gt Bool gt Bool int ames oma S 7 Boolean gt Bool Page 75 of 90 SmartSketfes E User Manual D18a 2004 10 07 P Santos A Stork pe Realtime Primitive Boolean Bool Real Realtime Primitive Boolean gt Bool gt Intersect Real int Color it black white Selection Material etc Editor 7 colors een Whees Wheels UI Adjust ment Wheels
61. racking and rendering hardware to run In this chapter the setup of the currently used optical tracking system is explained and the user is given information on which alternative tracking systems can be used Furthermore the various scenarios in which SketchAR can be used are detailed namely the back projection virtual table and augmented reality setup e Software Setup The software setup section mainly deals with the configuration of SketchAR for the three above mentioned scenarios The calibration of tracked artifacts and the many ways of editing and merging tracking input are explained in detail e Multimodal Interaction Techniques This section introduces the user to the main interaction techniques used in SketchAR which are usage of the tracked model artefact the pen interaction the two alternate menu forms PIP and PIE menus as well as speech I O and gesture based interaction techniques draggers sliders picking snapping and the usage of the spacemouse while in four view mode e Functionality This section covers all currently available features specifying which multimodal input techniques will invoke which functionality The features are grouped according to the following feature areas o File Operations Package Model Editing Editing Operations Geometry Creation Geometry Modification Supporting Operations Network Collaboration OQ 00000 The SketchAR Technical Report D18b elaborates on the user manual to describe
62. reate as many curves as you like Release the first button to complete the current curve and press and hold it again to create the next curve After creating all the curves press the second button to create the surface and complete the Net Surface operation Page 43 of 90 SmartSkdda E User Manual D18a 2004 10 07 P Santos A Stork Coons Patch from one 3D stroke Surface Coon Patch Select Surface Surface Coon gt Coon Patch Coon Select Select Coons patches from one 3D stroke is a way for creating complex free form shapes in a intuitive way a closed non self intersecting contour is covered using a Coons patch The mathematical concepts behind the surface generation which requires four oriented curves for the interpolation are completely handled by SketchAR in a transparent way The intuitiveness of the interaction method is reached by performing some reasoning on the input data the outline of the 3D stroke is automatically splitted into two pairs of oriented curves The resulting shape of the Coons patch heavily depends on the position of the split points This function has been improved in SkectchAR to provide surfaces that better conform to the user s expectations Starting form the splitting points using a simplified bounding box method the algorithm searches the respective neighborhoods relative maximum values of the first derivative along the curve Using such points avoids undesired wrinkles which appeared i
63. stribution immersive interaction framework 16 Opentracker distribution unified tracking system interface 17 Xerces distribution XML Parser 18 MS Speech SDK Speech I O 19 Spacemouse drivers four view support 4 Please see Annex A for references to all described hard and software Page 9 of 90 SmartSkdda E User Manual D18a 2004 10 07 P Santos A Stork 3 DVD Content The content on the DVD D17 is presented in more detail The main directory contains a general README file and three directories e Docs e SketchAR e Stb DOCS In the Docs directory the user finds documentation on SketchAR such as this handbook as well as the SketchAR Technical Report STB In the Stb directory the user finds all supporting software in general runtime DLLs that does not have to be installed and will run from this directory once SketchAR is started The only exception here is TGS Open Inventor which has to be downloaded and installed for the machine where SketchAR will run and the Microsoft Speech SDK if a user wants to use Speech I O functionality in SketchAR SKETCHAR In the SketchAR directory the user will find all necessary files to execute SketchAR as well as the binary itself in decompressed form Batch files have been created as well as sample configurations for the different setup types as there are Passive Stereo Back Projection Active Stereo Table Projection and Desktop usage as well as
64. t build a speech event firing source node with a specific command set Page 23 of 90 SmartSkdda E User Manual D18a 2004 10 07 P Santos A Stork 5 3 User Kit Configuration The userKit is a config file in OpenInventor format In this file you are able to configure Input devices of all users pen amp PIP and set the configuration of rendering output like stereo Camera model Video background The Devices are the same in all our configurations active passive stereo and desktop Only the DisplayKit settings vary from configuration to configuration 5 3 1 Devices All tangible user devices have a virtual match which can be configured in the following UserKit iv file UserKit userID 8 in Desktop and 10 in Stereo Mode userID 8 pen Benk station 7 geometry File name graphix smartPen3 iv pip PipKit station 1 pipSize 0 3 0 2 0 01 sheetAreaUpperLeft 0 05 0 05 sheetAreaLowerRight 0 95 0 95 offset Transform translation 0 0 O rotation 1 O O 1 57 geometry File name DefaultPip iv 5 3 2 Display The DisplayKit settings vary according to the rendering configuration that the user chooses be it a desktop configuration a passive stereo or active stereo 5 3 2 1 Desktop For autostereo displays the desktop configuration file is used to configure SketchAR display DisplayKit stereoCameraKit File name defaultCameraKit iv station 2 headlight TRUE headlightIntensity 1 0 backgroundColor 0 0
65. tart taping actions stop to stop taping Taping is a well known technique designers use to create and modify characteristic lines of a model on a white board In SketchAR this technique has been translated into a virtual taping feature and been improved by adding a new finger tracking module to OpenTracker enhancing the realistic behavior of virtual tape drawing Page 39 of 90 SmartSketebey E User Manual D18a 2004 10 07 P Santos A Stork Figure 27 Virtual Tape Drawing While taping the user is in an eyes busy hands busy situation since he uses both hands for his task Therefore speech input and output is used to switch between drawing and not drawing as well as choosing a color for the line and optionally its thickness The advantage again is that the user can focus entirely on his task and his view is not obstructed by pop up menus or interaction devices Speech in output for Tape Drawing Virtual tape drawing is activated by pressing the Tape Draw button in the Curve menu on the PIP sheet Each of the users tracked forefingers is matched by a small virtual sphere 3D cross Between the spheres there is the virtual tape Figure 28 Virtual Tape Drawing Finger Tracking The left forefinger is used to glue the tape on the imaginary wall and the right forefinger is used to control the tangent Now the user may start by saying the colour he wants to use e g black white and then the user says
66. technique presented before the curves need to be represented as splines Conversion from point sets to splines is also achieved using the ACIS library To combine the oversketch curve with the destination curve the destination curve is extruded to a surface The extrude path has to be parallel to the constraint plane and can either be vertical or horizontal as shown in Figure 38 curve extrusion horizontal extrusion surface destination curve vertical extrusion surface Figure 38 Two possible curve extrusions The selection between vertical and horizontal influence is done by the user In this way he selects which side profile stays unchanged Then the oversketch curve is projected onto the surface of extrusion The result of this projection is the oversketched curve The sequence in Figure 51 explains the steps from the user s point of view The user starts with sketching a curve of the hood in side view He switches to top view He makes an oversketch defining the side part of the hood The resulting oversketched curve is created The user switches back to 3D view to see the result rese O Page 52 of 90 SmartSketebey E User Manual D18a 2004 10 07 P Santos A Stork Figure 39 Constrained Oversketching 7 4 3 Editing Control Points Edit Edit Pen Control EditCPs Points Nowadays a vastly used tool for curve and surface modification in CAD CAS software is control point modification Although
67. tential gestures As a side effect of that implementation the third button originally used to trigger gesture recognition could now be used to activate the pie menus To offer implicit gesture support the Studierstube event loop had to be intercepted so gesture recognition would be active with any operation anytime and react to gestures when it made sense The decision when to start and stop the Cyberstilo positional information capture is done based on the average speed of the movement of the pen A gesture is in general initiated with a rapid movement and ends with a total halt of the pen Therefore automatic capture of pen positions is triggered when exceeding a certain threshold speed and stopped when going below another threshold speed The resulting list of pen positions is then sent to the 3D extended CALI Gesture recognition engine which analyzes the sequence and outputs the gesture found Page 30 of 90 SmartSketchey E User Manual D18a 2004 10 07 P Santos A Stork Figure 17 Gesture based Input 6 7 Draggers amp Sliders In some cases the 6 DOF of 3D input devices provide too much freedom to the user 3D Widgets have proven useful to enable 3D interaction with 2D input devices Following the idea of 3D widgets we have extended OpenInventor by 3D event enabled 3D widgets that map 3D events to lower degrees of freedom Our dragger can be moved arbitrarily in space by picking the centre sphere or it can be moved along
68. the necessary update frequencies around 120Hz or passive stereo vision requiring two separate independent graphic outputs Page 11 of 90 SmartSkadaus User Manual D18a 2004 10 07 P Santos A Stork Active Stereo Displays In this case frames for right and left eye are multiplexed on one output port of the graphics board This requires the board and the display to support high refresh frequencies which should be around 120Hz so each eye will be able to visualize its content with 60 Hz Generally active stereo displays require users to wear shutter glasses because they have to actively switch between left and right frames The display types may be monitors or projectors Passive Stereo Displays Here graphics boards will render left and right frames in parallel thus in general sending their output to two regular mono projectors which will then overlay their projection images as to provide a stereo view using pol filters in front of their objectives and requiring the user to use pol filter glasses as well 4 1 2 Virtual Reality Setup For styling the output device should provide high quality images presence in the virtual workspace and immersion We decided to use a semi immersive table like display with a diagonal of 1 7 meters which allows creating e g parts of a car body in scale by appropriate hand gestures and arm movements In stereoscopic mode the user wears a tracked pair of shutter glasses and the scene is rendered a
69. the Move button in the Edit menu on the PIP sheet or activate the pie menu using the third button go to the submenu Edit select the Move button and release the third button To use the Move operation pick a shape with the top of the pen press and hold the first button and move the pen Release the pen to complete the Move operation To activate the Copy operation press the Copy button in the Edit menu on the PIP sheet or activate the pie menu using the third button go to the submenu Edit select the Copy button and release the third button To use the Copy operation pick a shape with the top of the pen press and hold the first button You can copy and position the duplicated shape by moving the pen Release the first button to complete the Copy operation 7 5 3 Delete EE All gt Erase delete All Delete Edit gt Edit gt Rectangle deleting en EEr ER Select Select Selected objects can be deleted from the scene To delete all shapes in the application use the button Erase in the Edit menu on the PIP sheet or activate the pie menu using the third button go to the submenu Edit select the Delete All button and release the third button To delete only several shapes use the Delete Select operation To activate the Delete Select operation press the Delete Select button in the Edit menu on the PIP sheet or activate the pie menu using the third button go to the submenu Edit select the Delete Select button and release the thir
70. u Curve select the according button and release the third button To start the curve operation press and hold the first button of the pen and draw the curve Release the first button to complete the Curve operation 7 3 2 Eraser Pen During the styling process the designer usually makes different sketches until the shape has the desired appearance In traditional CAD systems and in general in 2D design environments the user takes a rubber tool to correct his drawings or he may change the position of control points to modify a curve Also an undo function is commonly used to correct errors or retry an operation This means that in traditional tools the user has to change modes between drawing and deleting The idea of the new interaction metaphor eraser pen is to combine the creation and deletion process in just one tool Moving forward the user creates a stroke by just reversing the direction he is deleting the stroke partially reverting again automatically bring him back into the drawing mode Figure 25 So drawing and deleting is seamlessly integrated See also tape drawing where a similar approach has been taken LE y aa l 4 mms K SA mh ke E K Figure 25 Eraser Pen In that way we have combined the creation and deletion process rewriting the rubber and pencil metaphor in just one tool The correction is done in real time if the pen inverts its direction for a second time the drawing pro
71. y affordable With this new function we provide a way to draw a curve on a plane which is arbitrarily positioned in 3D space using the table itself as drawing surface Using a tangible physical plane a projection plane is arbitrarily positioned in space When activating the pp2table function the projection plane is rotated to the table surface so that the designer can directly draw on the table Furthermore all geometry between the work plane and the user s head position is clipped away On the left Figure 59 you see the projection plane positioned in the scene On the right side you see the rotated view clipped at the plane position in orthographic view mode e d y Rotate plane to surface eg RT dn Zeg Figure 47 Work Plane to Table 7 6 5 Mirrorplane Mirrorplane Tools gt Tools gt Plane Mirrorplane Mirror Pen plane To create symmetric shapes we introduced a mirror plane concept The mirror plane can be used in two modes e Free positioning using a tangible physical plane or a dragger e Positioning it contrained to the center axis of the car Page 63 of 90 SmartSkadaus User Manual D18a 2004 10 07 P Santos A Stork Figure 48 Drawing in mirrored mode Shapes are immediately mirrored while creation time if the mirror is active This is implemented by activating a second creation operation that processes the mirrored pen positions events The Mirrorplane operation
72. ze the scene press the Model Toggle button in the Tools menu on the PIP sheet While the scene is frozen you are able to move the scene with a dragger To freeze unfreeze the view press the Head Toggle button in the Tools menu on the PIP sheet 7 6 8 Camera Zoom Move Camera Tools gt Zoom Move Camera Sliders In the immersive modeler the virtual camera is attached to the user s viewpoint That way the user can examine the model from all sides and can go nearer and farer away to zoom in and out However sometimes the stylist may need to change the view without moving the head for example an interior designer wants to have a view from the inside of the car Therefore we added support to pan the camera and zoom in out 7 6 9 Ortho View Ortho View View gt View gt Pen OrthoView OrthoView Page 65 of 90 SmartSketches User Manual D18a 2004 10 07 P Santos A Stork For curve input stylists often prefer to have an orthographic or mono view of the scene In SketchAR you can simply switch between those views Use the OrthoView button in the View menu to switch between mono and stereo View 7 6 10 Four View 4 Side View View View FourView FourView Stylists usually create four side drawings of selected concepts These drawings are used as a basis for the CAD modeler to create a 3D model Therefore we added a function that switches into a 2D orthographic viewing mode and displays the car from each si

PDF 3.4MB - SmartSketches

Contents

Download Pdf Manuals

Related Search

Related Contents