Home

Deliverable Template IMAT

1. Authoring Inde xing omposing scenario Figure 1 Scenario tool and relation with user and other tools The use of the scenario tool in the IMAT process is the following The user specifies the lesson scenario using the scenario tool He stores this scenario in the database of fragment The tool has linked to each instructor or learner action three instructional search indexes Fragment Description Type Material Use Knowledge Type and a topic index the X of being able to maintain system X He uses then the retrieval tool to access to the stored scenario For each instructor or learner action he can perform a fragment search and add some domain and technical index and select useful ones He uses its own authoring tool to construct instructional content using the fragments retrieved Version 2 Page 31 of 48 ESPRIT 29175 IMAT Ki R IV 2 Scenarios and guidelines version 2 As specified in section 10 an initial mark up must be performed manually before to be able to obtain relevant fragment to requests using instructional indexes 11 2 General tool description The scenario tool supports the user in defining a scenario by combining pre defined instructional patterns The scenario is gradually elaborated by selecting adapting or creating the scenario elements learning goal knowledge type instructional strategy instructional activities instructional steps and actions Depending on the choices
2. Task No Yes If not can you specify what is not 1 2 3 System image 2 Is anything e g a toolbar or an exit button represented at an unusual location Task No Yes If it is can you specify this 1 2 3 System image 3 Is there any action e g selecting a menu option on which you didn t receive feedback Did you do anything that had no visible result Task No Yes If there is can you specify this 1 2 3 System image 4 Is there anything that needs interpretation For example you cannot see what an icon or a button stands for you don t know what it should be used for Task No Yes If there is can you specify this 1 2 3 System image 5 Is anything represented in a way that it suggests an impossible action Does something look in a way that doesn t suit the underlying function For example a button is not disabled grayed out while it should be Task No Yes If there is can you specify this 1 2 3 System image 6 Is everything represented consistently Do two things look the same but have different meanings Task No Yes If not can you specify what is not 1 2 3 System image 7 Is everything represented consistently Is something missing while it is represented elsewhere e g in an equal or comparable window Task No Yes If not can you spec
3. Faster image retrieval Direct link between pictures Direct link between pictures and text The new AFPA requirements With respect to the AEI amp l tool To be able to see all the retrieved images using a panel of thumb images An image should appear without clicking on the CROP button A tool working without bugs and crashes To be able to search images on Content Keyword Right domain keywords used by the tool A tool easy to use for non English speaking people as AFPA trainers With respect to document analysis An Analysis Tool working on all the Technical Manuals from Renault following the manual ontology provided and or an new ontology if the structure is different The new ETRA I D requirements With respect to the AEI amp l tool To be able to edit create and delete the fragments and manage them to update the technical manuals With respect to document analysis The fragments need to be improved size and search terms With respect to the scenario s A standard lesson structure independent of the equipment and easy access and navigation to on line training material 12 2 2 General user requirements However users started with a larger set of requirements at the beginning of the project which were elaborated during their experiences with the tools and the IMAT concepts In the deliverables of the user workpackages WPI II and III these requirements were stated I
4. Summary Conclusion Once an instructional goal knowledge type instructional strategy and instructional activities are set and refined into instructor and learner actions a framework for an instructional scenario a standard lesson can be developed A final mapping from learning goal and instructional activity to fragment description type is made Version 2 Page 23 of 48 ESPRIT 29175 IMAT Ki R IV 2 Scenarios and guidelines version 2 9 7 Fragment description types Fragment description type is an attribute property of a fragment that reflects the perspective from which a fragment was written and it contains a description of the content of a fragment For example a fragment was written from a structural perspective describing a systems physical features and the content concerns a definition of a component The following top level fragment description types are identified Object of fragment is Structural overview System components System connections Functional overview System sub function Tasks procedures overview Type inspection Type preventive maintenance Type corrective maintenance Type subsidiary 9 8 Instructional scenarios Instructional scenarios are described below Each learning goal is mapped to fragment description type and instructional strategy The overall learning goal is Being able to maintain system x One level below that the learning goal Being able t
5. The questions about the lesson material the content questionnaire revealed that one respondent had spent 100 hours working with the tool including installation he gained 20 to 30 of his time creating instructional material 150 screens in Toolbook Another respondent had spent 16 hours with the tool while he would have needed 40 hours to create the instructional material 40 sheets in Powerpoint without the tool The third respondent did not answer the question Within the content questionnaire questions about the search and index vocabulary revealed that cannot search for image fragments based on the Domain index Part of the Domain Related vocabulary is not there There is noise in the Content Keywords Questions about the fragments revealed that there were Some irrelevant results e g asking for a fragment about the back of the seat reveals a fragment about the opening roof Version 2 Page 34 of 48 ESPRIT 29175 IMAT Ki R IV 2 Scenarios and guidelines version 2 12 2 User requirements 12 2 1 Updated requirements from the users After working with P1 each user partner has adjusted their original requirements making them specific with regards to the tools see Barnard et al 1999 Lorrain et al 1999 and Sebastian et al 1999 The new RNLAF requirements see for a detailed description Verstegen et al 2000 Search for parts of images based on Topic Select and cut parts of images
6. an instructional strategy is chosen specifying how to teach instructional activities are chosen specifying what actions the instructor and the learner should take in the training process instructional activities are refined into instructor and learner actions specifying individual actions as well as material use the way the material from the technical manual is going to be used in the training material the instructional scenario is filled in specifying a standard lesson template including what kind of fragments fragments with a certain description type specifying from which perspective the fragment was written should be retrieved from the database The user can use pre specified tables of learning goals knowledge types instructional strategies and instructional activities and instructional scenarios with a mapping to fragment description type In these tables it is indicated which combinations of goals knowledge types strategies activities actions and fragment description types are appropriate 3 Fragments from the technical manual are automatically indexed on high level description types The user can additionally index fragments on knowledge type and material use The user can import the fragments in an authoring environment Evaluation is an important part of the IMAT work The users RNLAF AFPA and ETRA worked with the first IMAT prototype tools from these experiences user requirements evolved which will hav
7. Choose an instructional strategy Determine instructional activities Define accompanying instructor and learner actions Select useful fragments from the database and transfer them to an authoring tool The instructional ontology or maintenance training ontology supports the actions that have to be taken to come from a high level scenario to instructional material by defining concepts and mapping them onto each other Products are syllabi presentations or CBT 9 1 Learning goal A learning goal describes the desired outcome of an instructional curriculum in terms of knowledge skills and attitudes that are necessary to carry out certain functions or tasks In the kind of technical training we are dealing with in IMAT most training is aimed at the following top level learning goals High level goal being able to maintain system x Being able to reason about system x Structural knowledge of system x hierarchical decomposition Functional knowledge of system x hierarchical decomposition Being able to perform all maintenance tasks within job To perform inspection task y list y1 yn Version 2 Page 16 of 48 ESPRIT 29175 IMAT oe R IV 2 Scenarios and guidelines version 2 Perceptual knowledge normal deviant appearance Declarative knowledge about safety procedure sequence reporting Skill in performing inspection To perform preventive maintenance task z list z1 zn Declarative knowledg
8. investigated and discussed In P1 there was no guidance available on organisational issues For P2 this guidance will be available in the organisational memory tool This tool will not be fully operational but will help users to organise their processes and to store the right kind of feedback 12 2 3 Further evaluation The user workpackages will start working with the P2 tools in order to produce training material which will be subsequently used in the training The outcomes of these evaluations will be gathered and analysed In the reports of the user workpackages R 1 3 R II 3 and R III 3 the specific experiences of the three different user organisations will be evaluated and discussed In Deliverable R IV 3 which will succeed this deliverable the experiences will be aggregated and compared with the requirements and expectations in this section Version 2 Page 39 of 48 ESPRIT 29175 IMAT Ki R IV 2 Scenarios and guidelines version 2 13 Bibliography and references Barnard Y F Riemersma J B J amp de Hoog R 1998 The use of IETMs in training Report TNO TM 1998 A074 Soesterberg The Netherlands TNO HFRI Barnard Y F Kabel S C Beije L Van Ooijen H 1999 Training requirements RNLAF IMAT deliverable R I 1 Barnard Y F Riemersma J Kabel S Desmoulins C amp Grandbastien M 1999 Scenarios and Guidelines version 1 IMAT deliverable R IV 1 Chen H 1995 A methodology for characterizin
9. is constructed Word PowerPoint Authorware Toolbook There are five kinds of relation between the technical manual and the instructional material 1 Separate instructional material is created by using the fragments from the technical manual The instructional material is separated from the manual and can be used stand alone For example a syllabus or a CBT A set of instructional fragments is created from the database of fragments from the technical manual The fragments are selected from the technical manual on the basis of predefined lesson structures lesson scenarios The lesson structure and the selected fragments can be stored The specific actual lesson material can then be created by using the fragments and further elaborating and adapting the standard lesson structure Instructional material is created with a direct link to the electronic technical manual In this case the instructional material can only be used in combination with the electronic manual From the instructional material dynamic links are made with the manual for example a PowerPoint slide with hotspots which are linked to the relevant section in the manual The instructional material is a didactic layer on the electronic manual In this case the manual is enriched for training purposes with notes advice assignments more elaborate explanations etc Another situation is feasible in which there is no real instructional material but the learner can search
10. of the user an appropriate selection is offered to user in each step For example if the user has determined an instructional strategy he she will be provided with the list of instructional activities which belong as described in this document to this strategy The tool itself determines the appropriate description type using the tables presented in this document 11 3 User interface requirements The following requirements have been made about the user interface The use of the software must be very simple and quick in order to provide an immediate result easy to browse and understand The tool must be flexible The user has then no obligation to use the predefined scenario elements and can create its own elements He can reorder the steps and actions of a pattern scenario He can instantiate actions with free text There is an infinite undo on the successive choices made For each scenario element learning goal knowledge type instructional strategy instructional activities instructional steps and actions a direct access to its definition is provided 11 4 Tool implementation To help the final integration of the IMAT tools the scenario tool is implemented using Microsoft Visual Basic From the beginning to the end of the tool use the scenario framework is displayed Then the user has at each moment a complete view of the scenario in progress displayed as a sheet in landscape format At each step of the tool use
11. there is can you specify this 1 2 3 Keystroke 4 Are there different ways using different keystrokes e g menu option and double click to obtain a single result e g open Task No Yes If there are can you specify these 1 2 3 Process aspects These questions are about series or sequences of user actions the way things are done Process 1 Is the order of actions right An example of a set of ordered actions is you first open a tool then load a file then edit the file and save it Task No Yes If the is not right can you specify it 1 2 3 Process 2 To attain a certain end e g add instructional index do you think too many actions need to be performed Could this end be attained more directly For example suppose that to save a file you d have to click save specify a name confirm the name and close the window These could be too many actions to perform the task of saving a file Task No Yes lf too many actions need to be performed can you specify these 1 2 3 System image aspects Version 2 Page 43 of 48 ESPRIT 29175 IMAT Ki R IV 2 Scenarios and guidelines version 2 These questions are about the appearance of the retrieval tool the way things are visualized and the consistency thereof System image 1 Is everything displayed on the screen e g the font visible and legible
12. users created training material by using fragments from the technical manual database They then further adapted the text and pictures added comments explanations animations etc using an authoring environment In the period the users worked with P1 it was not possible to go beyond this kind of production However users have come up with ideas to further integrate training and manuals for example allowing trainees to consult the database directly integrating training material used in the classroom and electronic documentation used in the workshop e the production of electronic material instead of paper based material and transparencies The users produced electronic material in different forms Instructors and trainees both have material available electronically For the coming months this process will be intensified The users will try to implement more multi media features hyperlinks and animations Not only the production of electronic training material has been worked on also the distribution and storage of material will be done electronically Users are currently working on setting up an electronic distribution and updating mechanism Because of the experiences with the first two prototypes users now have real life experience with electronic material and organizing the management of the material e better quality of material The users are busy with the process of improving the quality of the training material Requirements in this are
13. AE label EE 9 7 INSTRUCTIONAL MATERIAL TO BE CREATED WITH IMAT c0sscsssseseeees 11 BHIGH LEVEL SCENARIOG 0 ccccscccsssecccsseccsseeeseceeesenssesenseeuesseenaeeeeesseeesesenaees 12 Q9DEVELOPMENT OF INSTRUCTIONAL MATERIAL c cccsssccssscnssesensseensseeas 16 9 TL ARMING El UE 16 Di 2KNOWIEUGE Ty PO oars siese sets cc ese ec ce seat e asec aeaaea aaa eSa aaa anaa aa aaae aa aaa Eaa aan eah aaa cA a e daana eaaa 17 9 3instructional Sategy seinninn nananman aaan 18 9 4lnstructional activitieS asannnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnannnnn SEENEN ege 19 9 5Instructor actions and Learner ACtiONS cscceeeeecesnseeeeeeceeeenseneeecneeaeaseseseceeeanaenseseeeeaeaseeeees 20 OG Materiel US Circo EE 23 9 7FragMment description types cceceeeeceseeeeeeeeeee ee eeseeeeenseeee een seeee seen aneeeeeseeaeeeeneeeseenaeseeenseeeeenseceeeenees 24 9 8lnstructional SCON E 24 10 SEARCHING FOR FRAGMENTS IN THE DATABASE AND INSTRUCTIONAL ET 29 11 SGENARIO TOOL sissessscsisssitasiasiedtnsecdiaaisnendaeseswbiceodsnndeeasdceceeseceanesseceiadaceteceiatanandtesdeis 31 11 1 Use of the Scenario Tool in the IMAT proceSS sssssssuneennnnnnnnennnnnnnnunnnnunnnunnununnnnnnnnnnnnnnn nnna 31 11 2 G ralt old scriptioi s sess dee ences aaae aea e Seea aa aeara ge N AE aaaea aeaaaee aaae ians 32 11 3 User interface require Ment cceeccceceeeneeeeeeeeeeeseeeeeeeneeeeeeseeeeeeseeseseeeesee
14. IMAT ESPRIT 29175 4 oY Integrating Manuals and Training Scenarios and guidelines version 2 Author s Work package Deliverable Type Deliverable Number Responsible org Availability Contractual Date Actual Date Version Pages Author e mail Yvonne Barnard TNO HF Suzanne Kabel TNO FEL Johan Riemersma TNO HF Cyrille Desmoulins LORIA Monique Grandbastien LORIA IV Common Services R R IV 2 TNO P 15 11 00 27 11 00 Final 48 ybarnard tm tno nl Copyright 1999 The IMAT Consortium Abstract ESPRIT 29175 IMAT Ki R IV 2 Scenarios and guidelines version 2 In this document we define the ways in which the IMAT scenario s for maintenance training can be used effectively to develop training material A maintenance training vocabulary provides the building stones for a scenario while a scenario is a prototypical framework for training material The evaluation of the previous prototype tools resulted in the tools described here to support the user in creating training material Keyword List scenarios maintenance training ontology scenario tool evaluation Version 2 Page 2 of 48 ESPRIT 29175 IMAT Ki R IV 2 Scenarios and guidelines version 2 1 Table of Contents 1 TABLE OF CONTE IN GS EE 3 ZV ERSION CONT tel E 5 SEXECUTIVE SUMMARY EE 6 BINT RO DUG TION BEE 7 5PURPOSE OF SCENARIO DEVELOPMENT csssccsssccsseennecesseeenseenessesesenseseees 8
15. Or the fuel stream can be shown by letting a coloured stream go through the different pipes Hereto the developer selects from the database drawings and photos which show the component to be treated If the material is finalised it is printed out and given to the learners It is also possible to create computer based training programs for example by using authoring languages like Authroware and Toolbook using fragments from the technical manual as content In the second option a developer of instructional material prepares material to be further customised by a instructor or learner The instructional material is then constructed in two phases first the developer selects material from the database and next it is customised by the instructor who is responsible for the lessons This scenario is suitable for a centralised training institute where specialised people either domain specialists or educational experts take care of the production of instructional material For example standardised PowerPoint presentations are created This material is given to the instructors maybe in a training centre who have the freedom to customise the material This can be done by further adding features to the presentation by adding personal comments etc It is also possible that the learners have the possibility to change the material for example when they use electronic syllabi or workbooks They can change the material by adding their own annotations fill in an
16. Page 20 of 48 ESPRIT 29175 IMAT BS R IV 2 Scenarios and guidelines version 2 Construct Questions Ask Questions Explain Next lessons Show How a task is done Give Instructions Assignments Example Evaluate Own knowledge Answer Performance Diagnose Lack of knowledge Faults The following learner actions are identified Comprehend Learning goal Question Directions Summary Hints Instructions Assignments Accept Learning goal Memorise learn by heart Content presented Memorise Instructions How a task is done Read Explanation Listen to Explanation Study Explanation Feedback Illustration Relate Explanation to existing knowledge Recall Content Reason Answer Question Version 2 Page 21 of 48 ESPRIT 29175 IMAT BS R IV 2 Scenarios and guidelines version 2 Elaborate Apply Hints Acton Hints references Observe How a task is done Do Assignments Instructional activity is refined in terms of the combined concepts in the instructor and learner actions in Table 6 Instructional Activity Instructor and Learner Actions Setting goals Instructor Present the current learning goal Learner Comprehend and accept learning goal Illustration Instructor Present il
17. a are integrating text and pictures Version 2 Page 37 of 48 ESPRIT 29175 IMAT Ki R IV 2 Scenarios and guidelines version 2 being able to add commentaries and explanations making available for trainees different kinds of material or different ways of navigating through the material in order to cater for individual needs allowing trainees to navigate easily between training material and manual creating material suited for both training and work developing tests and evaluations in electronic format allowing to follow a signal or highlight a connection in schemes and diagrams making a clear distinction between hierarchical levels in the schemes and diagrams zoom in and zoom out on details The IMAT tools cannot directly meet most of these requirements because they are in the area of authoring training material The IMAT tools only allow searching and retrieving fragments from the manual interfacing with an authoring tool in which the training material has to be produced However by using the P2 IMAT tools developers are much better equipped so that they can start to work on these issues Within the IMAT project we expect to satisfy most of these requirements concluding the project with better quality training material It is still an unsolved issue how learners and engineers could navigate both through the IMAT database and the electronic manual itself although out of the scope of IMAT intere
18. al after comments from the partners Version 2 Page 5 of 48 ESPRIT 29175 IMAT Ki R IV 2 Scenarios and guidelines version 2 3 Executive Summary In this document we define the ways in which the IMAT scenario s for maintenance training can be used effectively to develop training material A maintenance training vocabulary provides the building stones for a scenario while a scenario is a prototypical framework for training material The evaluation of the previous prototype tools resulted in the tools described here to support the user in creating training material In the organisations addressed by IMAT dealing with training production of training material maintenance tasks operation of complex technical equipment and technical manuals three main categories of stakeholders can be distinguished learners instructors and developers of instructional material Scenarios are developed and used as follows 1 A high level scenario is determined This is an organisational choice Six high level scenarios are predefined describing the use of manuals for the creation of instructional material and in training specifying the type of training and the formality of training 2 Depending on the high level scenario the instructional design for maintenance training is further specified alearning goal is set specifying what to teach aknowledge type is chosen specifying what kind of knowledge the learner should gain
19. aled that with respect to the keystroke aspects users found that Itis not clear how to cut text or an image Itis not clear how to select part of an image Itis difficult to select an attribute by double clicking With respect to process aspects users found that To see a picture you have to CROP or FRAME first want to see the picture or the query result right away The following system image aspects were noted Itis not clear what the DONE button in the MARKUP window stands for OK is preferred The FRAME CROP and MASTER COPY buttons have the same result There are no COPY and PASTE buttons It s hard to see what kind of media is selected text image or other Concerning navigation aspects users had the following remarks The MARKUP window and the FRAGMENT RETRIEVAL window are so similar that you don t always know where you are in the system When the system is searching for fragments it s not telling you that is busy With respect to mental model aspects users said the following expect to be able to copy and paste into Pagemaker and Toolbook expect to be able to select and cut parts of an image expect values in the fields LEVEL and ELEMENT in the FRAGMENT RETRIEVAL window expect to be able to search images on TOPIC expect to be able to search on TYPE and SCOPE prefer the terms ZOOM IN and ZOOM OUT above SCALE
20. an Diego USA Sebasti n V Garc a M Kabel S C 1999 Training requirements ETRA I D and Customers IMAT deliverable R III 1 Verstegen D Barnard Y F Kabel S C Van Ooijen H Staalstra J 2000 Interim report on use of P1 tools by the RNLAF IMAT interim report Version 2 Page 40 of 48 ESPRIT 29175 IMAT Ki R IV 2 Scenarios and guidelines version 2 14 Appendix A P1 AEI amp I tool evaluation form INQIM EEN Prototype VErSIONiccssiicewee nnciciceie cone The assignment This assignment is based on three tasks retrieve material from the database add instructional markup to a fragment and paste it into training material Each task is refined into a number of actions to be carried out by the respondents printed in boldface A user manual for the P1 tools is provided which can assist the respondent in carrying out the actions in the assignment Task 1 retrieve material from the database Run Jasmine Start the retrieval tool Connect to the server Select a context Retrieve a text fragment 5a Specify search criteria 5b Retrieve the fragment s Retrieve a picture fragment 6a Specify search criteria 6b Retrieve the fragment s Task 2 add instructional mark up to a fragment Take a text fragment and look at the Markup Add instructional markup 8a Add knowledge type 8b Add material use Repeat ste
21. can be laid down in formal or more informal ways the latter for instance in the form of personal notes Version 2 Page 15 of 48 ESPRIT 29175 IMAT Ki R IV 2 Scenarios and guidelines version 2 9 Development of instructional material In this section we concentrate ourselves on the development of instructional material The high level scenarios provide a description of the ways in which instructional material can be produced from fragments of the technical manual The instructional material produced in the high level scenarios can be described in a more formal and detailed way using the maintenance training ontology for an extensive description of the instructional ontologies and a definition of the terms used we refer to deliverable O2b Second Didactic Ontology It contains the following top level concepts Learning goal Knowledge type Instructional strategy Instructional activity Instructor and learner action Fragment description type These concepts are similar to the steps a developer of instructional material has to take in Instructional Design methods Gagn Briggs amp Wager 1992 and which are supported by instructional design tools such as Designer s Edge See reference The complete ontology is described in the next subsections To come from a given high level scenario to instructional material the following actions have to be taken Seta learning goal Determine the knowledge type
22. cenarios can be derived from the training requirements documents generated in workpackage l Il and Ill New innovative scenarios have to be derived from the expertise of the partners in the area of educational science and from exploring the possibilities offered by the IMAT tools We distinguish three levels of scenario descriptions 1 General free text descriptions of high level scenarios for the use of manuals for the creation of instructional material and in training 2 Instructional scenarios consisting of specific frameworks or building blocks to support the developer of training material and the user of instructional material These building blocks specify what kind of fragments the developer should look for and how to use these fragments in the further authoring and training process These building blocks can be combined into more complex scenarios 3 Instructional mark up to be attached to fragments of the technical manual in order to specify the kind of knowledge and instructional use of the fragments Version 2 Page 8 of 48 ESPRIT 29175 IMAT Ki R IV 2 Scenarios and guidelines version 2 6 Stakeholders In the organisations addressed by IMAT dealing with training production of training material maintenance tasks operation of complex technical equipment and technical manuals many stakeholders can be distinguished Stakeholders are learners instructors developers of training training material manuals manufactur
23. ck the right fragments from the database Version 2 Page 7 of 48 ESPRIT 29175 IMAT Ki R IV 2 Scenarios and guidelines version 2 5 Purpose of scenario development Scenarios describe the way in which the IMAT tools can be used to produce instructional material from a technical manual The purpose of providing IMAT users with scenarios is to support them in the development of instructional material and in the organisation of the training process There are several reasons for providing support in the form of scenarios To provide on the one hand standard ways of creating instructional material and organising training in order to make this process more efficient while on the other hand allowing for customisation according to the specific needs of a user To provide standard good practice ways of training especially for those instructors and developers of instructional material who are inexperienced or who have little background in educational science To show instructors and developers of instructional material how to set up new innovative ways of training To work towards further industrialising the process of training development in technical domains In order to fulfil these support roles we have to define scenarios in a standardised format giving users ample opportunity for customisation We have to take into account scenarios which are directly related to the current practices of the user organisations in IMAT These s
24. cts concern your ideas and expectations of the tool For example did you expect the retrieval tool to have this functionality The aspects either concern the retrieval tool the user or the interaction between those Keystroke Mental aspects model Process System Navigation aspects image aspects Each aspect is covered by a number of questions For each task you have performed in the assignment these questions have to be answered The tools questionnaire contains close ended questions with answer categories yes no and open questions to specify answers of closed ended ones Version 2 Page 42 of 48 ESPRIT 29175 IMAT Ki R IV 2 Scenarios and guidelines version 2 Keystroke aspects These questions are about menu options buttons mouse clicks etceteras Keystroke 1 Are there any intended actions like open or cut difficult or impossible to carry out using the menu options buttons mouse clicks and keyboard Task No Yes If there are can you specify these 1 2 3 Keystroke 2 Are there any actions like close or save you can perform with the menu options buttons mouse clicks and keyboard you are unfamiliar with Task No Yes If there are can you specify these 1 2 3 Keystroke 3 Is there an action e g double click that has different results e g select and open Task No Yes If
25. e about preventive maintenance Declarative knowledge about safety procedure sequence reporting Skill in performing preventive maintenance To perform corrective maintenance task z list z1 zn Declarative knowledge about corrective maintenance Declarative knowledge about safety procedure sequence reporting Skill in performing corrective maintenance Being able to do subsidiary tasks Topical knowledge Calibrate maintain maintenance amp diagnostic tools 9 2 Knowledge type Knowledge is information acquired and stored in an organised manner in the mind Knowledge type stands for the knowledge that is necessary for a learner to appropriate in order to gain the learning goal The following top level knowledge type concepts are identified Declarative Knowledge Perceptual knowledge Reasoning skills Procedural knowledge Performing skills Attitudes Some learning goals go well together with some knowledge types In Table 3 learning goal is mapped to knowledge type Percep Decar Reaso Proced Perfor Attitude tual ative ning ural ming JS knowle knowle skills knowle jskills dge dge dge Being able to reason about system x Structural knowledge of system x hierarchical decomposition x x Functional knowledge of system x hierarchical decomposition x X X Being able to perform all maintenance tasks To perform inspection task y li
26. e main requirement is of course the tools should run without bugs and crashes Other requirements in the usability area are to easily navigate through all the technical material available Version 2 Page 36 of 48 ESPRIT 29175 IMAT Ki R IV 2 Scenarios and guidelines version 2 be able to see all the retrieved images using a panel of thumb images be able to search fragments including images using keywords and topic to select and cut parts of images having direct links between text and pictures search terms should be meaningful and correct fragments should have the right level of size images should have a good level of visibility the possibility to adapt the vocabulary used in the tools for non English speaking users In P1 these requirements were not yet completely met It is the expectation that in P2 no big problems will be occur in this area thus allowing the user to produce training material in an easier way than before IMAT 2 Producing new kinds of training material There are four areas in which IMAT brings innovation to the users with regard to producing new kinds of training material 1 integrating training material and technical material 2 the production of electronic material instead of paper based material and transparencies 3 better quality of the material 4 structured material and standardized lessons e integrating training material and technical material The
27. e to be met by the second version of the IMAT tool IMAT can support the user organizations with new ways of producing training material producing new kinds of training material and provide new ways of organizing training Version 2 Page 6 of 48 ESPRIT 29175 IMAT Ki R IV 2 Scenarios and guidelines version 2 4 Introduction In this document we define the ways in which the IMAT scenario s for maintenance training can be used effectively to develop training material A maintenance training vocabulary provides the building stones for a scenario while a scenario is a prototypical framework for training material The evaluation of the previous prototype tools resulted in the tools described here to support the user in creating training material As the IMAT tools are under development it is difficult to envisage all kinds of possibilities they offer to users There is also outside the IMAT project and the associated work of its partners little actual experience with the use of electronic manuals in training Barnard et al 1998 Jorgensen et al 1995 Kribs et al 1996 Morris amp Dickason 1996 Although we see a great diversity of opportunities for the outcomes of the project we will have to limit ourselves in the creation of scenarios of use in order to ensure that we come up with practical but innovative solutions Scenarios should take into account the short term and the long term possibilities to incorporate technical manua
28. eeeeseeseeeeseeeeeeeseeseneneets 32 11 4 Tool implementation ceseeeceeseee ee eeeeee eens seen eee eeenee ene eeee see neeeeeseeeeeeseeseeesesaeeeeeseeseeeeesaeeneeesesenens 32 Version 2 Page 3 of 48 ESPRIT 29175 IMAT Ki R IV 2 Scenarios and guidelines version 2 12 EVALUATION bssscssinsssntidantsais caniacsananeddsentwestinlansmawewsauuecetiuavaneutueubiuirserenesauautwwasueadie 33 12 1 TOOIS CVAIUATION 0 cc cece ce eeeee eee eeeee eee eeeeeee tee eeeeeeeeeeeeeeeseeeeeeseneeseseeeeseneeeeeseeseeeeseseneeeesaeeeseeseneeeeseeneees 33 BC Setup DEE 33 T2 Be 33 12 2 User requirement cseccccesseeeceeesneeeeessneeeeeseneeeseeennee sees seeseeeeseaeseeessneeseessneeeseesaneeseensaneeseseeneee 35 12 2 1Updated requirements from the userg nnns nnn 35 12 2 2General user reqUIFEMENS cccecceceeeeeeeeeceeeeeeeeeaeeeeaeeceaeeeeaaeesseaeeseaeeesaaaeseeaeeesaeesaeesseaeaeeaes 35 12 2 SF inn EE TEE 39 13 BIBLIOGRAPHY AND REFERENCEG 00 ccccccceeeeeceeeeeeeeeeeeeeeneeseeseneeeeneeeeened 40 14 APPENDIX A P1 AEI amp l TOOL EVALUATION FORM ccccceeseeeeeeseeeeeeeeeeee 41 Version 2 Page 4 of 48 ESPRIT 29175 IMAT Ki R IV 2 Scenarios and guidelines version 2 2 Version Control Version Date Changes to previous versions 13 9 00 Creation of document 9 11 00 Update and extension of document 14 11 00 Sent out to partners for comments 22 11 00 Minor changes after discussion with CNRS 27 11 00 Fin
29. elling Coaching Reflection Articulation Scaffolding and fading Exploration Some knowledge types go well together with some instructional strategies A mapping is provided in Table 4 Version 2 Page 18 of 48 ESPRIT 29175 IMAT Ki R IV 2 Scenarios and guidelines version 2 Articulatio Reflection Exploratio Modelling Coaching Scaffoldin n n g Perceptual knowledge X X Declarative knowledge X X X Reasoning skills X X X Procedural knowledge X Performing skills X X X Attitudes X X X Table 4 Knowledge type mapped to Instructional strategy 9 4 Instructional activities Instructional activities stand for the activities performed by instructor and learner Instructional activities distinguished in maintenance training are Setting goals Illustration Presentation Explanation Asking questions Providing workspace Instruction Demonstration Presenting tasks Providing examples Providing reminders Providing hints Evaluation Feedback Providing references Summarising Instructional strategies can be further detailed by crossing them with instructional activities They are mapped to each other in Table 5 Version 2 Page 19 of 48 ESPRIT 29175 IMAT oR R IV 2 Scenarios and guidelines version 2 Articul Reflect Explo
30. ence in the domain often lacking practical experience 3 Developers of instructional material Prototypical IMAT developers of instructional material are e Specialised developers at training centres who have as their main job the development of multi media instructional material e Instructors who have as their main job to provide training but who develop their own material they have no special expertise in development e Instructors who adapt and augment standard training material probably coming from a training centre to fulfil their own needs 4 Training managers The training manager takes care of the management of training material performing such tasks as determining who is authorised to make changes in the material taking care of the database etc In the P2 tools several options are built in for updating material and authorizing access Version 2 Page 9 of 48 ESPRIT 29175 IMAT Ki R IV 2 Scenarios and guidelines version 2 NB The stakeholders are described in terms of roles it is possible for an actual person to fulfil more than one role Version 2 Page 10 of 48 ESPRIT 29175 IMAT Ki R IV 2 Scenarios and guidelines version 2 7 Instructional material to be created with IMAT From the IMAT fragments different kinds of instructional material can be created The format and medium differ paper document electronic slides CBT etc as well as the authoring tools with which the instructional material
31. erial is developed However in the course of time experiences with different types of tasks can be stored in organisational memories for later use as examples of solutions for comparable problems as the one at hand Prior to doing special or complex tasks the engineer can prepare by studying the technical manual He she has to select using the IMAT tools and study all relevant information concerning the components involved the procedure described the relevant safety regulations and the tools to be used It depends on the level of sophistication of the technical manual how much guidance and explanations have been added to it in the form of job aids expert systems etc and thus whether all required information and guidance can easily be found In non standard or not foreseen problem situations the engineer has to fall back on general knowledge and skills to devise the solution path him herself or in collaboration with peers A large repair job may be planned or simulated with the aid of a technical manual by a repair team of different specialists discussing the different options and learning about possible ways to proceed In the ideal situation it may also be possible to make specific queries to organisational memories thereby profiting from earlier experiences of other engineers confronted with the same or comparable problems These experiences Version 2 Page 14 of 48 ESPRIT 29175 IMAT Ki R IV 2 Scenarios and guidelines version 2
32. ers of equipment managers training managers operators and maintainers of equipment logistics personnel and probably many more In the IMAT scenarios we will restrict ourselves to three main categories of stakeholders learners instructors and developers of instructional material The restriction is not made because we do not see the importance of the other stakeholders but because the focus of the IMAT project is on the re use of technical material for training purposes These three categories are the direct users involved in this process and the IMAT tools offer them support The stakeholders in focus are 1 Learners Prototypical IMAT learners are e Employees of a company who are trained in order to learn new tasks or to update their knowledge and skills working in a technical domain such as maintenance and repair of complex technical systems e Learners who are trained in a training centre in house or a public centre to qualify for a technical profession in maintenance or repair e Experienced employees of a company working as maintenance and repair engineers who have to perform tasks in which they are not proficient so they need on the job and just in time training 2 Instructors Prototypical IMAT instructors are e Technical engineers with ample domain expertise and experience in performing technical tasks who have a limited didactic background e Teachers with a background in education who have limited theoretical experi
33. g computer based learning environments Instructional Science 23 183 220 Designers Edge versie 3 5 Allen Communications South Lake City Utah USA Gagn R M Briggs L J amp Wager W W 1992 Principles of Instructional Design 4 ed Forth Worth etc Harcourt Brace Jovanovich College Jorgensen E J Fuller J J Rainey S C amp Post J P 1995 Potential benefits to Navy Training programs resulting from increased use of interactive electronic technical manuals Phase 1 Initial Evaluation of IETM Applicability to Schoolhouse and Worksite Training Functions NSWCCARDIV TR 96 016 Naval Surface Warfare Center Carderock Division Bethesda Kabel S 1997 System Evaluation Framework Graduation work for University of Amsterdam Faculty of Social and Behavioural Sciences Social Science Informatics Kabel S amp Riemersma J 2000 Second didactic ontology IMAT deliverable O2b Kribs H D Mark L J Morris B A amp Dickason D K 1996 A survey of interactive Electronic manuals used in education and training Report of the Navy Personnel Research and Development Center NPRDC TN 96 35 San Diego USA Lorrain G Bertozi Y Karrer F 1999 Training requirements AFPA IMAT deliverable R II 1 Morris B A amp Dickason D K 1996 Evaluation of the Interactive Electronic Technical Manual Automated Classroom Report of the Navy Personnel Research and Development Center NPRDC TN 96 39 S
34. gments itself All questions are open questions Questions concerning the lesson material Approximately how many hours have you been working with the retrieval tool Can you provide an estimation of the time you would have needed to create the same lesson material without the use of the retrieval tool How much lesson material have you created number of pages or screens What kind of lesson material have you created PowerPoint show Word document or Questions concerning the search and index vocabulary Are the correct terms provided to search for and index the fragments If you think that incorrect search and indexing terms are provided can you specify these Are the terms provided to search for and index the fragments understandable If you think that index and search terms are not understandable can you specify these Are the terms provided to search for and index the fragments detailed enough If you think that the terms are not detailed enough can you give an example of these terms and the desired details Question concerning the search mechanism Do queries result in too many or too little fragments If they do can you give an example Questions concerning the fragments Are the fragments too big or too small If they are can you give an example Are the fragments of the right composition If they are not can you specify the desired composition Did you need to edit the fragments a lot or did you mostly j
35. goal To use the system partly Learning Goal Instructional Fragment Description Type Strategy To use documentation gt Topical knowledge Articulation Goal task Procedure Conditions Reflection When to start Preconditions Safety measures Tools means material Description Steps Lists Decision points Test for completion Picture norms gt Skill in using documentation Modelling Coaching Table 15 Scenario for learning goal To use documentation Learning Goal Instructional Strategy Fragment Description Type gt Topical knowledge amp diagnostic tools Calibrate maintain maintenance Goal task Procedure Conditions When to start Preconditions Safety measures Tools means material Description Steps Lists Decision points Test for completion Picture norms gt Skill in Articulation calibrating maintaining Reflection Modelling Coaching Table 16 Scenario for learning goal Calibrate maintain maintenance amp diagnostic tools Version 2 Page 28 of 48 ESPRIT 29175 IMAT Ki R IV 2 Scenarios and guidelines version 2 10 Searching for fragments in the database and instructional mark up In the last chapters we explained how concepts from the maintenance training ontology are mapped and defined By selecting and adapting scenarios the following properties of useful fragments are determined knowledge type material use derived from instr
36. h the use of the fragments addressing questions such as is the picture enlightening for learners is there a mistake in it etc For P2 an organisational learning ontology is provided Database of C65 as ae ES Version 2 Page 30 of 48 ESPRIT 29175 IMAT Ki R IV 2 Scenarios and guidelines version 2 11 Scenario tool Providing a software tool to help the users specifying scenarios has a twofold aim in the IMAT context It obviously help them formalising and structuring teacher and learner activities during the lesson But more than that it aims to help them specifying the fragments they wants for a lesson only by describing this lesson This second aim represents the specifying by the use paradigm Thus the scenario tool global functionalities are twofold first to help specifying a scenario second to provide an automatic link from this scenario to a set of relevant fragments The general approach used for IMAT tools architecture is to provide the user a maximal flexibility in the use of these tools This implies particularly that he she can be helped by some smart tools or directly use the basic tools Like the others the position of the scenario tools follows this approach 11 1 Use of the Scenario Tool in the IMAT process The use of the scenarios and related tools is described in figure 1 Database of Fragments fragments NEF Scenario with search Components Retrieval tool Scenario tool
37. ify what is not 1 3 Navigation aspects These questions are about steering support moving through the system Version 2 Page 44 of 48 ESPRIT 29175 IMAT Ki R IV 2 Scenarios and guidelines version 2 Navigation 1 Did you at any moment have the feeling of not knowing where you are in the retrieval tool For example did you feel the urge to check while working how the open window is called in order to remind you where you are and what you were doing Task No Yes If you did can you specify when 1 2 3 Navigation 2 When accomplishing a certain task did you at any moment find that the retrieval tool should have told you what to do while it did not For example do you think that the retrieval tool should explicitly tell you what to do with a query result Task No Yes If you did can you specify when the tool should have told you what to do 1 2 3 Version 2 Page 45 of 48 ESPRIT 29175 IMAT Ki R IV 2 Scenarios and guidelines version 2 Mental model aspects These questions are about a possible mismatch between the users ideas of the retrieval tool and the image that is provided by the retrieval tool Mental model 1 Do you agree with the terms used in the user interface of the retrieval tool Task No Yes If you disagree can you specify at what point 2 3 Mental model 2 Are you fami
38. ing the authoring process To summarize the user can use the scenario tool for searching fragments only after an important indexing work using the AEI amp I tool 2 Inverse use The developer can use the Scenario Tool to initiate the instructional indexing by searching for relevant fragments for a given scenario without the instructional indexes because there are none or few ones keeping in mind these instructional indexes and then index the retrieved fragments with these scenario instructional indexes He can also obviously index without using the scenario tool in this inverse way This approach is more driven by the instructional point of view of the development work in which the main task of the developer is not indexing 3 Extensive use It is also possible to index nearly all the material with instructional mark up before an authoring process This might be useful in a situation where a central training centre has a database which is to be used on location in order to produce training material locally In this case someone at the centre is devoted to do the indexing Version 2 Page 29 of 48 ESPRIT 29175 IMAT Ki R IV 2 Scenarios and guidelines version 2 Fragments can have more than one value for knowledge type and material use For example a fragment might be used as an explanation as well as an illustration There is another kind of mark up which can be added to the fragments This mark up comes from experience wit
39. l decomposition gt Naming of the sub function Articulation System sub function identification name gt gt Goals why is it there Articulation Goal sub function gt gt Functions sub functions what does it Articulation System behaviour LO do description gt gt Implementation how does it do that Articulation Dataflow Process states gt gt Functional links Articulation Functional scheme gt gt Recognition of normal deviant functional states Articulation Coaching Normal deviating process states can be added to all fragments Annotation Table 8 Scenario for learning goal Functional knowledge of system x One level below the overall learning goal Being able to maintain system x the learning goal Being able to perform all preventive maintenance tasks within job is defined Table 9 to 11 are about this learning goal Learning Goal Instructional Fragment Description Strategy Type To perform inspection task y list y1 yn Articulation Identification label gt Perceptual knowledge normal deviant Articulation Depiction appearance Coaching Normal deviating gt Declarative knowledge about Articulation Goal task Procedure safety procedure sequence reporting Reflection Conditions Preconditions Safety measures Tools Steps gt Skill in performing inspection gt gt In taking precautions Modelling Coaching gt gt In disassemble a
40. l only generic Knowledge and skills will be acquired Use of specific systems and documentation can serve as illustrative examples in Version 2 Page 12 of 48 ESPRIT 29175 IMAT Ki R IV 2 Scenarios and guidelines version 2 the process directed at acquiring more general knowledge about technical concepts principles generic maintenance tasks and skills and the use of documentation in general Insofar material is drawn from electronic technical documentation this will be a subset of the use in scenario s 3 and 4 It is also possible to develop computer based training material which can be used by the learners directly for example using fragments from the technical manual as content in an Authorware or Toolbook program Scenario 2 Development by instructor for initial general training This scenario resembles scenario 1 only in this scenario the instructional material is developed by the instructor who directly uses the material In scenario s 2 in general only generic knowledge and skills will be acquired Use of specific systems and documentation can serve as illustrative examples in the process directed at acquiring more general knowledge about technical concepts principles generic maintenance tasks and skills and the use of documentation in general Insofar material is drawn from electronic technical documentation this will be a subset of the use in scenario s 3 and 4 By decentralized direct development by the inst
41. liar with the terms used the retrieval tool Task No Yes If you are not can you specify with what terms 2 3 Mental model 3 Do you think that terms in the retrieval tool are used inconsistently Task No Yes If you do can you specify these terms 2 3 Mental model 4 Do you agree with the content of the retrieval tool Do you think that the retrieval tool does what it should do like for example enable you to add the instructional mark up Task No Yes If you disagree with the content can you specify why 1 2 3 Mental model 5 Do you have different expectations of the retrieval tool Were you surprised by the availability of certain functionalities in the retrieval tool or did you expect them to be there Did you expect certain functionalities in the retrieval tool which were not there Task No Yes lf you have different expectations can you specify these 2 3 Version 2 Page 46 of 48 ESPRIT 29175 IMAT Ki R IV 2 Scenarios and guidelines version 2 The content questionnaire This questionnaire provides the possibility to comment on the content of the retrieval tool There are four groups of questions One group is about the lesson material you have created one concerns the search vocabulary you were provided with to search and index the fragments a third group of questions is about the search mechanism and a fourth group concerns the fra
42. ls in training Most short term scenarios will be instructor based in the long term just in time distributed and web based training scenarios will become feasible We have to take care that our potential users will be able to recognise the benefits for their training in the current situation while on the other hand giving them a perspective on new ways of training This document is part of the IMAT P2 work It describes instructional ontologies maintenance training vocabularies how to create a scenario how to use the scenario tool and the results of the P1 evaluation For an extensive description of the instructional ontologies and a definition of the terms used we refer to deliverable O2b Second Didactic Ontology The structure of this document is as follows Purpose of scenario development Stakeholders in the scenarios Instructional material to be created with IMAT High level scenarios Development of instructional material Instructional mark up The scenario tool 1 The IMAT tools consist of a Document Analysis tool an Indexing and Retrieval tool and a Scenario tool The Document Analysis tool enables the user to separate and index a technical manual and to store fragments in a database The Indexing and Retrieval tool enables the user to search for fragments and to index them with instructional information The Scenario tool enables the user to specify an instructional scenario standard lesson in order to pi
43. lustration Learner Study illustration Elaborate Presentation Instructor Present the content Learner jMemorise learn by heart content presented Explanation Instructor Provide explanation Learner Read Listen to explanation Study explanation Relate explanation to existing knowledge Asking questions Instructor Construct and ask questions Learner Comprehend question Recall content or reason and answer question Providing workspace Instructor Provide workspace Learner Use workspace Instruction Instructor Give instructions Learner Comprehend instructions and memorise them Demonstration Instructor Show how a task has to be done Learner Observe how a task is done and memorise Presenting tasks Instructor Give assignments Learner Comprehend assignments and doing them Providing examples Instructor Give example Providing reminders __ Instructor Provide reminders Providing hints Instructor Provide hints Learner Comprehend hints and apply them Evaluation Instructor Evaluate answer Diagnose faults Learner Evaluate own knowledge Diagnose lack of knowledge faults Feedback Instructor Provide feedback Learner Study feedback Comprehend directions Version 2 Page 22 of 48 ESPRIT 29175 IMAT Ki R IV 2 Scenarios and guidelines version 2 Instructional Activity Instructor and Learner Actions Providing references _Instructor Provide references Learner Act on references Summarising Instructor Pre
44. n this chapter we will give an overall impression of the kinds of user requirements put on IMAT and discuss how far these were met by the experiences of the users working with the P1 Version 2 Page 35 of 48 ESPRIT 29175 IMAT Ki R IV 2 Scenarios and guidelines version 2 toolset and what we might expect from P2 For more details we refer to the deliverables R 1 1 R 12 Rll R I 2 RI and R III 2 A full evaluation of P2 will be performed in the field tests of P2 resulting in reports of the three user workpackages at the end of the project Three broad sometimes interdependent categories of user requirements can be stated 1 New ways of producing training material 2 Producing new kinds of training material 3 New ways of organizing training 1 New ways of producing training material New ways of producing training material are made possible by the IMAT tool in terms of less time needed and easier ways to produce training material e Less time needed to produce training material Development of training material in the technical domains in which the user partners are working is a time consuming task Re use of material from the technical manual should speed up this task because the developers of training material have to spend less effort in searching for content The IMAT tools should provide quick ways to find the right fragments from the technical manual needed in the training material the facility to find releva
45. nt fragments which describe a topic for example a picture and a text even if they originate from different documents quick ways to export the fragments to an authoring environment In this area much has already been achieved by P1 The IMAT tools allow technical documentation becoming electronically available segmented and stored in a database These fragments are quickly be found and exported to an authoring environment Users already claim that they could speed up their development process by 20 40 in the last trials It is not only possible to produce material quicker it is also easier and quicker to change material making it up to date and geared towards specific needs For P2 no fundamental changes are foreseen but because of the better quality of the tools and the more technical material being analysed the development process could even take less time e Easy ways to produce training material This requirement has a strong relation with the first one the easier it is to produce training material the quicker it becomes Re using material from the technical manuals already makes life easier for the developer The IMAT tools should facilitate the development process further allowing for development by less specialized and or less experienced developers In P1 still many user unfriendly features and bugs existed However if the tools were properly running the users were able to use the tools without great problems For P2 th
46. o reason about system x is defined Table 7 and 8 contain scenario s for this learning goal Learning Goal Instructional Fragment Description Strategy Type Structural knowledge of system x hierarchical decomposition gt Naming of sub components in the system Articulation Identification component name number gt gt Structural description Articulation Definition Location component gt gt Perceptual knowledge about sub Articulation Depiction View angle components recognition Front Side Rear Top Bottom Skewed Cross section gt gt Location of sub components Articulation Sub components Exploded view gt gt Structural lay out Articulation Structural scheme gt gt Recognition of normal deviant appearance of Articulation Depiction normal deviating components Coaching properties gt gt Structural links Articulation System connections Identification connection name number Description of connection wiring piping Location connector Version 2 Page 24 of 48 ESPRIT 29175 IMAT oR R IV 2 Scenarios and guidelines version 2 Learning Goal Instructional Strategy Fragment Description Type can be added to all fragments Annotation Table 7 Scenario for learning goal Structural knowledge of system x Learning Goal Instructional Strategy Fragment Description Type Functional knowledge of system x hierarchica
47. p 5 and retrieve the fragment you just added markup to 9a Specify search criteria for knowledge type and material use 9b Retrieve the fragment Task 3 paste the fragment into training material Version 2 Page 41 of 48 ESPRIT 29175 IMAT Ki R IV 2 Scenarios and guidelines version 2 Open Powerpoint Word or another authoring tool of your choice Create a new lesson in your authoring tool Drag a fragment onto your new lesson Repeat step 12 as many times as you want Save the resulting lesson and add it to this evaluation form The tools questionnaire This questionnaire is based on different aspects of the retrieval tool These aspects are explained below Keystroke aspects Keystroke aspects concern single actions performed by a user For example Is the single action open implemented consistently Or do you have to double click one time while you have to use a menu option another time Process aspects Process aspects are comparable to keystroke aspects but they concern sequences of user actions For example the way something is done like open edit save System image aspects System image aspects are about the way things are visualised in the retrieval tool For example Is everything legible and at the right spot Navigation aspects Navigation aspects stand for steering problems For example you dont know where you are in the retrieval tool Mental model aspects Mental model aspe
48. pe gt Topical knowledge Being able to do subsidiary tasks Goal task Procedure Conditions When to start Preconditions Safety measures Tools means material Description Steps Lists Decision points Test for completion Picture norms gt Skill in installing To install a system Articulation Reflection Modelling Coaching Table 12 Scenario for learning goal To install a system Learning Goal Instructional Strategy Fragment Description Type gt Topical knowledge to start up shut down a Goal task Procedure Conditions system When to start Preconditions Safety measures Tools means material Description Steps Lists Decision points Test for completion Picture norms gt Skill in starting up shutting Articulation down Reflection Modelling Coaching Table 13 Scenario for learning goal To start up shut down a system Version 2 Page 27 of 48 ESPRIT 29175 IMAT BS R IV 2 Scenarios and guidelines version 2 Learning Goal Instructional Strategy Fragment Description Type gt Topical knowledge To use the system partly Goal task Procedure Conditions When to start Preconditions Safety measures Tools means material Description Steps Lists Decision points Test for completion Picture norms gt Skill in using the system Articulation Reflection Modelling Coaching Table 14 Scenario for learning
49. r Modelli Coachi Scaffol ation ion ation ng ng ding Setting goals X X X X X Illustration x Presentation X X Explanation X X Asking questions X X Providing workspace X X Instruction X X Demonstration X Presenting tasks X X Providing examples X X X Providing reminders X Providing hints X X X Evaluation X X X Feedback X X X X Providing references X X Summarising X X X X Table 5 Instructional strategies mapped to Instructional activities Once learning goal knowledge type instructional strategy and instructional activity are decided a number of refinement steps and finally a mapping to the fragments is made To be more precise instructional activities are refined into individual instructor and learner actions and learning goal and instructional activities are mapped to fragment description types 9 5 Instructor actions and Learner actions Instructional activities are refined into instructor actions and learner actions Instructor actions stand for everything the instructor does within a certain instructional activity while learner actions stand for everything the learner does within a certain instructional activity The following instructor actions are identified Present Learning goal Illustration Content Summary Provide Explanation Workspace Hints Feedback References References can be to a Glossary Index Lexicon Bibliography Annexes Version 2
50. rieval tool in an assignment and fill in a questionnaire about different aspects of the user interface and functionality of the retrieval tool the tools questionnaire followed by an additional questionnaire about the content of the tool the content questionnaire The assignment was based the following tasks retrieving the material from the database adding instructional markup to fragments and pasting it into training material In the tools questionnaire different aspects of the retrieval tool are evaluated These aspects are Keystroke aspects System image aspects Mental model aspects Process aspects Navigation aspects Each aspect is checked with a number of questions A system evaluation framework Kabel 1997 was used to draw up the questions This system evaluation framework provides a number of heuristics to check for each aspect The content questionnaire that follows upon the tools questionnaire contains questions about the lesson material the search and index vocabulary the search mechanism and the fragments The complete evaluation form is included in Appendix A 12 1 2 Results Note that some of the results are problems that needed to be solved in the document analysis trajectory or in the area of ontology development rather than in the AEI amp I tool Version 2 Page 33 of 48 ESPRIT 29175 IMAT Ki R IV 2 Scenarios and guidelines version 2 The tools questionnaire reve
51. rt safety procedure sequence reporting Reflection Preconditions Safety measures Tools means material Description Steps Lists Decision points Test for completion Picture norms gt Skill in performing corrective maintenance GSZ gt gt In taking precautions Modelling Coaching gt gt Making problem description on system level Modelling Coaching gt gt Generating possible causes at system level Modelling Coaching gt gt Making problem description at functional Modelling level Coaching gt gt Generating possible causes at functional Modelling level Coaching Version 2 Page 26 of 48 ESPRIT 29175 IMAT Ki R IV 2 Scenarios and guidelines version 2 Learning Goal Instructional Fragment Description Strategy Type gt gt Making problem description on deep level Modelling Coaching gt gt Generating possible causes at deep level Modelling Coaching gt gt Testing possible causes Modelling Coaching gt gt Repairing Modelling Coaching gt gt Testing correct functioning Modelling Coaching Table 11 Scenario for learning goal To perform corrective maintenance Within the learning goal Being able to perform all preventive maintenance tasks within job the learning goal Being able to do subsidiary tasks is defined Table 12 to 16 concern this learning goal Learning Goal __ Instructional Strategy Fragment Description Ty
52. rtain order and the presentation is ready to be used in the classroom It is also possible to store personal annotations of the instructor with the PowerPoint presentations In this way the instructors can have their own electronic annotated presentations this is also very handy if a second instructor has to take over the lessons Scenario 5 Special and complex tasks For special and or complex tasks a form of on the job training seems most optimal In this scenario instructors can prepare assignments on the level of tasks which are based on cases The trainees have to find information about components the right procedures to diagnose a fault and to fix the identified faulty component The assignments can be enriched with annotations by the instructor in the form of hints guidelines reminders and so on In this way training material can be presented to trainees for training on the job in their own time and pace When in some way the learners results are being kept track of further cases can be suggested which are tuned to diagnosed weaknesses of the learner Implementation can be in the form of an electronic assistant Another possibility is to present cases as assignments without enrichment The instructor has then the role of coach or even collaborator Coaching takes the form of helping and guiding the trainee in doing the job Scenario 6 Just in time preparing for special and complex tasks In this scenario no specific instructional mat
53. ructor the instructor is able to customize the material to the specific class he she is teaching A combination between scenario 1 and 2 is possible centralized development of the basic material and further adaptation by the instructor locally Scenario 3 Development at a centre for qualification training In this scenario a developer at a centre develops instructional material to be used by the learners directly by teachers at the centre itself or on location by instructors The material can be paper based for direct use by the learners and or in the form of a prepared presentation A developer of instructional material develops paper based material about a certain piece of equipment and its components for direct use by the learners In this situation the developer copies pictures or texts from the database to a presentation program such as Microsoft PowerPoint The developer searches with the IMAT tools for a general description of a component a description of its function a picture with the location of the component and a picture of the electronic scheme of the component The developer can add more information to the fragments found in the database The transparencies can be augmented by remarks or explanations parts can be coloured etc By using programs like PowerPoint it is also possible to make presentations that are more dynamic For example an image of the fuel tank can be build up by letting the different components appear one by one
54. sent a summary Learner Comprehend summary Table 6 Instructional Activities mapped to Instructor Action and Learner Action 9 6 Material use The nouns in instructor and learner actions represent the way fragments are going to be used in instructional material For example the learner action study illustration indicates that the fragment that is going to be put in the training material will be used as an illustration Although this material use attribute property is not explicitly used when creating a scenario it is recognised as part of the maintenance training ontology Fragments can be indexed with instructional characteristics knowledge type and material use to enable instructional re use without necessarily specifying a scenario This idea is also implemented in the tools the indexing and retrieval tool enables the author to index fragments with knowledge type and material use attributes The scenario tool does not let the author specify material use although material use is implicitly specified in instructor and learner actions The following material use concepts are identified Material use Pre instructional Introduction Definition Illustration Instruction Learning goal Instruction learning block Explanation Example Reminder Hint Guideline Warning Reference Feedback Testing block Question Answer Assignment Consolidation Anecdote
55. ssemble system Modelling components for inspection Coaching gt gt In identifying faults Modelling Coaching Version 2 Page 25 of 48 ESPRIT 29175 IMAT oR R IV 2 Scenarios and guidelines version 2 Table 9 Scenario for learning goal To perform inspection task y Learning Goal Instructional Fragment Description Strategy Type To perform preventive maintenance task z_ Articulation Identification label list 21 zn gt Declarative knowledge about preventive Articulation Goal task Procedure maintenance gt Declarative knowledge about Articulation Conditions When to start safety procedure sequence reporting Reflection Preconditions Safety measures Tools means material Description Steps Lists Decision points Test for completion Picture norms gt Skill in performing preventive maintenance gt gt In taking precautions Modelling Coaching gt gt In disassemble assemble system Modelling components for maintenance Coaching gt gt In doing maintenance Modelling Coaching Table 10 Scenario for learning goal To perform preventive maintenance task z Learning Goal Instructional Fragment Description Strategy Type To perform corrective maintenance Articulation Identification label gt Declarative knowledge about corrective Articulation Goal task Procedure maintenance gt Declarative knowledge about Articulation Conditions When to sta
56. st y1 yn Perceptual knowledge normal deviant appearance X Declarative knowledge about safety procedure sequence reporting x x Version 2 Page 17 of 48 ESPRIT 29175 IMAT Ki R IV 2 Scenarios and guidelines version 2 Hercep Declar Reaso Proced Perfor Attitude tual ative ning ural ming js knowle knowle skills knowle skills dge dge dge Skill in performing inspection p g Insp X X To perform preventive maintenance task z list z1 zn Declarative knowledge about preventive maintenance Declarative knowledge about safety procedure sequence reporting Skill in performing preventive maintenance To perform corrective maintenance tasks list c1 cn Declarative knowledge about corrective maintenance x Declarative knowledge about safety procedure sequence reporting x x Skill in performing corrective maintenance Being able to do subsidiary tasks 1 n Key Topical knowledge X Skill X X Table 3 Mapping table Learning goal to Knowledge type 9 3 Instructional strategy Knowledge is communicated to the learner by means of instructional strategies Instructional strategies refer to the effective way of teaching or to characteristics of an effective learning environment The main instructional strategies see in particular Chen 1995 which can be distinguished in technical training are Mod
57. sting possibilities arise in this area allowing integration between work and learning e structured material and standardized lessons The users required standardized lesson or material structures which are independent from the specific system parts They want to use these structures to create easily and quickly new lessons and material for other and additional systems or systems parts They also wished to provide instructors with more structure and standards so new instructors could easily take over and instructors would have more guidance in how to give the lessons The scenario tool described in this deliverable should provide this kind of support In P1 the scenario tool was not yet operational With P2 the users will have the opportunity to create lessons and material guided by the scenario tool They can store and re use the lesson structures We expect that the use of the scenario tool will fulfil partly the need for more structure and standardization 3 New ways of organizing training The users in the IMAT project are in the process of changing the way in which training is given to he trainees By computerizing training material and by producing training material that was not available before training procedures change This also put requirements on the life cycle of training and the documents involved Users hope to improve the knowledge circulation cycle between maintainers and trainers and between users and manufacturers of the s
58. swers to questions or do assignments etc Version 2 Page 13 of 48 ESPRIT 29175 IMAT Ki R IV 2 Scenarios and guidelines version 2 For large groups of trainees the selected and modified material can be embedded in a CBT package or CBT programs can be developed which use the technical manual as a source of domain knowledge by means of dynamic links This means great savings in development cost Challenge is to provide for options to retain the integrity of links in the course of time when many small changes in the technical manual itself are made Scenario 4 Development on location for system qualification training The instructor prepares a presentation about a component in the equipment In this situation the instructor copies pictures or texts from the database to a presentation program such as Microsoft PowerPoint The instructor searches with the IMAT tools for a general description of a component a description of its function a picture with the location of the component and a picture of the electronic scheme of the component The instructor can add more information to the fragments and augment them by adding remarks or explanations colouring of parts or connections or making them more dynamic etc In this way the instructors can make a presentation which is tailored to their own ideas They can easily make changes if it turns out that some pictures or texts are not clear for trainees The transparencies can be arranged in a ce
59. the manual for relevant information using the IMAT tools if necessary enhanced with a learner oriented interface The learner can use it to do assignments which ask for information from the technical manual or to learn how to use the manual The IMAT project is concentrated on the first two kinds of instructional material This is also due to the current training needs of the user partners Options 3 and 4 are closely connected with the technical manual as it is delivered for maintenance purposes see Barnard et al 1998 for an elaboration of these options IMAT did not take these options into account Option 5 is out of scope for IMAT at the moment but it is worthwhile to keep in mind for future use of the tools Version 2 Page 11 of 48 ESPRIT 29175 IMAT Ki R IV 2 Scenarios and guidelines version 2 8 High level scenarios At an organisational level a high level scenario is often already present like the situation in which an instructor develops his her own material and uses it for initial training As organisations change an organisation can decide to adapt its current high level scenario to that change The dimensions and factors that play a role in the origination and change of a high level scenario are described in this section Note that the decision for a high level scenario is not supported by the Scenario Tool Of all possible dimensions one can use to develop scenario types the next two seem the most versatile ones T
60. the user instantiates a new component of the scenario in progress The remaining choices for this component depends on choices made for previous components This dependency is calculated using the mapping tables described in this document and masked to the user in the tool The scenario tool furthermore contains help files describing different scenarios in more detail and giving guidelines about their suitability in different situations in plain text Version 2 Page 32 of 48 ESPRIT 29175 IMAT Ki R IV 2 Scenarios and guidelines version 2 12 Evaluation Two types of evaluation took place an evaluation of the P1 indexing and retrieval tool and an evaluation of the user s experience with creating the training material with the tools In addition users adapted their requirements The results are interpreted and used for improvement of P2 Both evaluations and the new user requirements are described in the next sections 12 1 Tools evaluation The P1 indexing and retrieval tool AEI amp I tool was evaluated at the three user sites At each user site one evaluation took place Respondents had to work with the indexing and retrieval tool in an assignment and fill in questionnaires about different aspects of the user interface and functionality and about the content of the tool The set up and the results of the tools evaluation are described in the next two sections 12 1 1 Set up Respondents had to work with the indexing and ret
61. uctor and learner actions fragment description type topic to be taught It is now possible to search in the database for fragments that are marked up with these fragment attributes For example one can look for a fragment which is about component X topic about how to take precautions knowledge type is a reminder material use and has a task procedural description of a condition fragment description type In the document analysis process the fragment description type is derived automatically to a large extent Most fragments are therefore indexed on fragment description type Knowledge type and material use on the other hand have to be indexed manually by the developer Therefore it might be possible that not all attributes have values so the developer has to search for fragments based on a subset of values The three ways of initiating the instructional indexing process 1 Standard use If a developer first starts working with the database of fragments the knowledge type and material use attributes will have no values During the selection of fragments the developer can mark up the fragments with this information This means that the knowledge type and material use tags are added to the fragments So the next time a developer uses the database he she can use these search terms In the indexing retrieval tool which is part of P2 the retrieval and the indexing process are integrated it is possible to perform the mark up dur
62. ust cut paste Did queries deliver irrelevant results If so can you give an example Version 2 Page 47 of 48
63. ype of training Initial general training for the first qualification as maintenance engineer Qualification training for a certain type of equipment Training for performing special tasks by already qualified personnel Scale and formality of training Centralised development of training materials by specialist developers On location development of training material by instructors Just in time use of documentation to train for specialised tasks crossing these two dimensions one gets six scenarios Centralised On location Just in time Initial general 1 2 System type qualification 3 4 Special tasks 5 6 Table 1 Type of training crossed by Scale and formality of training To further elaborate for each scenario some of the roles and functions have been filled in This results in Scenario Developers Instructors Peers 1 Develop Produce Teach Collaborate 2 Develop Produce Collaborate 3 Develop Produce Produce Collaborate 4 Develop Produce Collaborate 5 Coach collaborate Collaborate 6 Collaborate Table 2 Roles and functions for scenarios The scenarios can be described as follows Scenario 1 Development by developer of instructional material for initial general training In this scenario a specialised developer develops the instructional material The material is used by the instructor In scenario 1 in genera
64. ystems manuals deploying tools and organisational structures Requirements in this area are to organise authentication and authorization on producing and changing documents at different user levels to organise official changes at a central level in the organisation while allowing for flexibility to make changes at the lowest level by the instructors Version 2 Page 38 of 48 ESPRIT 29175 IMAT Ki R IV 2 Scenarios and guidelines version 2 to organise version control of documents to note and to store mistakes in the manual to facilitate the recording and storage of experience gained to feedback comments to other levels in the organisation to feedback errors detected to the manufacturer of the technical manual to update easily the fragments of the technical manual and the training material In P1 little of these requirements could be addressed We expect that in P2 more will be possible Adding and storing notes and feedback is taken care of in the tools Also version control is implemented There is a possibility to update fragments However the difficult problem of updating both the technical manuals and the training material has not yet been solved satisfactorily We have to experiment with the P2 tools in order to see in how far this will cause problems for the users Other requirements have a strong relation with organisational aspects In the GEM sessions with the users these issues were

Deliverable Template IMAT

Contents

Download Pdf Manuals

Related Search

Related Contents