User Guide - Mohawk College
Contents
1. amp amp amp amp 5 9 9 E 1 2 amp 22 m 9 t lt 4 lt 69 c m SeSeSSESSSE 0000000000 9 9 9 97 9 9 9 9 1 m dert AMTROM CORPOR ALL RIGHTS RE 14495 MC STUDENT NUMBER 20500999508 LAST Coe oC EEE Eo q 0 om a gre o GeOO09996989C CAPITALIZATION MARK E FEED THIS DIRECTION E AAN NAAA ANAND Sample Student Response Sheet DATE INSTRUCTOR T Cn E 5 0 O O 8 0 6 1 9 18 8 lt lt SS S2S3SSSS8 6 E 6 3 Y 5 5 6 5 9 5 6 2 feo lt lt lt lt lt lt SE amp Cd f cb 3 amo 62 9 82 00 E 5 E 8 53 9 5 5 8 5 6 5 63 5 9 G2 9 S 9 9 S APR Ed cd Cd Ed ed Ed ed 6 SLI 50 s amp s 9 9 E a 62. Y 5 0 0 5 6 53 6 6 5 2 2 2 E E lt 4 lt c C C C CO C CO Wem oem A m m 0000000009009 8 amp 89 Ee Cn queo We qum aem e em gem ar 8 2 8 5 8 5 80 0 0 5 9 9 9 9 6 5 E O 88 8 jalo 0 O ou B c o c we c cm 62 9 22 9900008980 99 9 89 6 89 2 5885885852 mm mm Em mm mm 2 3 2 3 3 29 C wm 9 9 C ml 8 9 2 8 s 5333885888 2000000000 8 9 8 em 0000000000 AA 5 co 0 0D cO c2 RR OR OR X n 09000000000 0000000000 0000000000 amp 9 G 9 9 amp 9 EREEEBERES i 8 3 2900
3. Test Scoring System Option Sheet An Option Sheet must be completed for each group of test scoring answer sheets submitted 1 Number of Answer Sheets submitted 2 Number of questions to be marked Max 200 3 Penalty for Incorrect Answer 4 Reports 1 Individual Results Analysis 2 Individual Results Summary 3 Question Analysis 4 Histogram 5 Item Analysis 6 Topic Analysis T Student Handout _ S Survey Analysis Please check M required reports Default reports are 1 and 3 LIE IE JE 0000 5 Individual Results Report Sort Order N Order by Surname default R Order by Rank 5 Order by Student Number Blank no sorting done Print Surname Yes or No Print Student No Yes of No 6 Report Heading Any Information you print below will be included in one heading line on each page of the reports You may want to include your name class test name etc 7 Topic file name if data is not included with answer sheets submitted by Date 10 Sample Master Sheet DATE INSTRUCTOR T F t ur 0 10 005008 E 0 000 a 55 6 55 se 8 7 4 lt CO So G CD C3 ur 5 5 8 a 8 a 8
4. B o E O O O O O E E 5 000506005000 8 amp 6 9 89 9 9 2 amp 9 9 3 9 9 9 9 9 Se00000000 e O O 5 5 E 5 89 5 89 89 89 EE GGG 3 9 C S 9 GJ amp 89 6 9 T Tine 595 s O 8 O O O 0 000000 a E 8 8 E falo feo lt lao a gt 59268 56 58 6 6 ua 8 amp 55852885885 er Y O 0 0 00000000008 000000090009 5 e s Te RFRERRRRERES 5 e E O 8 E 5 5 5 000000 E ex ee E lt lt lt 2 tajo la amp amp 0000000000 85 9 85 9 89 e 89 e amp 3 9 3 S S in iB in ib Big 00900000000 88 G 8 amp 8 0000000000 amp GG 9999 9 59939258928
5. a 55 3 E c 5 08106500 0 0 0 9 G2 bou D Be oo C 0000000000 amp 5 5 8 6 5 8 5 89 5 amp amp amp S 8 GO S 9 S 591995585888 6 E E a 5 6 a 8 5 5 5 6 5 9 5 6 89 6 e e lt lt lt lt lt lt Gi eB cb tO t5 UO c9 cO OR 2 8 8 E 8 E E 5 83 9 59 85 5 85 5 a 85 3 22222228282 ERRRERERRES Y 8 5 5 5 5 5 6 5 9 5 6 9 5 e E 5 8 9 lt lt i d 33 3535R 6552 8 e O O 5 9 5 6 5 88 e e O e 0 lt ele in I iB i5 IB IB IB IB 0000000000 E 9 amp 8 8 3 8 G 5 9 5 89 9 89 89 22222282988 SII 0000000000 6 8 8 amp 8 6 B 9
6. 00 Co C O 0 Soe OOO Goce Adjusted Score 27 00 28 00 21 00 27 00 26 00 28 00 17 00 34 00 20 00 25 00 32 00 27 00 24 00 19 00 32 00 26 00 26 00 21 00 27 00 19 00 33 00 24 00 24 00 34 00 32 00 25 00 26 00 26 00 29 00 11 00 25 00 28 00 24 00 25 00 19 00 32 00 24 00 27 00 25 00 27 00 25 00 33 00 25 00 32 00 26 00 27 00 21 22 Question Analysis Report Objective Test Scoring System Number Answer Weight Baco B3 o CO Q0 e uommpzizmgmaoumaoaouomaooumu gt 10 00 Righ Wrong Unans 0 00 9 18 00 2 4 00 0 000 38 7800 11 2200 0 0 00 000 33 66 00 10 2000 7 1400 33 66 00 17 3400 0 000 12 00 0 000 13 26 00 3 6 00 28 56 00 22 44 00 0 0 00 86 00 0 0 00 0 000 0 000 43 86 00 7 1400 0 00 8 00 28 56 00 0 000 8 1800 28 56 00 22 44 00 0 000 28 00 34 68 00 2 4 00 000 34 68 00 16 3200 0 00 12 00 B 16 00 7 1400 28 58 00 29 58 00 21 42 00 0 0 00 6 00 T 1400 2 5800 5 1000 28 5800 21 42 00 0 00 44 00 0 00 6 12 00 2 400 22 4400 28 56 00 000 12 00 9 18 00 0 000 21 42 00 8 18 00 41 82 00 0 0 00 2200 35 70 00 2 4 00 2 400 35 7000 15 30 00
7. E E E E E E E E lt lt 4 lt 4 amp amp 0 1 0 00 000 E 501 00 000 a y O 5 m 20 E E amp amp S 8 amp SSRSSESRRRS 4 E UY 5 6 E Te a co ip 59 59 2 59 eomm lt lt lt 4 JE 3 3 c A MD 52 51 9 color 5 5 a M o TESTS quuin e NE M E a 6 E 2 2 008 a e 8 e 6 E 8 0 E E 20 2 C92 C9 92 82 3 5595925858858 E 8 5 8 5 a 8 8 5 9 6 6 9 5 6 6 e lt lt 4 lt lt lt lt ch SD 58 co OP 3 3 89 22 5 E 6 9 B E E 5 9 3 9 6 9 9 9 9 85 8 5 5 89 6 alo 9 9 9 2 9 S ERXERERERRS a000000000 8 amp 8 8 E 8 8 000000900009 3 9 9 E 89 8 9 amp 9 amp 90099990999 9 E C 2 GO B G2 B 5 G9 9 9 3 3 3 5 6 E G3
8. nem Information Technology Division College Objective Test Scoring System User Guide Version 8 1 Revised October 20 2000 Table of Contents Guidelines Tor Optical Markings bent dis 1 OF Submissions tor Test Scoring een 3 nn Pe E 5 3 1 POT 221 eee o 5 de airada ITEMS 6 3 3 41 9585 X 6 3 4 O P 7 3 5 Student Response 2 2 1 22 06 600000000000 8 cH ee Y 9 SO AAA m A Rn eRe 9 Sample Master 10 Sample Wagh M 11 o VO EO AS 12 Sample Student Response 13 Sample Data Sheet REED 14 4 ae a PRO O 15 4 1 Summary FOO Y 15 4 2 Individual Results Analysis 15 4 3 Individual Results Analysis Summary 16 4 4 Question Analysis 16 4 5 xl sc 16
9. Average correct answers Average wrong answers Average unanswered Average adjusted score Average percent Standard Deviation Number of students marked Maximum possible score Thefiles used inthis scoring run eo posl Ter This report must be supplied in the event of any problems and a rerun is required so that the necessary files may be recovered The files are retained for one week after which time the original answer sheets are required Individual Results Analysis Report This report shows the raw scores for each individual and can be printed in surname student number or rank percent sequence or unsorted depending on the options chosen on the Option Sheet The printing of the student number and surname is optional Answers Each answer is represented by its position in the boxes on the right hand side of the report There are ten answers per box forty answers per line Each answer is represented by one of the following Correct answer Blank Unanswered A B C D orE The actual incorrect answer as given by the student Note If there are multiple answers for one question the machinery randomly selects and records only 1 answer While it attempts to detect the darkest mark this cannot be guaranteed Scores The total raw scores for correct answers wrong answers and unanswered are calculated by comparing the correct answers given on the master answer sheet to those marked by the student on the student answer 4
10. 205 2 In the OFFICE USE ONLY block the W in the first line must be filled in LAST NAME A 9 a 8 8 8 8 8 E B B Awol By default all questions are given a weight of one Customized weights can be OFFICE USE ONLY assigned by entering marks in the response answer portion of the sheet according to the following chart Answer Weight gt 02 0 null question If for some reason a question is to be excluded from the test scoring run a Null Question may be indicated by coding the E box on the 1880008 t5 E Weight Sheet for the appropriate question The corresponding 280008 200008 question on the Master Answer Sheet should NOT be left blank any 360006 13 6 bubble can be marked but the correct answer is recommended The 4 14 computer will treat this as an Unanswered question 5 15 In the example on the right all questions will be assigned a weight of 6 OO 16 D E 2 Box A with the following exceptions 780000 171040008 e Questions 8 17 will have a weight of 1 by default 8 B c D E 318 B C D E Question 20 will have a weight of 4 Box C 9 48806 10 E 8 C D E 20 a Y 0 E e Question 12 is a null question Box E and will be treated as unanswere
11. 8 G 000000000090 amp 89 9 8 amp amp 8 amp 39 53335858383 00900000000 E 3 8 8 8 C 000000900900 5 9 85 89 89 89 9 2222522982 SSSSSSSSNSS 09000000000 9 8 9 B 9 8 C 9 8 3 660 9 60 60 9 699 59 0 5 83 990000900 2 9 G2 9 G9 9 8 G 8 2220228228988 58858858829 e SES amp amp 59 92 89 NEN 5 62 EJ 5 63 2 2 2 5 amp 9 9 9 9 9 9 9 65 6 9 89 2 8 2 E 2 E 2 9 G2 2 e os 9 9 9 5 5 5 82 89 8 pm amp 9 6 9 amp 588585858599 E 22 m 0 O 0 O O 5 00 mw G9 9 Mi Teh 98 5 EEERERERRS ES e amp 2 2 2 E m G8 9 9 amp 9 9G 9 s GE 8 E 5 c 9 9 amp amp 9 5882885 y Se00000000 G000000099 266
12. S E 5 SABERES 2 2 62 E 9 8 E 8 9 8 8 8 5 9 9 9 9 9 5 9 89 8 gt T 0d co Foe oco C 2 2 2 8 amp 8 8 6 8 8 9 E 9 9 2 900908 amp 8 amp 9 3 2 44 OF OF CVM ua 8 0100 00050008 08 10005000 55 0 58 E 8 5 4 5 lt 4 Sia ss Bs wee aque Cu ue q uu 6 8 0 6 8 E 8 5 8 6 5 6 E 0 ze E Gagaga ee ao cS gg Poy qum go p m 200098900999 3 9 9 65 59 89 8 6 5 89 5333838588 mmm 9 9 00 0000 ml 9 C 9 9 9 G5 52 223388388 2 2 8 8 9 6 9 9 2 0 C 9 9
13. 0 00 84 00 0 000 0 000 0 000 47 94 00 3 6 00 0 000 80 00 0 0 00 2 400 0 000 45 90 00 5 10 00 0 00 6 00 43 85 00 4 800 0 000 43 86 00 7 1400 00 0 00 0 000 45 9000 2 400 45 60 00 5 10 00 0 0 00 4 00 0 0 00 1 2 00 0 000 47 94 00 3 6 00 0 000 8 00 6 1200 35 70 00 3 600 35 7000 15 30 00 0 000 0 00 5 10 00 4 800 38 7800 39 7800 11 22 00 000 0 00 000 43 600 6 1200 43 86 00 7 1400 0 0 00 62 00 4 8 00 T 14 00 3 600 31 62 00 18 38 00 0 0 00 Sample Histogram rer E 04083 18 0 16 0 14 0 13 0 11 0 9 0 7 0 5 0 4 0 2 0 Students 1 1 7 18 12 10 1 Absolute 000 380 7 60 1140 15 20 1900 2280 2860 3040 34 20 38 00 Range 379 7 59 1139 1549 1899 2279 2659 3039 3419 37 99 23 24 Sample Item Analysis and Reliability Report Objective Test Scoring System Analysis and Reliability Key A Sep 11 00 Number 2 Key C 27 23 315 31 Upper 27 13 students Middle 46 24 students Lower 27 13 students Sample Topic Analysis Report Barney Rubble B 328872 failed Overall Test Average 71 05 Part 1 Individual Scores Part 2 Topic Map 25 26 Topic Statistics Topic Passed Passed Average Mathematics 15 30 00 66 67 Physics 40 80 00 70 00 Quantum Mechanics 12 24 00 44 50 Celesial Na
14. 3 4 4 4 5 4 6 16 sheet The adjusted score is calculated by the following formula Total Adjusted Score Y Right Wrong Penalty Weight 4 1 The percent or rank is calculated by the following formula Adjusted Score 100 Percentage Possible Score Individual Results Analysis Summary Report This report is identical to the Individual Results Analysis except that the answer analysis is not displayed this report may be more suitable for posting results as the actual answers are not displayed Question Analysis Report This report prints one line of summary information for each question on the test Each line contains the following Question number Correct answer Weight Total responding to answer A Total responding to answer B Total responding to answer C Total responding to answer D Total responding to answer E Total responding with correct answer Total responding with wrong answer Total unanswered Histogram This graph indicates the number of students that fall into each of the eleven percentage ranges as indicated across the top of the graph Item Analysis and Reliability Report This report divides the class into three groups upper 27 middle 46 and lower 27 showing the percentage of students that answered each of the five choices A B C D or E The degree of difficulty of the question D is calculated by the following formula T U27 Right L2
15. 43 860 4 800 0 000 0 000 3 600 0 000 0 000 45 90 2 400 0 000 47 940 2 400 0 0 00 1 2 00 0 0 00 0 000 2 400 4 800 6 120 3 700 3 600 0 000 2 400 0 000 5 100 4 800 38 780 0 000 1 200 0 000 0 000 43 860 6 120 0 0 00 5 100 31 620 4 800 7 140 3 600 0 000 5 1 Sue Raw Scanner Data When test is scanned using the optical scanner the output data is stored in a file before it is passed to the scoring section of the Test Scoring program This raw data is usually retained for about a week so that re runs of the test can take place in the event of an error or a different set of options are required Occasionally instructors may wish to do use a different or enhanced system for scoring and analysing their tests In this case the raw data may be useful This data is stored in a colon delimited file in the following format Field Name Start Position Length Comments Record Type 1 1 m Master w Weight t lopic blank Student Response Comment delimiter 2 1 Student Number 3 10 delimiter 13 1 Last Name 14 15 delimiter 29 1 Initials 30 2 delimiter 32 1 Answers 33 200 It is important to note that the interpretation of each of the fields is dependent upon the record type These interpretations are described in Section 3 Input of this guide Raw Scanner Data that is used by the Scoring section must be preceded by a comment record that appears as follows Scanner Data File Version 8 1 This record is used as a cros
16. B 9 8 6 5 S 3 3 3 9 3 3 9 MTree Cc gy 7 12 2 e 000 0005 60 0 000506000 5 O E T WT be CO CD C3 am 63 6 3 00000000009 9089909988 9898898988 Gems 5989259585228 i ir an e E 0000000000 8G S S SEG E E E E E E le 2952528258859 5 127 52 5 3 1 8 8 E 6 5 5 5 gt 8 8 0 5 8 5 s 2 s 9 lt 89 2 amp s FORD Roe Re Rl Re Re CB 20000000000 8 8 8 8 E 9 000900090009 5 89 9 3 9 5 90220 85 8 El 60000000000 gt O a e 5 lt 9 lt 4 lt ip d iB i IB GB ID IB E 5 O O 8 55 9 O 55 6 6 9 5 gt 22 55 22 Ti C
17. C em 5 89 8 9 lt S 9 9 29000000000 8 E 9 E E 9 5 9 G9 9 9 9 93 9 0909 8 9 5 5 2000000000 8 E 9 E 8 E E wm 90 Ow 5 5 8 6 E 5 S 9 3 S 9 S wm 592208522 0000000000 8 E E E E 5 E 5 E 5 3 3 E 9 9 9 9 wm 9 5 9 5 89 69 8 8 8 gt 8 amp amp 8 2 S 9 9 i5 iB IB IBI IBI kin S um 24321 ALL RIGHTS A de BCANTROM COR STUDENT NUMBER LAST NAME Jm SS gt 51 EA BICEEEEDEEIE p E ec x Oe E CAPITALIZATION MARK E FEED THIS DIRECTION SEANTRON Sample Weight Sheet INSTRUCTOR 5 Ex O O O 9 5 9 0 9 5 9 5 eese E E E 6 E E E E E 933935338 5 5 5 O 00 9 5 e en em E E E E a e lt lt lt 6 lt 53353385838
18. Donald Duck D 8474 19 1 id 328734 Elray Jetson E 6842 12 0 DCAA 1 331228 Felix the Cat F 6842 13 7 0 2600 BBE 329436 Foghorn Leghorn F 5263 12 0 2000 1 445393 Franklin Turtle FT 5526 12 331184 Fred Flintstone F 71 05 2700 17 453569 George Jetson G 5000 11 8 0 1900 00 w 326078 George of Jungl 8684 17 3 0 3300 1 a 328732 Goofy D 13 7 2400 887180 Hagar Horrible H 6316 0 24 00 455917 Huckleberry Hau 8947 18 2 3400 961876 Jughead Jones 8421 1 a O 327432 Lil Abner L 6578 6 0 250 333570 Lite Lulu LL 6842 13 7 2600 326386 Worth 6842 0 280 331610 Mickey Mouse M T7632 14 0 290 327832 Minnie Mouse M 2855 7 13 o 1100 7 DB E Et 333574 Pikachu P 6578 13 T 0 2500 331508 Pluto D 73588 18 4 0 2800 454840 Pogo of Swamp 6316 13 O 2400 324667 Palliwag L 65 78 14 6 0 25 00 333089 Sailorman PS 5000 11 8 0 1900 329598 Porky P 8421 17 3 0 3200 444185 Reggie Mantle R 63 16 13 7 0 24 00 333905 Richie Rich R 14 B 0 27 00 332976 Road Runner R 6579 13 7 0 2500 329104 Rocky Squirrel R 8947 18 2 340 332423 Roger Ramjet R T7105 B 2700 804821 Snaggle Puss 5 65798 14 2500 340471 Tweety Bird T 8
19. It is important to note that accuracy of the results obtained from the Objective Test Scoring System are totally dependant on the quality of the marks on the student response sheets All marks on a sheet must a be made with a soft pencil and b fill a substantial portion of the bubble Any marks falling outside the bubble may cause erroneous results Who to contact The Receiving and Shipping department supplies the blank answer sheets to all users To obtain these sheets please contact 575 2172 Receiving and Receiving If you have any questions about the test scoring system please contact 575 2199 Information Technology Help Desk 3 1 The test scoring system requires the following input materials 1 2 3 4 One Option Sheet per test supplied by the requester One Master Sheet per test supplied by the requester One Weight sheet per test if required supplied by the requester Student answer sheets flat undamaged and all oriented the same way When Topic Analysis is requested either of the following a One fully completed Topic Response Sheet for each topic or b Atopic analysis file name entered on the Option Sheet 6 One Computer Services Work Order per test scoring request appropriately completed and listing any special requirements All materials items 1 5 as required relating to each test must be bound together in a non destructive way e g put in filing folders large elastic bands or paper cl
20. as follows Topic Name Pass Percent Student Percent for Topic Blank This blank field is used to separate topic groups and provide slightly better readability in soreadsheet format The first record of the Extended Statistics file is a record which provides a description title of the fields in the record Some spreadsheets and database programs allow this first record to be imported as the column titles Topics Data File This file is used mainly to internally store topic analysis data within the Test Scoring System However when pre arrangements have been made with Computer Services it may be more efficient especially when the same data is used frequently or in repeated runs of the same type of test to provide this data in a file rather than using Topic Sheets One record for each topic must be provided Each record is in variable length semi colon delimited format Field Name Comments Topic Name Up to 17 Characters delimiter Pass Percent 1 3 numeric characters between 1 and 100 delimiter Question List This list is series of individual question numbers or question ranges separated by commas Spaces may be used between fields to increase readability A question range is a beginning question number and an ending question number separated by a hyphen For example the question range 3 5 specifies the topic includes questions 3 4 and 5 An example of a valid Question List is 1 3 5 7 9 20 which specifies
21. depending upon the user s requirements The answers to the questions while not optional can be left blank if the student does not know the answer to a question The individual sections of the student response sheet should be filled out as follows Last Name Fl and Student Number Responses Answers A maximum of fifteen 15 characters may be used to denote the student s surname This name will appear on the Individual Results Report provided Item 5 Print Surname of the Option Sheet is marked Y If this area is unused do not request the Individual Results Analysis or Individual Results Summary in surname sequence choose rank sequence or student number sequence if student number has been selected When the Last Name is processed by the Test Scoring system the first letter will always be automatically capitalized By default the 2nd y rob B A 2 Y 0 09 C 20 C9 69 69 OW ga car IOS 5 di 1 6 gt CO S C9 G9 F7 al C N 9 69 through 15th letters will be set to lower case These defaults can be overridden by selecting any additional letters that should be printed in upper case and marking the appropriate capitalization bubble at the bottom of the LastName Fl and MI columns The First Initial and Middle Initial blocks may be utilized to identify students w
22. of students that have passed the topic 2 Theclass average and standard deviation achieved for the topic 3 The highest and lowest marks given in the topic Student Handout The Student Handout provides information on their personal results of the test and a perspective how they have performed in relationship to the rest of the class If Student Handouts are requested via the options one handout is generated for each student in the class Each handout is broken down into 5 sections as follows 1 Statistics display the student test results and includes the following e Percentage Raw Score and Maximum Score attainable e The number of Right Wrong and Unanswered questions The penalty for a wrong answer and the total penalties accumulated 2 Answers displays for each question the Student Answer the Right Answer and the weight associated with the question 3 Topic Analysis displays the Topics covered in the test For each topic the student s percentage and the questions that relate to that topic are displayed An asterisk before the topic indicates the student failed the topic 4 Class Averages displays information related to how the class as a whole performed and includes the following e Overall Class Percentage Standard Deviation and Adjusted Score e The Average Right Wrong and Unanswered questions 5 Histogram graphically shows the student s performance relative to the rest of the class and includes e The base histo
23. 0000000 5 E E 8 9 E 8 amp 89 9 89 89 89 69 89 65 4 4 G C 4 4 4 SSSSSSSSSE y Goog m 888900008 C w amp amp amp 2 E e gt Lu e 2 p e LAST NAME dE OFFICE USE ONLY MARK CAPITALIZATION veosouoe 2 GG 9600960 seo Oe 2 ZEIG IE 200 En a EEE IE M eo eec Soc EEE 352225222222222222225222222 Te 11 12 Sample Topic Sheet DATE INSTRUCTOR TESI F T T F 2000000000 00000000009 0000009909009 00000000009 7972979252218 0000000000 8 8 E 8 00000000009 9 s amp s amp 9 s 3 3 59689085899 5 10 0 84 8 e O O E 5 5 9 3 8 amp s 2 lt A SN RN NN NN 22 2 G2 E gea 668 8 3
24. 060606069 GG G5 zn08muzas5s2g q dil di y LEE LL ALL RIGHTS CORP LE e E un OFFICE USE ONLY MARK CAPITALIZATION E ao CD 3 3 C G C CD G0 C2 D B C3 00000050008 ci 13 Sample Data Scanner Data File Version 8 1 4 41 4 4 H HH Topic Analysis Data 75 65 65 65 65 65 80 Mathematics AAA Physics Quantum Mechani cs AAAA Celestial Navig tn AA Basket Weaving AAA Physics po AAA Etiquette AA Master and or Weight master weight ACABCCEDBCCBBCDADEDB AAAAAAA AAAEAAAA AAC Student Data 331610 327832 327070 359128 329104 333574 324687 325622 329598 331184 328672 445393 340471 7455917 446337 325725 332976 325961 328732 331509 329638 446865 329436 453569 328734 331228 329980 297210 577449 961440 17834 961876 444185 328975 14 Mouse Minnie Mouse Donald Duck Bullwinkle Moos B ACBBECEDBECBBCDADEDB Rocky Squirrel R ACABCCEDBABBBCDADEDB Tweety Bird Huckleberry Hou H ACABCBEDBBCBBCDADEDB Yogi Bear Booboo Bear Road Runner Wiley E Coyote WE ACDBCBEDBECBBCDADEDB Betty Boop ACABCCDDABCABCAACEDB M ACEAABCDDBBBBDABCEDD DD ACABCCEDBECBBCDADEDB Pikachu P CDDBCCEDBCBBBCDDEADB Polliwag
25. 4 6 Item Analysis and Reliability 16 4 7 TOC ATI E URN m UU Tt 17 4 8 Uden ANOO REIP 17 4 9 Survey Analysis Repor M 18 cue S A HM 19 Sample Option and Summary 19 Sample Individual Result 2 2 2 60600 000000005 20 Sample Individual Results Summary 2 2 21 Sample Question Analysis 22 VETE 23 Sample Analysis and Reliability nnn 24 Sample Foo ey qiu ReDal M 25 Sample Student 00000000002000 27 Sample Survey Analysis 27 22 2 0 000000000000000 28 5 M EU e ll 29 5 1 Scanner DA iia nee td DRE RB do UR i GERI AU 29 5 2 A SNCS lt lt BE oo dense UE 29 Da SS see 29 522 Extended O AUS Tm mw 30 5 3 TOSS E V 30 Guidelines for Optical Marking The Objective Test Scoring System uses an optical scanning machine to read stude
26. 684 17 3 0 3300 17834 Veronica Lodge 6579 13 7 0 250 325961 Wiley E Coyote 8421 1T 3 0 448337 Yogi Bear YB 6842 14 2600 448885 Yosemite Sam 7105 14 0 27 00 20 Individual Results Summary Report Student Number 297210 961440 333160 328672 303213 328975 577449 325725 325158 326225 325622 359128 878757 329638 329980 325116 327070 328734 331228 320436 445393 331184 453569 326078 328732 887180 455917 961876 327432 333570 326386 331610 327832 333574 331509 454840 324687 333089 329598 444185 333905 332976 329104 332423 804821 340471 17834 325961 446337 446865 Baby Huey Barney Rubble Beetle Baily Betty Boop Blondie Booboo Bear Boris Bedinoff Broom Hilda Bugs Buuny Bullwinkle Moos Charlie Brown Elroy Jetson Felix the Cat Foghorn Leghorn Franklin Turtle Fred Flintstone George Jetson George of Jungl Hagar Horrible Huckleberry Hou Jughead Jones Lil Abner Lttle Lulu Mary Worth Mickey Mouse Minnie Mouse Pikachu Pluto Pogo of Swamp Polliwag Popey Sailorman Porky Pig Reggie Mantle Richie Rich Road Runner Rocky Squirrel Roger Ramjet Snaggle Puss Tweety Bird Veronica Lodge Wiley E Coyote Yogi Bear Yosemite Sam zl F Erromonq gmmmuocggoguomgmumomgpoemss 5 lt lt lt 68 42 71 05 C C Co CO Ch Cn h 0 0 00 Co CO
27. 7 Right Difficulty No of students U27 L27 The validity of the question V is calculated by the following formula Validity U27 Right L27 Right The interpretation of V and any contributing factors that may affect it is left to the user s discretion 4 7 4 8 Topic Analysis Report The Topic Analysis Reports s purpose is to allow the users to evaluate a student s performance in each topic or subject area that was covered on a multiple choice test In order to evaluate each student in particular topics the instructor must complete a Topic Sheet for each topic covered on the test Two reports are created The Topic Analysis Report is divided into 3 sections as follows Individual Student Topic Report This report shows the performance of each student in the topics covered by the test and shows the following mb The student s Name Initials and Student Number 2 Thestudent s Overall Test Average 3 A list of Topics containing the Topic Name the student s percentage achieved for the topic and if the student as failed the topic an asterisk beside the Topic Name Topic Map This report lists the topics covered in this test and the questions related to each topic Topic Statistics Report This report lists the topics covered in this test and summarizes the classes performance for each topic The report displays one line for each topic and contains the following information 1 Thetotal number and percentage
28. B Leading zeros are not necessary and the Bar 1 7 data can be left justified by marking the response answer section of the sheet Amark in any of the ABCDE bubbles designates that the question is to be applied to the topic Generally it is recommended that only the A bubble be used so that errors and confusion are reduced The following rules also apply Multiple marked bubbles will cause an error and result in the test being unscored Caution should be used to ensure that 1 0 E a question is used for ONLY one topic 2 12 4 D E Multiple topics for a question are not 3 13 E allowed and will result in the test being 4 9 5 14 E unscored 5 4 0 0 0 15 4 5 E D E 620506060 16 3 8 9 5 E Any topic may have as many questions 7 17 4 E as there are in the test However ifthe 8 A 0 E questions are distributed unevenly 9 19 4 E throughout the test the maximum for 10 4 0 20 E the number of questions each topic can be easily exceeded Generally topics should be grouped using consecution questions As a rule of thumb no more than 16 questions or question ranges should be used Question ranges are consecutive questions that belong to a topic 3 5 Student Response Sheet Each student writing a test must fill in a Student Response Sheet Three areas of the answer sheet are optional LastName FI amp MI and StudentNo
29. EDBCBBBDDADEDA AEABABECEACBBCDADEDB AEBBCBEDAECBBCDADCED BB CDABCCDCAACABCDACEDB LL ACABCBBEBEBBBCEAEEDB S ACABEBBEBECBBCDADEDD L ACDBACDABACBBCDADEDE D ACABBCEDACCBBCDADDDB RR ADDBACCCBACBBCDADEDB ACDACCCDACEBDCDACEDC ACABECEBBACBBCDADEDB ACDACCCABCCBBCDABEDD 4 1 4 2 The Output of the Test Scoring System is controlled by the options requested on the Option Sheet the scoring and reporting options listed on the Option and Summary Report should match those requested on the Option Sheet In addition to the Option and Summary Report there are six optional reports available from the test scoring system All reports are optional however the Individual Results Analysis and the Question Analysis Report are produced by default unless otherwise requested The reports are described in the following sections Options and Summary Report This is a one page report that is produced for each scoring run It shows the following 1 The options that were in effect during the test scoring run as specified by the user on the Option Sheet The total number of students is indicated at the bottom of this page is the number of sheets actually scored and if there is a difference between this number and the number supplied on the Option Sheet the results should be carefully checked to ensure all students are included 2 A summary showing the statistical information calculated during the scoring of the test
30. L ACABEBBEBECBBCDADEDD Bugs Buuny BB ACDBACDABACBBCDADEDE Porky Pig P ACABBCEDACCBBCDADDDB Fred Flintstone F ADDBACCCBACBBCDADEDB Barney Rubble B ACABADDAAECBBCDADEDB Franklin Turtle FT ACDACCCDACEBDCDACEDC ACABECEBBACBBCDADEDB YB ACDACCCABCCBBCDABEDD B CCAAECEDAECBBCDADEDB R CCABCCEDDEDBBCDAACEB Goofy D AEBBCCEDAEBBABDADEDB Pluto D ACEBCCEDAECBBCDADEDA Daffy Duck D CDABBCEDBCBBBDDADEDA Yosemite Sam Y AEABABECEACBBCDADEDB Foghorn Leghorn F AEBBCBEDAECBBCDADCED George Jetson G DDABCCBDAACBBCDABDEC Elroy Jetson E CDABCCDCAACABCDACEDB Felix the Cat ACABCBBEBEBBBCEAEEDB Dagwood Bumstea D ACDBEBCEDBCBBCDADADA Andy Capp A ACDBCCEBDBCBBCDADCAB Blondie D ADABECECDABBBDABDEDE Archie Andrews AA CCAAECEDAECBBCDADEDB Veronica Lodge V CCABCCEDDEDBBCDAACEB Jughead Jones JJ ACDBCBEDBECBBCDADEDB Reggie Mantle R AEBBCCEDAEBBABDADEDB ACEBCCEDAECBBCDADEDA 887180 332423 326225 333089 303213 333570 804821 327432 325116 333905 878757 333160 326078 325158 326386 454840 Roger Ramjet Broom Hilda Popey Sailorman PS DDABCCBDAACBBCDABDEC Beetle Baily Lttle Lulu Snaggle Puss Lil Abner Daisy Mae Richie Rich Charlie Brown C ACABADDAAECBBCDADEDB Baby Huey B George of Jungl G Boris Bedinoff B ACABCBEDBBCBBCDADEDB Mary Worth M of Swamp P AEBBCCEDAEBBABDADEDB Hagar Horrible H CDABBC
31. d It is important to note that all wrong answers have their weight multiplied by the penalty from the Option Sheet and the result is deducted from the adjusted score 3 4 Topic Sheet A Topic Sheet is a specially coded Response Sheet However the meanings of the individual fields are different and described as follows 1 Topic Name 2 Flag 3 Pass Percent 4 Questions SB Si SEG A 3 IHHHHHRT A maximum of fifteen 17 characters may 1 3 4 3 be used for the Topic Name by marking the g 7 Lastname Fl and MI columns When the Topic Name is processed the first letter will always be automatically capitalized By default the 2nd through 17th letters will be set to lower case overridden by selecting any additional letters that should be printed capitalization bubble at the bottom of the LastName Fl and MI columns In the Office Use Only block the T in the first line must be filled in for each Topic Sheet These defaults can be in upper case and marking the appropriate The Passing Percentage is entered by using the first three columns of the Student Number field The following rules apply The questions that are covered by a topic are entered Percentages greater than 100 are invalid and will result in the test being unscored 0 16 16 00 00 00 Only whole number percentages are 2 E 1 allowed decimals are not permitted B GB d T
32. gram giving the distribution of the class results Mean Class Average Marker e Student Score Marker showing where the student lies within the class results 17 Survey Analysis Report The survey analysis report provides information on the number of respondents to each question Its primary use is for survey s in which there is no correct answer or answers are based upon opinion No master is required to run a Survey analysis a survey analysis is requested this is the only report produced One line of summary information is printed for each question on the test Each line contains the following Question number Total responding to answer A Total responding to answer B Total responding to answer C Total responding to answer D Total responding to answer E Total unanswered NO A Sample Option and Summary Report Options in effect during this run Number of answer sheets submitted 50 Number of answers marked 20 Penalty 0 00 Reports produced 1 Inidvidual Results Analysis 2 Inidvidual Results Summary 3 Question Analysis 4 Histogram 5 Item Analysis 6 Topic Analysis 7 Student Handouts S Survey Analysis Report sequence N Surname Sumame printed Yes Student Number printed Yes Weight Results are weighed Averages Right 1408 Wrong 5 92 Unanswered 0 00 Adjusted Score 26 24 Percentage 69 05 standard Deviation 13 03 Maximum possible score 38 Files used Input Scanne
33. h C cub cb cub cub ES y pe y 200000000 5 amp E 8 8 one 5 Ge 5 6 2222226288 BSSSESRERSES E 000 800008 5 8 O 1 006 3 3 f C em 5 5 5 2298028228288 HASARARAAS e 8 6 9 209 lt lt 4 lt 9 lt aoe Wu A 2220998998 9 89 9 2 89 ESE 5885883533 mm 0 60 O 0 O Cha 000000 5 6 9 a e 0 5 amp amp 5 8 5 amp s 5883885838 E GJ 6 C 2 2 86 E 9 6 _ OG E cocida 5 qum cum qum cms qum 6000000000 0 E 3 E9 C 5 5 5 6 5 96 86 aE 96 96 96 6 96 96 if EREERERERS
34. hat answer sheets are counted correctly and all oriented the same way Incorrectly submitted jobs will be returned unscored Master Weight if required and Topic if required sheets must be submitted for each group to be scored Special arrangements can be made to accommodate a large number of tests requiring the same Master Weight and or Topic sheets by contacting Computer Operations in advance of submitting the tests 2 1 Method of Submissions for Test Scoring Before submitting test scoring to the Computer Services Department for processing be sure the following sheets have been properly completed where applicable 1 Option Sheet 2 Master Answer Sheet 3 Weight Sheet 4 Topic Analysis Sheets 5 Student Answer Sheets When several groups of tests are to be scored individually and then consolidated as one large group be sure to complete the option sheet for the consolidation run and supply a master and weight and topic sheets if required for each group before submitting to the Computer Services Department Submit all test scoring input along with a Computer Services Work Requisition sheet to the Data Control Clerk in Room C122 Turnaround time should not exceed twenty four hours except in the case where sheets must be returned to the user for corrections Reruns of a job can be supplied for up to 1 week if the control sheet is submitted with the request After 1 week all sheets must be submitted with the request Note
35. ips Do not use staples or any other method which will cause the sheets to be damaged or curled Option Sheet Option Sheets are provided by the Computer Centre for all test scoring users An Option Sheet properly completed to reflect the desired options must accompany all batches of answer sheets submitted to the Computer Centre It is necessary to count the number of student answer sheets being submitted This count does not include the master weight or topic sheets The option sheet is used primarily to allow the user to have a choice of how the test is scored and what reports are printed However questions 1 and 2 are used to validate the test by providing the operator with an indication that the counts the scoring system computes are accurate If these items are not entered the test will be returned unscored The following default values are set for the remaining options 3 Penalty for incorrect answer 0 00 4 Reports produced 1 3 5 Individual Results Report Sort Order N ame Print Surname Y es Print Student No Y es 6 Report Heading Blanks The above default values may be changed to suit your requirements We strongly suggest that 6 Report Heading be filled in to identify the test Tests submitted with blank option selections other than 1 and 2 will be scored using default values 3 2 3 3 LAST NAME Master Answer Sheet The Master Answer Sheet is a specially coded Response Sheet and is used to sup
36. ith a same surname and different first names Both initials will be printed in upper case with no capitalization override available If the user wants the student number to appear on the Individual Results Report a student number must be coded in the student number area on each sheet and Item 5 Print Student No on the Option Sheet should be marked Y Student Number must be completed to have the Individual Results Analysis or Individual Results Summary printed in student number sequence Questions are answered simply by filling in the appropriate bubble A B C D or E Only One answer per question is acceptable If multiple answers are marked one will be randomly selected and passed to the scoring system A question may be left unanswered by leaving all bubbles unmarked It is important to note that when incorrect answers are penalized leaving a question unanswered blank avoids the penalty for that question 1 2 8 E 3 4 8 4 98 5 B 7 E 8 9 a 8 10 0 8 88 STUDENT NUMBER 000080000 989080000 amp 8 o8 2o 9 5 98987 6 100800 12 9 amp 9 D E 13 9 amp 9 D 8 DE 14 9 E 8 D E 1500080 168080068 1706080 1800800 80 3288008 20 9 amp 9 U E Sample Option Sheet Information Technology Division
37. nt response sheets As with any mechanical system the accuracy and quality of the output is directly dependant on the quality of the input In order to insure accurate scoring results the following guidelines must be adhered to 1 Use only a soft 2 or HB pencil to mark the response sheets Ballpoint pen or felt tip markers produce very erratic results Erase errors completely and cleanly Be sure that there are no eraser crumbs on the response sheets as they will plug the mark sensors Caution should be exercised so that the marking of one bubble does not slur into another causing unexpected results The bubbles on the response sheets must be filled with a clean dense mark If the mark is light or does not substantially fill the bubble it will not be read In addition itis very important that the centre of the bubble is well filled Awell marked bubble with a hole in the middle will result in an incorrect marking Examples Incorrect Markings A Not enough of bubble marked Centre of bubble not marked h Erasure not complete Correct Markings Perfect The top of the form is not scanned and may be used to identify test class instructor etc if desired Multiple responses to a question are not valid The scanner will not feed torn crumpled folded or edge damaged sheets Jobs submitted with non feedable sheets will be returned unprocessed Users are responsible to ensure t
38. ply the correct answers If a survey analysis is to be run this sheet is not required Two conditions must be met to ensure that a Response Sheet is considered to be a Master sheet S 0 0 0 0 0 0 0 pj pi 1 The word MASTER must be codedi in the first six j boxes of the area marked LAST 2 In the OFFICE USE ONLY block the M in the first line must be filled in DE 0 4 9 9 9 9 Alar 8 6 E E E 8 6 E E E c c 16 fc c c c fc in OFFICE USE ONLY The correct answers must be marked in the response answer area of the sheet Embedded blank answers are not permitted except on null questions Only one correct answer per question is allowed More than one Master in a test batch is not allowed and will result in the test being unscored Weight Sheet The Weight Answer Sheet is a specially coded Response Sheet and is used to apply different weights to each answer of the test The weight sheet is always optional and if it is provided when only a survey analysis is requested the sheet will be ignored Two conditions must be met to ensure that a Response Sheet is considered to be a Weight sheet pj p p 1 The word WEIGHT must be coded the first six TE TII boxes of the area marked LAST NAME 3
39. r data file C data score SampleData Topics data file Topics data included in scanner data Output Statistics file 19 Sample Individual Results Report Objective Test Scoring System Individual Results Analysis Student Student Raw Score Adjusted Number Name Int Percent Right Wrong Unens Score 297210 Andy A 71 05 14 6 0 27 00 D BDB RUE T 861440 Archie Andrews 7368 15 5 0 2600 1 3 5 o 333180 Baby B 55 26 12 B 2100 Da C A E D C C 320672 Barney Rubble B 71 05 34 6 o 2700 303213 Beetle Baily BB 6842 12 B 0 2600 0 DA 1 328875 Betty Boop E 7368 18 4 0 2800 577449 Blondie D 4474 10 10 1700 D E CDAJB DAB E 325725 Booboo Baar B 1358 15 5 2600 G AE AE 4 325158 Boris B 8947 18 2 HO Re 0 0 326225 Broom Hilda B 5263 12 B 2000 1 J 325622 Bugs Buuny BB 6578 14 o 25 00 DA 1 359128 Bullwinkle Moos B B4 21 17 3 0 32 00 1 1 BTHTBT Charlie Brown C 2700 ADDAAE 329638 Daffy Duck D 6318 14 6 2400 B D A 1 329980 Dagwood Bumstea D 5000 11 9 1000 DEBCEDB A A 325116 Daisy D B4 21 17 3 0 32 00 ee 0 327070
40. s check to ensure that the file contains the correct version of scanner output Statistics Files From time to time instructors may wish to do further analysis of their students results using their own tools such as Excel Lotus 123 or a data base package To facilitate this the Test Scoring System can create the following files 5 2 1 Basic Statistics File The basic statistics file contains data in fixed sized fields and no delimiters between any of the fields The definitions of the fields are as follows Field Name Start Position Length Student Number 1 10 Last Name 11 15 Initials 26 2 Percentage 28 8 No of Right Answers 36 9 No of Wrong Answers 41 5 No of Unanswered Questions 46 5 Student Answers to Questions 51 200 29 5 2 2 Extended Statistics File 30 The Extended statistics file contains variable length data records for each student Each record contains ASCII delimited fields suitable for importing into spreadsheet programs such as Excel Quattro and Lotus or database programs such as Access and Paradox All fields are separated by commas and enclosed in double quotes The definitions of the fields are as follows Field Name Comments Student Number Last Name Initials Percentage No of Right Answers No of Wrong Answers No of Unanswered Questions Student Answers to Questions No of Topics The No of Topics signifies the number of topic groups that follow this field Atopic group consists of 3 fields
41. the topic contains question 1 3 5 7 8 9 and 20 An brief example of the contents of a Topics Data file is as follows Trigonometry 70 2 5 8 10 14 16 Calculus 60 1 3 4 9 11 13 17 20
42. vigatn 0 0 00 35 00 Basket Weaving 45 90 00 88 67 Physical Ed 44 88 00 80 67 Etiquette 28 56 00 7400 Part 3 Topic Statistics Sample Student Handout October 20 2000 Objective Test Scoring System Student Handout Andy Capp A 297210 Statistics Class Averages Histogram Percent 71 05 Percentage 69 05 c Adjusted Score 27 00 Standard Deviation 13 03 13 Maximum Score 38 Adjusted Score 26 24 Right 14 Right 14 08 975 Wrong 6 Wrong 5 82 UnAnswered 0 Unanswered 0 00 we 0 325 10 20 30 40 50 80 100 Percent Mathematics 66 67 78 100 00 65 4 8 Quantum Mechanics 25 00 65 7 10 Celesial Navigatn 50 00 65 11 12 Basket Weaving 100 00 65 13 15 Physical Ed 66 67 65 16 18 Etique 50 00 19 20 27 28 Sample Survey Analysis Question Number 0 C Cn Co Po rna No Per No Per No Per Per No Per No 39 780 0 000 8 180 2 400 0 000 000 000 000 33 660 10 200 7 140 0 0 00 28 560 6 120 0 000 13 260 3 600 0 0 00 140 43 860 0 000 000 000 0 000 9 180 4 800 28 550 0 000 8 180 0 000 0 000 14 280 34 680 2 400 0 000 0 000 0 000 6 120 8 160 7 140 29 580 0 000 6 1220 3 600 140 29 580 5 100 0 000 20 400 22 440 0 000 6 120 2 400 0 000 4 280 6 120 9 180 0 000 21 420 0 000 0 000 11 20 35 700 2 400 2 400 0 000 600 47 940 0 000 0 000 0 000 0 000 3 800 45 900 0 000 2 400 0 000 0 000 0 0900 3 600
Download Pdf Manuals
Related Search