Home

ePEN Supervisor User Guide

1. 6916_01_Q02g 16 nem BSB ota Hot Gunin a mane 17 ena S N iN n rt Hot Gunin a 6916_01_O02h 16 e960 eee OF at es 0 6916_01_O03ai i 14 wae OT ate mana meamea O J o Y Y y 6916_01_O03aii ti 20 mem ae a a o 21 ameo ee pen a TI 8816 01 6316_01 etib Feo pete _J 0 D 39 fiA Ai BS N iN h Hot Gunin a 24 RHE M Ei 60 eN h Hot wuni fh a Je Pa EA G V1 AF b r Tae I IS i i 26 em es CC 27 mem hata a sa a 26 591601 euman Rows aore a Tj 6516 01 alg T ET EIA Nol cueita ka H 4 HA DE Qualification new lal Eig Ready nal 60 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 Any column that has been filtered will be indicated by a filter arrow that has changed colour Filtered Column Reader Date As signed a Non Filtered Columns You can apply multiple filters to any spreadsheet by clicking on the filter arrows for each column and selecting your filter requirement Removing filters To remove filters click on the filter arrows which have changed colour and select All from the dropdown list 61 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010
2. Performance Evaluation Network Select from the list of Enae responses marked or Project 55215 Item 5521 _02_Q28 t Select All NAA New Search Selection a re F p Scorer ID mackja Number of Responses 20 Cntena Sel Document ID Employee Review Date uJ 0168003994008 00000001 mackja 2006 10 09 a 0168000817203 00000001 mackja 2006 10 09 a 9168003994007 00000001 mackja 2006 10 09 Copyright 1999 2006 NCS Pearson Inc All rights reserved ag PLN helpline free phone 9200 169 9202 edexcel bee The figure below shows the functions and options available to Supervisors in the reviewing screen Figure 43 Reviewing a Response J NCS Pearson o PEN Scoring Microsoft Internet Explorer provided by Edexcel i File Edt View Fewkes Took Help i Document ID o O Roterence Librar 4 7 3 50 i 1 Wig 173g j Regular j A Lange iub of pepcoro cors 3 80 and holds 200 g A Regular iuh af popcom cosis 3 9 ond hodat Marking Grid FE tub m the beller value for MOBEY Rob save thal the 200g Lar bait 2g Rapala mb ja the berner value for mney F Whe is connect BPulo rete fee Aruet hh aek Explain the reasons for your answer You mna abnor all vaur aarin T MENNE mi ME k AFIA A a F ejh 2 sunisme A The Marker s a2 EE comments if any TOA e submitted on Sunjo 1 Not Nig 1 dugi li i atk t h e i te m i Hi z af 10 8001
3. Once you have logged in you will be in the ePEN Information and Training Site By clicking on the link you will have access to both the ePEN marker and supervisor courses It is essential that you complete the marker course and the online standardisation course if appropriate Supervisors will also need to complete the additional ePEN supervisor s training course s These online training courses will enable you to familiarise yourself with ePEN functionality relating to marking and also the online standardisation process The online training courses are available for use throughout the marking period Once you have completed the course s return to the Homepage and click on Here to confirm a short questionnaire will appear for you to complete and submit 3 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 Marking In The Practice Site When you click on the Practice Site link on the landing page you will be taken to the Practice Site login page The username and password provided and the initial login details for the Practice Site will be sent to you prior to the marking period When you login successfully to this site you will be prompted to change your password Please be aware that any password change Is site specific and will not be carried forward to any other site Therefore you will be required to enter the username and password provided twice once for each site Practice and
4. Below is an example where the Marker is tending to be harsher in their marking than the Senior Examiner Although this only indicates a small number of items you may wish to feedback to the Marker to alert them to the fact that they are marking Q2 harshly You may also need to backread some more of this Marker s work for this item 6XXX Marker Level Marker All Markers Item Differences GCE 6xxx_0x_Q02 Date Trait True Actual Difference Type Document ID Score Score 16 06 2007 Q02 3 2 1 Validity QC0184300072259 16 06 2007 Q02 2 1 1 Validity QC0184300072896 16 06 2007 Q02 2 3 1 Validity QC0184300072897 17 06 2007 Q02 4 3 1 Score 0184500345799 17 06 2007 Q02 3 2 1 Validity QC0184300072259 In the first row of data the Marker has awarded the response 2 marks whereas the True Score is 3 marks Therefore the Marker is 1 mark from the True Score Feedback to markers 22 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 Backreading items Score By noting the Document ID and accessing the Backreading screen you can enter this number into the search criteria view the item and send a message along with the image to the examiner as necessary Please note that Messaging is not possible if it is a Validity item However you can still view the item in the Backreading screen but you won t be able to send a message This report can also be exported to your PC and viewed offline To do th
5. Performance W N Evaluation 3 Network Praject S521 Gi Duy Reasi End Task ee keis Type pa Select the item you wish to caprino 19 08 nes peanon ne ML man m backread iore E rire After selecting the item you wish to backread a further set of selection criteria will appear 28 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 Figure 33 Backreading Selection Criteria a Backreading Query B1 Microsoft Internet Explorer provided by Edexcel File Edit view Favorites Tools Help kJ You can select a Team Q sax X 2 x a A P Search 5 Favorites E2 7 LJ 3s and Scorer from the drop Address 4 http 159 182 137 199 28080 epen webapp BackreadingController v Else Links down menu Performance W gt eape Ba eading Amend the maximum Evaluation A e S number of responses to Network return Project 5521 01 Item 5521_01_001 v Score Type Operational Team Selecta Team Scorer Select a Score Adjacency All v Document ID Number of Responses 20 Trait Select a trait scroll down v So rt by el t h er Previously Backread No Sort By Document ID A Doc ume nt ID or For a specific wiltteasd search use in the Document ID field Late st Score Ti me ex 9915 will retrieve all Doc atIDs that start with 9915 Copyright 1999 2008 NCS Pearson Inc All rights reserved ePEN Helpl
6. 5 different responses Reset Search Click on Search to retrieve responses When you revisit this function for a second time for the same item the system Is likely to return the same sample of items from the same search Note If you add a digit after the sign in the Doc Id field e g 3 your search will only return documents with the number 3 in their ID code This may be useful if more than one person Is escalating into the pool and you want the search to return different responses 40 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 Figure 51 Resize the candidate response Ow x 2 P A Serch Ie Ports E 2 Ga gt Candidate response is displayed Compete Marker grid and click on Escalate if the response is suitable Click on Skip to move to the next response 5521_01_001 a 2 Appt Cont nes soory viewer Windiwedviewer Apoiet carted O reer You will now be presented with copies of candidate responses for you to propose marks for and Escalate Once you have assigned a provisional True Score the response can then be proposed by clicking on the Escalate button You will then be asked to confirm that you wish to escalate the response and categorise the response as either Practice or Qualification If you do not think the response is suitable then click on Skip and the next response will appear Draft annotation can be added when the
7. l4 4 1 ofi gt Pi Find Next Select a format x Export 2 Bi Qualifying Summary Report By Marker 10 Jan 2007 10 02 25 Qualifying Summary Report By Marker 6663_01_901 41 Agree 6663_01_ 01 42 Agr me 6663_01_001 B Agree sesonon festem for mr asaonmscone onoommmomry Shows average 6663_01_Q02 bM Agree 6663_01_ 02 b41 6663_01_Q02 b42 a g r m e n t a C r 0 SS a item Marker Qc Set Date tem Marker QC Set Date EC T nanona soono Aa vui the et per tral http 10 153 21 59RepotSewgfLA Management Reports GQJan2007 VW Item Details Set name 11 0 ERA The Examiners Report Assistant ERA is a new functionality of ePEN introduced in Summer 2010 It is an electronic template which helps the Principal Examiners to structure their examiners reports It can be accessed from the Available Tasks menu on the Welcome Screen For more details please refer to ERA user guidelines 55 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 12 0 Frequently Asked Questions and Trouble Shooting Please contact helpdesk with your queries however some may be resolved by reading the information below Has the version of ePEN changed for this No it is the same version of ePEN that was marking series used since March 2008 however the View Progress screens have been updated If necessary please click on the Download and Install ePEN link on the Lan
8. 0 O08 mhf a yn Sc ae T i The Review screen will display the selected item a set of image control tools so that you can adjust the image for better viewing and the marking grid to be completed Note Items in Review have not been marked even if the marking grid was completed before the item was sent to Review 34 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 Once you have reviewed the response you can select from the following options actions shown in the figure below Figure 44 Review Menu Options Submit Score Submit and Retain Escalate Send Mail Belezas You can send a response to the Pull to enable the script to be viewed where candidates have written outside the viewable area Close Viewer When complete vou need to click on Close Viewer 7 1 Messaging in Review You can respond to a review item with a message to either the person who sent the review select Users on Response your team select Users on Response selected members of your team select Users on Response your own Supervisor select Principal Examiner Team Leader Attaching an Image of the Response to the Message If you wish to give feedback to a Marker you can type your message and attach the image of a specific item To do this you must send the mail before clicking submit score The Marker will then be able to see the image of the response but not the marks associated wit
9. 5 Sd Local intranet You will now need to access the Annotation Tool so that you can add the annotation for that item To do this you will need to click on the ePEN Annotation Tool button on your Start bar NB You must not use the button that reads update annotation beneath the submit score button This button is not integrated with the annotation tool Figure 63 Applet com ncs scoring viewer2 WindowedViewerApplet started Internet o start Inbox Microsoft ia ePEN Screenshots tay ePEN Supervisor U Z NCS Pearson e PE ePEN Annotation T WS e Click on the ePEN Annotation icon on your Start bar to open the Annotation Tool To open the Dialog Box select File and create amend annotation or click on the Icon Figure 64 J HCE earns PIM searing Micinta Iniarmet Lageharat provided igi dears Ges GJ t of past homie ou g 3 2 ge iW d ho a PEM Aamiin Tanl Aizian Inietet Lepleiee goovided by Diria Peace select the content te epen Paper be Fa ii of denih of bacterin mn jo ted Eain Aai Bie onio Perana for Future Lise mi iiir ep i Sg a rts at a Cope rng paya Hip elec Sheng Pepper _ ceed oor bee cen i of Tel we crt 0 Aii g ra yo unk near e p Trur Sia a pee MT start K iH aj a To create or amend existing annotation you can search for a particular response that has been pro
10. Examiner The figure below shows an example of a Marker Differences Report Figure 24 Marker Differences Report Display 21 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 gt Reports 01 Microsoft Internet Explorer provided by Edexcel G v tpit 153 211 235 Desboad Deshboardhepots amataskinderslAsd 6SI TIIRA EMEA IAEI I TAADSI CS60hpid Bernie 251 vy 9 XK P Fle Edt Yw Fgvertes Job Heb WR BS 1 edencel Onine Post Rents Revert 01 y A D e D SO1I6E 1F Marker Differences Level GCSE Marker M Test markertest item SO16F 1F OO03 Bice 02092029 Qdc i 0 TREET A Vakity 0c0233300000370 Otte i 1 ore 0239200000971 Page tof lt gt Copyngnt 1999 2009 NCS Pearson Inc All nghts reserved ePEN helpine freephone 0800 169 9202 edexcel Done Q trewrat R gt The columns in this report show you the following information The date the item was marked mm dd yyyy The name of the trait row of the marking grid e g Q1bii Q1biii The score assigned by a Senior Examiner either for a Validity Item or a backread item The mark awarded by the Marker The difference in marks between the examiner s mark and the Senior Examiner s mark on a trait level This indicates whether the item is a Validity Item or a Backread item Score This identifies the script ID within ePEN Case Study 1 Marker Differences Report
11. Please note that there are many reports available but these are only used for monitoring Online Standardisation activity and are described in detail in Section 10 of this document Marker Statistics Summary Marker Differences Validity Calculator Validity Item Monitor Reviews Pulls and Adj udications 5 3 1 Marker Statistics Summary This report will help you to monitor the quality of the marking for an individual or group by displaying statistical information on the accuracy of both Validity Items and Backreading From View Progress select the Reports tab The Reports default page will then be displayed To run a report you must first select the paper the role AA or Non AA the Marker and item required You may choose to select All items or All Markers for a paper Figure 21 Marker Statistics Summary Settings Reports Please select a report Marker Statistics Summary Paper 5016F 1F v Role Type ALL vi Team Leader ALL v Marker M Test markertest v ItemType ALL v Item ALL v To generate a Marker Statistics Summary Report 1 Select Paper 2 Select Role Type optional selecting AA role will exclude non marking roles 3 Select Team Leader optional allows you to limit the list of Markers to your team 4 Select Marker or All 5 Select Item Type e g Expert or All for all item types 6 Select the Item 7 Click on the View Report button 18 Prepared by Online Assessment Company
12. Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 Unmarkables Sometimes you still cannot assign a mark even though you can see the whole script This may be because for example the writing is too faint or coloured ink has been used If this is the case then you need to make a note of the Document ID together with the item name e g 5521 01 Q32 These details together with your AA number should be emailed to the following address online unmarkables edexcel com This script will then be sent out as a hard copy to a White Mail Examiner for marking traditionally Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 38 9 0 Online Standardisation Activities This section should only be read if you are a Supervisor on a unit which is to be standardised online Online Standardisation is a remote Standardisation process undertaken in place of holding a traditional Standardisation meeting The aim of the activity remains the Same to ensure a well founded and common understanding of the requirements of the mark scheme that can be applied reliably Using the ePEN system it is possible to prepare Standardisation material and make this available online Markers then have to complete three stages Familiarisation Marking Marking Practice Sets and Marking Qualification Sets As soon as candidate scripts are scanned and items become available on ePEN no
13. QOlai GENERAL 10000 10000 Edit ae Q02 GENERAL 10000 10000 Edit raahen Q03a GENERAL 10000 10000 Edit Q03b GENERAL 10000 10000 Edit Q03c EXPERT 0 50 Edit Q04a GENERAL 10000 10000 Edit Default Cap At Current Q04bc EXPERT 70 70 Edit Default Cap At Current Q05ab EXPERT 10000 10000 Edit Defau At Current Use the Default Q05c EXPERT 10000 10000 Edit Default Cap At Current option to change Q06 GRADUATE 0 10000 Edit Default Cap At Current QO7ab EXPERT 10 50 Edit Default ap At Current the Cap to equal Qo7ed EXPERT 12 70 Edit Default Cap At Current the allocation all Edit Default Cap At Current Copyright 1999 2009 NCS Pearson Inc All rights reserve Use the Cap at Current Option to change ePEN helpline freephone 0800 169 9202 the cap to equal the number of Items that have been marked edexcel You can change the cap either through the Edit Default or Cap at Current options Once selected the Edit option will allow you to change the cap to any value greater than the amount already marked but lower than the Allocation Use the Update function to submit your change or use the Cancel option Figure 20 An Amended Cap Item Type Cap Allocation Q 7ab EXPERT 10 50 Edit Default Cap At Current If you wish to increase the cap to equal the Allocation then you can choose the Default option as follows Using the Default Button to Change the Cap Item Type Cap Allocation Q07ab EXPERT 50 50 Edit Cap At Current If you wish th
14. S521 01 G01 tem Total 11 4 6 5 0 In the example above J aninemark has sent 9 items to review of which 4 are still waiting to be dealt with by the Supervisor This Marker has sent the greatest number to review when compared to any other Marker If a particular Marker is sending many items to review it may be that they need further guidance on a particular aspect of the mark scheme which the Supervisor could provide If one item is always being sent to review by different Markers it may be that the candidates have written outside of the viewable area If this is the case it may be that you need to allocate more time to resolving these queries in Pulled Paper 24 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 5 3 4 Validity Reports There are two reports which are used primarily by the Principal Examiner of a paper to ensure that the correct number of Validity Items are available during the marking period and that the Validity Items are appropriate Validity Calculator Every paper on ePEN requires a specific amount of Validity Items to be available to the examiners The number required for each item on a unit is confirmed by the Qualification Delivery and Award Manager QDAM The Validity Calculator report can be used to track how many items have been proposed and commissioned To view the report select Validity Calculator the drop down menu and then the Paper The report
15. against the cap Normally the cap should equal the allocation If the cap is less than the allocation it has been amended by a supervisor In addition to the marker view of Marking Progress supervisors can also see how many supervisory marks they have submitted in the Supervisory Marking column This number includes Frontreading scores and Backreading where the marks have changed Note The number of responses available will update as more scripts are scanned into ePEN Therefore the Overall Responses Completed By Item percentage complete will also update 5 1 1 Allocation of Items Before Standardisation Familiarisation Marking After the examination has been sat a small number of scripts are scanned to enable you to view candidates responses prior to pre standardisation At this stage all available markers will have the default allocation of 10000 and a cap of 10000 for each item to allow them to mark using the provisional mark scheme to note any issues e g Item Item Type Progress Through Cap Allocation Supervisory Overall Responses Marking Completed By Item Q03c EXPERT 0 37 10000 10000 0 42 750 5 1 2 On the day of the Standardisation activity All the marks in ePEN are reset This means that any marking undertaken prior to the reset is deleted Your Progress Through Cap and overall responses completed column will be reset to display O until marking starts Please note that any items sent to review will not be reset Supe
16. ch fhe Hominatiena Search acalatable Reaponsen Wer Sahect fem Seji dc Pool Dw a Ratchh a S 1 Click on QC pools Clear Oc Momrinaticns Dat Vitii l nj i 4B Applet com MEEBO emer airis en Ape shorted D nirre When you click on QC Pools the following panel will be displayed Select the item from the list select QC pool as Validity and the Commission State should be Proposed Note you can select All in theses search parameters if you need your search results to be less specific Figure 12 Searching for Proposed items 5521 01 000 4 2 Select the QC Pool as Validity Oc Pool Practice Qualification or Select All if applicable Search Qc Pools 1 Select the item or Select All tem Commission State Select 3 Select Proposed for the Commission State or Select All for a search for all states Reset Search Clear Oc Pools Searcl Once all your search parameters have been selected the ecrzrzeee reste ciate button will become available Note The options in the Commission State drop down menu can be defined as shown below Responses that have been submitted into the Validity Pool for consideration Proposed aa as Validity Items Approved Responses that have previously been approved or commissioned into the Validity Pool Deleted Responses that have been deleted from the Validity Pool Retired Responses which have been retired from the V
17. each stage proposed or commissioned can be tracked using the Validity Calculator report in ePEN Not all senior examiners will be asked to help with this task If you have been sent a message the option to Mark Student Responses is not available You must select Messages and read your mail first If your paper is Online standardised you must complete the Practice and Qualification Sets for the item before you have the option to Mark Student Responses 57 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 Appendix 1 Sequence of Events The list below is to be used as a guide to the sequence of events that Supervisors may be involved with It is by no means meant to cover all aspects of the marking process More detailed information can be found in the Online E38 39 booklet Before pre standardisation Receive passwords to Practice and Live Marking Complete online training courses and access the Practice Site Day after the exam access Edexcel Gateway for a copy of the Question paper and pre standardisation mark scheme Access live site after examination to practice marking live responses Familiarisation Marking Traditional Standardisation Activity Pre Standardisation Meetings use photocopies of either whole scripts or items Agreement of standards marks awarded Mark scheme updated and posted on Edexcel Gateway Markers and Supervisors notified of assigned
18. enter the website address of ePEN as advised by Edexcel Download and Install ePEN link Helpdesk telephone al Local intranet Q pack Z x a A P Search Pe Favorites L2 y4 7 v ha 3 Address a http www examZscore com edexcel OPEN Online Training Site Pract ce Site Download and Install ePEN Live Mar Level Administration GCSE LTE Copyright 1999 2007 NCS Pearson Inc All rights reserved ePEN helpline free phone 0800 169 9202 If you have any issues with the installation please contact the helpdesk on 0800 169 9202 This number can be found in the bottom left hand corner of many ePEN screens The installer will also capture some information about the PC you are using An internet explorer window will open displaying this information and giving you the opportunity to send this information to Edexcel You do not have to submit your details but if you do it will allow us to greatly improve our support if you should have any problems using ePEN We will only pass on your details to approved third part support personnel if it is necessary and with your permission we will not pass your details on to anyone else 2 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 Once you have installed the components for ePEN and have established an Internet connection you are ready to log into ePEN To log into ePEN or th
19. essential that Senior Examiners frontread and propose Validity Items The Marking Centre Manager is responsible for commissioning the Validity Items for these items Validity Item Monitor This report contains statistical information on the commissioned Validity Items marked by examiners It allows you to see if the item is performing as expected This report enables you to quickly target the specific Validity Items where there isa low level of agreement with the true score It is recommended that this report is used in conjunction with the 0 Marker Statistics Summary and Marker Difference reports The report shows performance by trait including the count variance and scores given by Markers against each validity response To access this report you must select Validity Item Monitor in the drop down menu in the reports tab The figure below shows the report page Figure 30 Validity Item Monitor Selection Reports End Task Please select a report Validity Item Monitor v Paper 5016F 1F v Item Type ALL v Item S016F_1F_Q01ai vi Select the paper and item required before generating the report Figure 31 Validity Item Monitor Display Reports 01 Microsoft Internet Explorer provided by Edexcel i E hip iili SLA hbs d hes apps hens sce Sade A 4 PRLS SB HA PS i Eida riie bl wo ite X Bl Pie Edt yew Fyni Jook Hap A fuet fe Gl mee bees Gtk e ease select a report Walebty Hem M
20. items have been proposed and prior to the pre standardisation meeting Figure 52 Frontreading Escalation Dialogue Screen FE facalate Rocpone Ea oo ce Select the required m ot Categorie Escalation Option Practice Qualification or Validity Select Gc Pool Typa See PTASIE E iilid widiy For practice Items you can add draft annotation at this stage by clicking on the Annotation Text radio button and typing the available space Provide an anmointion Cyptional Figure 53 Sample Annotation Salect Ge Poal Typa Salect Clasaiicaiien Practice Mol Categariril Provide a reaen Oplional 1000 characters mas z Sample draft Annotation Anmelation URL i p gt Click on Escalate 41 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 Note You can also escalate from the Review or Backreading function if you see a suitable response Once you have escalated the response you will return to the image of the candidates response To view the next candidate response you must click on and repeat the process until sufficient items have been escalated for both Practice and Qualification This process must be completed for each item on the paper that will be standardised online Note For non online standardised papers the option to escalate to Practice and Qualification is not present Note the Reserve for future use classific
21. on to send the item to the Validity Pool You will then return to the candidate response screen To view the next candidate response you must click on skip Note the Reserve for future use classification should only be used by Principal Examiners to select responses for ERA Examiners Report Assistant For more details on this process please refer to ERA user guidelines 4 3 How to Commission The Commissioning Frontreading task is used to approve the Frontread items and make them available during the live marking This means that the candidates responses must have already been selected and escalated into the Validity Pool as described in 4 2 How to Frontread and Escalate From the Welcome screen select Commission Frontreading from the Available Tasks menu Searching for Validity Samples Once you have accessed the Commission Frontreading screen you need to search the Validity Pool for the frontread responses by selecting the item from the Current Items menu 9 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 Figure 11 Commissioning Item Select Screen T NCS Poarson amp FIM Scoring Microsoft Intermed Explorer provided hy Edexcel Fie EG View Favorites Teoh Help Gu x F pl tech gp Pevertes 4 4 loa z3 htp 159 LaL 127 FAAO epenvebapp toner ret Elo h Tear
22. s tab When the Marker s tab is clicked the marking grid will self populate with the original scores given Then submit the score using the Submit Score button as shown in the diagram below Figure 39 Agree the Score DCS Pearson e PEN Scoring Microsoft Internet Explorer provided by Edexcel Fle Ect Veer Favorite Took Help Ge Q x 2 Fi po Search sp Favertes A E m 3 Akies hitpe 159 182 137 99 See epa Poaceae rr aan OOT AE apelet comuncs scoring viewerz Wirda enap started Click on the Marker s tab to populate the marking grid with the Scores Then click on Submit Score button to confirm the marks If you are backreading a multi trait item and do not agree with all of the marks the Marker has given the item you can amend the marks on a trait level as shown below Figure 40 Changing the score on a trait level T NCS Pearson e PEN Scoring Microsoft Interne Explorer provided by Edexcel i File Edt View Favos Tools Hep a O x 2 Gh Eh apres Qh G 4 mL i HHS iH E EE EEEE EEEE iapanesasiataiz Bisan JHE H Pt ii z ARESE Hix e z wl 1 s521 D2 Q26 ype C ompleiescore Type Dackea esne 20335 1 wi Score OFta o26b Send To Review z t o Skip Sened Mail Escalate Clete Viewer iF it l 1 The Supervisor s amended score is shown as
23. trait level shows True scores awarded to each response in the Set and any deviation from True Scores by Marker Note blank equals perfect agree 54 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 The Practice and Qualification Calculator reports give details of the number of Sets Required the number of Sets Proposed and the number of Sets Approved for each item on a paper The reports can be run per paper and should be used during the Pre Standardisation meeting to track progress Figure 79 Practice Calculator vi 552101 v Shows number of Sets at each state as well as 11 02 2008 12 43 45 the number of Previous Page next Pae E responses required assigned to approved sets 5521 01 Practice Calculator Level GCSE ltem No Sets No Sets No Additional Sets ta be No Responses No Responses Assigned Proposed Approved Approved Required to Approved Sets 2 1 11 4 521_01_01 1 521_01_Q03c 521_01_Q04c 5521_01_Q07 521_01_ 09d 521_01_ 10c 521_01_ 12c 521_01_Q13a 521_01_ 13d 521_01_14 Sey ab ab bss be ab Pepe e fF AA Ff N N B B O O GO O O O O O Se oy keki pani pes pes Ea n caa Cia ens ce Ne ees a a ee ao O O O O O O a O The Qualifying Summary Report by Item give details of the date on which a set was taken and the average agree per trait for all sets attempted on the paper Figure 80 Paper 6663 01
24. 0 items 50 53 x100 96 were in complete agreement with the True Score 4 or 3 Validity Items were marked below the True Score and outside of adjacency Non Adj Low column E Marker has marked 37 Validity Items 86 or 32 items 32 37x100 86 were in complete agreement with the True Score 11 or 4 Validity Items were marked below the True Score and outside of adjacency Non Adj Low column 3 or 1 Validity Item was marked higher than the True Score and outside of adjacency Non Adj High column There is no indication on this report as to how many marks the Marker is away from the True Score This information can be found on the Marker Differences Report This report does tell you the that the Marker is within Adj acency Adj Case Study 2 Interpreting the Backreading columns in a Marker Statistic Summary Report The report above shows the backreading results for 3 Markers A to C on an item which has 2 traits i and ii Marker Trait Low fAMarker i 22 o 95 5 fAMarker ii 22 0 100 0 BMarker i 2 0 95 s Bme i 22 5 90 s evare 1 2 0 t00 0 ewer i 2 0 100 0 A Marker has been backread on 22 responses made up of part i and part ii Part 95 or 21 times 22x95 21 the Supervisor agreed with the Marker s score for part 5 or on one occasion the Marker s score was higher than the mark awarded by the Supervisor Part ii 100 of the Mar
25. 000 motes F W eat g zs EXPERT o 10000 10000 i350 045 GENERAL om sree 10000 om i ov 250 3 E Internet 10 7 From this screen you can view your own marking progress the cap the total allocation and the overall progress of marking on the paper for each item you have been assigned to As a supervisor you can also access this information for an individual in your team or on a team level by selecting All from the drop down menu By selecting team members from the drop down list you can see what the individuals have marked to date their percentage Contract Completion as well as the overall progress of marking against the number of responses scanned for each item Progress Through Contract The default view will show your contract and allocation values This information is available as soon as ePEN is live Supervisors can also monitor their team by using the dropdown list to select individuals Figure 17 Monitoring an Examiner Prog I Progress Through Contract s Through Contract End Task User marker test markertest Expert Examiner Paper 5016F 1F Faper Contract Role Contract Contract Completion Marked Contract Type Amount Date Progress 5016F EXPERT Expert 11968 17 08 2009 120 iF Examine The individual The Contract Completion This displays the number of contracted allocation is Date for the paper is responses the individual has displayed displayed dd mm yyyy marked so far Marki
26. All rights rewired a EN helpline free phone ooo 169 202 edexce l Select the criteria in the following order to view a specific Marker s sent review items Select This allows the system to begin filtering all the responses based on the item item selection For example the category called Mark Scheme will be chosen to allow Step 2 Select Markers to submit for review responses for which they have difficulty in Category applying the mark scheme If you do not select from a category the search will return all selected responses regardless of the category To review responses from a particular Marker select their AA number from the drop down list that will be displayed or leave the scorer field blank to return all Marker responses for the item Step 3 Select a scorer You can quickly change your search setting in the window above using the button 33 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 Once the search criteria that you require have been entered click on the Query button to display the search results A list of responses that match your search criteria will be displayed Select any or all of the responses and click Review Figure 42 Selecting the Responses to Review F Review Selection 04 Microsoft Internet Explorer provided by Edexcel x fie g yem Fgvertes Jook thi a Ow g x 2 H P Saach Pg Favores ia amp 5
27. Confidential v1 0 Authorised By Peter Challinor October 2010 Figure 22 Marker Statistics Report cel ae lt Once generated the report will be displayed etre sestim Note You may need to use the Previous and Next page arrows to view all the results f Reports 01 4 Use this arrow to maximise the screen taskIndex 1 amp sid 66173144 E366 4394 4E93 C17 4486 1 CS608pid 18mid 261 Gay JE File Edit View w amp o9 x fa GX am p age G Tools End Task View Report Reports Please select a report Marker Statistics Summary v Paper 5016F 1F v Role Type ALL arkertest v Select a format v Export 4 i a Marker Statistics Summary Yo ucana SO Paper Level GCS Item 5016F_1F_Q01ai ex p 0 rt t h S 5016F 1F 1 T T T T o report in several a u m formats OQ0tai o Read Time Rate Validity Low Low Read Low Agree hours responses Reads per hour Principal Examiner T test m test markertest 6 0 00 1221 4 o 0 0 o o ou g lt amp Team Leader ALL Mi Marker v Validity Backreading v Copyright 1999 2009 NCS Pearson Inc All rights reserved ePEN helpline freephone 0800 169 9202 edexcel Internet R 100 When you first run this report at the beginning of the marking period we recommended
28. Gon aka A anann Genad Semele Honinaions Search Fata Responses AA ny i i m F rpi F i gfe ror fanen peanae y eere um m A mm ee Te S561 01 O02 Qualticat Ned Categorized fi EEN Ot Pool a 5501 0 007b ius paiay TT sSicbect Al m Dock x ao 5561 01 002b jGuaitital NotCateporred A ocmi r aa 5561 01 002b Guaiical Hil Categorized 6 jotia Beee esetet a A M al to 17 S861 01 G02b Ouai Nel Caligonted 4 STA E h t d p m m Hack 43 S861 01 G02n Practice MoCategonzed 4 at Decld r ac escalate response A ey a r D men Maa is listed with details of n n whether it is a Validity Contre Practice or Qualification item J Chgae Viewer sc e E Applet com nes scoring vieres WindoreadvieserApolet started A Sd Local intranet To view the response click on one of the items in the results list The candidate response along with any annotation will be displayed Figure 57 SMCS Paareon PEM Scoring Microvell Internal taplorer provided by Ldoxcel Il Pie fdt ew Favorte Took Hep Fg Q O h a G Psa fzr O Gy wh Biisi EE hips lipractke ncamulscone comfepen mebanp itomederrlet T f QcNomimatons O i You can now rem QePool Classification Score Doch b deel 1 5561 01 ooze Pracice et Categorized 2 jotia Ari revi ew amend the score EELEE ree TETT i gan 3 SSG oi Way cagot A Ej d 4 S860 O02
29. ID boxes e g 4 in the Document ID field This will return all the documents with a number 4 in the DOC ID This may be of use where there are more than one person selecting validity responses on the same item Once you have made your selection and clicked Query a table showing a list of responses will be displayed Figure 8 How to select responses to view F Front Road Sulaciien ETE E provided by Edexcel Ble dk Yew Favorites Took lbe Ar Qu OGD Pawn frr a 2b wy Pelle ek A Fithool praectL eres irp eu com iipeniebape frontresdingsuerr do 1 To select all of the responses click Select All Performance m A Frontreading Evaluation Network SelectAll Merina ai Has Search End Task Fro ntre al Preqeect 352 lz Trem 3521 02 QUT Fation Batch 4t Humber of Documents 20 Criteria sel Document M FAS Batch DD Stack UD KRespouse Stabe CL DL AEDN0RS ESSE INO est WOMIT 1 Complete m O jnesooessig2 talne NEE witii 1 Unscored 2 OM lo1esoosssr799 FROOOS 1380 OOOO A 1 Unucered Or select Items individually O japesootesraoo al miei Mimered O jotesoosserace FROG 31 Coccrcncrtc l Tinsconed L m jeaoossaTaos FROM S71 oon 1 Lingcored eS ee eee EE aE ee O jn esoosse7sn FROM 1386 witii 1 Une cnedl O eleeogsse7aos FRODU 1392 artuta l Tnicsred O OERO YITEDE FROO04 388 whogoat l Tnieered ma MLE N Al IOWA Eo Ow a a 1 Tirra srar d Please note that if you revisit this function for a secon
30. Live A sample of non subject specific responses from a previous series is available for marking on the Practice Site This site should be used to test if ePEN has been installed correctly on your PC and also to familiarise yourself with the functionality within the ePEN system This site is available throughout the marking period 3 0 How to Log into ePEN live marking Open Internet Explorer and enter the internet address assigned to you This will take you to the Landing or Home Page From here first select the level of qualification you wish to mark GCE GCSE The page will then display the administration subject group associated with that level Select the relevant administration subject group and click the Enter ePEN button Figure 3 ePEN Landing Page or Homepage E ePen 05 Microsoft Internet Explorer provided by Edexcel File Edit view Favorites Tools Help Q pack z x a Ay Ss Search Pe Favorites Ea 3 Address http fwww exam2Zscore com Default aspx First select the Level of Qualification e g GCE or GCSE edexcel OPEN i Then select the Serr Shae Administration subject GCSE English GCSE November 2007 Mathematics GCSE November 2007 grou p Mathematics iGCSE November 2007 i Science 360 GCSE November 2007 Enter button Enter ePEN Copyright 1999 2007 NCS Pearson Inc All rights reserved ePEN helpline free phon
31. Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 10 0 Online Standardisation Monitoring and Reports Once the Reset has taken place Markers can access the Practice and Qualification Sets Markers must ensure that they have the final version of the mark scheme before attempting any of the Sets The progress of all Practice amp Qualification is available in report format on ePEN There are 3 key reports which are available on ePEN which allow TL PEs to monitor the progress of Markers on Practice and Qualification Sets QC Practice Report QC Qualification Report Cumulative Qualification Report 10 1 QC Practice and QC Qualification Reports To access the reports within ePEN from the Welcome Page you need to select View Progress and then the Reports Tab The Report Screen Figure 74 showing the default Segre erento IE E Select the report and the paper required you will then need to click on generate report For a non specific search you should leave the other search parameters as All For a more specific search you can select from the options available in the drop down menu for each category Figure 75 The QC Practice and QC Qualification Reports give details of each Marker assigned to a particular item and whether they have attempted the set passed the set how many sets have been seen how many responses within that set were seen and what their agree percentage was It also n
32. Prae Mot Cabtgonzed i eon R i an or annotation as 550i 01 020 Guaita Not Categorized och t B 550i 01 O0 b fvaidiy No Catagonzed 7 M 7 S561 01 O07b Practies Nol Categorzed U foci E necessa ry SSR 01 G02 jOusmesi No Catagonzed H EEN LOG UA B 5561 01 02b Muakital Hol Categorized mitt l a Bae 0 obze fuam Ho Categorieen I Tam aa 5561 01_0 b Guaita Nol Categonzed 6 EEN ALAL D ji a23 S861 01 002b Guaca Mo cawgonzed 14 aci 4 13 iaus Pracboe Boor steguraee 4 eu Laai ff LOCC A 4 E 4 E A 7 r o 5 7 i 7 T a 7 w S eTeiAIcin e RISTA aiA Tun Og SUN GUO gun aT THF O ET JB Appt cor nes scoring rieres Windoresdileserapelt started 4 E Local intranet 43 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 9 6 Uploading Annotation using the Create Annotations option from Available Tasks The Create Annotations option should only be used if you have specific symbols or images that are necessary to feedback effectively to the Markers Images and symbols that do not appear on the keyboard can not be loaded into ePEN without using the Create Annotations function We would expect that the vast majority of annotation will not use the Create Annotations option How to upload annotation for an Item using the Create Annotations Option To upload annotation and review the item at the same time you will need to open the ann
33. Score for a set Shows the average percentage within adjacency agree of the True Score The figure below shows a QC Qualification Report Please note that this report is often over more than one page so you may need to use the navigation tool to review the whole report You can also export this report in a choice of formats so that you can review them offline Shows number of Sets and responses seen Figure 77 QC Qualification Report tem Mame Reader Ba Date Attempted Date Qualified Set 526 Set Passer SAUS SE M_a experti Qverridden i 574 _01_00 macko Successful 1102 2008 13 08 42 1402 2008 13 09 20 1 1 55 04 a0 shehab Started Of 02 2008 15 46 00 0 54 4 04 03c experti Overridden 0 0 0 574 04 G03c expert Not Started 0 0 0 s51 _01_204c yamina Mot Started 0 D 0 55 01 a04e U Mot Started 0 0 0 Qualification Status shows if Shows the date that Shows whether Marker has the set has been the Set was first qualified by date Shows average started successful or not attempted attempted Agree for Set 53 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 10 2 Cumulative Qualification Report The Cumulative Qualification Report gives details of the true scores awarded to each response within a set and a comparison with the marks awarded by the Marker This report can only be run on an item by item basis This report is available via the Welcome Page selecting V
34. a in the marking grid The original Marker s score isshown asa inthe marking grid When a mark is changed by a Supervisor it will be their scores that will be carried forward as the candidate s mark Prepared by Online Assessment Authorised By Peter Challinor Company Confidential 32 v1 0 October 2010 7 0 Review The Review feature allows a Marker to send a response they cannot mark to a Supervisor together with an explanation as to why they are unable to mark it Note After ePEN is reset after familiarisation marking any items in review will remain there until marked by the Supervisors on the paper From the Available Tasks menu of the Welcome screen select Review Firstly select the item you want to review and then the following window will appear Figure 41 The Review Sample Selection Screen J Review Query H Microsoft Internet Explorer provided by Edexcel De Cae Yew Pyvirtes Took EES x X A Supervisor can select A Review what to review from this Ferformance Evaluation window based on a variety Network on of selection criteria eg Project S52 2 Trem 5521 _02_ 07 Query Ae zer End Task Te a m Sc O re r 1 O r Cat e go ry Team Selecta Ty StOrer Select a Soon Batch LD Document LD Number af Responses 20 Category Select a Catagory Por a non specte search enter Yo m any of these bekis Copyright 1999 2008 NES Pearson inc
35. alidity Pool responses 10 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 Searcl Once you have clicked the rre E ei button you will see a folder with the item name under QC Pools To view the list of contents click on the magnifier glass symbol next to the Item folder icon then again onthe next to the Validity n approved responses folder icon and then on the icon Proposed folder icon The n in this case represents the number of responses that have been escalated into the Validity Pool Each proposed response is listed as a QC number You can click on any of the responses listed to view that response and approve them it into the Validity Pool Figure 13 Proposed items TET eT EEE TEE EEL EE EE EEE CETL T RTE TERETE EEEE EEEE ic Pools 5521 01_G09b QC Pools folders expanded E Validity 0 approved responses i Proposed O aco1s43000rg1 97 O aco1s43000791 98 O co1s430007sEr2 O aco1s430007sErs3 O aco1s430007sera O co1s4300078ers Responses ready to commission approved into the Validity Pool After clicking on a response QC number a window similar to the one below will appear You can resize the window in order to view the candidate response more easily Figure 14 Commissioning to Approved 3 NCS Pearson e PLN Scoring Micreseft Internet Luplorer provided by Ldexcel Selected re
36. an filter the report ESS EES SES a Firstly highlight the row amp ls Gh Greate we F p ma e FEE a ll bas ah containing the column Oc SAS SMe Poem Mees i amo Gy titles This is where the Ca Ba a adta A Se get He itenly ath ches 1 Ah A ee i filters need to be C w z s AKE T E 5 Q Qualica on naw applied 5 960l ot Gualiticaaen Repor Progen Himst Bas Haram Tinie Assigned Bale Guid i Ba Passe Respereea Sain Agran Ponce 7 Bion HiH ara 140i t hot Gade 9 S916 S16 01 Fa EEF hot Gaui 11 Sm AiG H Gra hot Gauche 13 Siem Eme 041 Gria 7100 HOLT VE IESE H uaii 1 GLE SERIO AE i7 Ein SAR i oda WIE hii Guakirri fi 1h it HiH Sara hot Gade Fi ke plea SG H Ga e DOT 1711 35 ST I eno 160 a a a 32 enem E16_04_ code 400 Hot Gausitesd 2 ae ee E r e a5 nim Bni i om KoE Guidad 25 amem a miam SAR M amib DE Fry S AE SOT Oe m 6m SiH Fe 7 Woa e HADE Qualifications rew jal 1 59 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 Next click on Data on the menu at the top of the screen Ey File Edit wiew Insert Format Tools Data Window Help You will see the following dropdown menu Src Form Advanced Filter Subtotals Validation Highlight the filter option to see the sub Table menu Click on t
37. antas Paper 2010 1H ka Them Tree S0716F iF Validity Item Monit r Level GCSE item S076F_WF_O03 SD Tred Ho Tees Loew Hon Low Api Perlez High 2d igh Han Treo Doone 7 Barked Ad Apit h Ad T OPII TI H 1 a 0 bog a 0 10 he 1 100 it D ir g I bag Pierae nannies r Page 1044 Se SS SHEMI PHH ta Oe ee AE E P E PEN helpiree freephone 0800 189 9202 7 7 trary edexcel Crore d iieri A e 26 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 The columns in the Validity Item Monitor report contain the following information The Validity Item identification number E g QC500012345 The trait name e g Q2cii Q2cill The total number of times so far that this Validity Item has been marked The number of times so far examiners have awarded a lower mark outside of adjacency The number of times so far examiners have awarded a lower mark within adjacency in relation to the True Score expressed as a percentage The number of times so far examiners have agreed perfectly with the true score as a percentage The number of times so far examiners have awarded a higher mark within adjacency in relation to the True Score expressed as a percentage The number of times so far examiners have awarded a higher mark outside of adjacency The True score assigned to the trait of the Validity ltem The number of times so far a particular mark has been aw
38. applicable Please remember that the Marker will have a copy of the mark scheme with them so it may be preferable for the annotation to make reference to the mark scheme Including page number rather than replicate large sections 9 5 How to review responses annotation that have been escalated From the Welcome Screen select Commissioning Frontreading Then from the QC Commissioning screen select the item from the Qc Nominations Item menu Figure 54 Q O dG Se r O 2 458 US Give pa z at Bie Beware Seven Select the n a required Item TEs S from the drop OS 42 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 Select Select All for both Qc Pool and Classification to return all the frontread responses for this item Then click on Search Figure 55 Search Qc Nominations Item 5561 01 Q02b Oc Pool lt Select All gt Classification lt select All gt Once you have entered your search parameters Doc Id click on Search l Reason Reset Search Clear Oc Nominations The search results will appear in the Qc Nominations window Figure 56 SMCS Poareon PEM Scaring Microwell Internal txplorer provided by Eduscol Pe Cet Yee rois Took Hep ee Be mrt order of the list by item Classification clicking on the column 4 Oc Pool Oise ftp ae 7 a J aE mi a Se p eeng headings 3 RNI ANAN il nagarien t corga
39. arded as a percentage 6Xxx Validity Item Level GCE Item Monitor 6xxx_01 Q01la Cumulative Frequency Distribution Document ID Trait No Low Low Ajd Perfect High Adj High True 1 2 3 Times Non Adj Agree Non Adj Score Marked A Ao QC0184300069429 Q0la 3 0 0 0 0 100 0 0 0 0 0 3 100 QC0184300070260 Q01a 0 0 0 0 75 0 0 0 25 0 1 75 25 QC0184300069941 Q0la 0 0 0 0 50 0 0 0 50 0 1 50 50 QC0184300069719 Q0la 0 0 0 0 100 0 0 0 0 0 3 100 QC0184300069903 Q0la 0 0 0 0 66 7 0 0 33 3 0 67 33 QC0184300069924 Q0la 0 0 0 0 66 7 0 0 33 3 0 67 33 QC0184300069925 Q0la 0 0 0 0 66 7 0 0 33 3 0 67 33 The Validity Item Monitor report shows each validity response commissioned and a detailed overview of how a Marker performed against each item Problematic Validity Items should be easily identifiable using this report and the appropriate action taken For example a problematic Validity Item could be retired In the example above QC0184300070260 has been marked 8 times by various Markers 6 of the scores 75 have agreed with the True Score 2 of the scores 25 have been marked generously and outside of adjacency High Non Adj Note if a Validity Item is retired or deleted a replacement must be commissioned back into the system to ensure that the correct number are available to the examining team 5 4 Help If you select the Help Tab you will be able to access the activity flow charts electronically There are also links to Edexcel Gatewa
40. assword reminder link fal 2 iterrest Login Once you have entered your details click on the button Please note that your user name and password are case sensitive Changing Passwords Please note that you will need to change your password the first time you log into any ePEN site and that you may be required to change your password during the marking period to maintain security If you have forgotten your password you can click on the link on the Password Reminder Link to receive details of your password via email If you enter your login details incorrectly 3 times the system will lock your account If this happens please contact the helpdesk to re activate your account The Welcome Screen Once you have successfully logged into ePEN you will be taken to a welcome screen as follows Figure 5 Welcome Screen PPIH Welcome E coe Gere oer provided Ery T iewe E T o Hetzer raU E E r aa e ar a D Bie pH Bee Barer E Hii a ay D PIH vehi N ia B eo oh The Items available to you are displayed el Welcome in a drop down menu Performance Evaluation Network Pa virga Emauit Araibis reca Meaty e LADIJ DTi atie Phe x Ar athebthy Tasks TES IEI0 Gbagi tea Your available tasks are displayed in the drop down menu Cepaight i RPI RCS Poco ke HMrigib romid aA Hegia be gera CODD fi 000 eee H a a taal a ie Moora Message of the Day and Admi
41. at have been proposed To do this you can click on the ePEN icon in your start bar or click on an area of the screen away from the active pop up box see diagram above 44 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 Once in the Welcome Screen select an item from those available in the drop down list Then select Commissioning Frontreading from the Available Tasks drop down list Select an item Performance Evaluation Network T Welcome j Peter Challiner Then select fhere A commer Commissioning Coma yar Frontreading from the drop down menu Fror ndrer Luar hilaris reari gjire ETETE You will need to search for the responses as previously described by using the Qc Nominations area of the commissioning screen Enter the search parameters and then click Search Figure 61 Search Oc Nominations r Item 5561 01 Q02b ae Classification lt SelectAll gt v Doc Id le O a Reason Se ooo O i l a d Anae coe en coreg emma freee Epir ote Sy cena pre The escalated responses are then listed in the Qc Nominations area To view a response click on a Document ID and it will be displayed with the proposed True Score and any existing annotation if applicable Underneath the image the proposed True Score will appear If the True Score Is incorrect use the marking grid on the right hand side to enter a new true score an
42. ation should only be used by Principal Examiners to select responses for ERA Examiners Report Assistant For more details on this process please refer to ERA user guidelines 9 4 Guidance on the Contents of Practice Set Annotation Feedback on all the responses within a Practice Set is provided to the Marker in the form of annotation and the True Score Annotation is only necessary for Practice Set items Annotation should be prepared whilst responses are being proposed for Practice Set This task of proposing annotating the Practice items should be undertaken during Familiarisation Marking prior to the pre standardisation meeting This annotation should indicate how the True Score has been assigned to that particular response so that the application of the Final Mark Scheme can be understood by the Marker Markers are advised to read the annotation carefully especially where there Is a difference between the mark they awarded and the True Score The breadth and depth of annotation will vary from item to item but should always be clear and concise Note Some items may not need to have any annotation because the mark scheme covers the issues Annotation for items with a greater available mark range may need to Include the following e The band level if appropriate e Aclear explanation of where the marks have been awarded and how the True Score was arrived at e Reference to the mark scheme e Reference to the Assessment Objectives if
43. aved Figure 67 The window supports normal text and limited special characters The window also supports tables so small sections of a mark scheme or assessment objectives for example could be added although it is preferable to make reference to the mark scheme rather than cut and paste large sections 47 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 There are Image Editor and Import icons on the toolbar The Image Editor icon will open another window which allows you to add hand drawn text and graphics Click on the Image Editor vi icon and a new window will appear Use the tools to create the desired image Figure 68 E TOniina yfer EIRA Web Page Daley Zea lst Once the images have been created click on the Save ICON METEN Cal E Llelel Draw any images required for annotation using mm 5 the tools on the left amp right of the pete fhe theme jepen lo ji o ajel Jadni W n d OW Once any images have been created click on the Save al icon The image created will automatically be added to the annotation window Figure 69 PIWTUL Tater Uicresett indernet Leptoces provided by deere m ot w e w feoti e r Q x 2 pua yrn Gree E Dm 2o Spe sem Sa ee joa ofa eto amp bd E EEFE The image D ea A1 tE automatically appears in the annotation window fene Sd ace niest To Import a previously prepared imag
44. can then be generated by clicking on View report Figure 28 Validity Calculator Display FA Raporis 01 Microvell Infernal Explorer provided by Edexcel EEEE E T sage Thaskinedex Ltgkbe A MPBOAD 3EG4 4346 E E AT TIENE E Fe EA Yew Fyrtio Test Hele A e Bft m DQ wt hem Gtk Please select a report alerty caleulater Paper ROEGF LF at lhem type ALL ed Validity Calculator Level GCSE Pripis GENERAL Ls GENERAL 0 GENERAL o GENERAL o EXPERT GENERAL 0 EXPERT o EXPERT i a 6 o o o 1 o EME i EvaroT G fi Copyright 1599 2009 HOS Pearson Ine All nights reserved PEN helpline freephone O800 164 2202 edexcel E inteme on The columns in the Validity Calculator report contain the following information Item name e g 5562 01 Q1 General Graduate Expert The number of responses that have been frontread and escalated into the validity pool The number of responses that have been commissioned from the validity pool and are available to examiners Figure 28 tem term Type Mo Responses i Ses ed Approved soe _01_201 EXPERT 1 1 5921 _01_202 GENERAL 0 0 5521 _01_2034 GENERAL 0 0 5521 O01 _203b GENERAL 0 0 5921 _01_203c GRADUATE 0 0 The number of Validity Items proposed and commissioned is also monitored by Edexcel 25 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 For General items Type 3 it is only
45. d click on Submit Score 45 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 Figure 62 SMCS Paareon 2 PEM Scoring Microwell Internal taplorer provided by Ldoxcel Fils fdt ew Fortes Took Heb O O HAO Lawn frm O 2 2 w Ls Bibis 6 htrsc ipraction ecamdscon com epen mebsop iiomedercdet bee gt eg lng iy i 4 a I i i Selected item OcPool Classification Scone Doch a j556i 0i G02b Pratite Mot Cateporited t C01 CET Nolcategoned amp O01 response IS 3 6561_01_G03b aiai Not Categorzed 14 Gen i 4 s0 O02 Prate Mot Calegorzed 2 GON i 7 displ ayed 5 as 0i G Guaita NotGatngorzed oon Co a S561 01 007b vandi NotCategorized 7 pcm ne eee LL O FIT s ti 7 S561 01 007 Prarten Nol Categorzed U ecco e S861 01 007b Qualfiecal Noi Catagonred U jacm p 5 ae S561 01 002b fuatical Nel Calegorized S acii 100 EAEn GO Emeen E If necessar ou io 5361 0i 002b Jauamkal No Gategonted 2 jecn y y aa 856101 025 Guata Mol Categonted 5 ati R a3 550 G0 Guanicat NotCatogonsed 4 ati Eoo MIRNA ee OE can amend the a3 S861 01 ban Pree MotCategonsed d Gent i __ y Ji OE a p A z z 7 r sn see Mmaa aa True Score am mas a a a ekl E 3 coat i we a Updale Clasasificalion IH S05 1_01_ Oat arw a E 4eplet com nes sconng vere WinceedviewenApolet started AP
46. d time for the same item you should change your search parameters otherwise the system will return the same sample of items as before For example if you select 20 items for the first search and 20 for the second search for an item the second search will list the same 20 responses as in the first search Therefore we advise that you search for a larger Sample on subsequent searches on the same item for example 40 and not select the first 20 for review 7 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 Copies of the candidate responses will then be displayed for you to select validity responses from Typically a validity response should clearly demonstrate the application of the mark scheme be an example of a commonly seen candidate response not be difficult to read and should not be out of clip If the response displayed is not suitable for use as a validity response you should click on to view the next response If the response is suitable you should award a provisional score and click pn Escalate Figure 9 Escalating a suitable validity response J HES Pearson a PEN Scoring Microsoft Internet Explorer provided by doxcol Pie Edt View Favorite Took Hep iy ant a T n i I g a G2 x Search iP Paworites amp a a E LJ 4 te a bse jip ferent air r gE p g P Errar a PE ii Describe the long profile of the River Thames We Riser Than
47. ding Page to install ePEN Error Message page cannot be displayed Please ensure you are connected to the during installation where the user needs to internet during installation For assistance upload PC specs installing the helpdesk can be contacted am getting an error with J ava Please contact helpdesk with any queries Should my username be the same each series The first 6 digits of your username are your Assessment Associate number and should not be different However the last three digits may change from series to series Please refer to the user details sent to you by Edexcel Locked out of ePEN Please contact the Helpdesk to be unlocked Your Supervisor cannot unlock your account Forgotten password If you have forgotten your password please use the Forgotten Password link on the login page Areminder will be sent to your email address ePEN has asked me to change my password From time to time ePEN will ask you to during the marking period change your password This is for security reasons The number of items marked on View There can be a delay approx 15 minutes progress does not look correct before the View Progress screen is updated Note The data shown includes both valid and invalidated responses This means any items where your score has been changed in backreading will be included in the total in previous sessions this has been subtracted A marker allocation has changed before the Marke
48. e Gee Sil fa oOmb e Dih aude JB blip Vhelirypctessebilt fepen_ordfediorfedior ass Eje Daler us ins l l thal oo EERE Esa BA te The uploaded image will Thess atest bor don D OCOO0RI RI DO004S on 1522_1F_ 0b a utomat i cal ly a ppea r i n gn the annotation window Candidate demenstrated a good krowledor by cowering bath poini raised a the mark scheme pg F edexcel A PEARSON C MPANY SS ieencing leaming charging bees edexcel advancing leaming changing lives aan See A PEARSON COMPANY edexcel reer P Format ed Local intranet Once the annotation for the Response is complete further annotation can be Tu Se ae a prepared by clicking on the new document icon This will automatically close the current annotation and open the dialog box so you can select the next document ID Repeat the instructions above This process must be repeated for all Responses within a Set before the Set can be commissioned in the Commissioning Frontreading screen in ePEN Once all the annotation has been prepared this window can be closed by clicking on the Close x icon 49 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 Note The system only allows one annotation per Response Document ID This means that that if Responses are nominated for use as both Qualification and Practice Items the same annotation will appear Once all the annotations for a given Set are c
49. e 0800 169 9202 You will then be taken to the ePEN login screen see Figure 4 ePEN Login Screen Instructions and details on how to use these pages are provided with your user account details that will be sent to you prior to the examination series Please note that your user name and password are case sensitive 4 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 Figure 4 ePEN Login Screen F MCS Pearson eP EH image Scoring B1 Microsoft Internet Explorer provided by Edexcel i fe Edt yew Fyi jok Hp LJ Gea x 2 Gh see serene C S la 2 ess B bene 199 102 1371 Fe aaea ape Hj Eo M he Edexcel will assign you a user name and password You must enter this here Performance Evaluation Network Microzett s Entermet Explorer verag aut be used to accent Sopbcation CLICK HERE FOR PASSWORD REM H you enter your pasword micemecthy three imes your account wil wutomabealy be locked and you wil need to coctact the AA Hefodesk on 0800 169 9202 te be unlocked Therefore f you have already ined unsuccessfully to log on pero tees and are sll unsure of pour password 1 is recommended bat you cick on the render ink above The ink ved ema your current password to pou for you bp s on ph third altempt and asid you hiana to contact the Hapdi Copyright 1952 2008 HES Pearson lnc All rights reserved SPN Melplinetres phone 055 169 t22 edexcel z P
50. e Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 The general Type 3 items on a paper also require Validity Items approximately 10 15 on each item The Marking Centre Manager will commission these into the live marking 4 3 How to Select Validity Items From the Welcome Screen select Frontreading from the Available Tasks menu Figure 7 The Frontreading Sample Selection Screen 1 Select the item from the drop down list T Front Reading Query 04 Microsoft Internet Explorer provided by Edexcel fe Et ye Fpa Tonk be Lid an gt x J fi pl rh IF Pareri gA eda gt ga a E Akie a hip iipraid mahing eu com pentvebapodMomeservet 2 Enter a sign into one of Performance e Frontreading the empty rectangular Evaluation S A f Network 7g boxes Eg in the Batch ID box Project 55210 ltem Scleceaniem Hatch LD Shack OD Noms f FAS ID Document ID ma E 1 I 7 a 4 Click on Query Topu uest enter an ID mits at least one of the Babe ID Stack ID PAS ID and Deeement ID fields te necceeshlly query che database Fer a nen spece search enter Y in any of these Gelde Copyright 1eea Sed6 MCS Pearson imc All rights reserved OPEN helpling nee phone 0400 1464 Baga 3 Enter the number of responses you want to choose from Default is 20 Note To refine your search you can enter a sign followed by a number into one of the
51. e Online Training Site or the Practice Site you will need the following information Internet Address URL Username Password These will be sent to you by email and letter before you are asked to start marking Your email provider s spam filter might occasionally hold the email with your login details in your J unk email folder We would therefore advise you to check this email folder 2 1 Online Training Site and Practice Site When you access www exam2score com you will be presented with the ePEN landing page homepage as shown below Figure 2 ePEN Landing Page or Homepage e a Mate ESC g p g i ATE Link to Online Training Fle Ek ye Favorter Tools Hee J FEL J x E of J Serth e Favortes o gt E J 3 Ajeress hapi ee marking daana ul ap gje Link to Practice Site containing historical Items Copright 1777 2007 HCS Pearson IME AN rights recerved BPEH helpline tree phones oo 167 F202 E Done W internet Online Training Site When you click on the Online Training Site link on the landing page you will be asked to provide your Edexcel Gateway username and password Your username will be your email address If you cannot remember your password you can regenerate it by clicking on the Forgotten your password link on the login page at www edexcelgateway com If you are a new examiner and have not used Edexcel Gateway before you will be emailed a username and password shortly after accepting your contract
52. e cap to equal the number marked you can choose the Cap at Current option Item Type Cap Allocation QOo7ab EXPERT 0 50 Edit Default Cap At Current You can combine the user and item selection option to change caps on a paper for a single item for an individual e all items for an Individual e all users with a particular role use not advisable e all users on a particular item type use not advisable To display the result of the selected search criteria in the Allocations Tab you must click on the A button Note The use of the All options in the drop down menus is not advised Where you wish to cap a single item for all individuals or all items for all individuals you must do so in consultation with your Principal Examiner and QDAM Allocations can not be amended by Supervisors and any issues with the allocation for a particular individual should be directed to the helpdesk 0800 169 9202 17 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 5 3 Reports In order to monitor your teams performance you will need to access the statistical evidence gathered on each Marker throughout the marking period To do this you need to select Reports within the View Progress screen We also recommend that you maximise the viewable screen by minimising the menu on the left There are 5 reports that will assist you in carrying out this task which are described in more detail below
53. e file or more lengthy text click on the Import t icon Another window will appear prompting you to find the file to be imported This option should be used for previously prepared files containing specialist symbols or foreign language text for example 48 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 Figure 70 Ou a p JO emh ipanen P bal LJ 8 Peach iB rete a e a Ean Fas Edit Fornat ca ae ae Se a ae ee 0 a H E 7 U 8 E the oo ESEE ESN B Ar He This a t st for doc D QO COO0BMIOO0DMS on 1522 1F_ Cosa ere Use the Browse inne button to search for the file to be Comhidate demonstrated a good knowledge by coverina both pomis r sed m the mak heme pe 5 tetas td uploaded Select a file to upload Bani _Sancul_ Upload WAY Ee TA mar o i fepen cma ec Sd Loca treet Once the path to the image appears click on the Upload button iG oma Wa Local mbvareet Use the Browse button to select the file and then click on Upload A pop up message will appear to let you know that the file has been imported successfully Figure 71 Microsoft Internet Explorer A File uploaded Click OK to proceed The image will now appear in the Annotation window Click on the Save E Icon Figure 72 ST HTML Edvtor Microsoft Imernet Explorer provided by Edexcel Boggy Fie Edt We Foverbes Took Hele Po Ge gt ix 2 Gh psn Grev
54. edexcel advancing learning changing lives ePEN Supervisor User Guide ePEN 4 6 2 Issue 3 A PEARSON COMPANY Edexcel Limited Registered in England and Wales No 4496750 Registered Office One90 High Holborn London WC1V 7BH _ _ _ Contents 1 0 Introduction 2 2 0 Getting started 2 3 0 How to log into ePEN Live Marking Site 4 4 0 Validity Items 6 5 0 View Progress and Reports Updated from November 2009 13 6 0 Backreading 28 7 0 Review 33 8 0 Marking Pulled Papers 36 9 0 Online Standardisation Activities 39 10 0 Online Standardisation Monitoring and Reports 52 11 0 ERA 55 12 0 Frequently Asked Questions and Trouble Shooting 56 Appendix 1 Sequence of Events 58 Appendix 2 Using Filters on Exported Reports in Excel 59 Note This guidebook should be read in conjunction with the following documents ePEN Marker User Guide ePEN Marker Activity Flow Charts ePEN Supervisor Activity Flow Charts Online E38 39 Edexcel is one of the leading examining and awarding bodies in the UK and throughout the world We provide a wide range of qualifications including academic vocational occupational and specific programmes for employers Through a network of UK and overseas offices Edexcel s centres receive the support they need to help them deliver their education and training programmes to learners For further information please call our GCE line on 0844 576 0025 our GCSE team on 0844 576 0027 or visit our web
55. ewer M creselt Internet Explorer provided by Edexcel oS m BOD RE Se Cee Navigate through the _ A pages until you find the p m page s of the script 1 Fenta ner with the selected item s annie wi or page i puis NO re response You can also use the drop down menu to go straight to the relevant page Armi al rapeciom pia 5 Once you are able to identify an accurate mark for the item you can then return to the selection screen by toggling Use the keys alt and tab or using the Start menu bar and click on the Score Pulled Papers button This will take you to the marking screen where you can enter the accurate mark into the mark grid and submit score Once you have submitted the mark you will need to close the window that shows the full script Figure 48 Mark Pulled Paper Screen T neS Pearson e PIN Scoring Microsoft Internet Caplorer provided by Cdewcel he Dat Vee Piet Sok Hip g ane Document ID to be noted ing if you still can not assign er a score I A Large tub at popeorn come 5 80 A Regular tub of pop cL Taya thal the 200 Large tub is the beiter value for money Linda says that the 175g Regular tub i the better value fier money Who ta comro Explain the reasons for your anracr re eee eee ee 6 sesis 000 PMG A aA a Assign a Mark and submit score Oof 10 S521 02 O28 Than Megibece Core Tye Sooreresponceld 2031 E ED i 37
56. gs lacia a orasle Crea rapidly ARCHLCADE AA laei Gk KOM ANCE IAIL Mae gA adeh 2A DIN po ANA Ae IO Provisional marks Click Escalate to put the response into the Along Hae nues Are lait iz below Cer E E J validity pool amp sia FOG PMG Alas A o 2 3 Excalale CHO 4 aii l e Send Hal Cluse Views a Andi 3 of 10 647 Of O04 ai ars a Applek com ner scoming viewed Windewed ieved pplet started fl Local nraet Once you have clicked Escalate a pop up box asking you to confirm your decision will be displayed Click either Yes or No as appropriate Escalate Response x AN This tem will be nominated for use as a QC Response This cannot be undone Are you sure Java Applet Window 8 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 Once this is confirmed categorise the response as validity from the drop down list in the screen below Figure 10 Validity Escalation Dialogue Screen Fa escalate Response Select Classification Not Categorized 7 Select Oc Pool Type It is not necessary to _ enter text for validity Provide an annotation Optional canteen responses Annotation URL O Annotation Text Note You can also escalate from the Review or Backreading tasks if you see a Suitable response for the Validity Pool Cancel Java Applet Window Then click
57. h it For further guidance on how to use messaging please refer to the Messaging section in the Examiner guidebook 35 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 8 0 Marking Pulled Papers Markers may have sent items to Review because a candidate has written outside the defined region clip area allocated for the candidate s written response hence part of the response cannot be seen in the Mark Student Responses screen When you view this item in Review you may still not be able to mark the item for the same reasons i e it may not be possible to assign a mark to the response without viewing the whole page or indeed the whole script Typically you will see these responses if your Review search specifies the out of clip category If you agree that there may be some unseen work that could affect the final mark but which cannot be viewed you must send the response to Pull If you are able to mark the response in Review you are advised to do so 8 1 Accessing the Items in Pull To mark these out of clip responses select Pulled Paper Scoring from the Available Tasks menu of the Welcome page Figure 45 Score Pulled Papers Item Selection J AFEN legis Query Micresof baerne Explorer provided by Edexcel Performance Evaluation Network L Diny Piit End Taik Project SEZI 01 Thonn S571 _01_ 901 Bate ID Ste Dog ID Di Priority Al
58. he AutoFilter option Text to Columns Consolidate Group and Outline be iA PivotTable and Pivothart Report Import External Data Project Hame Io Filter arrows i Hi will then appear on the highlighted row on the report Clicking on the filter arrows will display a dropdown list showing all the data contained in that column Scroll up and down the list and select the data you want to view By selecting one of the options in the dropdown list the spreadsheet will automatically show all rows that contain that data in the filtered column EN Microsoll Excel QC Qualification W70 mew 1 MB e coe m poet armor Toole pee wnd tb a tem Name jam Arial 0 e 7 u ESSAAT AlE aA B e lf e Fooie gor Fl Ciuman and Settineriden_tiesktoplor Qualification eit _ All aie OS AOE SRS RE To B 8 7 Bom B ee Ca Sg So el Ee a A H gel he ep th Changes ioe o et a z 3 z 6916 01 _O01b AG Qualification ne 6916 01 O01ci gt 6916 01 O0tcii 5 eregi Than Report oe aie FAF 6916 O1 O01 Perspect Merten Ham He Fisvacker Di e Acer Tial Casak iad Tai TAn Sal Paced Fister Se imen Per er Aspe C a 8 GE T T E E E E i i g RARD RIR M EM a i 3 mE 6916 _01_90z24 g lam S2075 CETA 6916 01 _0 b weer iam of 6916_01_Q02c 12 neo 6316_01_Q02d p 13 mem Sn E 6316_01_Q02e 14 Enem 6916_01_Q02f 15 891601 EEM Ge
59. iew Progress then the Reports Tab From the Reports Menu you will need to select the Cumulative Qualification Report from the drop down menu Once you have selected the report you will need to select the paper and item you wish to monitor The report displays the following information The number of times so far the examiners have awarded a lower mark outside of adjacency The number of times so far the examiners have awarded a lower mark within adjacency in relation to the True Score expressed as a percentage The number of times so far the examiners have agreed perfectly with the true score as a percentage The number of times so far the examiners have awarded a higher mark within adjacency in relation to the True Score expressed as a percentage The number of times so far the examiners have awarded a higher mark outside of adjacency A display of Markers score against the True Score but only if it differs from the True Score The example below only shows one Marker for the given item please note this report does not detail what the pass criteria has been set at This information is available at Pre Standardisation Figure 78 55271 01 O07 Examiner Set Date Trait a panes Lowy Aci ea High Adj High ae scores aaa True scores Set 1 144 2 2007 mor 0 expert2 Set 1 16 01 2008 oy 33 33 D gt 1 Mark33 Set 1 1442 2007 oy 100 ced ia dd Shows Perfect agree and Adj agree percentages at a
60. inefree phone 0800 169 9202 edexcel You can select Previously Backread Yes No or Both amp Done m 1 D Internet We recommend that you select a Marker in the Scorer field as this will return a sample of marked responses for this item for that Marker Please be aware that you can only select the Marker if they have marked the item Please note that if you leave the default Number of responses set at 20 you will always view the oldest 20 items marked by an examiner If you wish to backread the last 10 items that a examiner marked please enter a number of responses greater than the number of items marked by the individual for the question in the search field This means the last 10 they marked will then be displayed e g If an examiner has marked 50 responses as long as you put in a number greater than 50 the search results will show the last responses marked You also have additional options of selecting for example by Team or Trait These and the other additional parameters can be used if you wish to further limit your search within a particular range Trait this allows you to select Backreading by any row in a marking grid You can also choose to sort your search results by Document ID number or Latest Score Time The Latest Score Time will find the most recently scored items for the selected Marker s Figure 34 Number of Responses and sort by selection Number of Responses Sort By Lates
61. is select one of the available formats e g excel or Acrobat file select export and save to your PC 5 3 3 Reviews Pulls and Adjudications The Reviews Pulls and Adjudication report allows Supervisors to keep track of how many items are in Review in Pull or in Adjudication waiting to be marked Only Supervisors have access to the items waiting to be marked in review so it is important that these are dealt with throughout the marking period and that this report is checked on a regular basis The figure below shows the criteria page for this report when it is first selected Figure 25 Reviews Pulls and Adjudications Selection Select the Reviews Pulls and Adjudication Report from I Reports the drop down menu Please select a report Reviews Pulls and Adjudications 7 View Report Paper SO16F 1F v Item Type ALL wj Marker M Test markertest v gt gt id 4 jfi gt bi 100 v Find Next Select a format vi Export 4 You will need to select the paper the item type and the Marker or All before clicking on View Report The figure below shows an example of a typical report if the All option has been selected Figure 26 Reviews Pulls and Adj udications Display F Reports 01 Microsoft Internet Explorer provided by Edexcel AE i Rep 0 USI oho Dacha deporte asco ad eh PLA A 01 AA a ed ee Do The All option all
62. issioning Frontreading option from your available tasks For details on how to select Validity Items from the Commissioning screens please see Section 9 4 1 Selecting Validity Items Frontreading As a senior examiner you may be asked to help with selecting candidate responses that may be used for validity The selection of Validity Items is a two stage process Figure 6 Frontreading and Escalating Approving or Commissioning By Frontreading and escalating responses for an item a validity pool is created This contains the suggested responses with the provisional True Scores Once the final mark scheme has been agreed the Principal Examiner makes a final selection of the responses from the validity pool and verifies the marks for these responses These responses are then commissioned into the live marking and will be used as validity responses 4 2 How many Validity Items are required per item The Principal Examiner and or any nominated senior examiners will be notified of the number of validity responses required per item by the QDAM for each unit The number of validity responses required for each item may vary as it depends on the entry and the number of Markers with access to each item The Validity Calculator report within ePEN can be used to track how many items are in the proposed or commissioned state More information on the reports available in ePEN is contained in Section 5 of this document 6 Prepared by Onlin
63. items Number of items for initial sample agreed to be submitted within 48 hours All Familiarisation Marking items reset in ePEN on the day of standardisation Markers and Supervisors mark their samples for each item View Progress marking tab Supervisors can see when samples are completed Backread samples and feedback to Markers Marking Period Focus is on Monitoring and Supervising the team Tools available include View Progress Reports Backreading and Messaging Marking Review and Pulled items Ensuring progress and quality of marking is maintained and milestones achieved Or Online Standardisation Activity Familiarisation Marking period queries regarding mark scheme to Supervisors Frontread and propose Practice and Qualification responses and upload annotation Pre standardisation Mark scheme updated and posted on Edexcel Gateway Pre standardisation True scores and annotation agreed Sets created and Commissioned Approved by QDAM All Familiarisation Marking items reset in ePEN after pre standardisation Markers access and score the Practice Set s for each item and receive feedback Markers contact Supervisors after Practice Sets for feedback from Supervisor Markers access and score the Qualification Set s for each item and receive feedback Markers proceed to live marking Supervisor monitor and feedback to Markers Marking Period Ensuring progress and quality of marking is maintained and milestones achieved 58 Prepared by O
64. ker s scores were agreed by the Supervisor B Marker has also been backread on 22 responses Part 95 or 21 times 22x95 21 the Supervisor agreed with the Marker s score for part 5 or on one occasion the Marker s score was higher than the mark awarded by the Supervisor 20 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 Part ii 90 or 20 times 20x90 20 the Supervisor agreed with the Marker s score However 5 or on one occasion the Marker s score was higher than the Supervisor s score and 5 or on one occasion the Marker s score was lower than the Supervisor s score The Marker Statistics Summary report is updated throughout the marking period and as more items are marked the more the statistics can indicate how a Marker Is performing This report should be used to monitor your team s marking quality and should help you prioritise your backreading The report can also be exported and saved to your PC This will allow you to review and reformat the report whilst offline Appendix 2 has details of how to do this 5 3 2 Marker Differences Report Where further investigation is required the Marker Differences report shows a more detailed view on the performance of a Marker It contains statistics on both Validity Items and any backread items which have been marked by a senior examiner It displays all the occasions where a Marker s marking has differed from
65. l shosponise z 20 Dore oe Erberk 8 2 Marking a Response in Pulled Papers Select an item from the Select Item drop down menu Please note that you do not need to know the specific Document ID all these will be listed automatically Figure 46 Selecting the Script to Pull Select item then click on Query Project 552101 EEEE A ace Batch Dc ID You do not need to pridt AI Responses 20 change any of the aaaea i default settings 36 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 Once you have click query the results page will be displayed as shown below view score Pulled Papers End Task Query String Item 5521 01 _Q01 batch priority All document o maxResults 20 Priority BatchID DocID Select a Document ID by poddane 0i 4 3000765668 clicking in the radio button so e ogoogo 0184300078669 it is green Note you can only i 00000002 0184300078685 select one document at a time r 00000002 0184300078686 You can then click View to O1 00000002 0184300078698 display the whole candidate script Upon making your selection you can now view a complete script containing the item you initially selected By clicking on the view button you will see a separate window which will display the candidate s answer booklet page by page Figure 47 The View Script Screen J Oecument Vi
66. mackow J Ll IES TOOT TS666 Mo 3 001 macko C jI TSS He J a macka C NBA Aer Ho 3 201 imackow Ll Dime No 3 01 mackom C 01EATIW TRETA He 3 01 mackywj ye i Gore E iter You may get an additional download as shown below the first time that you access a Supervisory function Figure 36 Supervisory Function Download fe Downloading jar file x Downloading from URL http epenpract3 epen eu com Epencib jar 30 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 6 3 The Backreading Screen Figure 37 Backreading Functions T NCS Pearson PEN Scoring Microsoft Internet xplorer provided by Cdexce E e i ror ER Ww Faverter Tete HD E7 Image manipulation txt x E di F aargh Wr Fart il d E La ass AE Prp p150 ci 27 TC epr ihr mating ort redler Eja rhe a You can see previous marks from Markers Hamster Cal Dog Assign your own marks a H v 5521_m_om iw a Ej Diri es aoe avers A ra ewer A iil 6 liri Once you have evaluated the response you can choose to Figure 38 Backreading Options Send To Review Close Viewer If you send an item from Backreading to Review the score will be visible 31 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 If you agree with the Marker s scores while Backreading we recommend that you click on the Marker
67. mpleted a Marker can move onto the Qualification Set s The aim of the Qualification Set is similar to the traditional first sample of marking CE1 but with the ability to feedback to each Marker on their performance immediately 9 2 Selection and Preparation of Responses for Practice and Qualification Sets Both Practice and Qualification responses should be selected with their purpose in mind It is not beneficial to select responses that are blank or especially difficult to mark The Principal Examiner together with Team Leaders Supervisors if necessary will pre select the responses for use in Online Standardisation during the Familiarisation period You can either use Frontreading or Commissioning Frontreading from your Available Tasks to escalate responses as either Practice or Qualification Validity responses can also be proposed at the same time from either of these screens see Section 4 0 of this booklet for details of how to use the Frontreading screen 39 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 9 3 How to Frontread Practice or Qualification Responses using Commissioning Frontreading and load Draft Annotation In version 4 6 2 of ePEN you can now frontread and escalate to the Validity Practice Qualification pool by using the Commissioning Frontreading screen If you prefer to use the Frontreading screens you can also escalate Prac
68. ndidate Response is fit for purpose e the annotations are appropriate clear and concise Once these have been checked the proposed responses must then be commissioned into Sets These Sets will contain x number of responses and Markers are expected to pass x number of Sets in Qualification The number of responses in a Set and the number of Sets required will vary from question paper to question paper and possibly from item to item The committee present at the pre standardisation meeting should be involved in deciding which responses are used to create the Sets If more than one Set Is created they should be of equal demand to ensure a fair Standardisation process 9 7 Setting of Qualification parameters This activity is normally undertaken by the QDAM Subject Leader The parameters for each set must be confirmed The Number of Sets Must See and the Number of Sets Must Pass refer to the number of whole set a Marker must see and pass In order to qualify to mark All online standardised papers must have one Practice set and one Qualification Set unless otherwise informed All Markers will be expected to see one Practice Set and pass zero Practice Sets All Markers will be expected to see one Qualification Set and pass one Qualification Set If a Marker does not pass the first qualification set then a second qualification may be created for them to have a second attempt This will be confirmed by the QDAM 51 Prepared by
69. ng Progress by Item This area of the screen shows the number of responses you have marked for each item and should be used to help with planning your marking This screen also displays your allocation per item The sum of your allocation per item will be the Same as your contracted responses for that paper You can view a marker s details by selecting them from the dropdown list 14 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 Figure 18 Monitoring an Examiner Marking Progress By Item V marking Progress By Item The markin o progress for the Item Item Type Progress Through Cap Allocation Supervisory Overall Response whole paper is also Marking Completed By fem displayed i e the Q03c EXPERT 18 18 100 ae i 17 250 total number of Qo4bc EXPERT 23 23 100 tmo i 22 250 responses marked by QO5ab EXPERT 28 100 100 i 29 250 all markers against Q0Sc EXPERT EEE Ooo ai A 51 250 n ee i l Q07ab EXPERT 0 100 100 0 0 250 available when you QO7cd EXPERT ein 100 0 0 250 first login this The item type is Individual s Allocation Number if items displayed marking per item marked in Note progress Backreading and number will be low it will increase as more scripts are scanned in Expert Type 1 for each Frontreading Graduate Type 2 item For a marker this Clerical Type 3 will always be 0 The Progress Through Cap column shows the number of responses marked
70. nistration details displayed 5 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 From this screen you can select an item from the drop down list of Available Items as well as the task you want to undertake from the list of Available Tasks This welcome screen may contain a Message of the Day for all users of the system Each time you log into ePEN you should check for any new messages 4 0 Validity Items Validity Items are copies of candidate responses that have been selected and marked by the Principal Examiner or designated senior examiner which are included in the normal marking allocation and are marked by Markers Validity responses are sent to Markers automatically at a predetermined rate typically 1 in every 25 within the marking allocation The Marker does not know which items are the validity responses and hence these are used as another means of Sampling the marking and inbuilt quality control The Markers score on the Validity response is then compared with the True Score pre assigned by the Principal Examiner A record of the level of agreement or variance Is available through a set of reports on ePEN These reports indicate the performance of each Marker on the validity responses and therefore the quality of marking This version of ePEN allows you to select and escalate Validity Items by choosing either the Frontreading or Comm
71. nline Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 Appendix 2 Using Filters on Exported Reports in Excel After the report has been generated you can export the data to your PC To do this you need to select a format for the data from the drop down menu There is a list of choices but Excel is recommended as this will allow you to easily format the data The formats available are as follows Select a format kad Select a format HTML with Office Web Components Excel Web archive Acrobat POF file TIFF file CSV comma delimited SML file with report data Once selected click on the Export button You will be prompted to either Save or open the file Click open if you would like to view the spreadsheet straight away Once you have exported an ePEN report to excel you can focus on a particular part of the report by using filters oo a ePEN reports will typically ee nn es contain data relating toa DeboN OAT wegener nimam o number of examiners and F a or items OC Qualiftaton new t 5 691601 OC Qualification Report Sicic x E4 gt gt r OOE ie l o colo o 0100 f 30752 k 1 OALA 60 Otte TA m 3 a lt 12 676m 01 _Gote 321045 e 120 105 Jeno 14 ems M 601 001a miars Quat ed ne ka r 4S ta 3 4 OOte 00S 16 6v1601 17 nem 18 eam colo o To easily view data relating to one examiner and or one item at a time you c
72. ocme GO Summer 107 French GCSE V4 62 Bed 1012 Copyright 199 2009 HEG Person bic All rights neserind Messaging PEN Helplineinee phone 0000 149 F200 Feit edexcel ge Gee The following screen will be displayed It is divided into two sections Progress Through Contract Marking Progress By Item Note the data displayed on this screen is updated on a regular basis 13 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 Figure 16 View Progress screen Proarece Through Contract 01 Microsoft Internet Explorar provided by Edexcel G E hipid 053 21 235 eachbosrd DeachbosrdMierking aprisa Ei E diversi SO eee iag l iel bali To mi nimise expand the menu Fle Edt Wew Forbes Tob Heb w ropes through contract 01 Ea ne ee E PEN TL test Tltest eGexcel im End Task View Progress Weer TL teat test Principal Examiner w Paper SOF iF Paper Contract Type Role Contract Amount Contract Completion Date Marked Contract Progress SU16F IF EXPERT Team Leader 3138 17 09 2009 E Ds SQ16F IF GRADUA TE Team Leader 1454 17 00 2009 F Dh To return to the Marking Progress By Item Welcome Page Item Item Type Progress Through Cap Allocation Supervisory Overall Responses Marking Completed By Item QOla GENERAL om sid of 1o000 i FiS mem MRi o 10000 canoes 10 250 See a eee o 10000 Ban 117230 EDO oDnG o 10
73. ons area displays information about the cap limit of marking that can be undertaken by a marker as well as the number of items allocated The capping function must only be used if you have concerns over a Marker s marking quality and you wish to prevent the Marker from accessing any further items If this is the case you must also notify your Principal Examiner and QDAM The cap can be changed either item by item or for all items for a particular Marker The capping function should not be used as a workflow tool as this could prevent a Marker from fulfilling their contractual obligations 16 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 Figure 19 Allocation tab Allocations 01 Microsoft Internet Explorer provided by Edexcel Use the filters to refine the search to select a specific Role eg Expert Examiner to select a specific Marker or ALL O v http 10 153 21 235 Dashboard Dashboardallocation aspx taskIndex 18 amp sid 6617314A E366 4394 4E93 C17AAB61C5608 amp pid 18 amp mid 261 vA File Edit View Favorites Tools Help w k 007 Edexcel Online Post Results Allocations 01 Allocations End Task View Progress Paper 5016F 1F Role All Y Item Type All User TL test TLtest PE Filter Lis Marking Progress Allocations Bice Item Type Cap Allocation ne
74. otation tool and ePEN To do this you must first select an Item from the Welcome screen and then the Create Annotations option from the available tasks Figure 58 T e PIN Welcome 01 Micresedt Internet Laplaver provided by Ldexcel Ferformance Evaluation Network cca p Welcome Select an item and B sasine Macon then Create Annotations from the Available Tasks Heesaqe dien A Seong Sudas Magponpeg fot Z Mercat Outlock PIN Supervisor U Sues Gids ED Mempsott iinet Y PEN Weera E fn 7 0 ma A pop up window with the annotation tool will appear on screen along with the Open Content Dialogue Box Figure 59 DV e PEN Welcome B1 Microsoft Internet Explorer provided by Edexcel G e Annotation Teo Sine ia Ean Eda e paea he Pavortes Wee i E ttn thee penretSdmegt fener Tie gal u aamin oe Content Web Page li Fila Fi Format B n 7 Usg ibe om jii To view the ePEN Welcome screen click on the ePEN button in the start bar or the ePEN bar at the top of the screen Copa Cancel Cpe a Lome Lee edexcel l Opang phg htp hebrep idg aan Farci lanen cme Ml Locel erariet iB tere interne if Start E Inher Mero inj PEN Sereen ull ePEN Sune Th PER Waka SB GP EN Annota X unbitind Paint for Rl eee Before you begin any activity with the annotation tool you will need to return to the ePEN Welcome Screen so that you can open and view the items th
75. otes whether they were successful or not in qualifying 52 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 Figure 76 QC Practice Qualification Set Report Seen Peta Vis mck eaiiaion Shale Date ASempeted Dale Quadted Set Geen eed P eee Heaporned Seen 8 86A gr e Percert SEN 01 Get C i Not Si artid o g f aari ii adi wiog Not Started o oO E sah Of oon beoabche Net Started f 621 iai packen Net Started o 821 01 O01 e ae Not Started y a a 55H Be cheer Met Started a 4 SE ii adi Denai Mot Started D a s571 01 Oot aeni DAN Mot Startod D a 5 SEM m 0i faperi Neat Started g 6 0 Se Oi _ oot expert Chearniddean i 55H od egi expert Not Started 0 4 SEH ii aii nperi Craridden Q s571 01 oot expert Mot Started D p a S521 01 Oot eapertd Net Started D o saM 01 Oot femal Net Started Venn 124008 i i D SEN ol Sot gebr aiin Nol Started f p i Saat ot _oon iene Net Started a 652101 oo riaria Mot Started l 0 55 gt 0i adi maeka Successful cert REE Te aiaei i i 3 E Note There is no pass criteria for the Practice Sets The report can be used to monitor when if Markers have completed their Practice Sets The report displays the following information Overridden Started Unsuccessful Successful Not OLS Shows the number of sets passed Shows the number of responses seen in the set s Shows the average percentage agree with the True
76. ows M k AR the report to display Wo Ee eoni pet enke E Reports 0i x m Do ab 3 Bape 3 tae a information on all the Markers for the paper TL test Totes edexcel ses End Task ajase select a report Renews Fulls and Adpedecalors Pager SOL6F iF kem Trs SLL 5509 9B Reviews Pulls amp Adjudications Level GCSE fern 600 SB Oc BR us mayer Pape Jai Fani Mii regent etl r ii bia fren TaT T Copyright 1999 200 NES Pearson Inc All nights nesernend ePEN helpline freephone OB 169 9202 edexcel Q nanm 100 23 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 The report will only show information for examiners who have sent items to review The columns in the report contain the following information The number of review items still waiting to be marked The number of items sent from Review to Pull The number of items in Pulled Paper waiting to be marked that need marking in adjudication These are dealt with by the Marking Centre Manager Only applies to clerically marked items The number of items Only applied to clerically marked items The number of items marked in adjudication These are dealt with by the Marking Centre Manager Figure 27 Adjudications Completed Outstanding Soe 01 0 testS T Tests 1 0 0 0 oO 0 5571 01 001 experti E Experti 1 0 1 a 0 0 5521 _01_201 janinemark J Janinemark g 4 5 5 i 0
77. pervisor will appear It is advisable that you check the Marking Progress By Item and see if Progress Through Cap is 100 and if the cap has not been set at a different value to the allocation If the marker is showing 100 against their allocation for that item they should mark a different item until all items show 100 When markers have scored all items assigned to them they have completed their contract At this point the system may allocate additional responses that have not been allocated to other markers This reallocation of responses from the unallocated pool will occur every 60 minutes once the markers has completed their contract This is a default setting and may be changed during a series Once the reallocation takes place the Marking Progress screen will show a new item allocation which will be larger than the original allocation They will now be able to continue marking any additional responses This may occur before the contract completion date as long as the contracted items have been marked Note The Progress Through Contract is not updated with the new allocation values 5 1 5 Allocation of Items After the Contract Completion Date On the day after the contract completion date 00 01hrs the allocation of any marker who has not managed to mark all of their assigned items will be available for the rest of the marking team Marking progress Is reset to display a default allocation of 10000 5 2 Allocations The Allocati
78. posed By using the drop down menus available you will need to select the paper the item the type Practice or Qualification and the document ID 46 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 To create or amend existing annotation Figure 65 lt Q Open Content Web Page Dialog AE Please select the content to open a Select the paper Ttem 5561 01 G02b 5561 01 0ec a Select the item 5561 _01_003a E Practice Select the Set type YP Qlualiication Document felony tet ennir Select the document ID that requires the annotation This is automatically ICO 1643500018171 GLICO 1643500015554 CICO 16430001864 7 populated with the contents of the QC responses folder In ePEN http iidashboarda examescore corr a Local intranet Click on the Open button The text for the annotation can be typed directly into the new window which appears or alternatively cut and paste from other applications MS Word Excel etc Figure 66 T cP EN Annotation Tool Microsoft Interne Explorer provided by Edexcel Piretcn gt ADA Pao Yom Que E SBW Aies iB herp hebyapra ding Japan em lior editor Ape ha Or err A ar i gt es Fie Lat priit tagg oc RE ee rsa BA oe Type in or cut amp p paste the required text for annotation Then click on the Save icon A pop up message will appear confirming that the content or changes made has been s
79. reated and saved return to the Commissioning Frontreading screen in ePEN By clicking on the Document ID s the annotation created will become visible in the right hand window Figure 73 J Htt Pear a PIH Scoring T pa e Irvieremt Ccgisres presided by demai 4 L ER a 5 re w Leal i i tcp rei Ea pa e The annotation e o including any images 1 i ere gt Boat M Er a L A will appear in the a eee a prove 5 j Proved ro i on at a right hand window ne Doc ID is clicked on i when the relevant JE Soret eo ny eee eee got othe 3 Sj Goce nb 1 It is essential that each Response in a Set is checked thoroughly to ensure that the correct annotation has been uploaded At the Pre Standardisation meeting the Responses in a given Set are checked to see that they have had their True Score agreed annotation uploaded and checked It is then that the Set can be created and commissioned 50 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 9 6 Commissioning Practice amp Qualification Sets This activity normally takes place at the pre standardisation meeting where the final mark scheme has been agreed The commissioning of sets will be undertaken by your QDAM Subj ect Leader Each proposed response in both the Practice and Qualification QC Response folders should have been reviewed to ensure that e the True Score is correct e the ca
80. rmally 24 to 48 hours after the examination is sat all Markers will have access to all the available items in order to familiarise themselves with candidate responses and the application of the mark scheme This time is Known as Familiarisation Marking as the marks awarded to any items will be reset prior to actual live marking commencing All Markers are expected to contact their Supervisor prior to the pre standardisation with feedback regarding the candidate responses seen during familiarisation along with any proposed amendments to the mark scheme Familiarisation Marking will end at 23 00hrs on the day before the standardisation activity as the paper will be reset i e any marking that has taken place will be rolled back so that the Online Standardisation process can commence followed by live marking 9 1 Practice and Qualification Sets In order for Online Standardisation to take place the Principal Examiner Team Leaders need to select and propose responses to be used as Practice and Qualification items This can begin to happen as soon as items become available on ePEN normally 24 to 48 hours after the examination is sat The aim of the Practice Set s is similar to the photocopied scripts that would have been used at a traditional meeting to provide feedback and details of how marks should be awarded All of the Markers allocated to any given item will see the same Practice Set s Once the Practice Set s for an item has been co
81. rs who have marked the contracted contract completion date number of responses for the paper before the contract completion date have access to extra items from the pool A marker allocation has changed after the Once the contract completion date has contract completion date passed all Markers cap and allocations are reset to 10000 to allow access to the unmarked items Cannot Close Viewer The common cause of this error is the use of pop up blockers Please contact Helpdesk for further advice 56 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 The paper have access to is not the same subj ect unit as will be marking Is my paper being standardised online How many Validity Items are required per item can t see Mark Student Responses as an option in the Available Tasks The practice site uses historical items from previous series This site should be used to practice using ePEN functionality and also to test that ePEN has been successfully installed on your PC Therefore you may not be allocated your subject Your contract will have details of whether your paper is to be standardised online or if you are required to attend a standardisation meeting Please contact the helpdesk if you are unsure The number of Validity Items will be notified to the Principal Examiner and any nominated Senior Examiners by the QDAM for each unit The number at
82. rvisors are required to mark these items 5 1 3 Allocation of Items After Standardisation After the reset has taken place and once a sufficient number of candidates scripts have been scanned into ePEN the allocation and cap values will be amended automatically The value of the allocation will add up to the number of responses that a Marker Supervisor has been contracted to mark The cap will automatically be set at the value of the allocation unless it has been amended manually e g Item Item Type Progress Through Cap Allocation Supervisory Overall Responses Marking Completed By Item oe T PeT 0 150 150 0 0 2300 15 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 Most supervisors will automatically be given an allocation of every item on a paper Please refer to your contract and or for guidance given at the pre standardisation regarding the marking you are expected to undertake 5 1 4 marking period During the marking period you should check the Marking Progress screen to monitor the progress of your team against their allocation and the overall item completion A supervisor s allocation of items should be primarily used to mark responses in Review Pulled Paper and for Backreading See section 6 7 and 8 of the booklet for more details on Backreading Review and Pulled Paper As a marker completes marking on a particular item a default message asking them to contact their su
83. site at www edexcel com October 2010 All the material in this publication is copyright Edexcel Ltd 2010 1 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 1 0 Introduction ePEN electronic Performance Evaluation Network is an image based marking system available online ePEN allows the user to mark sections of papers known as items Scripts are scanned into the system and divided into individual responses which correspond to each item The student responses are then displayed on the Marker s PC As the candidate responses are shown against the selected item the Marker can focus on the mark scheme for that particular item for as long as required Senior Examiners and Qualification Delivery Award Managers have access to real time monitoring quality control reporting and other functionality to help with the supervision of marking 2 0 Getting Started Before you can begin using the ePEN system you must have installed ePEN The Current version of ePEN is version 4 6 2 It is essential that you download and install version 4 6 2 You can do this by accessing the following website www exam2score com and clicking on the Download and Install ePEN link Once you have clicked on this link please follow the onscreen instructions Figure 1 ePEN Homepage ePen 05 Microsoft Internet Explorer provided by Edexcel File Edit view Favorites Tools Help You must
84. sponse displayed in marking window that can be resized Proposed True Score is displayed Submit Score and Commission buttons D Aiet com nes scoring veer Windkesndviewer Aggiet arted ioe The selected candidate response and the suggested True Score are displayed The True Score is the mark given through Frontreading and is usually based on the provisional mark scheme Therefore this score needs to be reviewed after the final mark scheme has been agreed At this stage the score can be changed if necessary If you agree with the True Score allocated and want the response to be a Validity Commission Item click on the the drop down list button and then select Approved from 11 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 To change the True Score you must remark the item Enter your new mark in the marking grid and then click on the button This will change the True Score and you will then need to click on the Commission button and select Approved If you decide not to use the response as a validity response because it is unsuitable you can click on Commission and then select Deleted or simply click on the next response QC number you wish to view The next candidate response will be displayed in the window 12 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor Oc
85. t Score Time Document ID LatestScore Time 29 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 The screen below displays the responses available to Backread based on your selection criteria Here you will see a brief history of the responses showing whether the response has been marked what marks have been given if the response has been second marked and who has done the marking You can now select any of the responses or all of the responses from this list and then click on the Backread button Figure 35 the Backreading Sample Return Screen Hackreading Selection B1 Meeresoll Internal Explorer previded by Edexcel Fle D yer Favorites Tools ely Ou O bi a tp Eeh sprees a a m So rass E ep 199 002 197 0 eperwebepe Dachreadngcontrober Ejs cae Ferformance Evaluation Network ad Backreading Sq _Bockrend NewSanech End Task Prgject 5521 l Trem S521_01_O01 Selection Date m Ka 00S Soire Type m 0 Storer ID mackowy Adjacent m AlN Freni Backread m Eachidtsigt By m Document Ii Criteria Menber Of Responses 20 Here you can select Select D ID First Score bss Furst os pene Seco Resolution KResolaton Eackreading Barkreading n d Vi d U a Ite MS O r sel ect Selection Document Tear Tore Tore s Backread i Scorer i Scorer Score Tran Scorer Score Trait Score Trait Backread Trait 1 Al n E EO ees No lt 01
86. that you initially select all Markers for an item on this report This report will help you to monitor the quality of the marking for an individual or group by displaying accuracy on Validity Items as well as Backreading The report displays the following information Examiner Name or Team Total or Item Total Assessment Associate number Name of the trait if applicable or item Total marked including validity Total marking time in decimals e g 1 75 is one hour forty five minutes Overall number of responses marked per hour Number of Validity Items marked responses scoring lower than the true score responses scoring lower than the true score but within adjacency responses agreeing with the true score responses scoring higher than the true score but within adjacency responses scored higher than the true score The number of items backread responses marked lower than Supervisor mark responses agreed with Supervisor responses marked higher than the Supervisor 19 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 Case Study 1 Interpreting the Validity columns in a Marker Statistic Summary Report Validity aa Total Non Adjacent Adjacent adj adj REEOR A Team Leader AMarker 53 4 0 96 0 0 D Marker 0 EMarker 37 11 0 86 0 3 FMarker 6 0 0 A Marker has marked 53 Validity Items 96 or 5
87. the True Score on a trait level This report indicates whether the Marker is tending to be more harsh or lenient or than the senior examiner s marking and by how many marks The true score for a backread item Is the score that the Supervisor has submitted Therefore this report can be used to identify a trend in marking It can also indicate areas where the mark scheme is being applied erratically Therefore the feedback to a Marker can include points about the particular trait which may be causing the Marker difficulty Note this report does not display the occasions where Markers agree with the senior examiner s marks Once the Marker Differences Report has been selected from the choice of reports the following screen will appear Figure 23 Marker Differences Criteria Selection Page Select Marker Reports Differences End Task Please select a report Marker Differences v Paper 5016F 1F i Role type ALL s Role ALL v Marker M Test markertest j Item 5016F_1F_Q01ai Select the search criteria for the report and click on View Report You can refine the details of the report by selecting the data in each of the available fields Select the Marker the Item and other relevant parameters before clicking on the View Report button You can refine your search by Role Type AA Non AA role or All By selecting AA the selection of Role is limited to the marking roles on ePEN e g
88. tice and Qualification in the same way as for Validity Items described in Section 4 0 of this booklet From the Welcome Screen select Commissioning Frontreading from the Available Tasks menu From the Commissioning Frontreading screen select an item from the Search Escalatable Responses area and click on search Figure 49 Commissioning Frontreading screen T NCS Pearson e FEM Scoring Microsoft Interne keplorer provided by bdemocel Fie Edt Yew Favole Took Help Er E F x El A F Search P Favorites fa o lal 3 Pakkess 4 heip 150 AS 137 UA Sa epen webapp Homse E Search Escalatable Oc Nominations 1 Itam Ue Pool passice loan Score Uog Wi Re Sp O n se S a re a C Soarch Uc Nominrbions Search Escalatable Responses hem Select x hem Select hoc hi a ee Pou Classsificalion Feabch id Dow Id Hy Sack id i Rason PAS Id ot Kasa Search Char Qe Nominations lose Viewer IB Applet COL RS ote ing eves 2 Aredia aoe aariad 2 Inherit Figure 50 Search Escalatable Responses tem Seea Select the item you wish to frontread Doc Id E Batch Id ES Note If you add a digit after the sign in the Doc Id field e g 8 your search will Stackid 4 CS only return documents with the number 3 5 in their ID code This may be useful if more PASId than one person is escalating into the pool and you want the search to return
89. tober 2010 5 0 Supervisor View Progress and Reports Supervisors can access all of the View Progress functions available to Markers as described in the ePEN Marker User Guide However they also have access to additional information Therefore a Supervisor can view the following areas in View Progress Marking Progress Progress Through Contract and Marking Progress By Item This is the View Progress default page Note Markers can only see details about themselves Supervisors can see the details of their team Allocations View amend details of the caps of individuals Reports View statistical reports for an individual Marker item team or paper Help View an electronic copy of the activity flow charts and link to Ask Edexcel and Edexcel Gateway To access the View Progress screen you must select View Progress from the drop down list of available tasks in the Welcome Screen 5 1 Marking Progress Figure 15 Accessing View Progress 2 e PIN Welcome 1 kicresoft Internet Explorer provided by Cdexcel E iali to eT himla ean ire corey genet eda ein ei been eed Pa cate Dr fed abaya Pde oh Ho N H Gils ga yes fpe Jak He w A a Welcome Performance Evaluation Network TL test Select View Progress Available hiema Heeeaqe Pa 1225 4AF _O0liv M fie De Available Tasks Oo 10 2000 Choose your opbon scrol down E Choose your opon scroll down Fen Sc
90. y and Ask Edexcel 27 Prepared by Online Assessment Company Confidential v1 0 Authorised By Peter Challinor October 2010 6 0 Backreading 6 1 What is Backreading Backreading Is one of the key monitoring tools available to Supervisors in ePEN It allows you to select a sample of marked items from your team of Markers so that you can monitor the quality of marking Supervisors should use backreading as the way of accessing a Marker s first Sample of marking to ensure consistent and correct application of the mark scheme The functionality to backread the work of a Marker is available throughout the marking period but you should be aware that if marking continues past the Contract Completion deadline marked items will be held on ePEN for a shorter period of time As such you have a shorter window of opportunity to backread Although the reports can indicate performance issues with a particular Marker or item the backreading functionality can assist you in pinpointing the issue by allowing you to re view a response and see the score the Marker had awarded it 6 2 Selecting a sample to Backread To start the backreading process select Backreading from the Available Tasks menu on the Welcome Screen Once you are on the backreading screen you must select your search parameters Figure 32 Selected a sample to Backread E Please note that the ee ee LI Backreading functionality a l k jin is no longer date driven

ePEN Supervisor User Guide

Contents

Download Pdf Manuals

Related Search

Related Contents