RMS User Manual 2-6-05
Contents
1. Risk Management System RMS A User Manual Modeling Solutions LLC www modelingsolutions net i a mT Miji mA i au jill i Nt ieee poe the paradigm shift in assessment Modeling Systems LLC Copyright 2006 All rights reserved the paradigm shift in assessment User Responsibility All data and information contained herein and any and all modifications supplements translations variations and compilations thereof the results are for use as supplemen tal information and are not meant or designed to be used alone as a determinative mecha nism For greater clarity user understands and agrees that the reports are not intended or designed to replace professional judgment in regard to any classification or custody deci sion for a given offender The user and or organization accepts the responsibility risk expense and liability for using relying upon administrating validating selecting modi fying supplementing varying manipulating and compiling the results and for providing supplying collecting selecting validating administrating and relying upon the data and information used to generate the results MODELING Solutions LLC retains the right to use any and all data with the exception of identifying information for product enhancements Copyright 2005 MODELING Solutions LLC and its licensors All rights reserved The user may not reproduce in any form except for its own internal purposes w2. Popuab Form the Identification box Ofender Sex Binh Ome Create Date Click the Offender Re Population hyperlink 3 An Offender Search window will open Itwill L list all previous assessments for a given offender number You can see the date and time a particular assessment 12 was done You may select one of the assessments and click on the Populate Form button RMS will complete the form with the items from the assessment you have chosen You will only need to change the data that has changed since the previous assessment 4 Press submit What If Follow these steps 1 Click the Input Form tab This will launch a new window with a blank RMS form 2 If you know the offender client number enter it in the Identification box Click the Offender Re Population hyperlink 3 An Offender Search window will open It will list all previous assessments for a given offender number You can see the date and time a particular assessment was done You may select one of the assessments and click on the Populate Form button RMS will complete the form with the items from the previous assessment you have chosen You will only need to change the data that you would like to what if 4 Press submit 5 Make a note keep track of the item an the assigned ID number For example if an offender has been modeled previously s he s ID number will have a suffix attached to the number For example offender roooo might l
3. a spreadsheet This option will allow you to select which pieces of data to export Itis possible to download all data generated by the RMS When you have selected your options click Save CSV Options Ensure that you have selected All Offenders or narrowed your offender search by some criteria in the Results Filter Click the Output tab at the bottom of the screen A window indicating a file is ready for download is opened Save the file Note Ifyou have not identified a set of offenders prior to getting to this option a file cannot be generated Some internet browsers will report that a Script is running slowly This is because RMS is trying to do what you have told it to do generate a useful file with nothing in it gt You must provide some sort of criteria for RMS to generate a file for downloading CSV naude Verties CSV Sot By Q9 V Naadi Arsesmern ON Comparatie OSeectes Brh Date J Modoed Fisk Gerda v Low tS Mocc Fue Hon ARGA Amewo Oa Port Output Pore oe 9 Save CSV Options 16 Results on the Screen To see the results of your Results Filter selections click the green Search Results Form button at the bottom of the results options All Offender Recones Show Al Ofanoa Pacon wets No Fits 7 Sucre Oats Range Sears Fouse Oat Range Samh MMOD YYY Rom Feo 2008 f from To 15 Fen 2008 Pa To Fisk Levent Soa Modes vaos fem i Recess jw 2 Veerce Fs Neeos Assewrant Samh to Oftercer see eas rom Mo
4. band Likelihood of recidivism information is sometimes created by grouping scores into several bands the lowest band of scores is described as low risk the middle band of scores is described as medium risk and the highest band of scores is described as the high risk In reality the likelihood of recidivism statistic associated with these bands is simply the number of recidi vists v s the number of non recidivists falling into the band it does not represent the risk pro pensity of a given offender Suppose a risk band reportedly has a 50 chance of recidivism As stated above this tells you nothing about any particular offender For illustration suppose instead of a band we have a bag that reports a 50 chance or recidivism The contents of the bag are comprised of 50 recidi vists and 50 non recidivists Imagine being blindfolded and then asked to reach into the bag and pull out your offender Obviously this isn t going to happen because your offender was never part of the group that generated the risk estimate As a result it s statistically clear as a bell that the 50 is not tied to your offender and does not reflect a behavioral recidivism propensity So what is the risk score tied to a risk band actually doing It s telling you the likelihood of correctly classifying an offender as either a recidivist or a non recidivist based upon their score or band Hopefully as scores increase the proportion of r
5. nearest neighbors that can provide us additional informa tion We can see their DOBs we see that all of the offenders with a similar risk factor profile were males we can see their actual risk 1 they stayed crime free for 3 years 2 they were convicted of a subsequent crime within two years The Percent column tells us how important the comparable offender was in building the risk simulation The col umns denoted with V P D W E O are the number of conviction the comparable of fender has had in their career Finally the years served are the number of years of incar ceration in a locked setting the comparable offender has had since their DOB and the year 2002 Needs Assessment The needs assessment information is derived from the RMS input form The needs assessment is based on the level of behavioral functioning presented The RMS needs assessment displays two types of data r Actual Ratings the ratings you entered on the RMS input form displayed in blue 2 Simulation Ratings the ratings generated from the overall risk factor pattern by the RMS modeling engine displayed in red Offender Needs Assessment ga d ACAD EMPL FANM RELA COMP EMOT ALCO ORUG MENT HEAL SEXS To close this window click on the Green X in the upper right hand corner 19 Printing an Offender Report A printed report can be generated for the RMS data presented on the screen Follow these steps 1 Click the Results Filter tab
6. page is divided into 3 parts a gen eral navigation pane on the left hand side of the screen a description of the RMS tool in the daa tl Le 10 middle and an RMS login pane on the right hand side This is where you will login to your ac count Login to the RMS tool Accessing the RMS tool requires that you enter your username and password and clicking the submit button Note the username and password are case sensitive If you have not received a username and password contact you supervisor to ensure that your department is authorized to access the RMS You will be presented with a Disclaimer screen Read the information In essence it re minds the user that the RMS information is to be one part of a risk assessment and should not be the sole determinant in the decision making process It also reminds the user that they are responsible for how the information will be used These are common sense reminders By clicking Accept you will be presented with the actual RMS tool Ra Maragernet Sysw AS OPorsjer Mecords RMS Orientation Show Ai Omas Recrus wh No Aws Surat Dew Farge Seam To the right is a screen shot of what you will see after you login This is the Results hia Filter area We will examine this more fully in the next few pages For now we want to amat CSV Renate Opu orient you to the look and feel of the RMS re marth enfas Por There are five tabs at the bot Y
7. recidivists Although we could cut the continuum at any score the best cut score attempts to classify the most offenders correctly as recidivists or non recidivists Clear 1988 provides an excellent primer on this issue Unfortunately people who use risk tools have inadvertently inserted a subtle but important misunderstanding We often hear people talk about the norm table ofa risk tool The meaning ofa norm table is straight forward it tells you how an offender s score falls on a continuum from the lowest possi ble score to the highest possible score in relation to any other offender s score Although norm tables come in different flavors most people are familiar with the term cumulative percentage This is the statistic that provides information about the running total of how many people have at least some level of score The obvious assumption is that as scores increase so does the likeli hood of recidivism Unfortunately a norm table reporting cumulative percentage information is not reporting actual outcome information It simply tells you how one offender s score falls in comparison to some other offender s score To make norm tables more useful a second piece of information called base rate is often overlaid on top ofa norm table that purports to give an s estimate as to an offender s likelihood of recidivism The base rate is simply the proportion of recidivists to nonrecidivists for a given score or
8. this manual and use the RMS tool This manual is meant to guide the user in using the RMS interface to help users develop an ap preciation of how to use the information that RMS provides and to better understand general risk assessment issues To that end we appreciate your interest in RMS and welcome you to the RMS user community An adage from psychology If you want to anticipate someone s response to a situation examine what that person did under similar circumstances in the past This sounds good in theory but unfortunately this only works if you can observe that person several times in similar circumstances to get a baseline of what they might do In the world of corrections we must attempt to create a baseline of what a client might do without the luxury of being able to let them out unhindered to establish a good baseline from which to make future decisions What to do Classical risk assessment Presently most risk assessment tools have altered the adage to read If you want to anticipate what an offender will do in a future situation compare what a bunch of other offenders with a similar risk score have done in the past This approach has served to enhance decision making over the years by better standardizing and objectifying the risk assessment process but it has the basic statistical limitation of treating everyone with similar scores in a similar fashion without consideration of the underlying risk factors ge
9. 2 Click on the offender for whom you would like to print a report 3 Click the Printer Icon next to the Graphic Needs button near the upper right of the screen gt Graphical Needs Assessment Printable Report 4 Click File then Print in your browser 20
10. a given score or range of scores Thus local norms are actually an attempt to compensate for a regional bias in the way offenders get their scores As we have seen above simply knowing an offender s score doesn t mean that you know the likelihood of correctly classifying the offender as a potential recidivist or non recidivist Some researchers might argue that you can simply transport the outcome data from a devel opment sample to your new local norms with statistics But this misses a significant point the decision to pursue local norms argues that your population s scores are fundamentally different than the development sample s scores Given this fundamental difference what degree of confi dence can you have in outcome data developed on a fundamentally different sample of offenders The answer is simple you simply don t know at least not until you have outcome data from which 7 to validate your local norms Validation is the missing piece to most local norms and validation takes time Local norms and RMS Local norms are essentially an on the average model that has been tweaked for a particular region The distribution of possible scores is calibrated to the region AND new outcome data is paired with the scores Local norms can help to increase classification accuracy because they are sometimes more reflective of the way your offenders get their scores A brief digression is in order Before it is assumed t
11. ar Mon put Fom Fosse Aw tom of the screen that access Modeled Values y Viona RMS Fetes Ork Amga Seach MMOON YYY Seah tr Ofercer Mooetod Farsuts Qrp Re uv 11 RMS functionality As a general rule the Results Filter tab will be your primary focus When selected it will display the Input Form Results Filter and the Model Results tab These three tabs will undoubtedly form the core functionality for most RMS users The Main Menu and Out put file tabs are primarily for future functionality Note You will also want to use only the navigation tabs and buttons in the RMS screen Do not use your browser Back or Forward buttons this will undoubtedly cause you to say things your mother wouldn t approve of These buttons do not control the RMS they will log you out of the RMS and will return you to the previous website you were looking at prior to launching the RMS Input Offender Data Rating an offender on the RMS items is straightforward Initial Assessment Follow these steps 1 Click the Input Form tab This will launch a new window with a blank RMS form 2 Complete all items Refer to the Scoring Manual for specific scoring criteria 3 Click the submit button 4 Repeat for next offender Reassessment Follow these steps r Click the Input Form tab This will launch a new window with a blank RMS form 2 Ifyou know the offender client number enter it in Qender Search Offender Nuniber 100 et Seam 5
12. deled Results The Modeled Results screen presents the data generated by the RMS When the window opens it will look like the one below verw z Otierder eth Dwle Gender Modeled Meleme Date Model Process 1P wres male jisa amaa R C Anatis Rapor toe 1740323 MALE jas m v HPR t2040 MALE 1 xosmas A HPA E20404 MALE hi mosmas v 12PA 1970 0531 MALE i177 moosapm A 12PR 19700531 MALE 200 moma v 13P0 1981 0616 MALE 191 20080727 R 3P0 19810515 MALE 148 0s07a7 v 140 1981 01 08 MALE 200 A 140 1981 01 06 MALE 147 2006 03 28 Y PN tTM MAI F im ae A Ofender GathDute Gende Actua fish Percent V P D WE O Years Seved V Viona P Mopy D Ong W vagon E Esape O Oter fons resto mmm un f oae opa 17 This window allows you to access three types of information I Offender Information table Risk and Violence scores 2 Comparable offenders used to create an RMS model 3 Needs Assessment information The Offender Information table presents the modeled results for recidivism and capac ity for violence The data in this table is straight forward The Modeled Process column presents the type of risk score recidivism denoted with an R the capacity for violence denoted with a V The Comparable Offender Information table presents the cases used to construct a model for the offender you selected in the Offender Information table No data will ap
13. ecidivists increases which in turn increases your chance of pulling a recidivist out of the bag In this illustration the bag had a 50 chance of giving you a correct classification of a recidivist and a 50 chance of correctly classifying a non recidivist When risk scores are put in this light you can probably see that many of the risk tools you are fa miliar with are actually attempting to tackle classification problems Unfortunately users often misunderstand the information the designers of the tools had intended to provide Anytime the outcome of interest is an either or scenario e g recidivist or non recidivist violent or not vio lent the risk tool is telling you the likelihood of correctly classifying someone as belonging to one group or the other group In essence the risk tool is telling you the likelihood of making a correct decision Any tool constructed in this fashion with bands purporting to provide a chance of recidivism in the neighborhood of X must be interpreted in the following way you have a X chance of correctly identifying your offender as belonging to one of two groups a group comprised of roo recidivists or a group 100 comprised of nonrecidivists The most obvious implication for supervision is half of the people in a middle risk band will recidivate Unfortunately we have no way of knowing which offenders are the recidivists What we do know is that offenders falling into this band wil
14. evise the adage to read Ifyou want to anticipate what someone will do in a future situation examine what ten other people with similar risk factor pat terns have done in the past under similar circumstances Zamble and Quinsey 1998 examined many of the static and dynamic risk factors associated with recidivism While recognizing the difficulty in doing so their work urges professionals to view recidivism as a process that should examine the risk factors in the context of each other We understand that a risk factor pattern is but a limited picture for any given offender but it is a picture none the less and it is substantially more than a simple additive risk score RMS uses an advanced set of statistical techniques that were historically unavailable RMS builds a model based upon the patterns from comparable offenders to represent the client being as sessed In order to build these models we have collected exemplars of offenders exhibiting the range of behaviors we are trying to anticipate recidivism capacity for violence and intervention effects We call these exemplars our reference library We mathematically select our compara ble offenders based upon the similarity of risk factor patterns to the risk factor pattern of our tar get offender The EBM technology allows us to use time tested variables that have been utilized in a variety of settings and jurisdictions to generate a much more accurate classification model for a gi
15. hat your region requires local norms some time should be spent assessing if offender behavior is different or whether it is your region s abil ity to detect criminal behavior Ifit the former local norms may be warranted but if it is the lat ter adjusting your norms to reflect your official detection rate as opposed to detecting criminal behavior will only serve to decrease your ability to protect the community Simply because a tool is consistent with your incarceration rate doesn t mean it is consistent with your crime rate A second issue with local norms is that not all offender patterns are equally useful when trying to classify any particular target offender It is very likely that some offender s data patterns are very different from your target offender s patterns yet these offenders are still incorporated into the traditional norm table and these offenders will influence the base rate of your outcome variable This will affect your ability to correctly classify your target offender RMS directly addresses what local norms are attempting to accomplish increase the likelihood that offenders who have similar risk factors are used to model what that offender might do upon release to the community This is accomplished because RMS was constructed with a pattern matching paradigm at its core Summary RMS was developed from the ground up to tackle classification problems To do this RMS in corporates example based modeling alg
16. in level of dysfunction These op tions are useful when trying to anticipate the number of offenders with a particular need OR in combination with the other filters identifying offenders for specific types of programs based upon the level of needs within certain time frames Ad Ofterster Mecords Offenders Filtered by Needs Adeo ocra Sats Empwymant franas Minerren Vera amiy Reators ps Con para To Prom To Pron be Save Needs Options Ofer Dag huiver Mortal A pity Haam Smul Betevear To search for a particular need check the box next to the need and set the range of need that you are looking for in your offender group The scale is actually the percentage of need listed on the needs plots When you have made your selections click Save Needs Options Note These search criteria are saved To clear these criteria uncheck the needs filter in the Results Filter tab See below 4 Fite Naach Acurerert te Eade Unique Onas Os Orty Foma CSW Rossts Otat 08 Soan Rests Fom Saah tr Orcii MerMew nptrFom Rosst Fim Qf Ro 15 There are two remaining search option r Enable Unique Offender D s This will restrict your search to the most recent as sessment for each offender 2 Search for Offender Search for a specific offender Output results to a File Format CSV Results Output This option is for exporting the risk data to another pro gram Most common is
17. ithout the written permission of MODELING Solutions LLC the paradigm shift in assessment Table of Contents RMS 3 User Manual 3 An adage from psychology 3 Classical risk assessment 3 RMS assessment 4 How does RMS provide greater accuracy 5 The missing piece to local norms 7 Local norms and RMS 8 Summary 8 Website Navigation 10 www modelingsolutions net 10 Getting to the Website 10 Login to the RMS tool 11 RMS Orientation 11 Input Offender Data 12 Initial Assessment Follow these steps 12 the paradigm shift in assessment Reassessment Follow these steps What If Follow these steps Selecting Results Output results to a File Results on the Screen Modeled Results Needs Assessment Printing an Offender Report 12 13 14 17 17 20 the paradigm shift in assessment RMS User Manual The Risk Analysis System RMS is a risk analysis system designed for corrections professionals by corrections professionals It takes information about an offender typically collected by a cor rectional system and applies an advanced mathematical approach called Example based Modeling EBM RMS uses EBM to generate a risk estimate for a given offender based upon the pattern of risk factors the offender possesses RMS s ability to use patterns to derive risk estimates for an offender give it a significant advantage over traditionally constructed risk tools You will begin to appreciate the advantage as you read
18. l consume the greatest amount of your su pervision resources This is understood when you realize that half of the offenders in this band are estimated to be roo genuine nonrecidivists and will be assigned exactly the same resources as the future recidivists There is no middle risk offender In summary arisk score from a tool like the ones described above help us take a slice of offend ers whom we have outcome information to determine the proportion of recidivists vs non recidivists at a score level similar to our target offender This information provides us with an estimate as to our chance of correctly classifying an offender as a recidivist or non recidivist No tice the word estimate Researchers use the word estimate to emphasize that although they have a degree of confidence in the data associated with their norm tables and the outcome data associ ated with the norm table researchers know that data samples can vary to some degree The missing piece to local norms Local norms are considered to be very valuable in most correctional systems But local norms are valuable only to the extent that you can make a compelling case that the population you in tend on using a risk tool on is fundamentally different from the tool s development sample Lo cal norms are often premised upon the notion that OUR offenders get their scores differently than YOUR offenders AND OR the proportion of recidivists vs non recidivists will be different for
19. nerating a score for a given offender wo The result of using scores in this fashion is to over classify the risk for some offenders while si multancously under classifying the risk for other offenders This happens because not all path ways to a particular score will carry the same level of predictive accuracy Andrews 1989 in the context of juvenile assessment stated The classic research is also near unanimous in suggest ing that there are several routes through which young people become at risk for delinquency Thus producing a list of risk factors does not suggest that any particular risk factor is always pre sent or that some factors not included in the lists may be very important to some people under some circumstances Andrew s insight provides an important key to the development of ad vanced risk assessment tools in general paying attention to the pattern of risk factors can en hance future tools Risk tools using scores as described previously essentially distill an on the average estimation as to the likelihood of correctly classifying an offender as a recidivist or a non recidivist for a given score But suppose you could remove pathways to a particular score that are less associ ated with recidivism so that they wouldn t water down the real estimation for a given offender To better understand this concept we need to revise our adage one more time RMS assessment To begin to understand RMS we must r
20. ook like this if more than one assessment has been done 10000 04 This indicates that four previous assessments exist What if s are simply another assessment to RMS 13 Selecting Results The Results Filter follows a logical flow It provides options that allow you to narrow search re sults As you select more options your search will narrow When you are done selecting the op tions press the Search Results Form button at the bottom of the screen to view your results on line o gt Search Results Form Search options 1 All offenders This option precludes any narrowing of the search You will see all rated offenders You must uncheck this option to narrow your search 2 Submit Date Range Search for RMS assessments between two dates For example you would like to see only your most recent assessments 3 Release Date Range Search for estimated release dates between two dates For example you would like to see who might be coming up for release and what their risk ratings have been 4 Risk Level Search Restricts the range of RMS risk scores to a band For example you want to identify offenders with scores above some criteria 5 Modeled Values Restricts to just Recidivism Violence or both For example you might restrict your search to only violence scores for offenders who might benefit from anger management programs 14 Needs Filter This option allows you to identify offenders with a certa
21. orithms EBM EBM identifies historical offenders most similar to an offender s risk factor pattern for whom we have behavioral outcomes and builds a modeled simulation of what might be expected for a given risk factor pattern i e whether the pattern is most associated with a recidivist or a nonrecidivist By incorporating EBM RMS automatically creates a local model based on the actual risk factors instead of geographic loca tion or simple summed scores Website Navigation www modelingsolutions net Welcome to the navigation section of the manual This section will organize itself around two primary functions entering arisk assessment and viewing the results RMS is a web based risk assessment tool that can be accessed in most standard internet browsers like Internet Explorer and Firefox that have a Macromedia Flash plug in For those of you in a Mac environment Safari works too Your computer should have Macromedia flash player 7 0 Most users find it useful to be at a computer when they read this manual However for those of you who prefer to relax while reading feel free just remember to take the manual with you the first time you actually navigate the website Getting to the Website The first thing that you have to do is get to the website It is located at www modelingsolutions net When the website opens you will see the screen captured below This is the front door to our current interface The opening
22. pear in this table until you select an offender in the Offender Information table Upon selecting an offender in the Offender Information table the Comparable Offender table will populate 0 28 of 28 Os oo Crap hkh Atasema Mrih Oste Gender Modeled fletease Dale Model Process 1OF 10744 MALE 153 2007 08 77 n Ponsa Repant 10P 19740 73 MALE 155 2007 05 27 TIPA 1062 014101 MALE 174 2005 09 29 n HPA 1962 04014 MALE 150 2005 00 29 v 12PR 1070 05 31 MALE 177 20s002 A 12PR 19790531 MALE 200 2005 00 26 13P0 10510516 MALE lt 9 20050727 A 13PD 1081 08 16 MALE 146 20050727 V 1430 erode MALE 200 20502 A 14 0 10810105 MALE 147 20060328 V ien TANA MAI F inn ATANT NA a Cre pamare Ofer reraton i Lagares Offender Guth Date Gender Actual Fisk Percent W P D W E Yeas Served View 10001 aoia MALE z lasa a le lo lo lo o lev A eg 10002 19511224 MALE 2 17708 5 5 2 0 0 2 767 PE a W Vooo 10003 19751212 MALE t 1770 1 1 o o o o loz E Escape 10004 imGe 22 MME 2 1250 a la lo o 1 3 i234 O Cre 10005 15 0 MALE z att 160 0 0 0 17 1250 10005 100 00 10 MALE 7 7 20 2 9 1 10 0 10 2 10007 10 0 MALE z 555 9 10 10 10 14 19 429 ieee ieee MALE 505 113190 19 190 242 1o09 1970 04 11 MALI r 7 0911 12 J arr 10019 1057 05 12 MALI q p 14 10 0 11 19539 In the screen above Offender 12PR is selected This is a male DOB 1970 Modeled Process is Recidivism with a score of 1 77 18 Offender 12PR has a group of
23. ven offender How does RMS provide greater accuracy Many readers have been to a risk assessment seminar or training class Invariably someone in the session will ask What does the risk score mean if an offender s score falls in the middle This is very good question because it drives to the heart of an important issue What is explained by the trainer is often misunderstood by the listener The listener typically interprets what is said as meaning that an offender with a score falling in the middle range presents a medium risk This conclusion is based upon the mathematical fact that the offender s score happens to be some where between the lowest possible score and the highest possible score on the risk tool regard less whether or not offenders with similar scores have the same risk factors But let s think this through Most risk assessment tools are comprised ofa list of risk factors that are related to a behavior of interest e g recidivism Put another way these risk factors are related to one of two categories clients who do recidivate and clients who do not recidivate there is no category of sort of re cidivate The range of scores from the lowest to highest possible score help us to select a point on the continuum of scores that will chop that continuum into two pieces The piece with the lowest scores typically represents the non recidivists whereas the other piece with the highest scores represents the
Download Pdf Manuals
Related Search