Session 7: Thinking with the End in Mind
Contents
1. Clinical Evaluation of Drugs for Treatment rea ATT e The goal for efficacy endpoints in IBS clinical trials is to assess the treatment effect on the core disease defining signs and symptoms of IBS in a well defined and reliable way e The PRO measure s should capture all of the clinically important signs and symptoms of the IBS target population e g IBS C or IBS D Irritable Bowel Syndrome Clinical Evaluation of Drugs for Treatment in Abdominal Pain If a drug is developed specifically to improve only one of the major signs or symptoms of IBS e g abdominal pain based on the drug s mechanism of action it is still important to assess the other important signs and symptoms to document that the drug has not negatively affected those components Range From 0 to 10 incrementing by 1 How would you rate your abdominal pain at Push button Left Label Dail its worst over the last 24 hours Enter a with numbers No abdominal Diary number from 0 to 10 where O represents shown in Pain y no abdominal pain and 10 represents very buttons l severe abdominal pain Right Label Very severe abdominal pain Irritable Bowel Syndrome Clinical Evaluation of Drugs for Treatment Stool Frequency Six different questions C S For IBS C the defecation component of the proposed primary endpoint can be evaluated by assessing stool frequency Bowel Movement pe you have a bowel movement to report since Note Y If phase Pr2. Other Measurement Properties Confirm conceptual framework with scoring rule Assess score reliability construct validity amp ability to detect change Finalize instrument content formats scoring procedures amp training materials Document measurement development Endpoint Model Parallel Development K 0 Sponsors should define the role a PRO endpoint is intended to play in the clinical trial i e a primary key secondary or exploratory endpoint so that the instrument development and performance can be reviewed in the context of the intended role and appropriate statistical methods can be planned and applied It is critical to plan these approaches in what can be called an endpoint model PRO instrument adequacy depends on its role as depicted in the endpoint model The endpoint model explains the exact demands placed on the PRO instrument to attain the evidence to meet the clinical trial objectives and support the targeted claims corresponding to the concepts measured CRITICAL PATH INSTITUTE Endpoint Model PRO Planned Analyses amp Hierarchy D We intend to determine the adequacy of clinical trial data to support claims in light of the prespecified method for endpoint analysis We usually view unplanned or post hoc Statistical analyses conducted after unblinding as exploratory and therefore unable to serve as the basis of a labeling claim of effectiveness A single hierarchy of
3. But the endpoint could be based on the single BPI SF item 3 worst pain over past 24 hours e EORTC QLQ C30 Has many questions and multiple domains Could we pre specify Time to Functional Deterioration as the Endpoint CRITICAL PATH INSTITUTE Also The Endpoint is MORE than Just PRO the Question Being Asked Kena CONCEPT PAIN Instrument Brief Pain Inventory Short Form 15 questions Endpoint a 7 EA 8 RR SANSE l eee BTT Pa 6 e O REAAL eran ETEA DAIN DA IATIO N usinge Iter na 2 Worse Pa nin past 24 hours FAIN FALLIATIUIN USINS ILC mo VWVOFSS Falll IN Vdol 26 NOUIS When an Instrument is Not the Endpoint Added Rationale is Needed be e lf the intent is to use a portion of an instrument as your endpoint strong rationale will need to be made for why this will still be a well defined and reliable assessment Measurement properties of the item domain type of analysis time to event responder analysis endpoint definition including threshold for response or deterioration concomitant medications etc e lt can be done but early consultation with the review division and SEALD is urged if you intend to consider this strategy CRITICAL PATH INSTITUTE From Instrument to Endpoint 1 PRO e Jakafi PRO endpoint Improvement in total symptom score consisting of 6 items Table 8 Improvement in Total Symptom Score Jakafi Placebo N 148 N 152 Number of Patients with 50 or Greater Red
4. Items and Responses and Instructions K e e User Manuals How the instrument should be scored How to we handle missing items missing scores if people go off study if people die When should it be collected When should data not be used Many other issues but more will be needed in the SOPs SAP protocol etc picture of all the documents that came with my house all the manuals etc Still did not tell me who to contact about electricity etc When study team members patients and others use a tool or an ePRO you are thinking of the User Manual Also think of it for the study designer data manager analyst Statistical Issues That May Affect p Results Multiplicity amp Non Inferiority K eesse e Type error inflation due to multiplicity What is are the score s being used Is this a single score composite endpoint something else Looking at items and a summary score e Non Inferiority f the study design is for non inferiority or equivalence how is that margin or bound set for the endpoint CRITICAL PATH INSTITUTE Statistical Issues That May Affect p Results Missing Data amp Subgroups K eraann e Missing data High percentage of dropouts e Is it the burden of the COA The ePRO Or something else Inappropriate imputation for missing values e Appropriate should take in to account tool development for example IRT or CTT e Inconsistency of results ac
5. endpoints as diagrammed in an endpoint model see Figures 1 and 2 in section III A Endpoint Model is determined by the trial s stated objectives and the clinical relevance and importance of each specific measure independently and in relationship to each other Endpoint Model Treatment of Disease X CONSORTIUM CRITICAL PATH INSTITUTE concept T Endpoint Indication Treatment of Disease X oe Supportive Concepts Improvement in Symptoms Signs of Disease X Primary Physiological Effect Secondary Symptoms Diary PRO Signs Diary PRO Physical Exam Physical Performance possibly PRO PRO Claim with an Indication Gai Concept Indication Treatment of Symptoms of Disease Y Supportive Concepts Other Treatment Benefit Link Endpoint Primary Total Disease Y Symptoms Score PRO Secondary Physical Performance Possible PRO Disease Y related Physical Limitations PRO CRITICAL PATH INSTITUTE Summary PRO Develop aspirational TPP identify potential claims and propose preliminary endpoint model Build a conceptual framework that supports the aspirational TPP Develop proposed scoring items l instrument that corresponds to conceptual framework Collect psychometric metadata and clinical data e g Phase 2b trials Align the conceptual framework scoring and potential claims Propose analysis endpoints that capture clinical meaning e g En
6. O data collected during a clinical trial and provide optimal information about the patient oerspective for use in making conclusions about treatment effect at the time of medical product approval Indication Claims PRO CRITICAL PATH INSTITUTE Hypothesize Conceptual Framework a Outline hypothesized concepts amp potential claims Modified Wheel Determine intended SEE j amp Spokes j Determine intended application characteristics type of scores mode amp frequency of administration Perform literature expert review Develop hypothesized conceptual framework Place PROs within preliminary endpoint model Document preliminary instrument development x Adjust Conceptual Outline Framework amp Draft recall s Instrument hypothesized Obtain patient input concepts amp other Generate new items Select recall period response H H options amp format potential claims Select mode method of administration data collection ey Conduct patient cognitive Collect Analyze amp Interpret interviewing Data Pilot test draft instrument one e lt Document content validity Prepare protocol amp statistical analysis plan S final endpoint model and responder Confirm Conceptual Framework amp Assess definition z Collect amp analyze data Other Measurement Properties Confirm conceptual framework with scoring rule Evaluate treatment response using TRE AiR E S cumulative distribution amp responder Assess score reliability c
7. Thinking with the End in Mind From COA Instrument to Endpoint SIXTH ANNUAL PATIENT REPORTED OUTCOME CONSORTIUM WORKSHOP April 29 30 2015 Silver Spring MD CRITICAL PATH si INSTITUTE Tn 1Gears Disclaimer PRO The views and opinions expressed in the following slides are those of the individual presenters and should not be attributed to their respective organizations companies the U S Food and Drug Administration the Critical Path Institute the PRO Consortium or the ePRO Consortium These slides are the intellectual property of the individual presenters and are protected under the copyright laws of the United States of America and other countries Used by permission All rights reserved All trademarks are the property of their respective owners CONSORTIU CRITICAL PATH INSTITUTE Session Objective PRO e Assessments of symptoms and function are important endpoints to document treatment risk benefit e The objective of the session is to discuss an approach that recommends that the endpoint development occurs alongside the instrument development _CONSORTIUM_ CRITICAL PATH INSTITUTE mare points Instruments Score s Session Participants PRO Moderator Jean Paty PhD Principal Advisory Services Quintiles Presenters and Panelists Paul G Kluetz MD Acting Deputy Director Office of Hematology and Oncology Products FDA David S Reasner PhD Vice President Dat
8. a Science and Head Study Endpoints Ironwood Pharmaceuticals Laura Lee Johnson PhD Associate Director Division of Biometrics Ill Office of Biostatistics Office of Translational Sciences FDA Elisabeth Piault Louis PharmD MA Principal Outcomes Research Scientist OncologyGenentech Endpoints in Cancer Clinical Trials Paul Kluetz M D Office of Hematology and Oncology Products U S FDA SIXTH ANNUAL PATIENT REPORTED OUTCOME CONSORTIUM WORKSHOP April 29 30 2015 Silver Spring MD CRITICAL PATH INSTITUTE Why is PRO so Foreign to Oncologists PRO as an Endpoint k re e We treat life threatening diseases where median survival can be less than one year Overall Survival Gold Standard Endpoint e We can Visualize our Disease and Objectively Quantify a Drug s Effect on the Tumor Radiographic response and time to progression e Oncologists and Statisticians have standard definitions and analyses for these endpoints CONSORTIUM CRITICAL PATH INSTITUTE But OS and PFS Aren t the Whole story PRO e There is broad consensus that accurately describing a patient s experience is important to patients and physicians e There has been difficulty in agreement on the optimal instruments collection and interpretation of PRO data e Efforts are underway to OPTIMIZE and STANDARDIZE PRO in oncology trials CRITICAL PATH INSTITUTE Which Questions Can PRO Answer PRO e Efficacy Does the dru
9. d of Phase 2 milestone meeting 44 You Have a Measure You Have a Tool Do You Have an Endpoint for the Study Laura Lee Johnson Ph D Division of Biometrics Ill Office of Biostatistics U S FDA SIXTH ANNUAL PATIENT REPORTED OUTCOME CONSORTIUM WORKSHOP April 29 30 2015 Silver Spring MD CRITICAL PATH INSTITUTE Struggle PRO e Measures e Endpoint usually uses some summary Time until an event defined by the measure Difference in scores e g Week 12 score Baseline score e Weekly average but have a daily diary but Amillion other ways to make an endpoint e Some make sense and can be analyzed and interpreted on label other endpoints are not as clear CONSORTIUM CRITICAL PATH INSTITUTE Responder Definition Responder Analysis k e To use or not to use a responder definition as a part of the endpoint e Impact on power and analysis methods e What if the responder definition changes e Responder sounds nice but is there data to say that the status change matters clinically Comparing responded by a certain time Time until become a responder e Can responder status change multiple times Something else Struggle PRO e Interpretation and analysis issues True for PROs PET scans etc e What is the difference to plan power and sample size in a trial Interim analyses and how to interpret CRITICAL PATH INSTITUTE Instrument is EVERYTHING Not Only PRO
10. d randomized clinical trials Pain with Bone Seeking Radioisotopes Total Symptom Score with Jakafi e Less experience with Time to Deterioration TTD analyses which have specific challenges Some have radiographic progression before symptoms worsen Would require follow up beyond progression Likely to be affected by subsequent therapies Higher risk of missing data e TTD can be done and can be supportive but work must be done to optimize these types of analyses CONSORTIUM CRITICAL PATH INSTITUTE Must We Assess ALL Endpoints in One Single Trial e The best population to study a drug s effect on disease related symptoms is a Symptomatic Patient e Many products are being tested in asymptomatic or minimally symptomatic populations e While TTD analyses can be done consider a second smaller trial enriching for symptomatic patients and optimizing design for PRO disease symptom palliation primary endpoint It All Comes Down to Risk to Patients PRO What should we all be worried about e A PRO result is labeled that is significantly biased and or is NOT clinically meaningful to patients false positive e Worst Case Scenario 1 False Positive PRO is the sole primary endpoint of a new drug or biologic anti cancer product Risk Ineffective drug is marketed to patients with toxicity and no benefit toxic placebo e Worst Case Scenario 2 False Positive PRO is labelled but drug has demonstrated e
11. e Treatment Y the most recent date time of the following 00 00 day of assignment or when subject records No more BM completion date time of most recent BM Diary completed in Pre treatment or 00 00 of the previous day If phase Treatment or Post Treatment Y the most recent date time of the following Completion date time of Eligibility Review Report or when subject record no more BM completion date time of most recent BM Diary completed in Treatment or Post Treatment phase or 00 00 of the previous day Note business rule applies Display all date times as DD Mmm YYYY HH mm Multiple choice choose one Offset 1 Ascending values Solid button CONSORTIUM CRITICAL PATH INSTITUTE Yes 1 If yes next screen No 2 If no go to st question daily diary CRITICAL PATH INSTITUTE Irritable Bowel Syndrome PRO k Clinical Evaluation of Drugs for Treatment rea aT ST Provisional Endpoints for IBS Clinical Trials e Weekly Stool frequency as measured by the number of complete spontaneous bowel movements CSBMs per week e Weekly Percent Change from Baseline in Abdominal pain intensity as measured by a numeric rating scale i e O to 10 that asks patients to rate their worst abdominal pain e Overall Responder A responder in greater than 50 of the treatment period weeks where response is both a decrease in abdominal pain intensity of at least 30 and an increase of at least 1 CSBM per week b
12. ence based on relations to other endpoints including predictive validity 29 CRITICAL PATH INSTITUTE Conceptual Framework s PRO The conceptual framework of a PRO instrument may be straightforward if a single item is a reliable and valid measure of the concept of interest e g pain intensity However single item measures of General general concepts that include Concept multiple items or domains rarely provide sufficient evidence to support claims about that general concept e g a global question concerning a functional disorder defined by clusters of specific signs and symptoms PRO Guidance CRITICAL PATH INSTITUTE Open Questions PRO k Measurement Properties CONSORTIUM Are the construct s underlying the patient reported outcome appropriate to the targeted mechanism of action Open Questions PRO b Measurement Properties CONSORTIUM CRITICAL PATH INSTITUTE Does the patient in the context of use attend to all the available symptoms 32 CRITICAL PATH INSTITUTE Open Questions PRO b Measurement Properties CONSORTIUM Do clinicians reference signs and symptoms outside of the content derived conceptual framework 33 CRITICAL PATH INSTITUTE Open Questions PRO b Measurement Properties CONSORTIUM Do clinicians reference all the signs and symptoms inside of the content derived conceptual framework 34 CRITICAL PATH INSTITUTE Irritable Bowel Syndrome p k
13. etadata PRO CRITICAL PATH INSTITUTE Characteristics of PRO instruments that are reviewed Concepts as elicited from patients Conceptual framework as documented in the patient population and in the words of the patients Content validity i e items and other measurement properties Administration mode as appropriate to the patient population Scoring of the questionnaire to create instrument Collection of analysis variables and derivation of analysis endpoints Note The FDA will review documentation of PRO instrument development and testing in conjunction with clinical trial results to determine whether a labeling claim is substantiated Psychometric Metadata PRO A few definitions related to measurement properties Content Validity Evidence that the instrument measures the concept of interest including evidence from qualitative studies that the items and domains of an instrument are appropriate and comprehensive i e saturation relative to its intended measurement concept Testing other measurement properties will not replace or rectify problems with content validity Necessary but not sufficient Construct Validity Evidence that relationships among items domains and concepts conform to a priori hypotheses concerning logical relationships that should exist with measures of related concepts or scores produced in similar or diverse patient groups e g discriminant and convergent validity Evid
14. ffect on tumor based on PFS or OS Risk Patients and physicians may choose the therapy over another more effective therapy that does not have a labeled PRO benefit CONSORTIUM CRITICAL PATH INSTITUTE How Can We Reduce This Risk Optimize and Standardize PRO b e Core Concepts to Measure e Narrow Group of Instruments for each Concept e Clear Endpoints with Meaningful Definitions e Clinical trial design assessments data collection e Data analysis Primary and Sensitivity Analyses e Data presentation in Manuscripts and FDA Label Linking Assessments amp Endpoints Measurement Properties David S Reasner Ph D lronwood Pharmaceuticals SIXTH ANNUAL PATIENT REPORTED OUTCOME CONSORTIUM WORKSHOP April 29 30 2015 Silver Spring MD Assessments amp Endpoints Measurement Properties GNTCAL ATHINSTUT Goal Treatment of Disease X or Treatment of the Symptoms of Disease Y Path Along the Way Patient as the phenomenon Patient as the instrument Psychometric metadata Well defined and reliable assessments Clinically meaningful endpoints i e relative improvement CRITICAL PATH INSTITUTE Patient Perspective s PRO By explicitly addressing the review issues identified in this guidance sponsors can increase the efficiency of their discussions with the FDA during the medical product development process streamline the FDA s review of PRO instrument adequacy and resultant PR
15. g provide superior improvement in disease related symptoms or functional deficits Pain Total Symptom Score Performance related outcomes More conducive to formal statistical analysis statistical superiority e What do patients experience while on therapy Adverse events from therapy PRO CTCAE Physical function Performance status Likely to be more descriptive in nature For This Panel Will Focus on Efficacy PRO e Efficacy Does the drug provide superior improvement in disease related symptoms or functional deficits Pain Total Symptom Score Performance related outcomes More conducive to formal statistical analysis statistical superiority CRITICAL PATH INSTITUTE PRO Efficacy Endpoints What is Important PRO to the Clinical Reviewers at FDA K KAE e FDA has reviewed the instrument and feels that it is adequate not perfect but fit for purpose e Questions asked make clinical sense in evaluating efficacy e Sponsor has provided data to support the rationale for the meaningful difference that creates the threshold for the endpoint event improved pain deterioration in function etc e There is a hypothesis and it is TESTED There is a pre specified statistical analysis plan for how the data will be analyzed CONSORTIUM CRITICAL PATH INSTITUTE When an Instrument is Not the Endpoint e Brief Pain Inventory Short Form Instrument BPI SF Has many questions
16. onstruct validity amp ability to detect change definition Document interpretation of treatment benefit Finalize instrument content formats scoring procedures amp training materials in relation to claim Document measurement development Dr err TR RT K s Labeling Goals s PRO CRITICAL PATH INSTITUTE Sections Instrument Targeted Claims or TPP Target Product Profile Endpoint Model The PRO Instrument s Conceptual Framework Content Validity Documentation Assessment of Other Measurement Properties Interpretation of Scores Language Translation and Cultural Adaptation Data Collection Method Modifications PRO Specific Plans Related to Clinical Trial Design and Data Analysis Key References Appendix A User Manual Appendix B Item Tracking Matrix Appendix C Transcripts PRO Guidance Appendix The Dossier eCTD 5 3 5 3 Target Product Profile A Strategic Development Process Tool Key Section List ae ee ee 9 PRP RP PPP KA Oe Se ee 2 Indications and Usage Dosage and Administration Dosage Forms and Strengths Contraindications Warnings and Precautions Adverse Reactions Drug Interactions Use in Specific Populations Drug Abuse and Dependence Overdosage Description Clinical Pharmacology Nonclinical Toxicology Clinical Studies References How Supplied Storage and Handling Patient Counseling Information 27 gt CONSORTIUM CRITICAL PATH INSTITUTE Psychometric M
17. oth relative to baseline final endpoint model and responder definition Hypothesize Conceptual Framework Outline hypothesized concepts amp potential claims Determine intended population Determine intended application characteristics type of scores mode amp frequency of administration Perform literature expert review Develop hypothesized conceptual framework Place PROs within preliminary endpoint model Place PROs within preliminary endpoint model Document preliminary instrument development Modify Instrument Change wording of items populations response options recall y period or mode method of administration data collection Translate amp culturally adapt to other STs UEC Evaluate modifications as appropriate Document all changes Collect Analyze amp Interpret Data Prepare protocol amp statistical analysis plan final endpoint model and responder definition Collect amp analyze data Evaluate treatment response using cumulative distribution amp responder definition Document interpretation of treatment benefit in relation to claim Adjust Conceptual Framework amp Draft Instrument Obtain patient input Generate new items Select recall period response options amp format Select mode method of administration data collection Conduct patient cognitive interviewing Pilot test draft instrument Document content validity ii Confirm Conceptual Framework amp Assess
18. ross subgroups Is it the interventions or the measure CRITICAL PATH INSTITUTE Other Topics from Reviewers PRO e When working on protocols and when asking FDA for advice keep in mind our Statistical Reviews include information on the following Breaking the blind can this happen by accident Unblinded or unplanned interim analyses Change of primary endpoint during conduct of the trial e Dropping items e Linking studies Dropping adding treatment arms Sample size modification Planned and unplanned study design adaptations e Usually this means to randomization etc Plenty of Other Topics to Consider PRO CRITICAL PATH INSTITUTE e Talk to Regulators e Talk to the statisticians who will design and analyze the data Many will say number Ok and not ask questions about the PRO itself Be careful and pro active e Start thinking in early development of the COA how it will be used in randomized longitudinal clinical trials Every validation study Qualitative work Every step needs to keep the end in mind Discussion Discussion PRO Moderator Jean Paty PhD Principal Advisory Services Quintiles Presenters and Panelists Paul G Kluetz MD Acting Deputy Director Office of Hematology and Oncology Products FDA David S Reasner PhD Vice President Data Science and Head Study Endpoints Ironwood Pharmaceuticals Laura Lee Johnson PhD As
19. sociate Director Division of Biometrics Ill Office of Biostatistics Office of Translational Sciences FDA Elisabeth Piault Louis PharmD MA Principal Outcomes Research Scientist OncologyGenentech _ 8 k B A JIC EAAS P R O Je S l w i 7 wee 7 fP a 4 CONSORTIUM g d g L J w em m 8 889 w T TQ iVviaac BC Na CRITICAL PATH INSTITUTE mi l w FEV EUW dl l aa iiD G Endpoint Subscale a 1 e 55 SERAI 1 6 7 9 10 13 16 30 32 eb 3 18 19 22 27 29 31 28 5 21 25 26 16 11 23 28 12 2 14 17 34 16 Overall OV All items 34 170 136 CONSORTIUM CRITICAL PATH INSTITUTE Irritable Bowel Syndrome Quality of Life PRO Measure IBS QOL e Subscales are scored through simple summative scaling s All items are negatively framed on a 1 5 point scale with the highest response scale equaling the worst quality of life s When scored all items are reversed so that as IBS QOL scores increase quality of life increases e All final raw scores are transformed to a 0 to 100 scale using the following formula sum of the items lowest score Scale Score ______YY Y Y Y x 100 possible raw sum range
20. uction in Total Symptom Score by Week 24 aoe oer P value lt 0 0001 e Jakati did not statistically signiticantly reduce the severity ot any of the 6 individual symptoms Those hypotheses weren t statistically tested The individual symptoms were DESCRIPTIVELY analyzed to show there was no one symptom driving the result FDA Label Jakafi CRITICAL PATH INSTITUTE If you want to show a reduction in p something it has to exist at baseline eo For our Common Radiographic Endpoints e Tumor at Baseline Either measure tumor response or measure the time it takes to grow progression Endpoint Objective Response Rate responder analysis e Definition 30 reduction in tumor measure Endpoint Progression Free Survival time to event e Definition 20 increase in tumor measure OR death from any cause e Ifyou have NO tumor you cannot assess a response rate and must use a time to appearance of disease Endpoint Disease Free Survival time to event e Definition Radiographic appearance of disease or Death CRITICAL PATH INSTITUTE PRO Endpoints are Not Different PRO e Symptomatic Patients gt Symptom Improvement Symptom Palliation e Minimally Symptomatic Patients gt Time to Symptom Deterioration CRITICAL PATH INSTITUTE Palliation or Time to Deterioration PRO e Successful oncology PRO labeling claims have been palliation endpoints with responder definitions conducted in blinde
Download Pdf Manuals
Related Search