BOA User`s Manual - University of Iowa College of Public Health
Contents
1. University Brooks S and Gelman A 1998 General methods for monitoring convergence of iterative simulations Journal of Computational and Graphical Statistics 7 4 434 455 Brooks S P and Roberts G O 1998 Convergence assessment techniques for Markov chain Monte Carlo Statistics and Computing 8 4 319 335 Chen M H and Shao Q M 1999 Monte Carlo estimation of Bayesian credible and HPD intervals Journal of Computational and Graphical Statistics 8 1 69 92 Cowles M K and Carlin B P 1996 Markov chain Monte Carlo convergence diagnostics a comparative review Journal of the American Statistical Associa tion 91 883 904 Gelman A and Rubin D B 1992 Inference from iterative simulation using multiple sequences Statistical Science 7 457 511 Geweke J 1992 Evaluating the accuracy of sampling based approaches to calculating posterior moments In Bayesian Statistics 4 eds J M Bernardo J O Berger A P Dawid and A F M Smith Oxford Oxford University Press Heidelberger P and Welch P 1983 Simulation run length control in the presence of an initial transient Operations Research 31 1109 1144 Jennison C 1993 Discussion of Bayesian computation via the Gibbs sampler and related Markov chain Monte Carlo methods by Smith and Roberts Journal of the Royal Statistical Society Series B 55 54 56 43 10 Raftery A L and Lewis S 1992a Comment One long run with diag2. be reduced if the chains were run to infinity To adjust for the sampling variability in the variance estimates the correction proposed by Brooks and Gelman 1998 is applied to the PSRF to produce the corrected scale reduction factor CSRF BOA also displays an upper quantile of the sampling distribution for the CSRF Users can control which quantile is computed via Option 1 in Section 5 3 Brooks and Gelman 1998 developed a multivariate extension to the PSRF known as the multivariate potential scale reduction factor MPSRF The MPSRF does not include a correction for sampling variability This statistic is relevant when interest lies in general multivariate functionals of the chain The MPSRF and the PSRF satisfy the following relationship maz PSRF lt MPSRF Computation of the reduction factors is based on analysis of variance and sampling from the normal distribution To avoid violations of the latter assumption BOA transforms any parameters specified to be restricted to the range a b to the loga rithmic or logit scale before calculating this diagnostic By default only the second half of the chains iterations 101 200 is used to compute the reduction factors Op tion 2 in 5 3 can be used to vary the proportion of samples from the end of the chains to be included in the analysis If the estimates are approximately equal to one or as a rule of thumb the 0 975 quantile is lt 1 2 the samples may be considered to have arisen from t
3. interval is always greater than that outside Consequently HPD intervals are of the short est length of any of the Bayesian intervals The algorithm described by Chen and Shao 1999 is used to compute the HPD intervals in BOA under the assumption of unimodal marginal posterior distributions The alpha level for the intervals can be modified through Option 12 in Section 5 3 HIGHEST PROBABILITY DENSITY INTERVALS Alpha level 0 05 Chain linet Lower Bound Upper Bound alpha 1 64596000 3 95041 beta 0 17817400 1 52666 sigma 0 37893400 2 46407 tau 0 00575839 4 45471 5 1 4 Summary Statistic This option prints summary statistics for the parameters in each chain The sample mean and standard deviation are given in the first two columns These are followed by three separate estimates of the standard error 1 a naive estimate the sample standard deviation divided by the square root of the sample size which assumes the sampled values are independent 2 a time series estimate the square root of the spectral density variance estimate divided by the sample size which gives the asymp totic standard error Geweke 1992 and 3 a batch estimate calculated as the sample standard deviation of the means from consecutive batches of size 50 divided by the square root of the number of batches The autocorrelation between batch means follows and should be close to zero If not the batch size should be increased Quan tiles are given after the ba
4. 150 First Iteration in Segment First Iteration in Segment Geweke Convergence Diagnostic Geweke Convergence Diagnostic JU Sigma met sigma meg NADA a Aa E Ory Fie A o 4 o a o g J g o 5 a D 4 o ASA y O NE A N vy t A ea o_o N oo ja T z 7 E E Dee yes ee iS a eae eee Se ees ay T T e T T T T T 2 T 0 50 100 150 0 50 100 150 First Iteration in Segment First Iteration in Segment Hai 17 Title Bayesian Output Analysis The options grouped under the Graphics heading control the general layout used to generate plots The following gives a brief description of each of these options 12 The device name used by S PLUS or R to open graphics windows Users should not modify this value 13 If set to TRUE all plots generated in BOA will be kept open oth erwise a value of FALSE indicates that only the most recently opened plots be kept open 14 The number of rows and columns respectively of plots to display in one graphics window 15 If set to TRUE only one chain is displayed per plot otherwise a value of FALSE forces all of the chains to be displayed on the same plot 16 If set to TRUE only one parameter is displayed per plot otherwise a value of FALSE forces all of the parameters to be displayed on the same plot 17 The title to be displayed centered at the top of all graphics windows 37 Chapter 7 Options Menu The Options Menu serves as a central location from which
5. 2 2 Delete Parameters Often times it may be desired to delete parameters that are not of interest in the analysis This may arise in cases where data other than model parameters were saved to the output file imported into BOA Alternatively the user may only be interested in functions of the original parameters Once the new parameter is created using the methods described in the following section the unnecessary parameter upon which it is based may be deleted Deleted parameters will speed up the manipulation of data in BOA DELETE PARAMETERS 1 2 3 alpha beta sigma Specify parameter index or vector of indices none 16 4 2 3 Create New Parameters BOA includes an option to create new parameters Most S functions can be used to create the new parameter Typically a new parameter is defined as a function of the existing parameters For instance suppose the user was interested in analyzing the precision parameter tau 1 sigma The following menu commands demonstrate how to create this precision parameter NEW PARAMETER 1 alpha beta sigma New parameter name none 1 tau Read 1 items Define the new parameter as a function of the parameters listed above 1 1 sigma 2 Read 1 items tau has now been added to the two datasets in BOA and will be available to all subsequent analyses 4 3 Display Working Dataset Selecting this option will display summary information for the Working dataset CHAIN SUMMARY IN
6. 96 for a more in depth review and comparison of these methods 5 2 1 Brooks Gelman amp Rubin Convergence Diagnostic The code for implementing the Gelman and Rubin 1992 convergence diagnostic in BOA is based on the itsim function contributed to the Statlib archive by Andrew Gelman http lib stat cmu edu The Brooks Gelman and Rubin convergence diagnostic is appropriate for the analysis of two or more parallel chains each with different starting values which are overdispersed with respect to the target distribution Several methods for generating starting values for the MCMC samplers have been proposed Gelman and Rubin 1992 Applegate et al 1990 Jennison 1993 The following diagnostic information was obtained for the line example BROOKS GELMAN AND RUBIN CONVERGENCE DIAGNOSTICS Iterations used 101 200 Potential Scale Reduction Factors alpha beta sigma 1 00316 1 00171 1 00841 Multivariate Potential Scale Reduction Factor 1 026086 Corrected Scale Reduction Factors Estimate 0 975 alpha 1 00596 1 03859 beta 1 00940 1 03702 sigma 1 01147 1 06440 The diagnostic originally proposed by Gelman and Rubin 1992 is based on a com parison of the within and between chain variance for each variable This comparison is used to estimate the potential scale reduction factor PSRF the multiplicative 24 factor by which the sampling based estimate of the scale parameter of the marginal posterior distribution might
7. Allocation in R R and S PLUS differ in the way they request memory from the operating system The amount of memory allocated to S PLUS is free to grow provided the computer s memory resources have not been exhausted In contrast a fixed amount of memory is allocated to R when it first starts up If too little memory is allocated R may prematurely run out of memory when trying to import large datasets The amount of allocated memory may be specified by appending the options vsize and nsize to the end of the command used to start the R program Type help Memory in R for detailed information on using these options The following error indicates that the vsize is too small Error heap memory 6144 Kb exhausted needed 125 Kb more See help Memory on how to increase the heap size whereas the following message is displayed when the nsize is too small Error cons memory 250000 cells exhausted See help Memory on how to increase the number of cons cells For analyzing extremely large datasets users may want to experiment with different values for these options until they find a combination that works with their data On systems with 128Mb of physical memory a reasonable starting point might be vsize 64M nsize 4M 42 Bibliography 1 Applegate D Kannan R and Polson N G 1990 Random polynomial time algorithms for sampling from joint distributions Technical report no 500 Carnegie Mellon
8. BAYESIAN OUTPUT ANALYSIS PROGRAM BOA VERSION 1 0 USER S MANUAL Brian J Smith January 8 2003 Contents 1 Getting Started 4 1 1 Hardware Software Requirements 2 4 12 2 harm OAc dey sae bg cae at Go pas eB ah al Og Scand haben bea 4 LS Installation sse 3 a i aa te dt a A es 4 1 4 BUGS Line Example Sg oes ee pe oe ee eh ee eS 5 2 Using the BOA Menu Driven User Interface 6 3 File Menu 8 oe dimport Data Men q ee baie ts AS dia 8 3 1 1 BUGS Output File og 2 8 6 6d eRe Gd ee es 9 ee Eo A Gis ora tng rege ai iea era Say Hein e Cid Parnas Gad Ean 9 3 1 3 Data Matrix Object soe 6 4 e962 A ee io 10 3 1 4 View Format Specifications rod sra ere 10 21 5 Data Opuons A a ed A e e ee e e 10 32 Load Sessione tease Se Beans SE ee eh SY Gok Ae Go ee he eae ke SE 11 o A A A eel a beds ge EEA 11 Bo EXI ORs Atila 11 4 Data Management Menu 12 4 1 Chains Menu 2 2 62 ee 12 4 1 1 Combine All Chains 2s ow ae aN aa Be A ed 13 4 1 2 Delete Chain 0 00000 000008 13 4 1 3 Subset Chains 0 0000 000000000084 14 4 2 Parameters Menu 0 0 0 00 0 eee 15 4 2 1 Set Parameter Bounds 2 2s cue A e ee 6S 15 4 2 2 Delete Parameters 000 000000000084 16 4 2 3 Create New Parameters 2 24 4 a64 4 84 244 e e242 24 17 7 8 4 3 Display Working Datas tins pu Ste oe eos Sen te es bs Sota ie ibi Sate dg ney 4 4 Display Master Dataset o o E A AsO SO acy Mie O ey wd aa
9. Back 7 Return to Main Menu Selection 3 1 1 BUGS Output File The two CODA output files generated by the Bayesian inference Using Gibbs Sam pling BUGS or WinBUGS program can be imported into BOA The output file containing the parameter definitions should be saved as a ind file whereas the file containing the sampler output should be saved as a out file BOA will expect these files to be located in the Working Directory See Section 3 1 5 for instructions on specifying the working directory Upon selecting to import BUGS output the user will be prompted to Enter filename prefix without the ind out extension Working Directory d bjsmith boa 1 linel Only the filename prefix should be specified BOA will automatically add the appro priate extensions and load the data from the linel ind and linel out files 3 1 2 Flat ASCII File BOA includes an import filter for general ASCII files This is particularly useful for output generated by custom MCMC programs The ASCII file should contain one run of the sampler with the monitored parameters stored in space comma or tab delimited columns and the parameter names in the first row Iteration numbers may be specified in a column labeled iter The ASCII file should be located in the Working Directory Upon selected to import an ASCII file the program will prompt the user to Enter filename prefix without the txt extension Working Directory d bjsmith boa 1 linel Spe
10. Basics 9 1 Output Display Options The options function in S PLUS and R can be used to control the format of the outputted text in BOA This should be done prior to starting BOA To set the number of significant digits to be displayed type options digits lt value gt The number of characters allowed per line can be controlled by entering the command options width lt value gt 9 2 Vectors in S Several menu selections in BOA prompt the user to input a vector of data Vectors in S can be supplied in a variety of ways The simplest way to construct a vector is with the concatenation function c c lt element 1 gt lt element 2 gt lt element n gt where the elements may be numerical or logical values or character strings Another means of constructing vectors is with the seq function seq lt starting value gt lt ending value gt length lt number of values gt Al or seq lt starting value gt lt ending value gt by lt step size gt where length is number of values in the vector and by is the spacing between successive values in the vector The operator which is a special case of the seq function can also be used to construct vectors This operator can be defined as lt starting value gt lt ending value gt seq lt starting value gt lt ending value gt by 1 For more detailed information about these functions consult the help systems in S PLUS or R 9 3 Memory
11. FORMATION Iterations Min Max Sample line1 1 200 200 line2 1 200 200 17 Support linel alpha beta sigma tau Min Inf Inf o 0 Max Inf Inf Inf Inf Support line2 alpha beta sigma tau Min Inf Inf 0 0 Max Inf Inf Inf Inf In this example the two chains have been combined Hence the Working dataset is a modified version of the Master dataset 4 4 Display Master Dataset Selecting this option will display summary information for the Master dataset CHAIN SUMMARY INFORMATION Iterations IA A A A A A A A A Min Max Sample linel 1 200 200 line2 1 200 200 Support linel alpha beta sigma tau Min Inf Inf o o0 Max Inf Inf Inf Inf 18 Support line2 alpha beta sigma tau Min Inf Inf 0 0 Max Inf Inf Inf Inf 4 5 Reset The Reset option copies the Master dataset to the Working dataset This undoes any modifications that were made to the Working dataset 19 Chapter 5 Analysis Menu The statistical analysis procedures are accessible through the Analysis Menu Anal yses are categorized into two groups Descriptive Statistics and Convergence Diag nostics ANALYSIS MENU 1 Descriptive Statistics gt gt 2 Convergence Diagnostics gt gt 3 Options 4 Return to Main Menu Selection 5 1 Descriptive Statistics Menu Options to compute autocorrelations cross correlations and summary statistics are available from the Descriptive Statistics Menu DESCRIPTIVE STATIST
12. ICS MENU 1 Autocorrelations 2 Correlation Matrix 3 Highest Probability Density Intervals 4 Summary Statistics 5 Back 6 Return to Main Menu Selection 20 5 1 1 Autocorrelations This option produces lag autocorrelations for the monitored parameters within each chain High autocorrelations indicate slow mixing within a chain and usually slow convergence to the posterior distribution LAGS AND AUTOCORRELATIONS Lag 1 Lag 5 Lag 10 Lag 50 alpha 0 141244 0 0935154 0 1139290 0 1150173 beta 0 240793 0 2157423 0 1174318 0 0240399 sigma 0 328217 0 1068319 0 0015778 0 0292616 tau 0 296238 0 1474137 0 0114918 0 0073957 Option 11 in Section 5 3 allows the user to set the lags at which autocorrelations are computed 5 1 2 Correlation Matrix This option returns the correlation matrix for the parameters in each chain High correlation among parameters may lead to slow convergence to the posterior As sociated models may need to be reparameterized in order to reduce the amount of cross correlation CROSS CORRELATION MATRIX alpha beta Sigma tau alpha 1 beta 0 205461 1 sigma 0 176248 0 312405 1 tau 0 034745 0 052883 0 468271 1 21 5 1 3 Highest Probability Density Intervals Highest probability density HPD interval estimation is one means of generating Bayesian posterior intervals HPD intervals span a region of values containing 1 a x 100 of the posterior density such that the posterior density within the
13. aa ia pate Mea 20 20 21 21 22 22 23 24 25 26 27 28 30 30 30 31 31 31 32 32 33 34 34 38 9 S PLUS and R Basics Al 9 1 Output Display Options y sa oe oh Bo a 41 A e EN 41 9 3 Memory Allocation in R A A we 42 Chapter 1 Getting Started 1 1 Hardware Software Requirements BOA has been successfully tested on S PLUS versions 3 4 6 0 for UNIX S PLUS 2000 for Microsoft Windows and R 1 2 0 for UNIX and Microsoft Windows 1 2 Obtaining BOA The program files and help documentation are freely available from http www public health uiowa edu 1 3 Installation The program source is located in the ASCII text file boa s S PLUS version OR boa r R version UNIX S PLUS and R users should source the appropriate file via source lt program directory gt boa s OR source lt program directory gt where lt program directory gt is the directory in which the program files are located S PLUS Windows users may open this as an S PLUS script file and run that file from the script window The BOA program need not be installed to the directory in which the data to be analyzed are located The location of the data can be specified from within the program 1 4 BUGS Line Example Output from the BUGS Line example is used to illustrate the capabilities of the BOA program The Line example involves a liner regression analysis of the data points 1 1 2 3 3 3 4 3 and 5 5 The proposed Bayesian mo
14. cify only the filename prefix The import filter will automatically add the exten sion and load the data from the linel txt file See Section 3 1 5 for instructions on specifying the Working Directory and the default ASCII file extension 3 1 3 Data Matrix Object MCMC output stored as an S object may be imported into BOA The object must be a numeric matrix whose columns contain the monitored parameters from one run of the sampler The iteration numbers and parameter names may be specified in the dimnames Upon selecting to import a matrix object the user will be asked to Enter object name none 1 linel BOA will import the data from the linel object in the current S PLUS or R session 3 1 4 View Format Specifications Selecting this menu item will display the format specifications for the three types of data that BOA can import BUGS Bayesian inference Using Gibbs Sampling output files ind and out files must be located in the Working Directory see Options ASCII ASCII file txt containing the monitored parameters from one run of the sampler file must be located in the Working Directory see Options parameters are stored in space comma or tab delimited columns parameter names must appear in the first row iteration numbers may be specified in a column labeled iter Matrix Object S or R numeric matrix whose columns contain the monitored parameters from one run of the sampler iteration numbe
15. del is yli N muli tau mult alpha beta x x i mean x with the following priors alpha N 0 0 0001 beta N 0 0 0001 tau Gamma 0 001 0 001 Interest lies in estimating the posterior distribution of alpha beta and sigma 1 sqrt tau The starting values for the parameters were varied to generate two parallel chains from the Markov chain Monte Carlo MCMC sampler The first chain linel was generated with the initial values of list tau 5 alpha 5 beta 5 seed 987654321 whereas the second chain line2 was generated with list tau 0 01 alpha 0 01 beta 0 01 seed 1234567890 Each chain contains 200 iterations The resulting output is available from the BOA website Chapter 2 Using the BOA Menu Driven User Interface A menu driven interface is supplied with the BOA It provides easy access to all of the command line function To start the menu system type Bayesian Output Analysis Program BOA Version 1 0 0 for UNIX R Copyright c 2001 Brian J Smith lt brian j smith uiowa edu gt This program is free software you can redistribute it and or modify it under the terms of the GNU General Public License as published by the Free Software Foundation either version 2 of the License or any later version This program is distributed in the hope that it will be useful but WITHOUT ANY WARRANTY without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE Se
16. e 0 0 50 100 150 200 0 50 100 150 200 Iteration Iteration Trace Plot sigma he 12 Parameter Value 0246 8 0 50 100 150 200 Iteration interpolate through the point estimates from the segments 6 2 3 Geweke Plot Plots the Geweke Z statistic see Section 6 3 for each parameter in successively smaller segments of the chain The kt segment contains the last number of bins k 1 number of bins 100 of the iterations in the chain Options 8 and 9 in Section 6 3 set the error rate for the confidence bounds and the number of bins included in the plot respectively Options 10 and 11 control the fraction of iterations covered by the windows used in computing the Geweke diagnostic It may be possible that some of the subsets contain too few iterations to compute the test statistic Such segments if they exist are automatically omitted from the plot The test statistic is plotted against the minimum iteration number for the segment 6 3 Plot Options 34 Bayesian Output Analysis Brooks amp Gelman Multivariate Shrink Factors 1 05 Rh 1 03 1 04 Shrink Factor 1 02 l 50 100 150 200 Last Iteration in Segment Plot Parameters 1 Number of Bins 20 2 Window Fraction 0 5 Density 3 Bandwidth function x 0 5 diff range x log length x 1 4 Kernel gaussian Gelman amp Rubin 5 Alpha Level 0 05 35 Bayesian Outpu
17. e the GNU General Public License for more details For a copy of the GNU General Public License write to the Free Software Foundation Inc 59 Temple Place Suite 330 Boston MA 02111 1307 USA or visit their web site at http www gnu org copyleft gpl html NOTE if the menu unexpectedly terminates type boa menu recover TRUE to restart and recover your work BOA MAIN MENU FO RK RK 1 File gt gt 2 Data gt gt 3 Analysis gt gt 4 Plot gt gt 5 Options gt gt 6 Window gt gt Selection Note the message given at startup if the menu unexpectedly terminates type boa menu recover TRUE to restart and recover your work There are a few instances where supplying the wrong type of data will crash the menu system Im mediately doing a recover will ensure that no data is lost Chapter 3 File Menu Selecting menu item 1 from the BOA Main Menu brings up the File Menu Options to import data load previously saved session data save the current session and exit the program are available from the File Menu FILE MENU 1 Import Data gt gt 2 Load Session 3 Save Session 4 Return to Main Menu 5 Exit BOA Selection 3 1 Import Data Menu BOA can import MCMC output from a variety of sources Data may be imported at any point in the analysis from three different sources IMPORT DATA MENU 1 BUGS Output File 2 Flat ASCII File 3 Data Matrix Object 4 View Format Specifications 5 Options 6
18. es the N 0 1 if the chain has converged Z values which fall in the extreme tails of the N 0 1 suggest that the chain in the first window had not fully converged The two sided p value outputted by BOA gives the tail probability associated with the observed Z statistic It is common practice to conclude that there is evidence against convergence when the p value is less than 0 05 Otherwise it can be said that the results of this test do not provide any evidence against convergence This does not however prove that the chain has converged 5 2 3 Heidelberger and Welch Convergence Diagnostic The Heidelberger and Welch convergence diagnostic is appropriate for the analysis of individual chains The following diagnostic information was obtained for the line example HEIDLEBERGER AND WELCH STATIONARITY AND INTERVAL HALFWIDTH TESTS Halfwidth test accuracy 0 1 Chain linet Stationarity Test Keep Discard C von M Halfwidth Test Mean alpha passed 200 O 0 208584 passed 3 01256 26 beta passed 180 20 0 267382 passed 0 83503 sigma passed 180 20 0 391229 passed 1 05928 Halfwidth alpha 0 0605627 beta 0 0561035 Sigma 0 1026402 Heidelberger and Welch s 1983 stationarity test is based on Brownian bridge theory and uses the Cramer von Mises statistic If there is evidence of non stationarity the test is repeated after discarding the first 10 of the iterations This process continues until the resulting chain passes the test or more
19. he stationary distribution In this case descriptive statistics may be calculated for the combined latter 50 of iterations from all of the chains 5 2 2 Geweke Convergence Diagnostic The Geweke convergence diagnostic is appropriate for the analysis of individual chains when convergence of the mean of some function of the sampled parameters is of interest The following diagnostic information was obtained for the line example GEWEKE CONVERGENCE DIAGNOSTIC Fraction in first window 0 1 Fraction in last window 0 5 Chain linet 25 alpha beta sigma Z Score 0 251456 3 338157835 3 07513466 p value 0 801462 0 000843358 0 00210408 The chain is divided into two windows containing a set fraction of the first and the last iterations Options 3 and 4 in Section 5 3 allow the user to set the fraction of iterations included in the first and the last window respectively Geweke 1992 proposed a method to compare the mean of the sampled values in the first window to the mean of the sampled values in the last window There should be a sufficient number of iterations between the two windows to reasonably assume that the two means are approximately independent His method produces a Z statistic calculated as the difference between the two means divided by the asymptotic standard error of their difference where the variance is determined by spectral density estimation As the number of iterations approaches infinity the Z statistic approach
20. is limited to iteration 50 200 Users can verify that the subset was success fully constructed by selecting the option to display the Working dataset output not shown Thinning Thinning refers to the practice of including every k iteration from a chain Users can thin a chain by using the seq function when prompted to specify the iterations For example the following input will included every other iteration from the chain seq 1 200 length 100 A description of the seq function can be found at the end of the Appendix 4 2 Parameters Menu PARAMETERS MENU 1 Set Bounds 2 Delete 3 New 4 Back 5 Return to Main Menu Selection 4 2 1 Set Parameter Bounds This option allows the user to specify the lower and upper bounds support of selected MCMC parameters The parameter support factors into the computation of the Brooks Gelman amp Rubin convergence diagnostics SET PARAMETER BOUNDS 1 2 line1 line2 15 Specify chain index or vector of indices all di Parameters 1 2 3 alpha beta sigma Specify parameter index or vector of indices all 1 3 Specify lower and upper bounds as a vector Inf Inf 1 c 0 Inf In this example the variance parameter sigma has been restricted to only non negative values When no chain s is specified the default is to apply the change to all of the chains Likewise the default is to select all parameters and to set the bounds to Infinity Infinity 4
21. ke 3 Window 1 Fraction 0 1 4 Window 2 Fraction 0 5 Heidelberger amp Welch 5 Accuracy 6 Alpha Level Raftery amp Lewis 7 Accuracy 8 Alpha Level 9 Delta 10 Quantile Statistics 11 ACF Lags 12 Alpha Level 13 Batch Size 14 Quantiles 005 05 001 025 Oro O c 1 5 10 50 0 05 50 c 0 025 0 5 0 975 29 Chapter 6 Plot Menu Like the Analysis Menu the Plot Menu categorizes the available plots into a De scriptive and Convergence Diagnostic group Most of the options found under the Analysis Menu have a counterpart within the Plot Menu PLOT MENU 1 Descriptive gt gt 2 Convergence Diagnostics gt gt 3 Options 4 Return to Main Menu 6 1 Descriptive Plot Menu DESCRIPTIVE PLOT MENU Autocorrelations Density Running Mean Trace Back Return to Main Menu Selection 6 1 1 Autocorrelations Plot Plot the first several lag autocorrelations for each parameter in each chain 30 Autocorrelation Autocorrelation Autocorrelation 1 0 0 0 1 0 1 0 0 0 1 0 1 0 0 0 1 0 Bayesian Output Analysis Autocorrelation Plot alpha Tinet Autocorrelation Plot beta Tinet Autocorrelation Plot Sigma met 6 1 2 Density Plot Plot the kernel density estimate for each parameters in each chain Options 3 and 4 in Section 6 3 control the width and type of wi
22. lot Running Mean Plot alpha line Parameter Value Parameter Value T T T T T 0 50 100 150 200 0 50 100 150 200 Iteration Iteration Running Mean Plot sigma ine Parameter Value 2 4 6 8 10 0 50 100 150 200 Iteration the chains The remaining iterations are then partitioned into equal bins and added incrementally to construct the remaining segments Option 1 in Section 6 3 governs the number of bins used for the plot Scale factors are plotted against the maximum iteration number in the segments Cubic splines are used to interpolate through the point estimates from the segments 6 2 2 Gelman and Rubin Plot Plots the Gelman and Rubin corrected potential scale reduction factors see Section 5 2 1 for each parameter in successively larger segments of the chain The first segment contains the first 50 iterations in the chain The remaining iterations are then partitioned into equal bins and added incrementally to construct the remaining segments Options 5 and 6 in Section 6 3 control the error rate for the upper quantile and the number of bins respectively Option 7 determines the proportion of samples from the end of the chains to be included in the analysis The scale factor is plotted against the maximum iteration number for the segment Cubic splines are used to 33 Bayesian Output Analysis Trace Plot Trace Plot alpha line ie Parameter Value Parameter Valu
23. ndow used in the computations respectively 6 1 3 Running Mean Plot Generate a time series plot of the running mean for each parameter in each chain The running mean is computed as the mean of all sampled values up to and including that at a given iteration 6 1 4 Trace Plot Generate a time series plot of the sampled points for each parameter in each chain Autocorrelation Autocorrelation Autocorrelation 31 1 0 0 0 1 0 1 0 0 0 1 0 1 0 0 0 1 0 Autocorrelation Plot alpha mineg Autocorrelation Plot beta lineg Autocorrelation Plot Sigma meg Bayesian Output Analysis Density Plot Density Plot alpha line beta lied Density Density 0 0 Parameter Value Parameter Value Density Plot 15 al ae sigma ine Density 0 0 0 5 fi 0 2 4 6 8 10 12 14 Parameter Value 6 2 Convergence Diagnostics Plot Menu CONVERGENCE DIAGNOSTICS PLOT MENU 1 Brooks amp Gelman 2 Gelman amp Rubin 3 Geweke 4 Back 5 Return to Main Menu Selection 6 2 1 Brooks and Gelman Plot Plots the Brooks and Gelman multivariate potential scale reduction factor and the maximum of the potential scale reduction factors see Section 5 2 1 for successively larger segments of the chains The first segment contains the first 50 iterations in 32 Bayesian Output Analysis Running Mean P
24. ng chains may free up a substantial amount of computer memory The program prompts the user to select the chain s to discard DISCARD CHAINS 1 2 linei line2 Specify chain index or vector of indices none 1 The specified chain s will be immediately deleted from the Master dataset If the Working dataset has not been modified the chain s will be deleted from there as well If modifications were made to the Working dataset the user can copy the new Master dataset to the Working dataset via the Reset option If no chain is entered at the prompt no action is taken 13 4 1 3 Subset Chains Subsets of the MCMC sequences can be selected for analysis via the Subset option SUBSET WORKING DATASET Specify the indices of the items to be included in the subset Alternatively items may be excluded by supplying negative indices Selections should be in the form of a number or numeric vector Chains 1 2 linei line2 Specify chain indices all 1 c 1 2 Parameters 1 2 3 alpha beta sigma Specify parameter indices all te 2 Iterations IA A A A ttt Min Max Sample linel 1 200 200 line2 1 200 200 Specify iterations all 1 50 200 In this example both chains were first included in the subset Since the default is to include all chains this line could have been left blank Next the beta parame ter is excluded by supplying a negative sign in front of the selection Finally the 14 subset
25. nostics implementation strategies for Markov chain Monte Carlo Statistical Science 7 493 497 11 Raftery A L and Lewis S 1992b How many iterations in the Gibbs sampler In Bayesian Statistics 4 eds J M Bernardo J O Berger A P Dawid and A F M Smith Oxford Oxford University Press 44
26. p Why tp LA E Analysis Menu 5 1 Descriptive Statistics Menu lt eck a ci eo a ek go a 5 1 1 Autocorrelations ea ee eee eS RE RRS 5 1 2 Correlation Matrix De gard Gee a4 mb a He a ew A eA 5 1 3 Highest Probability Density Intervals 5 1 4 Summary Statistic t5 2 ue od BA eG ben Be Ske 5 2 Convergence Diagnostics Menu air ee Re 5 2 1 Brooks Gelman amp Rubin Convergence Diagnostic 5 2 2 Geweke Convergence Diagnostic 5 2 3 Heidelberger and Welch Convergence Diagnostic 5 2 4 Raftery and Lewis Convergence Diagnostic 93 Analysis o z e ga kent E ale ood eh ge aA Plot Menu 6 1 Descriptive Plot Menu a o 2 ee oe ee oe ee og ee 6 1 1 Amtocorrelations Plot Ya ei ad oie ed wt Soe el Ge ws A Density Ploter ae clo de ods OG AE te he BEARS chy ho AE ety S 6 1 3 Running Mean Plot 2002 3 a aa a a A eal ek ea GAA Trace Plotin 305 A ees Pit we Owe wee Ses 6 2 Convergence Diagnostics Plot Menu 0 6 2 1 Brooks and Gelman Plot 6 2 2 Gelman and Rubin Plot 6 2 3 Geweke Plot 6 3 e Plot Options s Pers a AA A o O te oe A Options Menu Window Menu 8 1 Previous Graphics Window lt 3 gees A ii 8 2 Next Graphics Window 0 ER AE A ta a 8 3 Save to Postscript A ine gs Sy es soa Gs Sa shite e Gy dt ites de wg 8 4 Close Graphics Window cios ae ee Be eS eS 8 5 Close All Graphics Window salada p
27. rs and parameter names may be specified in the dimnames 3 1 5 Data Options The Options menu item lists the values for the user settings used to import data Data Parameters 10 1 Working Directory home bjsmith 2 ASCII File Ext txt Select parameter to change or press lt ENTER gt to continue des Most users will want to specify the Working Directory at the start of their BOA session This directory should be set to the path in which the MCMC output files are stored Forward slashes should be used when entering the directory path 3 2 Load Session The Load Session menu item allows users to load previously saved work Enter name of object to load none 1 line 3 3 Save Session All imported data and user settings may be saved at any point in the analysis Users will be prompted to Enter name of object to which to save the session data none 1 line The session data will be saved to the specified S object 3 4 Exit BOA Select this item to exit from the BOA program Users will be prompted to verify their intention to exit in order to avoid an unintended termination of the program Do you really want to EXIT y n n 1 Users wishing to save their work should go back and do so before exiting BOA will not automatically save the user s work 11 Chapter 4 Data Management Menu BOA offers a wide range of options for managing the imported data Two copies of the data are maintained b
28. t Analysis Gelman amp Rubin Shrink Factors Gelman amp Rubin Shrink Factors 1 08 Shrink Factor 1 04 fi 1 00 fi J RA 1 15 Shrink Factor 1 05 fi 0 95 l 50 100 Last Iteration in Segment T T T T T T 150 200 50 100 150 200 Last Iteration in Segment Gelman amp Rubin Shrink Factors Shrink Factor EA 50 100 150 200 Last Iteration in Segment 6 Number of Bins 7 Window Fraction Geweke 8 Alpha Level 9 Number of Bins 10 Window 1 Fraction 11 Window 2 Fraction Graphics 12 13 14 15 16 Device Name Plot Layout Separate Plot Var Keep Previous Plots Separate Plot Chain 20 0 5 0 05 10 0 1 0 5 X11 FALSE c 3 2 FALSE TRUE 36 Bayesian Output Analysis Geweke Convergence Diagnostic Geweke Convergence Diagnostic NARA AAA A alpha ane SES Ss Se eS Se alpha ine o 4 o a o 5 2 9 8 o o 7 3 T 7 i gt nN 4 Ao E o o EE Ns IE OE be oS ee MS a So e 4 l T T T T T T T T 0 50 100 150 0 50 100 150 First Iteration in Segment First Iteration in Segment Geweke Convergence Diagnostic Geweke Convergence Diagnostic al E AAA a E S o beta tinet a a 0 ABS PAE AAA beta tineg o ai o ene o o 3 v a BS o 1 o N Mu a SS eee Se Se o N do o P o AAA ANO EA o T T T T T a T o 50 100 150 o 50 100
29. tch autocorrelation Finally the minimum and maximum iteration numbers and the total sample size round out the table 22 SUMMARY STATISTICS Batch size for calculating Batch SE and Lag 1 ACF 50 Chain linel Mean SD Naive SE MC Error Batch SE Batch ACF alpha 3 012561 0 597966 0 0422826 0 0308999 0 0242799 0 1413237 beta 0 790924 0 444957 0 0314632 0 0280210 0 0693133 0 0971101 sigma 1 182538 1 212897 0 0857648 0 0567591 0 1184687 0 3510225 tau 1 726886 1 507967 0 1066293 0 1274516 0 1306512 0 4207326 0 025 0 5 0 975 MinIter MaxIter Sample alpha 1 748050 2 98737 4 07063 1 200 200 beta 0 107853 0 81173 1 63205 1 200 200 Sigma 0 458036 0 86806 3 28955 1 200 200 tau 0 092636 1 32741 4 76732 1 200 200 Options 13 and 14 in Section 5 3 allow the user to change the batch size and the quantiles respectively See the Appendix for instructions on setting the number of significant digits and display width 5 2 Convergence Diagnostics Menu The Convergence Diagnostics Menu offers the user the following diagnostic methods CONVERGENCE DIAGNOSTICS MENU 1 Brooks Gelman amp Rubin 2 Geweke 3 Heidelberger amp Welch 4 Raftery amp Lewis 5 Back 6 Return to Main Menu Selection These are the most commonly used methods used to asses the convergence of MCMC output A brief explanation of each approach is given in the following sections Users 23 are referred to the work of Brooks and Roberts 1998 and Cowles and Carlin 19
30. tery and Lewis 1992b tests for convergence to the stationary distribution and estimates the run lengths needed to accurately estimate quantiles of functions of the parameters The user may specify the quantile of interest the desired degree of accuracy in estimating this quantile and the probability of attaining the indicated degree of accuracy Options 7 9 and 10 in Section 5 3 allow the user to modify these quantities BOA computes the lower bound the number of iterations needed to estimate the specified quantile to the desired accuracy using independent samples If fewer iterations than this bound have been loaded into BOA the following warning is displayed fom Warning ORK Available chain length is 200 Re run simulation for at least 3746 iterations OR reduce the quantile accuracy or probability to be estimated If sufficient MCMC iterations are available BOA lists the lower bound the total number of iterations needed for each parameter the number of initial iterations to discard as the burn in set and the thinning interval to be used The dependence factor measures the multiplicative increase in the number of iterations needed to reach convergence due to within chain correlation Dependence factors greater than 5 0 often indicate convergence failure and a need to reparameterize the model Raftery and Lewis 1992a 5 3 Analysis Options Analysis Parameters 1 Alpha Level 0 05 2 Window Fraction 0 5 28 Gewe
31. than 50 of the iterations have been discarded BOA reports the number of iterations that were kept the number of iterations that were discarded and the Cramer von Mises statistic Failure of the chain to pass this test indicates that a longer run of the MCMC sampler is needed in order to achieve convergence A halfwidth test is performed on the portion of the chain that passes the sta tionarity test for each variable Spectral density estimation is used to compute the asymptotic standard error of the mean If the halfwidth of the confidence interval for the mean is less than a specified fraction accuracy of this mean the halfwidth test indicates that the mean is estimated with acceptable accuracy The confidence level and accuracy can be modified through Options 5 and 6 respectively in Section 5 3 Failure of the halfwidth test implies that a longer run of the MCMC sampler is needed to increase the accuracy of the estimated posterior mean 5 2 4 Raftery and Lewis Convergence Diagnostic The Raftery and Lewis convergence diagnostic is appropriate for the analysis of indi vidual chains The following diagnostic information was obtained for the line example RAFTERY AND LEWIS CONVERGENCE DIAGNOSTIC Quantile 0 025 Accuracy 0 02 Probability 0 9 Chain linet 27 Thin Burn in Total Lower Bound Dependence Factor alpha 1 2 160 165 0 969697 beta 1 5 188 165 1 139394 sigma 1 2 160 165 0 969697 The diagnostic proposed by Raf
32. the options in Sections 3 1 5 5 3 and 6 3 can be accessed GLOBAL OPTIONS MENU 1 Analysis 2 Data 3 Plot 4 All 5 Return to Main Menu Selection 38 Chapter 8 Window Menu The Window Menu allows the user to switch between and save the active graphics windows WINDOW 1 MENU 1 Previous 2 Next 3 Save to Postscript File 4 Close 5 Close All 6 Return to Main Menu Selection The number of the active graphics window is displayed in the title of his menu In this example graphics window 1 is the active window 8 1 Previous Graphics Window Make the previous graphics window in the list of open windows the active graphics window 8 2 Next Graphics Window Make the next graphics window in the list of open windows the active graphics win dow 39 8 3 Save to Postscript File Saves the active graphics window to a postscript file The user is prompted to enter the name of the postscript file in which to save the contents of the graphics window Enter name of file to which to save the plot none 1 Only the name of the file should be given The file will be automatically saved in the Working Directory see Section 3 1 5 Microsoft Windows users may save the graphics window in other formats directly from the S PLUS or R program menus 8 4 Close Graphics Window Close the active graphics window 8 5 Close All Graphics Window Closes all open graphics windows 40 Chapter 9 S PLUS and R
33. y the program the Master dataset and the Working dataset The Master dataset is a static copy of the data as it was first imported This copy remains essentially unchanged throughout the BOA session The Working dataset is a dynamic copy that can be modified by the user All analyses are performed on the Working dataset The Data Management menu offers the following options DATA MANAGEMENT MENU 1 Chains gt gt 2 Parameters gt gt 3 Display Working Dataset 4 Display Master Dataset 5 Reset 6 Return to Main Menu Selection 4 1 Chains Menu CHAINS MENU 1 Combine All 2 Delete 3 Subset 4 Back 12 5 Return to Main Menu Selection 4 1 1 Combine All Chains Selecting this options will combine together all of the chains in the Working dataset Sequencing is preserved by concatenating together the different chains and then or dering the result by the iteration numbers in the original chains Note that this may result in a chain with multiple samples at a given iteration The resulting chain contains only those parameters common to all chains CAUTION Although possible to do so convergence diagnostics and autocorrela tions should not be computed for combined chains A combined chain is essentially a single chain with potentially multiple samples per iteration These analyses expect that a single chain has no more than one sample per iteration 4 1 2 Delete Chain Chains may be discarded when they are no longer needed Discardi
Download Pdf Manuals
Related Search