Home

Geostatistical analysis tools Diva User's Guide

Contents

1. 1 5 Table 4 8 Peninsula mesh parameters 5 SABE Y RIRE AS P ROSES OR K Figure 4 5 Peninsula case mesh SPA OONA IDO APODO ALZ DANS K UR VDO 60 40 VA EE SAR AV ZA Z 22 A 7 S e eU x N PS C VN LAVADERO I DS AAA OA T CRD WI AZ CV NV 2 o o 10 YS D e a N epnine1 Longitude E 4 2 3 Analysis The error field is the peninsula The region of influence of the data is separated by symmetric 4 o LS S 213 oo a DV Table 4 4 Peninsula analysis parameters N epnine1 40 Longitude E Longitude E Figure 4 6 Peninsula analysis and error fields 4 3 Interior sea Here the two domains are completely separated by a boundary of land 4 3 1 Data One data is located inside the interior domain and another one is in the West side of the exterior domain Figure 4 7 Lake case data Latitude N 75 4 E 2 E 0 Longitude E 4 3 2 Mesh The meshes are generated in the two regions Lm Surf coef Smooth num 05 15 3 Table 4 5 Lake mesh parameters DADAS ALSO SESS ESL ESESERSDA LR SES LA ATS SODA AROSA ERY VV MX A DIVINA D K AN PVA K K X e Figure 4 8 Lake case mesh ES o 3 L me IDR MDW KR 5 4 3 2 E 0 1 2 3 4 5 Longi
2. 6 The last Nm lines are the coordinates of the points of the last contour 2 4 0 0 100 0 100 100 0 100 4 40 40 40 60 60 60 60 40 Figure 3 2 Example of a contour file and its graphical representation 3 1 3 Finite Element mesh As explained in the first chapter the solution of 1 3 1 4 is computed with the help of a finite element method The mesh is built using the coastline provided by the user The main parameter ruling the shape of the mesh is the size of the element Le This length scale should be small compared to the scales of interest As in reality each element is the combination of three smaller third order FEM the actual resolution is rather L 3 When deciding the characteristic length of the elements you should be aware of the following facts e Contour segments cannot be much smaller than finite element length Le gt Le If your contour is too fine the tool divacck can be used in order to reduce the contour resolution e The correlation length should be larger than Le otherwise the grid is too coarse compared to the signal to resolve e When requesting a too fine mesh you might need to recompile diva to increase memory allocation programs written in Fortran 77 3 1 4 Parameters In each step of a Diva analysis many parameters can be changed according to the par ticular case you are treating If you work with the CL version these parameters have to be entered in the file param
3. In the Analysis window select the Data file you created with the extraction adria tic dat and the mesh you generated adriatic mesh Choose an output file name adriatic anl for example Diva 4 1 View Options Data Ctri d Mesh Figure 5 5 Mesh generation The analysis parameters can be chosen according to the fig 3 5 Save you configuration adriatic_analysis conf when all the parameters are set The resulting analysed and error fields are showed on figs 5 6 and Observe that the error field is lower where the data density is higher To get the graphical output you can use either the Matlabscripts diva_analysis m and diva_error m or the NetCDF visualization softwares as described in section Analysis 46 45 44 A w Latitude N A 40 39 12 16 Longitude E Figure 5 6 Adriatic sea analysed field Latitude N A zx Error field 45 44 A I A N 40 39 12 15 16 17 Longitude E Figure 5 7 Adriatic sea error field Some properties of Generalized Cross Validation Application to Diva 6 1 Introduction The purpose of this chapter is to describe the validations tools included in the Diva software We start with theorical notions about the Cross Validation followed by a section about the quality control of data The chapter ends with the application to Diva 6 1 1 Generalities Let us consider the vector d conta
4. e the Generic data output file name the generic name of the extracted data files Commands e Load and Save configuration once you have entered all the file names and the parameter value it is useful to save your configuration in order to gain time e OK performs the data extraction e Display Data File allows the user to check the output file 3 3 Graphical Users Interface Chapter 3 DIVA USE Figure 3 3 Data extraction window 33 3 3 3 The mesh menu Input e two or three dimension analysis e Contour description file e Bathymetry file name 3D case only file that contains the bathymetry of the region where you want to work regular grid in the GHER format e Depth file name 3D case only contains a list of depths and the corresponding data files e Uniform or Non Uniform mesh in the Non Uniform mesh case a different density mesh can be used in some parts of the domain e Reference length fixes the length of the side of the triangular elements touching an outline Surface coefficient controls the shape of the triangular element the value 1 5 pro vides triangles close to equilateral triangles Smoothing number leads to a more regular shape of the elements the value 3 is suggested Output The Mesh file name specifies the generic name of the file that will be generated through the mesh process The software creates three files 1 name mesh contains the topology
5. K PDA YY LEX A L VA VY a J M BECKERS JM BeckersCulg ac be D SIRJACOBS D Sirjacobs ulg ac be C TROUPIN ctroupin ulg ac be GeoHydrodynamics and Environment Research MARE O El ea a 20 S e e Koy u ea a lay E E Go Y a 20 R rn 2 a pen D ES E D GHER UNIVERSIT de Li ge Environment Research GeoHydrodynamics and Yi T Honorary Research Associate National Fund for Scientific Research BELGIUM 2 PhD Student Fonds pour la formation la Recherche dans l Industrie et dans l Agriculture F R LA BELGIUM 1 Diva theory 1 1 The gridding problem 1 1 1 1 1 2 Data analysis and gridding methodologies 1 1 3 Noise on data 1 2 VIM and its implementation Diva 1 2 1 1 2 2 Error field 1 2 4 Domain gridding 1 8 Comparison between the methods 1 4 Additional tools 1 4 1 Generalized Cross Validation GCV 1 4 2 1 4 4 Possible developments Interpolation vs approximation Formulation Advection constraint 1 4 5 DINEOF 2 Installation procedure 2 1 Tcl Tk 2 1 1 Windows 2 1 2 Linux 2 2 Diva software installation 2 2 1 Diva 4 1 content Contents 2 2 0 Command Line versionl 19 ee de a eee 19 2 3 Compiling POI 21 Geb e de uie e Se PD Si o n AR Sene Wy he Db ese icr d 22 2 4 Creation of shortcuts 22 IM p
6. EROS RCA ORO SS RAR AI TIRSO CRIS AO ORO 5 RS IO ROS ESERDE NO E OOO RS RES KOOKS FRE S E CR G N C L TSS K GE eA OOO RRA GDA a V C b El T S 504 AAPY OSI miS Y An NA G y L G VAI AN V Y G K R INN VY 3 VAN A fo X N AV PO P FA KN SN UN aN X Ad ah PRO DA A EI NY ASE iN SADA NS 2 D A CS SOS PLA O NA T E SR E x DOO PP Pa VSD COOKER Spe ES RAE AE G pe ERA RS S E ED igure ingle point case ARIS RR SAS NA SOS EO EEE Pa YATATA t e CAS DIEPPE Lc Ce Ed Do K EO TAPAAN AS RAE SAIS ie AS oO e SIS BEA TA CR a X X NS 5 A E C ifi Y VN CK AS jm iin A GS K G NA ADRIAN LA CH KK NA SA DO NI A FUA ERP A VPN LA 5a AR OOo S PERS AVAN ATAT AT aN S LE OD KSLA DER IAS A OOO ISS AS IRRADIA AC RL SCA SECOS EH d RAVAN Pc Ee aS aes a ra POS ESO PD LOS SAY E OCC EE EAA VAN A SS ra CAE S AAA OOOO SS GE POI S A EE RL R CRIRE PORT PRESSE KA ARR SES ROSE AOS R A A AA AAA AA Ya EOS SES OK ERO RSR ASE OC 2 G ES S S 5 E E Y el AN MARY VAN Y POO EV SL ROS LITIO ELOTE E PAY era TT ETAT RASPES OOS z Bos 1 RES COO PO BOSSES E SCC PE SSSR RES E AVE AVAVA E A Va VAT va ras VAN PA VA VA VAT AUS A VAT IS AXE Va VAV a RAS VAN aV AVAT AN aN aN ATA N VAV ZZ A S VaN aN 4 3 2 1 0 1 2 3 4 Si DA N NORN AY PA M 79 x g 4
7. Karafistan et al 200211 Karafistan A J M Martin M Rixen and J M Beckers Space and time distributions of phosphates in the Mediterranean Sea Deep Sea Research I 49 67 82 2002 Rixen et al 2000 Rixen M J M Beckers J M Brankart and P Brasseur A nu merically efficient data analysis method with error map generation Ocean Modelling 2 45 60 Rixen et al 2001 Rixen M J M Beckers and J T Allen Diagnosis of vertical veloc ities with the QG Omega equation a relocation method to obtain pseudo synoptic data sets Deep Sea Research I 48 1347 1373 2001 Rixen et al 2005 Rixen M J M Beckers S Levitus J Antonov T Boyer C Mail lard M Fichaut E Balopoulos S Iona H Dooley M J Garcia B Manca A Giorgetti G Manzella N Mikhailov N Pinardi M Zavatarelli and the Medar Consortium The Western Mediterranean Deep Water a proxy for global climate change Geophysical Research Letters 32 2005 DINEOF Alvera Azc rate et al 2005 Alvera Azc rate A A Barth M Rixen and J M Beck ers Reconstruction of incomplete oceanographic data sets using Empirical Orthogo nal Functions Application to the Adriatic Sea Ocean Modelling 9 325 346 2005 80 Alvera Azc rate et al 2006 Alvera Azc rate A A Barth J M Beckers and R Weisberg Multivariate reconstruction of missing data in sea surface temperature chlorophyll and wind satellite fields Journal of Geoph
8. domain where we Figure 4 1 Island case data Figure 4 2 Island case mesh SARL RRR TATAN RN IDAS SOCIAL R VN VOS K ASIN Se CI gt KISEKI ROCK DORA ISE OR LATA AL 1 5 Lm Surf coef Smooth num Table 4 1 Island mesh parameters Longitude E YN o aaas UAT IV E RRK PK N AROS NZNIS PRI AA RAH KIN 100 90r 80r 30 20r 1 o o 2 D N epnine1 N epnine1 You can observe that the mesh covers only the region if interest have water 100 90 80 70 50 50 40 30 20 10 4 1 2 Mesh Longitude E Fig 4 3 shows that the error is lower where the data coverage is higher but the analysis itself is not biased 4 1 3 Analysis Latitude N 0 amp c Ref Field Az Ay 15 100 Mean value 1 1 Table 4 2 Island analysis parameters Latitude N 70 80 90 100 10 20 30 40 50 60 60 Longitude E Longitude E Figure 4 3 Island analysis and error fields 4 2 Peninsula In this case we have a domain that separates the sea in two regions 4 2 1 Data There is one data on each side of the peninsula to the left of the peninsula the field value is 10 and to the right the value is 5 100 90r 80r 70r Latitude N 30r 20r 60 L 50r 40r Figure 4 4 Peninsula case data 40 50 60 70 80 90 Longitude E 4 2 2 Mesh Lm Surf coef Smooth num
9. 1 type configure prefix MYPREFIX where MYPREFIX stands for the installation prefix Example usr local plplot make make install 4 in the directory MYPREFIX share plplot VERSION examples type make plplot test sh to see some examples The complete installation procedure is described in the file INSTALL Diva Use 3 1 General We describe here what are the ingredients needed by Diva to perform an analysis and the way the input files have to be created 3 1 1 Data The data file contains three or four columns X Y value relative weight If the number of column is three the fourth column is assumed to take the value 1 e e 25 25 5 25 75 10 m E 75 75 5 M 75 25 10 ha e e 20 6 Figure 3 1 Example of a data file and its graphical representation 24 3 1 2 Contour The contour files are defined this way 1 The first line indicates the number of contour in the region of interest say M 2 The second line tells the number of points in the first contour say N1 3 The next N lines are the coordinates of the points of the first contour The con vention for the contour is that the land is on the right when you follow the points successively The contour is automatically closed meaning that the last point of a given contour has not to be the same as the first one 4 The following line is the number of points of the second contour say Na C
10. Normally interpolation and extrapolation works on anomalies with respect to a back ground field Eq 1 1 Diva allows you to work with different background fields depend ing on the analysis you want to perform e no treatment is applied the data you treat are already anomalies e the mean of data is subtracted from the data values e the linear regression plane is subtracted e additional subtracted semi normed field ao 0 and large L obtained by two chained Diva executions In particular when no treatment is applied with ay gt 0 the minimization forces the analysis towards zero when there are no data points in a distance comparable to L This is coherent with the idea of an anomaly only 1 At now only available with the Graphical Interface version 1 2 2 Error field For any analysis method it is important to have a quantification of associated errors For example you may want to use the relative error field to mask regions where you consider errors are too high We may expect the error field to be affected by 1 the data coverage the error field is expected to be higher where the data coverage is lower 2 the noise on the data In Diva error fields are calculated by analogy with Optimal Interpolation since analysis in O I is equivalent to analysis with VIM and since error field of O I equals analysis of covariance fields error field of VIM equals analysis by VIM of covariance fields In pra
11. effect to increase correlation along streamlines and decreases across streamlines fronts Be aware that error fields are in this case not equivalent to error fields of OI 1 43 DINEOF DINEOF stands for Data INterpolating Empirical Orthogonal Function It is designed to exploit repeated observations with missing data e g a series of collocated satellite images with clouds For more details about the DINEOF tool the reader is invited to consult Alvera Azc rate et al 2005 Alvera Azc rate et al 2006 http ocgmod2 marine usf edu DINEOF welcome html 1 4 4 Possible developments Some aspects of Diva are still under development Here are the improvements that will be developed in the future e 3D extensions from surface to deep layers through EOF calculated from data set DINEOF approach horizontally VIM Static stability constraints and other constraints Multivariate approaches easy in OI but more difficult in VIM Parameter calibration by cross validation in multivariate version Data query modules possibly through ODV Strategy for combined analysis of satellite and in situ data Installation procedure IVA is a software designed to run under Windows as well as under Unix The instal lation procedures are described for both operating systems 2 1 Tcl Tk The graphical interface was created with the help of the language Tcl Tk Tcl stands for Tool Command Language and is an interpreted language which
12. s ss ox ko wor m 3 4 E obo Eod oe Rod Ree kog Hia 22 ho EDR Nas ER ee ee sd 3 Ede 3 23 2B PIPIOt BDISE exo rra oe AGE wo echo En oe be Rh 23 24 31 Genera so gia ee RC EORR E E ARA d 24 ALL Diales seisa eee eS REDE EERE E x SRS SHEAR d 24 3 1 2 Contour ox oo ee 4 20008 RR RUE SOR RH RR oe 25 Lena hee te Behe Pe ee due d 25 3 1 4 PSM LL HS este AR Ke Ee oL Ede d ee eS 26 3 2 Command Line version 4 444 ee 28 3 2 1 Working directories 28 LT TTT 28 du Q tp t files i seek bx Ree kde Ra m ed D E 29 UG Ree Te E E S 29 3 3 Graphical Users Interface 4 64544 a bbe ede ee eS 31 3 3 1 Working directory 31 ogee ec E E we Soe ee ae ce ee 32 bd qel ded REM A oda EE dox e 34 EB lig AA a a E 36 boa Oe bee Sek od bo ao Oe eee dos d 37 ee ee eee ee 38 Dd ee e ee Sic ee ee d Yee ee d 38 jo ara ee eS ee BESS EO pO we 40 3 6 Display of outputs eee eo a eee AAA RO dere E 41 AA S 3 oc dm ER RT uRSE d sm goku Edessa 41 3 6 2 NcBrowse and Neview 42 4 1 Island 4 4 Noise 4 5 Single point Al AMAS e deze de deo Ge eR oues e RS o 4 2 Peninsula dJi DE Aa eee Istae a T2 ABUSE wee eb ee qe eU ur od OS eS ed RE de d de d dl DA se so ok Bi NM diode AA a ee A e Vd dos d REE AH ES RE eee ir A33 Analyse xen ek eR ek de a Bk ee aec fem he AAS Analysis on mon x Rom omo Rh A A5 l Data o s usage EGRE G3 4 5
13. 2 Meshl 45 3 Analysis 4 lt 2 2 44 Ba 4 5 A complete example with the GUI 9 1 Data Extraction 5 2 Mesh generation 9 9 Analysis i 22d m4 be x OR x BAS 6 Generalized Cross Validation in Diva 6 1 Introduction 6 1 1 Generalities 6 1 2 Ordinary Cross Validation OCV 6 1 3 Generalized Cross Validation GCV DA e E a E e a a ATI MESH DR X ITCRUM 44 44 44 45 45 46 46 AT AT 48 48 48 49 90 90 90 ol 51 51 52 52 53 93 54 99 6 2 Quality control OC of dat l 4 40406 dis RR he rrr O eere 6 3 Application and implementation in Diva llle 6 3 1 Diva formulation o amp 4 xo x 8 HAN hoe ea RAA ans 6 3 2 Implementation of result validation in Diva 0a aaa I Appendix A a a Bees eee TT A ara a eds da a e A eee sg ae aes ola a ees ema a o da isa e B Gallery Bibliography DINEQEJ EM a ica a ca o A ae eo ad RR Useful links Conditions of use Acknowledgements 66 67 67 67 68 68 69 70 70 13 75 80 80 80 81 82 83 84 Part I Diva theory dis chapter is dedicated to the description of the theorical background necessary to understand how the Diva software works As only the main ideas are summarized the reader is invited to refer to the articles mentioned in the bibliography 1 1 The gridding problem In oceanography a typical conce
14. 5 3 Analysis c Ref Field Ax Ay 1 1000 No 0 2 0 2 Table 4 10 Single point analysis parameters Analysis Error field 4 3 1 0 1 2 3 4 5 Figure 4 15 Single point analysis and error fields Latitude N 3 2 1 0 1 2 3 4 Longitude E A complete example with the GUI This chapter describes the procedure to obtain an analysed field step by step using the GUI Note that the first step data extraction is not necessary if you already have a data set at your disposal 5 1 Data Extraction Figure 5 1 Data extraction In the Data Extraction window choose the following files e Data Base Description File liste des e Generic data output file name adriatic data Then specify the time Years and Period and space domains where you want to perform the extraction If you use a contour to delimitate the region you want to work in load adriatic cont as Contour description file 93 Finally choose the Data type and nature as well as the quality criterion Once you have filled all the fields of the window press the Save configuration button This will allow you to load automatically the parameters you saved this way with the Load configuration command The extraction is performed by clicking on the OK button The data file is displayed with the Display data file command The result obtained with the same parameters as the fig 3 3 is showed on the fig To visualize the result ope
15. Because we will look for outliers instead of working with mean values and variances more robust estimators are the median and MAD median absolute deviation A bias in the analysis is likely to exist if S median s is not much smaller than one the typical values of s should be of the order of one if the misfit estimate is correct on average Instead of calculating the standard deviation of s we calculate the median absolute devi ation 0 defined as MAD s 1 4826 median s S 6 28 where the factor 1 4826 is introduced such that for a normal distribution the MAD is identical to the standard deviation Hence we can detect points qualified as outliers if they satisfy the inequality ls S gt 30 6 29 This normalized version of quality control to some extent corrects for inaccurate estimation of the expected misfits A 6 3 Application and implementation in Diva The latest version of Diva allows the user to perform generalized cross validation as well as quality control Two new modules were added to achieve these tasks dataqc and gcvfac 6 3 1 Diva formulation The Diva theory is extensively described in Chapter 1 We limit ourselves here to recall the base formulation When working with non dimensional space coordinates Diva consists in minimizing Je lld pai 6 30 de VV VVo a LUX Vo agL o dD D where y x yi is the field to analyse d is the field value at point x y L
16. and go into the directory msys home username tc18 x xx win Type configure to create the makefile Type make install to perform the installation Repeat the operations 5 and 6 in the directory msys home username tk8 x xx win Make sure the files tclshxx exe wishxx exe tclxx d11 and tkxx d11 are located in msys local bin If it is not the case copy them from the folder tc18 x xx To check if your installation is completed type wishxx in the msys shell Example type wish84 if you are working with the version 8 4 You should get a command interpreter Fig 2 1 Cygwin users assuming that Cygwin is already installed on your computer you can follow the previous steps by replacing msys home username by cygwin home username Where username is your personal directory 2This step and the following one may last several minutes so please be patient 2 1 2 Linux Under Linux the msys or cygwin installation is not required Before installing a new version of Tcl Tk you are advised to test if your computer already dispose of a version compatible with the graphical interface To this end follow the instructions given in section 2 2 3 If you get the fig then you do not need to install a new version of Tcl Tk If the operations from section 2 2 3 following the procedure hereafter to install Tcl Tk Installation of Tcl Tk under Linux 1 Download the most recent versions of Tcl Tk from http www tcl tk software tcl
17. can work on Mac PC and UNIX stations Tk stands for Tool Kit a set of tools allowing the creation of graphical interfaces For more details the following websites can be consulted e Tcl Developer Site http www tcl tk e Tcl Tk Software download http www tcl tk software tcltk choose html e Tcl Tk 8 4 Manual commands and libraries description http www tcl tk man tc18 4 2 1 1 Windows There are two possibilities to generate the graphical interfaces under Windows using the language Tcl Tk Cygwin or Msys Cygwin can easily installed or updated from http www cygwin com by running setup exe found there Cygwin is a complete Linux like environment including the possibility to use Tcl Tk Msys constitutes the minimal environment to be able to program in Tcl Tk under Win dows The msys installation is described in the following Msys installation 1 2 3 Download the compressed file from http prdownloads sourceforge net tcl msys mingw8 zip Unzip the msys_mingw8 zip Example in C Open the msys shell by double clicking on the msys icon MS DOS command file Tcl Tk installation Download the most recent versions of Tcl and Tk from http www tcl tk software tcltk choose html Unzip tcl8xxx src zip and tk8xxx src zip within the directory msys home username Copy the folders tc18 x xx and tk8 x xx contained in tc18xxx src and tk8xxx src into msys home username Open a msys shell
18. ct interpolation We have to differentiate the subjective analysis for which the way the approximation is performed are decided by hand and the objective analysis which is based on predefined mathematical operations The data assimilation uses in addition physical biochemical dynamic governing equations Choice of a method among the three previous methods our choice will be the objective analysis Indeed the subjective analysis is not sufficiently objective and the data assimilation depends on the region and the model Instead of working with the data themselves we will work with anomalies dq with respect to a background field pp plr pelr e r 1 1 The background field y is defined a priori and the anomalies are calculated with respect to this reference field examples climatological average linear regression theorical solution We assume that the anomalies can be expressed as a linear combination of the data t e p r wir 20 dj 1 2 where d is the data anomay at the point r rj and w is the relative weight of the data j Now the problem consists in determining the weighting functions Once the background field and the weights are known the field can be computed at any position r hence gridding is possible 1 1 3 Noise on data When measuring a field there is always uncertainty on the value obtained whatever the instrument and the field Noise not only takes into account instrumental error
19. ctice the data input of the analysis tool is a vector containing the covariance of data points with the point in which the error estimate is to be calculated 1 2 3 Kernel and correlation function Kernel functions can be examined by analysing a single point with high signal to noise ratio and no background field The exact function in an infinite domain is given in blue Differences between the two curves are only due to boundaries 0 9 Field value eo e o o o a a o A to e w N 0 1 L L L L L 2 2 5 3 3 5 4 4 5 5 Distance r L 0 L L 1 L Figure 1 3 Kernel function blue and analysis of a single point red 1 2 VIM and its implementation Chapter 1 DIVA THEORY Diva Analysis Figure 1 4 Result of a single point analysis with a unit anomaly and high signal to noise ration in a large domain The Kernel function can be used to calibrate Diva parameters ao a pu so as to fit observed covariance functions 1 2 4 Domain gridding The minimization of is actually performed by a Finite Element Method hence the need for generating a finite element grid Because the field is only defined in the water the minimization also works only within the contours defining the coastline Thus the grid generation has to be consistent with the coasts existing in the considered region The corresponding mathematical problem is referred to the Constrained Triangu lation More details about the finite eleme
20. ction is dedicated to the different methods available to create the contour coast cont and topography coast cont files necessary to perform an analysis with Diva 3 5 1 Coastline generation As seen in the first chapter a relevant asset of Divais the fact that it takes into account the real coastline and topography of the region of interest We explain herinafter the ways 38 you can produce a correct coastline file The description of the coastline structure is provided in section 3 1 2 By hand The first possibility is to build your file by hand having at your disposal the location of different points of the coast longitude latitude you create a file containing M contours with the th contour having being made up of N points i 1 2 N Be aware that some cases fig 3 9 are to avoid problems arise when crossing occur in the contour Also note that as the contours are automatically closed by Diva the segment joining the first and the last point may generate errors Figure 3 9 Example of improper contours Left crossing of two segments of a same contour up crossing of two different contours right first and last points of the contour generate a segment that crosses the other parts down two contours have a common segment Remember that you can use the tool divacck for checking and thinning of contours From topography divacont uses topo grd and TopoInfo dat see Section to see how to g
21. data file fort 44 and generalized cross validator value available in output gcvval dat produces the file containing the expected misfit value at data locations fort 76 to be used by lookfor outliers a for outlier detection Used in divaqcter calcestbis f from relative weight on data points data file fort 44 and the trace of the analysis matrix available in output gcvval dat produces the file containing the expected misfit value at data locations fort 76 to be used by lookfor outliers a for outlier detection Used in divaqcbis dbdb2diva f makes easier the extraction from a website findmin f from a list of values SNR GCV data variance found in fort 11 tries to find by local parabolic interpolation the minimum of GCV and the place SNR where it is found Points do not need to be ordered The output fort 12 contains the expected value of SNR in which GCV is minimal as well as the VARBAK value Used in divagcv fitlsn f from the data file fort 10 and information on coordinate change and reference field fort 11 tries to fit the Bessel covariance function to the data covariance Output fort 66 contains the correlation length and an rought estimate of the SNR The data covariance over all distances is written out in fort 99 The data covariance as a function of data distance as well as the corresponding fit over distances up to the correlation length are found in fort 98 Used in divafit griddef f cr
22. dinate of the 66 dx step of output grid 0 2 dy step of output grid 0 05 nx number of grid points 300 ny number of grid points 300 valex exclusion value 999 0 snr signal to noise ratio 30 first grid point of the output first grid point of the output in the x direction in the y direction of the whole dataset varbak variance of the background field If zero no error fields are produced i If one relative errors are obtained 0 Example file 3 1 1 param par The example file param par can be used as a model for your own use The signication of the parameters is discribed in the file itself and are also made explicit in the present section 3 2 Command Line version The CL version allows for batch processing and customization but is not practical for beginners This version is much better suited for repeated and automated treatment and will also benefit from latest developments earlier Important remark when working under Unix you may have to convert the compila tion scripts nakediva and makeall and the procedures files files with a name starting by diva located in diva 4 1 divastripped To perform the conversion use the command dos2unix i followed by the name of the file you want to convert Example dos2unix divacalc 3 2 1 Working directories The Command Line version is contained in diva 4 1 divastripped Four subdirectories exist inside divastripped 1 input data con
23. e located in diva 4 1 bin In this purpose you are offered two possibilities 1 Use the precompiled versions of the executables for Windows Linux and SGI which are provided in diva 4 1 bin Win32 diva 4 1 bin Linux and diva 4 1 bin SGI respectively Simply copy the executables corresponding to your operating system to diva 4 1 bin 2 Compile the Fortran programs for example if you need to modify internal parame ters to increase memory allocation Execute the file compileall located in diva 4 1 src Fortran This file makes the compilation of the Fortran scripts in each folder Mesh Util NC No P1Plot and Calc The makeall should not be modified You only have to adapt the files makediva in Calc and makeall in the other directories according to the Fortran compiler and the compilation options you are using Examples of makediva and makeall files are provided with the suffixes SGI LINUX and WIN All the executables a files are automatically moved to the directory diva 4 1 bin after the compilation 2 3 1 Tcl Tk programs As Tcl Tk is interpreted language you do not have to compile it The modifications you may have made will be taken into account once you will have launched the Diva GUI 2 4 Creation of shortcuts Here are the procedures to create shortcuts for GUI and for CL on your desktop DIVA 2 4 1 Windows e Copy diva ico divacl ico diva bat and profile all found in diva 4 1 install into th
24. e msys directory e Edit diva bat and change the line starting by start sh login c cd to adapt the path to the directory in which you installed the Diva tc1 files Example startsh start sh login c cd c diva 4 1 src Icl wish84 main tcl de rem exit and save the modified diva bat file e Edit profile and modify the third line according to your configuration Example cd c diva 4 1 e With explorer create shortcuts to msys diva bat and msys divastripped bat and move them to the desktop e Right click on the desktop shortcut Properties in General use diva 4 1 GUI instead of shortcut as text for the GUI version diva 4 1 CL for the CL version in Shortcut change icon browse to msys and chose diva ico divacl ico 2 4 2 Linux e Right click on your desktop and select Create launcher e Type application name DIVA as you wish command path to the wish executable path to main tcl e n permission allow executing file as program 2 5 PlPlot library The PlPlot library is a set of Fortran routines that allow you to obtain graphical outputs without resort to external softwares such as Matlab or NcView We describe hereinafter the procedure for the installation 1 download the most recent version of PlPlot from http plplot sourceforge net 2 uncompress and unpack the archive plplot 5 7 1 tar gz gunzip plplot 5 7 1 tar gz tar xvf plplot 5 7 1 tar gz 3 in the directory plplot 5 7
25. eates the file GridInfo dat of which the content is used for writing the NetCDF output lceleme f computes the mesh characteristic length lookforoutliers f checks if there is outliers in the data provided for the analysis topoprep f makes easier the generation of the topography using a collection of local measurements Remark grid definition for users is based on an origin which is the first grid point In the code xori and yori are defined by x xori iAz and is thus shiftet one grid space to the left compared to the user origin Input file fort 13 is modified from the param par grid information by griddef a accordingly while gridInfo dat contains the user grid as defined by param par A 1 5 NetCDF output src Fortran NC Three files allow to get an output under the NetCDF format LE netcdfoutputfield f netcdfoutputerror f netcdfoutput f The first two routines write the analysed and the error fields in two different NetCDF files anlysed_field nc and error field nc The last programs generates results nc which contains both the analysis and the error fields The information required for the coordinates xorigin y origin dx dy xend yend are read from GridInfo dat ovo 7 7 100 100 Example file A 1 1 GridInfo dat A 2 Input and output files The Fortran executables files ending by a work with input and output fort files generopt a e reads as inputs fort 10 coast file see exemp
26. ed is the working directory for the CL version see section 3 e doc contains the publications and the documentation You can also find the corresponding BibTeX entries in the file DivaPublications bib GUIwork is the working directory for the GUI version see section B e src contains the for the Diva analysis folder Fortran and the graphical interface folder Tcl It also contains Matlab scripts for plotting the meshes and the results of the analysis folder Matlab 2 2 2 Command Line version The Command Line version is directly usable through a shell in the divastripped direc tory 2 2 3 Graphical User Interface version To finish your Diva GUI installation you only need to set the paths according the location of the diva 4 1 folder on your hard disk 1 Go to diva 4 1 src Tcl edit the file main tcl and set the path to your diva 4 1 directory For example if you installed diva 4 1 in C set env DIVA GENERAL DIR c diva 4 1 bin SGI Unix Win32 divastripped divawork input meshgenwork output ghertonetcdf meshvisu Tcl GUIwork ANALYSIS BASE BAT CONFIG CONT DATA DEPTH DES MESH TMP install a compiled executables precompiled version for SGI precompiled version for Unix precompiled version for Windows command line executions and diva scripts w
27. ed to the module GCVFAC of Diva Even if A is not available Az can be calculated easily by applying the analysis tool to a random vector and retrieve the analysis of this random vector on the data locations Then the scalar product of the analysis of the random vector with the original random data provides the numerator of while the denominator is simply the squared norm of the random vector Generalized cross validator In order to make the error estimator robust we take the average over all data points and define the generalized cross validator as O Le NE Assuming temporarily e hence having all misfits with the same weight the gener alized cross validator is obtained o IRP aa ous N 1 trace A 1 N trace 1 A The GCV consists in minimizing O by changing the signal to noise ratio A is a global estimate of the analysis error variance In view of 6 5 and 6 17 the expected variance of the noise can also by assuming a spatial average corresponds to a statistical expectation be calculated as dee 1 L trace A 6 18 When the observational errors are uncorrelated but vary in space we should replace the T residual measure r d d d d by r d d R d d 6 19 to take into account the relative noise level For a diagonal matrix we can define weights w such that 2 2 1 6 20 6 20 E gt Defining the diagonal matrix W diag w the gene
28. enerate them from directory input to make contour files coast cont in directory output divacont creates a contour for each level defined in contour depth one value per line in directory input A series of coast cont 100nn will thus be created nn 01 cor responing to the first line each of those can later be copied into a coast cont for an analysis at the level of your choice In addition coast cont is created with a default value z 0 From a mask Simply look at contourgen f and create the mask as you wish Alternatively you can create a pseudo topography with adequate pseudo depth at which you draw the contour Mesh refinement For all coastlines a by hand approach allows to refine regions with increased finite element resolutions To use this create a coast cont dens file that contains NR number of subregions Li N1 length scale and number of points defining the subregions x1 yl coordinates xN1 yNi L2 N2 length scale and number of points defining subregion 2 Subregions do not need to define real contours only parts of the space are defined in which resolution is different Place this file into input and perform as usual Be sure you eliminate the file for other applications 3 5 2 Topography generation By hand Create a gridded file in the same format as the analysis fields fieldgher anl of topog raphy and call it topo grd The information on the grid s geographical dimensions are to be placed
29. f the average error and field variance we have N 1 1 1 N gt pa py trace R gt gece R 1 6 7 lp ne B 0 gt E 8 1 6 8 N N N The unknown parameter that controls the analysis is the ratio of the signal and noise variances Called signal to noise ratio 6 9 because matrix A depends only on A AQ B B AIR 6 10 Dividing both sides of Eq 0 00 by o and e we also find that A d d Ty Em 6 11 1 d d 2 o 6 12 so that knowing A we can calculate the signal and noise variances from the data values 6 1 2 Ordinary Cross Validation OCV We want to optimize the parameter by looking for its value for which the analysis has a minimal error For this reason we need to find a proxy norm that we will be able to minimize As the difference of the analysis with the true field is not available we will work with the difference of the analysed field at the data points with respect to the original data field 02 di diy 6 13 If we try to minimize this norm we will get an infinite signal to noise ratio and a perfect data analysis fit This is because the analysis at the data point is directly influenced by the corresponding data To avoid this inconvenient the solution is to calculate the difference of the data value with respect to the analysed field in which the data under investigation was not taken into account This is called the Ordinary Cross Validation To make this es
30. f the following programs menu tcl generates the menu bar for the Diva main menu with its sub menus File View Options and Help data tcl builds the Data window Diva File Data which permits the data extrac tion fem tcl generates the finite elements mesh diva tcl performs the analysis on the data set view tcl defines the procedures for the visualization menu Diva View option tcl provides the sub menu Options of the Diva main menu Diva Options help tcl builds the Help window in the Diva main menu Diva Help and provides useful information about the meaning of the parameters choice and the interface The program varlist tcl initializes the data in the dialog boxes and the default access paths dictionary tcl is a dictionary in English and French tablist tcl defines tables used in other subroutines entry tcl misc tcl and xmisc tcl define several procedures called in other tc1 files Gallery The following results were created with the data of the Diva Workshop participants Li ge 13 15 of November 2006 1 Atlantic Ocean and Mediterrranean Sea surface temperature 2 Eastern Mediterranean temperature 3 Barents Sea temperature 75 10 20 30 Longitude E 10 42 20 1 o e N apne 10 20 30 Longitude E 10 48r 46r 44 fi N 20 36 34L 32L 30 i o eo e N apne N epmine1 N epmi
31. format binary These files are opened and closes with the help of gzopen m and gzfclose m respectively These three files do not need to be modified Plots for CL version They are designed to create plots by reading information in the subdirectories of GUIwork By default the plots are saved in GUIwork PLOT To make a plot you just have to indicate the name of the case you treating Example adri atic and run the corresponding Matlab file The program will load information required to perform the plot in the other directories Example adriatic dat in the DATA directory adriatic mesh adriatic mesh mh2 and adriatic mesh mh4 in the MESH directory etc If you are working with another convention for the file names you just have to edit the m files according to the names you have chosen The names of the figures are automatically determinated Example adriatic mesh but you can modify them easily by changin the line s beginning by fileouti Name _ Plots for CL version As the name of the input files are always the same data dat for the data file coast cont for the contour file and param par for the parameter you do not need to adapt the names of the input files By default the suffix of the figures created is output Example out put_contour 3 6 2 NcBrowse and Ncview Diva provides analysed and error fields under the NetCDF network Common Data Form format It is an machine independent format to represent sc
32. g error field coord 1 0 mathpr 1 2 gt ITYP 0 gt ISYM 2 gt IPB topolo 1 158 gt Number of Vertex Nodes 426 gt Number of Interface Nodes 268 gt Number of Elements datapr 1 1 solver 1 0 stores 1 1 gt 1 if normal or seminormed final otherwise 3 esterr 1 stopex Example file A 2 5 fort 10 0 0001 gt alpha0 0 02 gt alphal Example file A 2 6 fort 12 0 0 gt xorigin yorigin 1 1 gt dx dy 100 100 gt nx ny 99999 gt Exclusion Value Example file A 2 7 fort 13 90 90 1 12 5663706 90 80 1 12 5663706 90 70 1 12 5663706 16 30 2 12 5663706 16 20 2 12 5663706 16 10 2 12 5663706 Example file A 2 8 fort 20 e produces as outputs fort 71 field value at data points given in fort 20 ascii format fort 72 error value at data points given in fort 20 ascii format fort 73 error at points given in fort 79 ascii format fort 82 field value at points given in fort 79 ascii format fort 83 field on regular grid ascii format fort 84 field on regular grid gher format fort 86 error field on regular grid ascii format fort 87 error field on regular grid gher format A 3 Tcl Tk code src Tc1 The Tcl Tk files generate the graphical interface The main file is nain tcl it generates the main window sets the paths to the Tc1 GUIwork and bin directories The different windows of the interface are created with the help o
33. hat can be found on the WWW server Diva is a Liege University software which will be further developed for Sea DataNet scientific data products in JRA4 activities All SeaDataNet products data and softwares are freely distributed to the sci entific community at the following conditions V The products should be used for scientific purposes only V Articles papers or written scientific works of any form based in whole or in part on data or software supplied by SeaDataNet will contain a suitable acknowledgement to the SeaDataNet programme of the European Union The related publications see bibliography should also be cited V The applications of the SeaDataNet products are on the full responsibility of the users neither the Commission of the European Communities nor the SeaDataNet partners shall be held responsible for any consequence resulting from the use of SeaDataNet products V The recipient of these data will accept responsibility of informing all data users of these conditions V The Diva software cannot be incorporated into another software package without permission Acknowledgements his report was written for helping oceanographers to work with the Diva software This would not have been possible without the help of scientists involved in Data Analysis projects We would like to thank the participant to the Diva workshop in Li ge November 2006 for their valuable comments to improve the
34. ientific data For more details please consult http www unidata ucar edu software netcdf The files written in this format can be easily displayed with the following tools e NcBrowse Linux and Windows available at http www epic noaa gov java ncBrowse e Ncview Linux and Windows Cygwin available at http meteora ucsd edu pierce ncview home page html data from analysed field nc Domain Selector float data y x valid min 10 0 float analysed_field nc local NetCDF File Browser eli mecs Noe File Edit View Window Help E Select Variable for Display File analysed field nc local data Dependent Variable x y Reverse not Hi o Degrees east MO G In ix analysed field nc 60 40 E o E l P En B o o a gt 20 40 60 x Degrees_east Figure 3 10 Plots of results with NcBrowse est cases dios test cases are idealized situations designed to show some properties of the Diva analysis and to allow the use to observe the influence of the variation of parameters 4 1 Island This case shows the influence of the data density and of the influence of a specific contour an island on the analysis 4 1 1 Data The field has a value 1 to the West of the island and a value 2 to the East Fig 4 1 The number of data to the West is four times the number of data to the East 44 i e
35. in TopoInfo dat The grid is simply an array i 1 M and j 1 N where the coordinates of the grid nodes are z 01 i 1 dz y y j 1 x dy The file TopoInfo dat contains simply x1 yl dx dy M N Look into dvdv2diva f in src Fortran Util how to write such files from within a Fortran code Topography extractor From https idbms navo navy mil dbdbv dbvquery html select Area and desired spacing then submit Chose CHRTR ascii format and save resulting file as topo asc into directory input Execution of dbdb2diva in divastripped directory will then create topo grd and Topo Info dat in output You can use matlab file topovisu m to visualize the topography Then for automated contour generation see subsection 3 5 1 move these files into input Topography generation with Diva You can use Diva itself to interpolate a topography from isolated depth measurements Such point data can be extracted for example from http topex ucsd edu cgi bin get data cgi With such a series of data x y depth saved in topo dat an execution of divatopo in directory divastripped creates topo grd and TopoInfo dat in output The analysis and output grid is based on the param par found in the input directory All topographies be they created by hand read or analysed can then be used for contour generation section 3 5 1 by moving the gridded topography topo grd and its metain formation TopoInf
36. ining the N data anomalies Objective analysis of d leads to analysed field with minimal expected error variance The analysis o at any location r is given by PI c B R d 6 1 where c is a vector containing the background covariance between the point in which the analysis is to be performed and all data point locations The optimal interpolation is based on the background covariance matrix B and error covariance matrix R of the data Let us call d the analysis vector at data points The two vectors d and d can be related by the expression Q Ad 6 2 98 where the matrix A used to perform the analysis at the data points is calculated accord ing to A B B R 6 3 The data covariance matrix is the statistical average of data products dd B R 6 4 where T it the transposed matrix or vector For uncorrelated observational errors error covariance matrix R is diagonal with a variance e for point i i e R diag c In that case we can show that the variance of expected misfit at point i is a 2 Ei 6 5 In practice covariance matrices are known only imperfectly their structure is often con sidered to be fixed but with imperfectly known amplitude In other words it is often assumed that B c B 6 6a R eR 6 6b 2 E 97 e 6 6c where matrices are fixed and non dimensional while the field variance o and the error variance e are imperfectly known By definition o
37. is the characteristic length of the problem and oo a and y are parameters that have to be adapted to the considered problem ao fixes the length scale L over which variations are significant to move the kernel function from one to zero maL 1 6 31 uL fixes the weight on data vs regularization As the signal variance is o and the noise variance ez we can write 2 uL An 6 32 2 a fixes the influence of gradients aL 2 6 33 where is a non dimensional parameter close to one if the gradients are to be penalized with a similar weight than the second derivatives 6 3 2 Implementation of result validation in Diva The overall signal to noise ratio is calculated as c X E 6 34 The relative weights on the data are then defined as z iL VU AT 6 35 To be coherent with the total noise we should in principle use relative weights on the data that satisfy 6 22 Hence defining w as 1 1 1 6 36 wo N 2 wi p the weights w w w are relative weights coherent with the noise In this case we can use 6 21 for the estimator In practice in Diva the relative weights can be retrieved by w u j where the average is a harmonic mean The minimisation and the optimum for are the same even if the weights provided by the user do not satisfy the condition 6 22 However the interpretation of O e and o might be different Diva GCV In Diva the matrix A is not e
38. le file with description case of a square island in a square sea fort 11 the parameters required for the mesh generation e produces as outputs fort 22 finite element mesh to be copied as fort 11 for the work of diva a fort 23 topological parameters Do not forget you need the NetCDF library if you want to compile these files The library needed by your system can be downloaded from http www unidata ucar edu software netcdf 2 4 00 100 0 100 100 0 100 4 40 40 40 60 60 60 60 40 Example file A 2 1 fort 10 Example file A 2 2 fort 11 1 0 0 947 10 0 40 20 0 161 49 139 92 398 90 583 157 557 93 455 95 584 158 293 95 447 78 562 158 584 Example file A 2 3 fort 22 158 426 268 Example file A 2 4 fort 23 diva a e reads as inputs fort 10 divawork organiser defining which modules to use and how including several parameters fort 11 finite element mesh copy of the fort 22 produced by generopt a fort 12 ay and o calculated as 2 1 Qa and a p 4 fort 13 caracteristics of regular output grid fort 20 data file 4 columns X Y data X Y u where H Am pe with A the signal to noise ratio fort 79 coordinates where analysed field values are requested 2 columns separated by space X Y fort 15 parameter varbak variance of the background field set as 0 will avoid calculatin
39. mation Output 1 The analysed field is stored in the an1 file 2 The error field is stored in the output file suffixed by error Analysis Data file name c msys home Charles diva 4 0 GUIwork DATA adniatic dat Load msyshome Charles diva 4 0 GUIwork DEPTH Mesh file name c msys home Charles diva 4 0 GUIwork MESH adriatic mesh Load c msys homezCharlesZdiva 4 0 GUlwork BAT Analysis output file name C msys home Charles diva 4 0 GUlwork ANALYSIS adriatic anl Load Compute error 1 0 Reference field Mean value C Lin regression C Semi normed 0 0 c Char length 5 S N ratio 10 C Alpha ofo 0016 Alpha 1 Mu 502654824 Output grid Nb 401 Nb y 401 Min longitude axis 12 step oz Max longitude 20 Min latitude y axis E y step 02 Max latitude 45 Excl value 38398 Selected depth Trans results Load configuration Save configuration Dk Cancel Display analysis file Figure 3 5 Diva analysis window 3 4 The Options menu Coordinates system DEA Coordinates system T Coordinate change deg to km Ho coordinate change Ok Cancel Figure 3 6 Coordinates system options 3 5 Generation of contour and Chapter 3 DIVA USE topography files Default diretories Figure 3 7 Default directories choice Default language pm e Figure 3 8 Language preference 3 5 Generation of contour and topography files This se
40. n a Matlab session go in the directory diva4 1 src Matlab GUI and execute the script diva contour Contour and Data 45 28 44 26 A I 24 Latitude N 5 EN 20 40 39 38 i 13 14 15 16 17 18 19 20 Longitude E Figure 5 2 Adriatic sea contour and data 5 2 Mesh generation Diva 4 1 View Options Figure 5 8 Mesh generation Select the input and output files for the mesh generation e Contour description File liste cont e Mesh file name adriatic mesh Decide the reference length surface coefficient and the smooth number you want to use The suggested values for that case are Reference length 0 5 Surface coef 1 5 Smooth number 3 These mesh parameters can be modified in order to see their influence on the mesh shape Once the parameters are chosen use the OK button to create the mesh You can examine the mesh file with the command Display Mesh File The graphic result is showed on fig For the visualization use the Matlab script diva mesh Mesh x 45 SQ NS SAR POSSE S2 44 43r ci Pae a SOSA S Cc w RRA 42L Kan EA E G AEAEE T ATTAT 3 c CEE E a AAA Ya pe ANA 2 IZDA KO TET TAN a V AVAA L E E E A 5 E VATA STA A SOOKE BE a SIDOR 4 AS CPR ZAR at ex 40 L 39r a LX IS g gt Te s A 38L i j dR OR ERNO K 13 14 15 16 17 18 19 20 Longitude E Figure 5 4 Adriatic sea mesh 5 3 Analysis
41. ne1 Longitude E Longitude E 3 epnii amp uo 3 epnii amp uo es 9c ve ce oe 9c 9c 9c 9c ve ce oe 9c 9c N epmine1 N epmine1 3 epniibuo 3 epniibuo ge ve ce 0 9c 9c vc ez 9c ve ze 0 ez 9c Ln r T T T T T T i E x vl srl SL ES ag E e 5 9L s e Z Li 2 2 GOL e L 4 fod GLb el 30 20 10 84r i N eo 80r 78r N epnine1 70r 68 66 30 20 82r 80r 78r i o we i ha pne i N M 70r 68r 66 10 40 Longitude E Longitude E 82r WWW WM VW XY AAA w M 80r 78r 1 o s E N epmine1 L N M 70 68r 66 10 Longitude E Longitude E Bibliography Diva Brankart and Brasseur 1996 Brankart J M and P Brasseur Optimal Anlysis of In Situ Data in the Western Mediterranean Using Statistics and Cross Validation Jour nal of Atmospheric and Oceanic Technology 13 47 1 491 1996 Brankart and Brasseur 1998 Brankart J M and P Brasseur The general circulation in the Mediterranean Sea a climatological approach Journal of Marine Systems 18 41 70 1998 Brasseur et al 1996 Brasseur P J M Beckers J M Brankart and R Schoenauen Seasonal temperature and salinity fields in the Mediterranean Sea Climatological analyses of a historical data set Deep Sea Research 43 159 192 1996
42. nt grid will be given in section 3 1 3 suorjegdepe ouros ILA o qe reAe x o1n3e9 o qe reA x orje1 9SIOU 03 RUSIS 2 0 IBU uOrje ol1OO T SIOH JO 1equinu A SIS eue 103 sjurod pui jo Jaquinu y squiod eyep jo 1oquinu Py IIM sisfippuo vop fo spoyjzaw juasaffip fO s21251423204047 T I 20H L N yes x SN x JOSNIG x 00T T A N x 16 WA et 20 T T 4 2 NTN TN TO 7 7 4 m PNPN x ULUSSI amp doi30stue AOD otad D 3 2 oSeurr sd Jem qe z3 urur POYI9N spoujeur vy uooA3oq uosrtreduio T 1 4 Additional tools Diva software is provided with addition tools designed for parameters adjustment and validation of results The tools from to are already available The others are under development 1 4 1 Generalized Cross Validation GCV The Cross validation consists in setting data aside for a global error estimate GCV applies analysis to random anomalies at the same locations as real data It allows also estimating global errors and calibration of the correlation length L and the signal to noise D 2 ratio Please refer to Chap 6 for more details 1 4 2 Advection constraint An advection constraint can be taken into account in the variational principle 1 4 For this purpose a velocity field has to be specified on a regular grid similar to topography for 3D version This will have the
43. o dat into directory input 3 6 Display of outputs 3 6 1 Matlab Tools to display the contour data mesh analysis and error fields are provided in the directory diva 4 1 src Matlab They are separated in two subdirectories CL for the Matlab programs adapted to the command line version and GUI for the graphical inter face The operations performed by the Matlab programs are the same in both directories but the input file names may be different The plots will be exported either in jpeg or in encapsuled postscript eps formats choose between parameter format out 1 for jpeg format format out 2 for eps format Four scripts allow you to create your plots 1 diva contour m 2 diva mesh m 3 diva analysis m 4 diva error m These four operations can be summed up by running diva plot all m Remember that these Matlab scripts aim to show you how to create plots from files generated by Diva Therefore do not hesitate to modify them and adapt them to your own need With diva analysis mask m you have the possibility to plot the analysis field with a mask corresponding to the points where the error field has a value lower than a chosen value You just have to specified the higher bound allowed for the error Example tres hold 25 results that the analysis field will be plotted only at the location where the relative error is lower than 25 Subroutines uread m performs the reading of files written in GHER
44. of the mesh 2 name mh4 contains the number of vertex and interface nodes and the number of meshes 3 name mh5 that takes into account the depth in the 3D case 3 3 Graphical Users Interface Chapter 3 DIVA USE Figure 3 4 Mesh creation window 35 3 3 4 The Analysis menu Input e Two or three dimension case e Data Depth 3D only Mesh and Bathymetry 3D only file names e Compute error computation of the statistical error map of a field e Reference field treatment to be applied on the data located in the mesh a None b subtraction of the Mean Value c subtraction of the Linear regression d semi normed analysis e Characteristic length L interpreted as the radius of influence of a data at a given point Signal Noise ratio c variance of signal variance of noise ratio e Qo a and u directly set or computed from L and c according to 1 2 4nc ao pan ai F2 H IP e Output grid parameters phi lambda origini end dphi dlambda limits of the grid where the resulting reconstruction is written The Exclusion value is used to fill the output matrix when a cell a point corre sponds with a point on land Selected depth the data for a given depth located in the list of data file name are reconstructed e Transient results determines the amount of information appearing to the screen while computing the data reconstruction Goes from 1 minimal information to 5 maximal infor
45. on 3 2 4 Running an analysis Simple run Copy the data you want to process into the subfolder divastripped input with the following names coast cont data dat valatxy coord param par Set your command line into the folder Divastripped and execute by typing divadress This will execute the four following procedures divaclean cleans up the working directories by removing the intermediary fort files from divawork and meshgenwork and the output files from output divamesh creates the finite element grid divacalc based on the existing mesh performs the analysis divaqcbis simple outlier detection using generalized cross validation output from divacalc Additional tools A few utilities are provided to help you calibrate the signal noise ratio and or the corre lation length e divafit a script that uses input data dat for a direct fitting of the covariance function In output file output paramfit dat the best estimates are given and could be used as parameter values for running Diva Estimates of the correlation length are rather robust while those of the signal to noise ratio are neither precise nor robust especially for large values The fit needs a sufficiently large data set Output file covariance dat is the data based covariance function column 1 dis tance between points columns 2 covariance column 3 number of data couples used to estimate the covariance Output file covariancefit dat allows l
46. ooking at the fitted covariance function column 1 distance between points column 2 data covariance column 3 fitted covariance divagcv a script that exploits a new Diva module gcvfac analysing random fields to assess the generalized cross validator GCV Input to the module is the number of random estimates required the larger the value the more robust the estimator Default value is 5 unless you change in divadress The script divagcv is an example how to minimize the estimator by changing the S N value but could be adapted to optimize other parameters as well Input file input gvcsampling dat contains the list of values for S N on which to try the estimator typically around the values provided by divafit During the divagcv execution error field calculations are disabled to reduce com puting time Output file output gcv dat contains the GCV estimator GCV column 1 S N column 2 GCV column 3 data anomaly variance and output gcvsnvar dat the best new estimate for the S N and VARBAK parameters divacck contour check In output output coast cont checked you find a thinned contour based on the length scale divaqcbis simple detection of outliers listed in output outliersbis dat with a last line giving an indication if this is normal or not Be aware that this detection is only valid as long as your S N and VARBAK parameters are correct Systematic unusual numbers of outliers might indicate a problem in
47. orking directory input files mesh generation working directory output files whre the NetCDF files are stored where the plots are stored documentation and related publications sources diva core for 3D in GUI mesh generation NetCDF visualization routine Utilities some post treatment in matlab Command Line version Graphical User Interface version files to generate the GUI working place for GUI analysis files data base bathymetry files configuration files contour files data files depth files data base description files mesh files temporary files elements for the software installation Figure 2 2 Diva 4 1 directory tree 2 In diva 4 1 GUIwork DES edit the file liste DES according to the location of diva 4 1 in your hard disk Example c diva 4 1 GUlwork BASE demo BOT pdf c diva 4 1 GUlwork BASE demo BT pdf c diva 4 1 GUIwork BASE demo CTD pdf 3 a For Windows Open an msys shell and go to the directory diva 4 1 src Tcl Type wishxx main tcl where xx refers to the version you are using to activate the interface b For Unix in the directory diva 4 1 src Tcl type wish main tcl to activate the interface 4 If the installation succeeds you will get a windows like Fig Diva 4 1 File View Options Figure 2 3 Graphical interface menu bar 2 3 Compiling The executables produced by the Fortran programs should b
48. ouVo Vp apy dD 1 4 D where e o penalizes the field itself anomalies e 0 penalizes gradients no trends e 0 penalizes variability regularization e U penalizes data analysis misfits objective Without loss of generality we can chose a2 1 homogeneous function Meaning of the parameters Writing Eq 1 3 and 1 4 in non dimensional form with V V we have x Wie a Tle uld eG ur L zvve LV Ve Ve Ve aop L db 15 and multiplying by L Nd Jia y uL ld e z y L vve VVo aL Vo Vo noD dD ja 1 6 Hence ao fixes the length scale over which variations are significant to move the function from one to zero maL 1 1 7 uL fixes the relative weight on data signal 0 vs regularization noise e 2 pL 4m AnS N 1 8 C Finally o4 fixes the influence of gradients aL 2 1 9 where 0 if no penalization and 1 if penalization is a non dimensional parameter close to one if the gradients are to be penalized with a similar weight than the second derivatives Weights on data A weight u can be assigned to each data d This weight expresses the confidence you have in a particular data It is expressed as a function of the signal to noise ratio and the correlation length c Ar when 1 In Diva the fourth column of the data input file if present allows to apply a different weight to each point Background field
49. ow the advanced user to adapt the code A 1 Fortran code The Fortran programs are sorted according to their purpose Analysis Mesh Generation Graphical Interface NetCDF and Utilities A 1 1 Diva programs src Fortran Calc The main file of the Fortran code is diva f this routine calls successively the subroutines enumerated below in order to perform a Diva analysis using the finite element method mathpr f describes the mathematical problem topolo f describes the topology of the finite element grid meshgn f generates a square finite element mesh on a regular grid datapr f input of data to be fitted by spline smoothing bcondi f Dirichlet boundary conditions to be fixed 67 constr f input of information for constraint implementation solver f builds and solves the global linear finite element system Stores f storage of the solution esterr f estimates the analysis error same grid as the analysis coord f coordinate change longitude latitude to x y and x y to longitude lati tude if requested gcvfac f estimates the analysis error by generalized cross validation dataqc f data quality check estimates of expected data analysis differences A 1 2 Mesh programs src Fortran Mesh To generate the coastline contourgen f can be used For the mesh generation 2 subroutines are used generopt f which generates a multi connex 2D mesh with the Delaunay triangulation contourcheck f
50. par see file 3 1 1 If you work with the GUI version the parameters are chosen as one goes along the analysis Here is a summary of the parameters you may have to modify General Coordinates system kilometers or longitude latitude Mesh Uniform or non uniform mesh density mesh constant or varying within the domain If non uniform mesh a density file coast cont dens has to be provided Reference length length of the side of the triangular elements Surface coefficient shape of the triangular element Smooth number smoothing of the mesh elements Analysis Correlation Length radius of influence of a data at a given point Signal to noise ratio relative importance of the signal variance with respect to the noise variance Reference field background field which is subtracted from the data field Variance of the background field Output grid parameters xori yori dx dy nx ny Exclusion value value used to fill the ouput matrix when a point corresponds with land Lc correlation length in units coherent with your data 10 icoordchange 0 if no change of coordinates is to be performed 1 if positions are in degrees and if you want to use real distances ispec output files required mean of data 2 regression plan if at least non aligned data provided 1 3 ireg mode selected for background field O null guess 1 3 1 xori x coordinate of the 15 yori y coor
51. ralized cross validator then reads a d W d d T N 1 trace AI where the weights should according to 6 7 satisfy y a N 6 22 2 The are the values provided as optional fourth column in data dat 6 2 Quality control QC of data Having defined the generalized cross validator we look for a criterion that will allow us to reject or accept a given data 6 2 1 Quality criteria The first possibility is to compare the actual value of the misfit with the expected standard X2 deviation a di leading to the criterion ld d gt 3A 6 23 with AP CV 1 Asi 6 24 which is the most expensive version if A is not explicitly known and must be evaluated by analysis of vectors such as e 00 010 0 If we replace A by its average a trace A we have a second criterion based on AQ L traco a 6 25 This version requires only a few analysis of a random vector if trace A cannot be evalu ated explicitly Finally in case the noise is not calculated from O another estimate is then according to 6 18 amp f _ 1 A z race A O 6 26 which we can calculate directly from the rms value of the misfit or residual and the generalized cross validator O This version can easily be used simultaneously with test 6 25 using the results of the GCV 6 2 2 Normalized version Let us consider the scaled variable s defined as 6 27
52. rn consists in determining a field y r on a regular grid of positions r knowing Na data in locations rj 1 Na This is called the gridding problem Fig 1 1 It is useful for many applications data analysis graphical display forcing or initialization of a model In this chapter we consider only two dimensional cases but generalization can be done to 3D and even 4D using distance in time but beware of autocorrelations as in seasonal signals Figure 1 1 The gridding problem 1 1 1 Interpolation vs approximation When treating the gridding problem two main techniques have to be distinguished 1 The interpolation which implies a strict passing of the solution by the points of data Physically this means that you assume you have no error on the data 2 The approximation or analysis provides a solution more smooth than the one given by the interpolation in this case the solution does not necessarily have to contain all the data points The solution passes near the data points so its shape is still influenced by the data This technique allows to take into account the error on data as well as to treat multiples data on the same location but with different values Figure 1 2 Interpolation vs approximation 1 1 2 Data analysis and gridding methodologies Because of data error and often close data points the considered field is always recon structed with the help of approximation never with a stri
53. sh and French are available 3 3 1 Working directory The GUIwork directory is made of the following folders ANALYSIS analysed and error fields BASE data base files BAT bathymetry CONFIG pre defined configurations CONT contour files DATA DEPTH DES data base description files MESH and TMP temporary files To easily identify the different kinds of files and to be in agreement with the interface conventions it is suggested to use the following extensions for the file names anl for the analysis files bat for the bathymetry files conf for the configurations files cont for the contour files dat for the data files des data base description files and mesh for the mesh files 3 3 2 The Data menu Input e the Data Base description file it indicates which data file names are used to search for the extraction of data e Period mmdd the days of the year when you want to extract data For example from 0901 to 1231 means from the 1st of September to the 31st of December e Depths the depths where the data are extracted e Contour description file describes the position of the coastline e Position specify the longitudes and latitudes to select a rectangular domain e Quality criterion two types of flags are introduced one for the whole profile profile quality flag and one for each particular observation observation quality flag For more details please consult the Diva Help menu Output
54. software the GHER team for its constant support the Fonds de la Recherche Scientifique FNRS and the Fonds pour la formation la Recherche dans l Industrie et dans l Agriculture FRIA for their funding the European Union for funding the SeaDataNet project 84
55. the parameter calibration S N L and or error field calculation VARBAK e divaqc advanced outlier detection see Section 6 3 for a detailed description Finally the typical execution chain for a complete Diva run can be summarised as 1 Preparation of a first input param par including coordinate change and reference field definition 2 divafit for a first guess of the parameters adaptation of input param par according to the output file paramfit dat and creation of input gcvsampling dat 3 divacck for testing the contour 4 divamesh to generate the mesh 5 divagcv for improved estimates of parameters adaptation of input param par according to the values given in gcvsnvar dat 6 divadress chains divaclean divamesh divacalc divaqcbis for analy sis as well as error field whenever VARBAK is not zero detection of outliers with the help of divaqcbis divaqc additional quality check of data listed in output outliers dat 3 3 Graphical Users Interface The Diva GUI is made of four main menus File View Options and Help fig 2 3 File and View are both made of three sub menus Data used for the data extraction from a data base Mesh which generates the finite elements mesh and Analysis which performs the field reconstruction using the variational inverse method VIM The menu Options permits the choice of the coordinates system Coordinates the directories and the language at now Engli
56. timate robust the analysis has to be repeated over a large number of data points increasing the computing cost so that OCV is generally too expensive to perform 6 1 3 Generalized Cross Validation GCV According to Craven and Wahba 1979 modifying the error estimate as follows d di Ay 6 14 allows to keep the data during the analysis and will reduce the computing cost In this formulation the denominator penalizes more heavily data points in which the analysis is forced to be close the data and accounts therefore for the self influence of the data point which is absent in the case of pure cross validation Computation of Aj When the matrix A is not explicitly calculated A can be obtained by performing an analysis with a vector e 00 010 0 zero on all data locations except at point i where its value is one This demands an analysis for every data point in which the estimator is constructed The associated computational cost can be reduced by replacing A by the average value and assuming 1 Ai c trace A 6 15 To avoid calculating all A and summing them up we can use the following estimate Girard 1989 1 TA py trace A 6 16 E 6 16 where z is a vector of random variables of zero mean For robustness the trace estimate can be repeated several times with different random vectors averaging of the different estimates The number of estimates is the parameter provid
57. tk downloadnow84 html 2 Uncompress and unpack the archives tc18 x xx src tar gz and tk8 x xx src tar gz within the directory home username gunzip tcl8 x xx src tar gz tar xvf tcl8 x xx src tar gunzip tk8 x xx src tar gz tar xvf tk8 x xx sre tar 3 Go into the directory home username tcl8 x xx unix 4 Type successively configure to create the makefile make to create a library archive libtcl lt version gt a and an interpreter application called tclsh that allows you to type Tcl commands interactively or execute script files make install to perform the installation of Tcl binaries and script files in standard places you need to work as a root to have all the permissions 5 Repeat the operation 4 in the directory home username tk8 x xx unix 6 Add tclsh and wish to the path edit bashrc and add home username tc18 x xx unix home username tk8 x xx unix 2 2 Diva software installation The two versions of Diva Command Line CL and Graphical User Interface GUI are installed by downloading diva 4 1 zip from http modb oce ulg ac be modb diva and unzipping where you want to install Diva on your hard disk Example in C The directory tree you will obtain is showed on Fig 2 2 Console File Edit Help win 1 Figure 2 1 Wish interpreter console 2 2 1 Diva 4 1 content The folder diva 4 1 contains five main subfolders e bin is where the executables are located e divastripp
58. tour s and parameters needed by Diva to perform an analysis 2 meshgenwork mesh generation working directory 3 divawork the working directory of the CL version where the Diva calculation is performed 4 ouput results for the user 3 2 2 Input files The data file data dat the coastline file coast cont and the file that indicates the points where you want an analysis valatxy coord Several examples are provided in the Diva distribution l This command should be available within you msys or cygwin distribution The generation of coastline file is explained in section 3 2 3 Output files A complete run of Diva generates numerous output file If you are only interested by the analysis results here are the files you shall have a look at fieldgher anl and errorfieldgher anl are respectively the analysis and the error fields in gher format on the regular grid specified by x yorig dx y and x yend fieldascii anl and errorfieldascii anl are the same as fieldgher anl and errorfieldgher anl but in ascii format valatxyascii anl and erroratxyascii anl give respectively the values of the anal ysis and the error fields at the points specified in the file valatxy coord fieldatdatapoint anl and erroratdatapoint anl are respectively the analysis and error fields computed at the data points i e the points from data dat The other files generated by Diva are described in the additional tools in the following secti
59. tude E 4 3 3 Analysis The analysed fields in the interior and the exterior region are not influenced one by each other The fields and error fields are disconnected le c Ref Field Az Ay 1 5 100 Mean value 0 1 0 1 Table 4 6 Lake analysis parameters 75 4 3 2 1 2 3 4 5 0 1 Longitude E Latitude N Latitude N E 0 1 Longitude E Figure 4 9 Lake analysis and error fields 4 4 Noise 4 4 1 Data Figure 4 10 Noise case data N epnine1 Longitude E 4 4 2 Mesh Lm Surf coef Smooth num 1 5 10 Table 4 7 Noise mesh parameters Figure 4 11 Noise case mesh w N epnine1 9 Longitude E 4 4 3 Analysis Lo o Ref Field Ax Ay 1 5 10 Mean value 0 1 0 1 Table 4 8 Noise analysis parameters 0 25 0 2 Z Z 3 3 0 15 0 1 l 0 05 E E 4 E 3 1 0 1 E 3 4 5 75 4 E E 1 0 1 3 4 5 Longitude E Longitude E Figure 4 12 Noise analysis and error fields 4 5 Single point 4 5 1 Data Contour and Data 5 2 4 1 8 3r 1 6 2 14 1 1 2 il EM Figure 4 13 Single point case 1H 08 data 2b 0 6 eas i i 04 L i 02 5 i L L L i L L 0 4 5 2 Mesh Lin Surf coef Smooth num 1 3 1 5 5 Table 4 9 Single point mesh parameters Vp MS DOG N PAN VN BONA DOAN X fs YyDOSCOEDEADOS ISSN SSSR ESOPO ROSSANA C EPA AT OLE CP Z AE CE ECO G ESA DOCS E AN AA PADOZAES
60. which eliminates identical points and checks the contour format A 1 3 GUI programs src Fortran Extensions The graphical interface requires some particular routines extract f interface to select or extract Data from MODB and MED formatted Data Bases fem3d f is based upon a mesh file and a bathymetry file regular grid in the GHER format it creates a result file containing information to determinate if a mesh is in land or is sea for a given depth concat f is used to create the 3D file from the 2D files stiff f generates the rigidity file fort 60 in the 3D case mask f masked the solution according to the bathymetry and the depth sum f writes the sum of two fields with same dimensions in GHER format substref f subtracts the reference field to the computed solution only available when using the semi normed reference field header2 f and visu f are subroutines to visualize the data the mesh and the solu tion requires the PlPlot library Two versions of visu f are provided one is located in diva 4 1 src Fortran PlPlot and the other in diva 4 1 src Fortran NolPlot The first can be compiled only if you have installed the PIPlot library on your computer for now only under Linux A 1 4 Utilities src Fortran Util alpha f computes the values of ay and ay calcmu f computes the coefficient y knowing the characteristic length L and the signal to noise ratio A calcest f from relative weight on data points
61. which is generally low but also e representativity errors what you measure is not what you intend to analyse e g skin temperature inadequate scales e synopticity errors when the measures are assumed to be taken at the same time e g data from a cruise Perfect fit to data is therefore not advised In Diva the analysis behavior is controlled via the signal noise ratio of data 1 2 VIM and its implementation Diva VIM stands for Variational Inverse Method This method was initially designed for climatology in that case you have generally high resolution vertical profiles sufficient profiles in all seasons but irregular horizontal coverage Brasseur et al 1996 Thus a spatial analysis on horizontal planes is needed Relatively large number of data points in each plane penalises Optimal Interpolation O L methods because these methods require the inversion of a N by N matrix i e a number of operations proportional to N3 see Tab 1 1 This is the reason why VIM resorts to the expertise in efficient finite element solvers Diva stands for Differential Integral Variational Analysis and is the implementation of VIM It is designed to solve 2 D differential or variational problems of elliptic type using the finite element method 1 2 1 Formulation We are looking for the field which minimizes the variational principle Nd J e 5 2 uj ld ej up lle poll 1 3 j l with lell f a VWVo VVop
62. xplicitly constructed but the product Ax consists simply in the application of the analysis in data locations The value of trace A can be evaluated and stored for subsequent quality control by and 6 26 For quality control 6 24 Aj must be evaluated by an analysis of the pseudo data vector e zeros everywhere except unit value in point 7 The analysis can benefit from the LU decomposition already performed for previous analysis Diva QC Quality control with Diva can be performed according to one of the three criteria 6 24 Or 6 26 respectively implemented in Diva with divaqc divaqcbis and diva qcter The corresponding outputs are given in outliers dat outliersbis dat and outlierster dat The modules divaqc also generate outliers normalized dat which contain in a sorted way from the most suspect data to less suspect the possible outliers from the normalized misfits test 6 29 By default the criterion used in divadress is divaqcbis but you can change it by editing the file divadress and replacing divaqcbis by one of the other quality test divaqc or divaqcter l The optional fourth column for data in Diva Part II Appendix Diva code pes engine of Diva is a set of Fortran 77 subroutines driven by local input fort files and local output fort files The GUI is driven by programs written in Tcl Tk see Sec A 3 This chapter is dedicated to the description of these programs in order to all
63. ysical Research accepted 2006 Beckers and Rixen 2003 Beckers J M and M Rixen EOF Calculation and Data Filing from Incomplete Oceanographic Datasets Journal of Atmospheric and Oceanic Technology 20 1839 1856 2003 Beckers et al 2006 Beckers J M A Barth and A Alvera Azc rate DINEOF re construction of clouded images including error maps Application to the sea surface temperature around Corsican island Ocean Sciences 2 183 199 2006 Miscellaneous Craven and Wahba 1979 Craven P and G Wahba Smoothing noisy data with spline functions Numerische Mathematik 31 377 403 1979 Girard 1989 Girard D A fast Monte Carlo cross validation procedure for large least squares problems with noisy data Numerische Mathematik 56 1 23 1989 Useful links Ek SeaDataNet Home Page es http www seadatanet org Diva Home Page http modb oce ulg ac be modb diva html GHER Home Page http modb oce ulg ac be Ocean Data View http odv awi de NetCDF http www unidata ucar edu software netcdf Tcl Tk http www tcl tk PIPlot http plplot sourceforge net Msys http www mingw org msys shtml Cygwin G http www cygwin com Matlab A http www mathworks com UR University of Li ge Mer http wiw ulg ac be Conditions of use Diva is a public domain graphical software and can be found on the GHER www server This program is subject to conditions of use t

Download Pdf Manuals

Related Search

Related Contents

  Baumatic BHG630SS Technical Drawing  GMS 10.0 Tutorial PHT3D – Transport and Mineral  User Manual - Instrumart  Bedienungsanleitung Instruction Manual  Toshiba Satellite 1955-S803 (PS195U  ATA Amplifiers Manual  L`AUTONOMIE Comment rendre les élèves de terminale STL plus  

Copyright © All rights reserved. Failed to retrieve file