Home

Acceptance Test Plan

image

Contents

1. SME LET Announcement of Opportunities 2009 Cal Val and User Services Calvalus Acceptance Test Plan Version 1 2 31 October 2011 Prepared by Norman Fomferra Calvalus Acceptance Test Plan ff Brockmann Consult GmbH Change Log Version Date Revised by Change Authors 1 0 16 09 2011 The initial version of this document N Fomferra 1 1 10 10 2011 E Kwiatkowska Level 2 processing Added another test case N Fomferra O Faber with NetCDf as output format Added another M Zuhlke test case that uses QAA as IOP algorithm Matchup Analysis Added better explanation for good pixel expression and good record expression Level 3 processing Fixed spatial resolution to be 9 29 km Fixed averaging period to be 10 days Trend Analysis Fixed compositing period to 32 days 1 2 31 10 2011 E Kwiatkowska Explicitely naming the L2W processor version N Fomferra to be used for the test namely 1 3 Added a description on how to change the colour mapping in images displayed by BEAM VISAT Using new L3 image screenshot the old one had artefacts caused by an invalid water mask used for the server side processing Added some minor improvements and fixed some typos E Brockmann Consult GmbH Calvalus Acceptance Test Plan Table of Contents De WUC LT EE E E E OE aaa EE 1 1 1 Papo cond ODE RE E EAE 1 1 2 Terme aid Del MORE se OG NG Iga NA GREG GE iain a angan aa De 1 1 3 ADDTEV ide li AO NT TAN NAGA RAN
2. 3 5 4 3 Expected Results Similar to Test 4 1 but with constrained output records The scatter plots shall now look similar to the following 22 Brockmann Consult GmbH Calvalus Acceptance Test Plan conc_chl NOMAD Kd_490 NOMAD coastcolour processing 1 3 SNAPSHOT CoastColour L2W coastcolour processing 1 3 SNAPSHOT CoastColour L2W n 13 a 1 49314 b 1 42149 n 4 a 0 04490 b 0 68269 50 0 6 AS 40 1 Da D 2 gt oa 30 ad 25 A 0 3 lt 20 a 20 st GE Fi C 15 2 A AE z A o oi A 5 4 a gt 0 66 0 01 0 5 10 15 20 25 30 35 40 45 50 00 01 O2 03 04 05 06 CONC_CHL in situ KD_490 in situ 3 6 Test Group 5 Level 3 Processing L3 3 6 1 Feature Description The production type Level 3 Processing L3 allows user to process a filtered input data product file set using a selected Level 2 processor to one or more Level 3 data products The result of the production is a zipped set of output files in a user selected EO data format currently BEAM DIMAP NetCDF GeoTIFF that can be downloaded by the user For the demonstration of the Calvalus system the Level 2 processor for L3 is again the CoastColour L2W processor same as for L2 and MA A description of the L2W output product and its processing parameters is given in annex A Following Level 3 Parameters need to be set up The L3 production type can generate many L3 output variables at the same time Users simply add a new variable using t
3. 77 CoastColour L2W Figure 12 Level 2 production in progress You should now see the processing status transit from SCHEDULED to RUNNING to COMPLETED After the processing completed production you should see the staging status transiting from SCHEDULED to RUNNING to COMPLETED The total processing time should not exceed 6 minutes and shall not take longer than 12 minutes The production request will produce 7 ZIP files that add up to 325 MB Note It is a normal behaviour for the Level 2 processing that it progresses quickly first but then slows down when it comes to the processing of the remaining files The reason for this is that the entire production is only completed by the time when the last L2 task completes And since every L1 input file will be processed on a dedicated node in the cluster the entire L2 production waits for the last task to finish The total staging time should not exceed 2 minutes and shall not take longer than 6 minutes Your production status shall now look similar to the following screenshot L2 Processing Match up Analysis L3 Processing Trend Analysis Manage Regions Droductions Production User Processing Processing Staging Status Time Status 20110916191816 L2_271ab00ad0861b Level 2 production using input path eodata MER_RR__1P r03 2008 MMV dd norman COMPLETED 0 04 59 COMPLETED Restart Download _N1 and L2 processor CoastColour L2W me norman 201109816191816 L2 271ab00ad0861b
4. 3 4 5 3 Expected Result The result shall be similar to the one of Test 3 2 the QAA algorithm may execute faster The geo physical contents of the generated product will be more or less different from the ones in Test 3 2 3 5 Test Group 4 Match up Analysis MA 3 5 1 Feature Description The match up analysis MA is a production scenario that compares reference point measurements such as in situ observations with corresponding extracts from Level 1 or Level 2 data The measurement points are taken from a user provided data table A record in this table may not only contain the geographical coordinate of a point but also any number of reference data in situ data or EO data and measurement metadata such as the measurement time For any variables contained in the data table which can also be found in the Level 1 or Level 2 data products the MA will generate scatter plots and provide a linear regression of how the reference data matches the data found in the data products at given points The following screenshot shows the MA parameters in the Calvalus portal Matchups are not done on single pixels but on macro pixels that include neighbours to the centre pixel that exactly corresponds to the given geographical point coordinate In the Calvalus implementation of the MA the macro pixels 1 span 5 x 5 normal pixels or more All pixels in the macro pixel are screened and a list is generated of values that ar
5. Figure 13 Level 2 production complete 15 Calvalus Acceptance Test Plan E Brockmann Consult GmbH 3 4 4 Test 3 2 3 4 4 1 Aim Test that Level 2 processing results can be downloaded and that the processing has been performed correctly 3 4 4 2 Set up You will need BEAM 4 9 for displaying the results of this and the following tests If you don t have it installed please go to the BEAM home page in order to get BEAM 4 9 first Note You may want to use the Unix tool wget http www gnu org software wget to download all the produced Level 2 output files at once 3 4 4 3 Procedure 1 Make sure the Manage Productions tab is selected 2 Click the Download button in the Result column of the production table A new window shall be opened in your browser It shall now show the outputs of the Level 2 processing You shall be able to download the listed ZIP files 3 Download one or more files and unpack them on your local computer Open the unpacked files in BEAM VISAT and inspect their contents The products shall contain all the L2W bands and datasets as described in Annex A 3 4 4 4 Expected Result The following screenshot shows the resulting Level 2 files You shall see a similar page in your browser showing 7 zip files comprising around 325 MB Calvalus staging area staging norman 20110916195044_L2_271ab00ad0861c Filename Size Last Modified L2 of MER RR 1PRACR20080601 090622 000026432069 00079 32698 0000 zip 26
6. Table Discrete colors BY TN Name conc_tsm Unit g m 3 Min 0 083 Max 68 877 Rough statistics sua il AE da siba ea More Options RK Oo R 21 d LO ng A Colour Manipulation i Na igation 1 conc_tsm Satellite coordinates 19 58 32 179M of 440M ol H In order to use the same colour palettes as in the screenshot shown above do the following in VISAT 1 Double click a band in order to open an image view 2 Open the Colour Manipulation tool window if you can t find it click on the icon in the main tool bar 3 Click the import icon button and select the palette spectrum cpd or spectrum_large cpd Click Open 4 Use the 95 icon button to adjust the value range to be colour coded 3 4 5 Test 3 3 3 4 5 1 Aim Test that Level 2 processing parameters can be changed and that retrieved results are different In this case we change the IOP retrieval algorithm of the CoastColour processor to use the QAA algorithm RD 8 3 4 5 2 Procedure The procedure is the same as in Test 3 1 with the exception of step 6 In the text area Processor Parameters scroll down the XML content until you reach the element lt useQaaForlops gt false lt useQaaForlops gt and change its value to lt useQaaForlops gt true lt useQaaForlops gt Then follow the procedure as described in Test 3 1 and 3 2 for downloading the results 17 Calvalus Acceptance Test Plan E Brockmann Consult GmbH
7. 3 3 2 Procedure 1 Click on tab Level 2 Processing or any other production tab 2 Inthe component Temporal file filter select By date range Enter various dates the number of days displayed at the bottom of the component should change accordingly 3 Select By date list Add and remove dates the number of days displayed at the bottom of the component should change accordingly 3 3 4 Test 2 3 3 3 4 1 Aim Test that the spatial file filter component works as expected 3 3 4 2 Procedure I Click on tab Level 2 Processing or any other production tab 2 Inthe component Spatial file filter select By region 3 Select various regions in the list box regions in the map should be selected and zoomed to accordingly 4 Select various regions in the map regions in the list box should be selected accordingly 3 3 5 Test 2 4 3 3 5 1 Aim Test that the L2 processor components work as expected 3 3 5 2 Procedure 1 Click on tab Level 2 Processing or any other production tab The list box in the Level 2 processor component must not be empty It shall comprise at least the CoastColour processors L1P L2R and L2W version 1 3 2 Select different list entries Different default processor parameters shall be displayed in the text area of the Level 2 Parameters component 3 Select different list entries Different processor descriptions should be displayed at the bottom of the processor list box 3 3 6 Test 2 5 3 3 6 1 Aim Test
8. A QAA see below using as input the output of GAC Graph Processing Framework of BEAM Hadoop Distributed File System Input output Inherent optical properties Level 1 Level 2 Level 3 data CEOS classification Leading Edge Technology Small and Medium sized Enterprises National Aeronautics and Space Administration Ocean Biology Processing Group Open GIS Consortium Quasi Analytical Algorithm see RD 8 E Brockmann Consult GmbH Calvalus Acceptance Test Plan Abbrev Expansion RR Reduced resolution SoW Statement of Work UML Unified Modelling Language WPS Web Processing Service OGC specification 1 4 References The following documents are applicable to this document EL ajaa SME LET 2009 CalVal and User Services Offer A3067 in response to 1 0 15 09 2009 Invitation To Tender AO 1 6037 09 F VS LET SME Announcement of opportunities 2009 Cal Val and User Services Calvados Requirements Baseline ESA 1 2 30 06 2010 SME LET AoO 2009 Brockmann Consult Cal Val and User Services Calvalus Technical Specification ESA 1 2 31 03 2011 SME LET AoO 2009 Brockmann Consult Fomferra N The BEAM 3 Architecture http www brockmann consult de beam doc BEAM Architecture 1 2 pdf Brockmann C Fomferra N Peters M Z hlke M Regner P Doerffer R A Programming Environment for Prototyping New Algorithms for AATSR and MERIS iBEAM in Proceedings of ENVISAT Symposium 2007 ESRIN Frascati Italy Fomfe
9. KANANG NA KN ANNA KAN NGNE MANAN AN AN Kh 2 1 4 PRETO ETC EE RE a GE OE ER 3 1 5 DOCUMENT OVE OW aaa aa aa EE dad na a BAN kaja BA gag ag EG dala EE n GR GE a aaa diajak 5 2 EE eer ER EE OE EE 6 2 1 CAINS POr ie ee EE GE a D gg ga oases a ga EG Do NGE oe De 6 2 2 Calvalus Command line TOOIS ees aa Nai KERANGANG GENENG TER BE KANE PEREP BA KANEA TA Aa AANE EA ga ee E 6 Be PCC WS aas pana aaa N EO EE ER EE EE EN ER EE EE EE TE 8 3 1 PN o ef 21 9 ice Fast PI eee a Kan ngae gag aa aga Ka EE ee ee ER GE ee ee ie 8 3 2 Test Group 1 Web Application Accessibility ccccccesccccesseccccesseceeeeseceeeeeeceeeeeseeeseeneess 8 3 2 1 Feature Description testes es bei ee ig ie be eo ee bo ee ie be ve EG ER 8 3 2 2 EA ME OE OE TAN ANK EE TAN AT EE TAN RN AA 8 3 3 Test Group 2 COMMON COMPONEMMS ccccesccscccsccssccsccseccsseseceseeseceseesecessesecesseseceseeaees 9 3 3 1 Feature DES ErIDUGR ses SE AG E SENG KAG KA Bg AGA BNN E AG DEN KAN TAA ON aaa gg Ba gan 9 3 3 2 MSS 2 dee te eae vs E E OE GA sate PR ee ee ED Ne Ge A Re ed 12 3 3 3 TSE 22 oa EE PA EE EE TE AT EEE A KN NEG 12 3 3 4 GSE ie RE EO EE OE OE OE OE TAN A ENT 13 3 3 5 is TE E A A E aires eutecteesensuieareaeaesnee 13 3 3 6 TE TA D aa aaa aa EE EE 13 3 4 Test Group 3 Level 2 Processing L2 cccccccccssssecccceeeseeccceeeeeeececseeuseceesseueeeeeeesaeneeess 14 3 4 1 Feature DescriptiOn sis SE EE N ES ee Ge Ge ob GE e
10. Parameters lets user specify the processor parameters used for a L2 production The default set of parameters is processor dependent and read form the processor bundle s metadata Currently the Calvalus system uses various CoastColour processors comprising a radiometric correction and pixel classification L1P an atmospheric correction using neural networks L2R and a L2 IOP retrieval using neural networks L2W which includes a parameter switch to perform the QAA IOP retrieval L2W comprises the L1P and L2R steps and also their outputs A description of the L2W output product and tits processing parameters is given in annex A Note It is planned to also integrate other processors into the portal in the near future For example the NASA I2gen and ESA MEGS ODESA processors 3 3 1 4 Output Parameters The Level 2 and Level 3 processing production TERA types generate data niga __ product files The emen output parameters Oo DEAMOIMAP component is primarily F Perform staging step after successful production es ee die EE a output EO data file format Currently BEAM DIMAP NetCDF and GeoTIFF are supported Optionally users can specify an output directory this is Figure 9 Output parameters especially useful if a result shall serve later as input to another production type For example you could perform a Level 2 processing into a dedicated directory Then for Level 3 processing provide that directory as input path to the L
11. amp Snat H ila t Ri N L1 Fila yf EE ES gt Spa ins a WE sears o L1 Eila gt g wn ng ma a Bins gt Temp Bins werd na L3 Temp Binning d L1 Eila ad va Rins p p Temp Bins L2 Proc amp Spat Reducer Task j Mapper Task b TA Formatting Staging L2 Prac amp Snat ila Ri Lari gt L2 Prac amp Snat gt Spat ae um ila ft Rins LE L2 Prac amp Snat gt ppa ens Li Filo L es BE Dw Spa sad 1 File I gt Spat Ri L L2 Proc amp Spat Mapper Task s L3 Temn Binning Temp Bins L3 Temp Binning ws gt gt Temp Bins Reducer Task J _ TA Report Figure 19 TA production type in Hadoop 3 7 3 Test6 1 3 7 3 1 Aim Make sure TA works with most parameters set to default values The test will perform a TA on 7 years of MERIS RR data for the South Pacific Gyre region as used by the ESA Ocean Colour CCI project Sas A Brockmann Consult GmbH 3 7 3 2 Procedure 1 2 3 4 5 6 7 10 11 12 Click on the tab Trend Analysis Select the nput file set MERIS RR L1b 2002 2010 In the Temporal Filter component select No filter Select By region in the Spatial Filter component select South_Pacific_Gyre OC_cci In the component Level 2 Processor select MERIS CoastColour L2W version 1 3 In the component Level 2 Parameters leave defaults In the component Level 3 Parameters add the varia
12. kill lt PID gt Kills the production with given identifier PID lie lip Prints out usage help _34
13. parameters is given in annex A Note The current CoastColour L2W processor v1 3 runs only under BEAM 4 10 which will be released by the end of 2011 For this acceptance test the CoastColour L2W processor v1 3 and BEAM 4 10 SNAPSHOT are made available L2W user documentation is provided in the CoastColour Product User Guide RD 7 see also http www coastcolour org documents Coastcolour PUG v2 1 pdf 3 4 2 Realisation using Hadoop For each file in the possibly filtered input product file set the Calvalus Li file MEEN f ar N eta system will create a mapper task ona File L Cee po Laila ila Fila dedicated node in the Hadoop cluster E eee gt DE L1 Filo _ _ gt L2 Filo The Hadoop processing engine will try TE K L2 Processor O mie to select the node according to the Mapper Task location of the data in the cluster so Figure 11 Level 2 Processing using Hadoop that the tasks will most probably work data local No reducer tasks are required for Level 2 processing In its current configuration 20 nodes cluster and in the ideal case no other tasks running the Calvalus system can perform a L2 processing of 20 files 20 times faster than in sequence on a single computer 3 4 3 Test3 1 3 4 3 1 Aim Test that a Level 2 Processing can be ordered and the resulting production completes in a timely manner 3 4 3 2 Procedure 1 Click on tab Level 2 Processing 2 From the list box Pred
14. plain text XML file conforming to the WPS Execute operation request see http schemas opengis net wps 1 0 0 wpsExecute request xsd OPTION may be one or more of the following e errors Print full Java stack trace on exceptions q gulet Quiet mode only minimum console output c config lt FILE gt The Calvalus configuration file Java properties format Defaults to C Users Norman calvalus calvalus config B beam lt NAME gt The name of the BEAM software bundle used for the production Defaults to beam 4 10 SNAPSHOT C calvalus lt NAME gt The name of the Calvalus software bundle used for the production Defaults to calvalus 0 3 201109 copy lt FILES gt Copies FILES to calvalus home lt user gt before any request is executed Use character to separate paths in FILES deploy lt FILES gt BUNDLE gt Deploys FILES usually JARs to the Calvalus BUNDLE before any request is executed Use the character string gt to separate list of FILES from BUNDLE name Use character to separate multiple paths in FILES install lt BUNDLES gt Installs list of BUNDLES directories ZIP or JAR files on Calvalus before any request is executed Use character to separate multiple entries in BUNDLES uninstall lt BUNDLES gt Uninstalls list of BUNDLES directories or ZIP files from Calvalus before any request is executed Use character to separate multiple entries in BUNDLES S
15. 1 10 26 46 ENT Yearly cycle South Pacific Gyre Ea 490 mean pn 29 58 kb Man 19 Sep 2011 1 0 z6 49 GMT following screenshot on the right Yearly cycle South Pacific Gyre Kd 490 median png 65 8 kb Mon 19 Sep 2011 10 26 50 GMT i Yearly cycle South Pacific Gyre Ed 490 rade pa 53 6 kb Mon 19 Sep 2011 10 26 50 GMT The following table shows Yearly cyole South Pacifie Gyre Ka 290 situ pry 14 7 KE Mon 19 Sep 2011 10 26 49 GMT examples of the genareted plots Yearly cyole South Pacific Gyre conc chl mean pn 37 0 kb Mon 19 Sep 011 10 26 48 GMT Yearly eycle South Pacific Gyrre conc chl median on 48 6 kb Mon 13 Sep 2011 10 26 49 GMT You shall be able to look at the Yearly cycle South Pacific Gyre conc chl mode prn 3 kb Mon 19 Sep 201 26 49 GMT Yearly cycle South Pacifie Gyre cene chil sisan nag 16 2 kb Man 19 Sep 2011 10 26 46 GMT same when opening the request html 12 1 kb Mon 19 Sep 011 10 26 50 GMT request html file that is generated request xml 5 0 kb Mon 19 Sep 2011 10 26 50 GMT as part of the report Timeseries for South_Pacific_Gyre Yearly Cycle for South_Pacific_Gyre conc_chl_mean 0 050 4 sd 0 025 4 0 025 aes Z a l soas Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan 2003 jJan 2004 jJan 2005 Jan 2006 Jan 2007 jan 2008 Jan 2009 jan 2010 Jan 201 Time Time 29 Calvalus Acceptance Test Plan B Brockmann Consult GmbH Timeseries for South_
16. 75 6 kb Fri 16 Sep 2011 17 56 25 GMT L2 of MER RR 1PRACR20080601 104658 000026432069 00080 32699 0000 zip 81550 1 kb Fri 16 Sep 2011 17 58 27 GMT L2 of MER RR _ 1PRACR20080601 122734 000026432069 00081 32700 0000 zip 8937 3 kb Fri 16 Sep 2011 17 58 31 GMT L2 of MER RR 1PRACR20080602 101519 000026432069 00094 32713 0000 zip 59661 5 kb Fri 16 Sep 2011 17 59 41 GMT L2 of MER RR 1PRACR20080602 115555 000026432069 00095 32714 0000 zip 41510 9 kb Fri 16 Sep 2011 17 59 58 GMT L2 of MER RR 1PRACR20080603 095209 000021342069 00108 32727 0000 zip 43394 3 kb Fri 16 Sep 2011 18 00 14 GMT L2 of MER RR 1PRACR20080603 112415 000026432069 00109 32728 0000 zip 87044 0 kb Fri 16 Sep 2011 18 00 51 GMT request html 7 9 kb Fri 16 Sep 2011 18 00 51 GMT request xml 4 2 kb Fri 16 Sep 2011 18 00 51 GMT The second file in the list the 81 MB one above shall look like the following in BEAM VISAT 16 E Brockmann Consult GmbH Calvalus Acceptance Test Plan File Edit View Analysis Tools Window Help Babli Ble RES 1 L2_of MER_RR__1PRACR20080603_095209_000 da Of MER RR 1 Ka 490 b Metadata L Flag codings D Tie point grids E Bands H A iop conc a B conc_chl E chiSquare E K_min E Kd 412 E Kd 4a3 E Kd 490 E Kd 510 E Kd 560 E Kd 620 E Kd 664 E Kd 680 DU Z90 max E turbidity mo Pl 0 O v BOOB a ote faa B 1 reflec 12 Editor Sliders
17. A multiband quasi analytical algorithm for optically deep waters APPLIED OPTICS Vol 41 No 27 Bryan A Franz Sean W Bailey P Jeremy Werdell and Charles R McClain Sensor independent approach to the vicarious calibration of satellite ocean color radiometry APPLIED OPTICS Vol 46 No 22 1 Aug 2007 Bryan Franz Methods for Assessing the Quality and Consistency of 18 01 2005 Ocean Color Products NASA Goddard Space Flight Center Ocean Biology Processing Group http oceancolor gsfc nasa gov DOCS methods sensor_analysis_m ethods html Janet W Campbell John M Blaisdell Michael Darzi Level 3 SeaWiFS Data Products Spatial and Temporal Binning Algorithms SeaWiFS Technical Report Series NASA Technical Memorandum 104566 Vol 32 Aug 1995 K Barker et al MERMAID The MEris MAtchup In situ Database ARGANS Limited http hermes acri fr mermaid doc Barker et al 2008 MERMAID pdf 2008 NASA OBPG Ocean Color Level 3 Binned Products http oceancolor gsfc nasa gov DOCS Ocean_Level 3 Binned Data Products pdf CoastColour web site http www coastcolour org ECSS E ST 40C ECSS Space Engineering Software European Cooperation for Space Standardization ESA ESTEC Noordwijk The Netherlands Bryan Franz OBPG 2gen User s Guide http oceancolor gsfc nasa gov seadas doc l2gen l2gen html Web site of the ESA Climate Change Initiative http earth eo esa int workshops esa_cci intro html OGC Web Processing Ser
18. D AN A low value in the product indicates a higher success in the retrieval and that the conditions which have led to the measured spectrum are in sufficient agreement with the conditions and the bio optical model used in the simulations for training the neural network A value above a threshold of spectrumOutOfScopeThreshold default is 4 0 triggers the out of training range out of scope flag Quality flags dataset from Lib product CoastColour L1P pixel classification CoastColour L2R atmospheric correction quality flags CoastColour L2W water constituents and IOPs retrieval quality flags The following table lists the flag coding for the lip flags dataset Bit Name 1 CC LAND CC COASTLINE GEWE OUE Description Pixel masked as land Pixel masked as coastline Pixel masked as cloud CC CLOUD BUFFER Pixel masked as cloud buffer CC CLOUD SHADOW Pixel masked as cloud shadow 2 3 4 CC CLOUD SPATIAL Pixel masked by spatial cloud filter 5 6 31 Calvalus Acceptance Test Plan id Brockmann Consult GmbH 7 CC SNOW ICE Pixel masked as snow ice 8 CC LANDRISK Potential land pixel SJ NGI ER BOE TK Risk that pixel is under glint The following table lists the flag coding for the I2r_flags dataset Bit Name Description 1 Not used 2 ATC OOR Atmospheric correction out of range 3 TOA OOR TOA reflectance out of range 4 TOSA OOR TOSA reflectance out of range 5 SOLZEN Large solar zenith angle 6 ANCIL Missing OOR auxilia
19. In the component Level 2 Parameters leave defaults 24 E Brockmann Consult GmbH Calvalus Acceptance Test Plan 10 11 12 13 14 In the component Level 3 Parameters add the variable conc_chl leave default parameterisation namely aggregation method AVG weight coefficient 1 0 fill value NaN Add another variable Kd_490 leave default parameterisation namely aggregation method AVG weight coefficient 1 0 fill value NaN Leave Good pixel expression l2w_flags INVALID Leave stepping period 30 change compositing period to 10 Change spatial resolution to 3 km pixel target size should now be 1230 x 930 pixels Change supersampling to 3 pixels Press the Check Request button You should see the production request now Now press the Order Production button to submit the production request After a few seconds you Shall be delegated to the Manage Productions tab of the portal The production progress can be observed now You shall see the production state changing from SCHEDULED to RUNNING and finally after a few minutes to COMPLETE A Download button shall appear by that time After pressing it the portal shall redirect you to a personal space from where the new resulting report file a ZIP can be downloaded 3 6 3 3 Expected Results The generation of the Level 3 Calvalus staging area staging norman 20110918181623 L3_271lab00ad0861d product should be done after 6 minutes it shall take no longer than Filena
20. OESMMUS AA noman COMPLETED ODE AA COMPLETED Restar Download p rod u ctl on req u est IS L NT and L2 processor CoastColour LAW Level 3 production using input path RUNNING submitted and the order accepted by the eodata MER RR IF TO ow mu woo lalang Calvalus server the l TET production is given a Delete Selected unique ID and Figure 10 Production manager displayed in the production table Accidently submitted productions can now be cancelled while in scheduled or running state Productions that are not used anymore can be selected and then deleted The production request that was used for the production can be displayed at any time by clicking on a row in Production column 3 3 2 Test 2 1 3 3 2 1 Aim Test that the input file set selection component works as expected 3 3 2 2 Procedure 1 Click on tab Level 2 Processing or any other production tab The list box called Input file set shall not be empty 2 Select different list entries Their regular expression paths should be shown in the text field below the check box named Use alternate input path 3 Select different list entries The file sets start and end dates should be displayed in the start date end date text boxes below the radio button No filter in the temporal time filter 3 3 3 Test 2 2 3 3 3 1 Aim Test that the temporal file filter component works as expected sd 2e E Brockmann Consult GmbH Calvalus Acceptance Test Plan 3
21. Pacific_Gyre Yearly Cycle for South_Pacific_Gyre 0 275 0 275 0 250 0 250 0 225 vas j E c is id 0 200 v 0 200 E z 9 0 175 S 0 175 t o ol 2 0 150 Y 0 150 0 125 0 125 0 100 0 100 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Time Jan 2007 Jan 2008 jan 2009 Jan 2010 Jan 201 Time 2002 2003 2006 2007 2004 2005 2010 2008 2009 Jan 2005 Jan 2006 Jan 2003 Jan 2004 30 E Brockmann Consult GmbH Calvalus Acceptance Test Plan Annex Annex A CoastColour L2W Processor version 1 3 CoastColour L2W Output Product The following table lists LAW specific geophysical variables and datasets Name 10p a pig 443 10p a ys 443 10p bb spm 443 10op a total 443 K min Kd A EUrbadi LY A90 max cone chl Come ye sim chiSquare li flags Lie flags dat Tag l2w flags Description Absorption coefficient at 443 nm of phytoplankton pigments Absorption coefficient at 443 nm of yellow substance Backscattering of suspended particulate matter at 443 nm Total absorption coefficient of all water constituents at 443 nm Minimum down welling irradiance attenuation coefficient Downwelling irradiance attenuation coefficient at A where A is one of 412 443 490 510 560 620 664 and 680 Turbidity in FNU Formazine Nephelometric Unit Inverted value of k_min Chlorophyll concentration mg m Total suspended matter dry weight gm tsm_conc tsmConversionFactor pasi
22. Production Job The server side process as a result of an accepted Production Request The production job therefore has a limited lifespan Calvalus Acceptance Test Plan Brockmann Consult GmbH Term Production Scenario Production Step Processing Step Processor Definition An operational end to end scenario implemented into the Calvalus system The core system contains production scenarios for L1 to L2 Bulk Processing L2 to L3 Bulk Processing for a Match up Analysis and Trend Analysis A single step in a Production Scenario that usually operates on Product Sets An atomic transformation of an input into an output by a Processor that usually operates on single Products An implementation of an algorithm that transforms an input into an output 1 3 Abbreviations The following abbreviations are used within the document without repeated definition Abbrev API BC BEAM cal val CCI CEOS CPU DEM ECSS EO ESA FRS GAC GAC QAA GPF HDFS I O IOP L1 L2 L3 LET SME NASA OBPG OGC QAA Expansion Application Programming Interface Brockmann Consult GmbH Basic ERS amp Envisat A ATSR and Meris Toolbox calibration and validation ESA Climate Change Initiative Committee on Earth Observation Satellites Central Processing Unit Digital Elevation Model European Cooperation for Space Standardization Earth Observation European Space Agency Full resolution swath GKSS Atmospheric Correction
23. ame parameters as the ones described in the previous chapter However the time range for a meaningful analysis will be typically many months and the compositing period is usually significantly smaller than the stepping period For example the TA automatically performed by the OBPG for the SeaWiFS and MODIS ocean colour products uses a Level 3 Parameters stepping period of 32 days and a prm PEN Weight compositing period of 4 days The spatial NEE WON BN ki daa ad Kd 490 h AVG_ML 0 5 NaN resolution is fixed to 9 28 km the Ta supersampling fixed to 1 Good pixel expression lw _flags MVALID Stepping penod 32 days Spatial resolution dd km petal In opposite to L3 production type the Compositing period 4 days Supersamplng piini temporal bin cells for a compositing period Number of periods 99 days n pa arget heg zu pixels are all aggregated and averaged So every an Figure 18 TA parameters compositing period results in a single value for each variable forming a time series over the entire time range of the analysis 3 7 2 Realisation in Hadoop The TA production type is implemented the same way as the L3 production type with the exception that the temporal bin cells output by the reducer tasks are all averaged again So every compositing period results in a single value for each variable forming a time series over the entire time range of the analysis L2 Prac amp Snat 1 File gt 5 at Rins LIE L2 Prac
24. an 20111027155753_L3_33a543b5948465 Filename Size Last Modified L3 2008 05 01 2008 05 10 zi 60022 5 kb Thu 27 Oct 2011 15 26 27 GMT 10 8 kb Thu 27 Oct 2011 15 26 27 GMT 5 0 kb Thu 27 Oct 2011 15 26 27 GMT This is how the global L2W IOP L3 product shall look in BEAM VISAT Ml 1 conc chl mean C Users Norman Downloads L3_2008 05 01_2008 05 10 6 L3_2008 05 01_2008 05 10 dim Sessior File Edit View Analysis Tools Window Help Rani BIS SGBwe4e ORE wie P 1 conc chi mean 1 conc_chl_sigma not saved VISAT 4 10 SNAPSHO NEE Al amp Ei 2 GCP D A Wa ZeZecA2 A E a TT EA sa NAN had z amp d 1 L3 2008 05 01 2008 05 10 ly be a BH 1 conc chl mean ea Es sk Bands ee Em N ER E num_obs 5 i 4 SEE N Es e gi E num_passes sitar a ati KE Bn f E conc_chl_sigma A E Kd 480 mean 38 E Kd_490_sigma EE N l 7 E Editor Sliders Table F Discrete colors pp a oy Name conc_chl_mean 2 E D Unit null Min 0 0 ah dh Max 45 295 pie 1003 h K 2 PD N wee a 1 conc_chl_sigma fe i Products id Pixel Info mi RD TEA PP A 1 conc_chl_mean WGS84 DD No pos 08 43 55 367M of 413M F 27 Calvalus Acceptance Test Plan d Brockmann Consult GmbH 3 7 Test Group 6 Trend Analysis TA 3 7 1 Feature Description The TA production type is used to create time series of Level 3 data It has therefore the s
25. atter task is used during the staging process to L2 Prac amp Snat gt sal ari L2 Prac_ amp Snat gt Spat ns n ila aici L1 Filo yf EE Spat Rins 7 12 Proc amp Snat r Spat Bins a 1 Fila ma EE L2 Proc amp Spat d L1 File Binning gt Spat Bins Mapper Task aand L3 Temn Binning gt L3 Formatting Temp Bins Staging L3 File s L3 Temp Binning Reducer Task Figure 17 L3 production type in Hadoop collect all the latitude bin ranges parts and compile the final binned data product The Calvalus implementation of the OBPG binning algorithm is very efficient The binning scheme is a perfect use case for the Hadoop map reduce programming model Data locality is in most cases fully exploited Level 2 processing is performed on the fly no intermediate files are written 3 6 3 Test5 1 3 6 3 1 Aim Make sure L3 works with most parameters set to default values The test will process MERIS RR Level 1 data of 10 days in Summer 2010 to Level 2 ocean colour products and then create a binned Level 3 product for the North Sea region 3 6 3 2 Procedure 1 Click on the tab Level 3 Processing Select the nput file set MERIS RR L1b 2002 2010 In the Temporal Filter component select By date range enter 2010 08 01 to 2010 08 10 2 3 4 Select By region in the Spatial Filter component select northsea 5 Inthe component Level 2 Processor select MERIS CoastColour L2W v1 3 6
26. ble conc_chl leave default parameterisation namely aggregation method AVG weight coefficient 1 0 fill value NaN Add another variable Kd_490 or tsm_conc leave default parameterisation namely aggregation method AVG weight coefficient 1 0 fill value NaN Change Good pixel expression to l2w_flags INVALID amp amp conc_chl lt 100 Leave stepping period 32 and also compositing period to 4 resulting in 99 periods Press the Check Request button You should see the production request now Now press the Order Production button to submit the production request In the Manage Productions tab you shall now be able to observe the production progress of the new TA production 3 7 3 3 Expected Results The TA will compute 99 periods for Calvalus staging area staging norman 20110919115952 TA_2flab0Qad08622 the South Pacific Gyre region and thus output 99 data points Calvalus Acceptance Test Plan Filename Size Last Modified 9 2 kb Mon 19 Sep 011 10 26 46 GMT including mean and sigma for the ee eee ee ee ode 47 5 kb Mon 1 Sep 2011 10 26 50 GHT geophysical variables The TA 36 8 kb Mon 19 Sep 2011 10 26 50 GHT 12 4 kb Mon 19 Sep 011 10 26 49 GMT should ta ke no longer than 0 5 h 23 1 kb Mon 19 Sep 2011 10 26 48 GHT and shall not exceed 3 h The 31 7 kb Mon 19 Sep 2011 10 26 49 Get 21 1 kb Mon 1 Sep 2011 10 26 49 GMT ouputs shall look similar to the 14 0 kb Mon 19 Sep 201
27. cation This document describes how users interact with the system and what they have to expect in terms of processing performance and of downloadable results The acceptance tests described in the following are referred to as end to end system tests During development of the Calvalus system other types of extensive tests have been performed by the development team The tests included unit level testing on around 200 Java test cases and integration testing where7 Java modules continuously pulled from a versioning system were build unit level tested and integrated on a build server These are not described here 1 2 Terms and Definitions The following terms describe concepts used in this document with a specific meaning Though the terms may be common in the earth observation domain some of them may be used differently in other documents and contexts Term Definition Product Files EO data used for both input and output of a Processing Step A product may be represented by one or more physical files Product File Set In this document a product set denotes a named collection of references to Product files Production Request A request to the processing system to produce something A production request operates on one or more Product Sets and may produce a new Product Set Production Request An incomplete Production Request that users may use as prototype for new Template processing requests e g by altering single parameter values
28. ccccssssecceceeseececeeeeeeeeseeeeeneees 34 E Brockmann Consult GmbH Calvalus Acceptance Test Plan 1 Introduction Calvalus is a technology study that investigates the use of Hadoop Map Reduce massive parallel processing on distributed data for the improvement of the cal val cycle of processor development and validation It also supports the evaluation of the temporal stability of the instrument by processing large time series for trend analysis 1 1 Purpose and Scope This document describes the plan for the final acceptance test of the Calvalus system It is a deliverable in the ESA project 22869 09 NL VS The acceptance test comprises end to end testing of the system that has been implemented according to the user requirements baseline AD 1 and the system technical specification AD 3 The focus is on a selected set of production scenarios that are typical for the cal val cycle of instrument and processor validation e L1 to L2 processing of satellite data including match up analysis against in situ data e 1 to L2 to L3 processing including long term trend analysis Typically these scenarios are computationally expensive and time consuming if large datasets are used Calvalus addresses this challenge Therefore the acceptance tests described in this document provide the step by step procedures that demonstrate that the Calvalus system meets its initial challenge The technical approach is described in the technical specifi
29. cluster so that the tasks will most probably work data local The Calvalus system can process a 1000 Level 1 input files in a few minutes because the L2 processing is only performed on sub regions given by the point records and the macro pixel s size 20 E Brockmann Consult GmbH Calvalus Acceptance Test Plan 3 5 3 Test 4 1 3 5 3 1 Aim Make sure MA works with most parameters set to default values The test matches NOMAD in situ data with two outputs of the current version of the CoastColour processor conc_chl Kd_490 using MERIS RR data of May July 2007 comprising 1490 full orbit files The output comprises extracted data records and scatter plots for matching reference and EO data variables 3 5 3 2 Setup The NOMAD in situ data file is provided for the test The following shows an extract of this file It is a plain text file using TAB separators ID LAT LONG SOURCE TIME CRUISE CONC_CHL KD_490 1595 61 599 62 598 NOMAD 2000 02 22 13 30 00 amlr2000 0 283 0 04498 1596 61 45 62 299 NOMAD 2000 02 22 17 00 00 amlr2000 0 118 0 03131 1599 61 3 59 59 NOMAD 2000 02 24 13 35 00 amlr2000 0 389 0 04595 1601 61 449 58 3 NOMAD 2000 02 25 13 30 00 amlr2000 0 13322 1604 61 149 57 003 NOMAD 2000 02 27 18 00 00 amlr2000 0 08816 3 5 3 3 Procedure 1 Make sure the Match up Analysis tab is selected 2 From the list box Predefined file sets select MERIS RR L1b 2007 3 Inthe component Temporal Filter select By date range Enter Start dat
30. cord 3 5 1 2 In situ point data format The MA production type currently can handle two types of point data format BEAM placemark XML files and TAB separated plain text CSV files 19 Calvalus Acceptance Test Plan Brockmann Consult GmbH The BEAM placemark files can easily be used to extract point data records from the Calvalus EO data archive Users can create placemark files by using setting pins into images using BEAM VISAT and then exporting the pins into a file Calvalus expects in situ data to be provided as a CSV file The following rules apply to valid in situ CSV files e Empty lines and comment lines introduced by a character are ignored e The first non empty and non commented line is the header line It contains the TAB separated names of the columns The following names all letter case insensitive have a special meaning o LAT or LATITUDE provide the geographical WGS 84 latitude of the measurement s point location This column is mandatory it must exist o LON or LONG or LONGITUDE provide the geographical WGS 84 longitude of the measurement s point location This column is mandatory it must exist o TIME or DATE provide the measurement s data take time in the format yyyy MM dd hh mm ss This column is optional however it must exist for applying the max time difference criterion o Column names that are letter case insensitive equal to a variable
31. cted region using S region Figure 4 Input file set Alternatively users can specify an input path in a text box The common use is to specify the path of an output of a previous production here The path pattern is also a regular expression and may contain the references to the settings in the temporal and spatial filters the year month and day S yyyy S MM S dd and the name of the selected region using region 3 3 1 2 Spatial and Temporal File Filters The files determined by the input file set can be further limited by specifying a temporal filter comprising either a date range or a list of single days Single days are very useful for testing L2 or L3 processing on a small subset of files before ordering a larger number of files that may take some time to process Temporal Filter Spatial Filter No filter No filter global By region Manage regions Start date 2004 01 01 acadia Coast Karte Satelit Hybrid amazondelta CoastColour FE End date 2008 12 31 aia aka O Nonn K 2 By date range balticsea CoastColou Start date 2008 05 01 beibubay Coast benguela CoastColour End date 2008 07 31 capeverde CoastColou 5 ey adha By date list centralcalifornia CoastCok AB N E chesapeakebay 2008 06 02 chinakoreajapan CosstColou 2008 06 03 dome_c CoastColour greatbarrierreef CoastColour TRE gulfofmexico CosstColour Google Nutzungsbedingungen 92 days Figure 5 Temporal file filter Figure 6 Spatial file filter Th
32. e eat bales CC EIE false Terai lag NAD IED false FORNO true screening The average salinity of the water in the region to be processed The average temperature of the water in the region to be processed The arithmetic expression used for cloud ice detection The arithmetic expression used for land detection Toggles the output of water leaving radiance reflectance The arithmetic expression used for land detection Toggles the output of water leaving radiance reflectance The arithmetic expression defining pixels not considered for water processing If enabled IOPs are computed by QAA instead of Case 2 Regional Concentrations of chlorophyll and total suspended matter will be derived from the lOPs The lower bound of the valid value range of N The upper bound of the valid value range of A The lower bound of the valid value range of BB The upper bound of the valid value range of BB The lower bound of the valid value range of APH The upper bound of the valid value range of APH The upper bound of the valid value range of ADG The lower bound is always O If selected the source remote reflectances are divided by Pl 33 Calvalus Acceptance Test Plan is Brockmann Consult GmbH Annex B Usage of the Command Line Tool cpt version 0 3 usage cpt OPTION REQUEST The Calvalus production tool submits a production REQUEST to a Calvalus production system REQUEST must be a
33. e 2007 05 01 and End date 2007 07 31 In the component Spatial Filter select No filter global In the component L2 Processor select MERIS CoastColour L2W v1 3 In the component L2 Parameters leave defaults for time being gt mm p p In the component Match up Analysis Parameters you need to specify an in situ file first Therefore press Add button found below the list box n situ and point data files A dialog shall pop up prompting for a file to be uploaded Browse the file NOMAD conc_chl txt that comes with the test data distribution After pressing OK the file shall appear in the list box Make sure the new entry is selected 8 Leave all MA parameters to their default values except for Grouping column Set this to SOURCE so that the name NOMAD will be used in the generated MA report 9 Scroll to the end of the page and press the Check Request button A dialog box shall appear showing the production parameters 10 Now press the Order Production button to submit the production request After a few seconds you shall be redirected to the Manage Productions tab of the portal The production progress can be observed now You shall see the production state changing from SCHEDULED to RUNNING and finally after a few minutes to COMPLETE A Download button shall appear by that time After pressing it the portal shall direct you to a personal space from where the new resulting report file a ZIP can be downloaded 3 5 3 4 Expec
34. e compared against the reference measurement data Match up Analysis Parameters In situ and point data files NOMAD conc chlta Add Remove Good pixel expression The good Dixel expressio TRUE value the pixel will be used for further analysis For BEAM processors you usually don t need to specify it because BEAM produc Macro pixel size Maximum time difference Filtered mean coefficient Grouping column attached to their geo physical output variables mple conc_chl lt 50 AND Kd Good record expression filtered mean rouping column must be a name 5 3 15 SOURCE n the header of the selected pixels hours Figure 14 Match up analysis parameters The screening and the matchup strategy is based on the multi sensor approach for the on orbit validation of ocean colour satellite data products as performed by the NASA OBPG for MODIS SeaWiFS CZCS and MERIS ocean colour products RD 22 The inputs to the Calvalus implementation of the MA are e EO data product file e g MERIS scene full orbit swath e In situ reference data file comprising point records A point record comprises the point s geographical latitude and longitude a measurement time and any number of other attributes including reference variables e g CHL in situ measurements e Macro pixel size e Maximum time difference between EO observation and point record Lee E Brockmann Consult GmbH Calvalus Acc
35. e ge Ge ee NG Di 14 3 4 2 Realisation using Hado Oi ee a RE E IA KK De N eo eo 14 3 4 3 NESU DL anga an A etree recente A aa a a A sane a aa na aaa Na E a aaa 14 3 4 4 iss de VEE TE OR OE RE DE EE E ES 16 3 5 Test Group 4 Match up Analysis MA ees ees ee ee ee ee ee RR ee ee ee ee EE ee 17 3 5 1 Feature Description sis SEE SE EE EN ee ee ke NG Ge DR Ge ne Ge eN Die 18 3 5 2 Raalisation M HIJO eene ee oe ie aa a ee ee ee 20 3 5 3 TE cee Ee ee E ee eee 21 3 5 4 fs EE ET TE RE OE ee NK ee 22 3 6 Test Group 5 Level 3 Processing 3 aie ui ssie sd sie ie Se se ei be Ge Ge ee ie 23 3 6 1 Feature IDES CHOU ON eraser S 23 3 6 2 Raalisati m In HaIdOOD see E 24 3 6 3 det ad Dimer ern re E a ee ee ene 24 3 6 4 id ER AAN ER EE EE EE 26 Calvalus Acceptance Test Plan B Brockmann Consult GmbH 3 7 Test Group 6 Trend Analysis TA sesse SKEER EG DR De EG eg ee ee Ee oa 28 3 7 1 Feature DeseripUO sod ik EG oe OE Go Gie EG vd EG EG Go 28 3 7 2 Realisation led se EE Ee ER ee Ge oe Oe 28 3 7 3 NE do RE OE EE TAN E AA EE AEAT E E 28 ANOR e E EE A N 31 Annex A CoastColour L2W Processor version 1 3 sees ee ee ee ee ee ee ee ee ee EE ee 31 CoastColour L2W Output Prod cese nienie Er 31 CoastColour L2W Parameters cccccccccccsssssseecccccccueessececcecceseeesseeececeessaeasseeeceeeessuaaaeeeeeeeeesaaas 32 Annex B Usage of the Command Line Tool cpt version 0 3 cccc
36. e spatial filter is used not only to filter out files but also to create spatial subsets of the input data before further processing takes place Users can define their own regions by using the region manager not part of the acceptance test 10 E Brockmann Consult GmbH Calvalus Acceptance Test Plan 3 3 1 3 Level 2 Processor and Parameters The Calvalus system has been designed to be easily extended to new data processors developed using the BEAM Graph Processing Framework RD 2 One or more compiled processors are packed as Java archive files JARs in a Calvalus processor bundle and installed on the Hadoop cluster using the Calvalus cpt command line tool The processors that are currently installed are shown in the Level 2 Processor list Level 2 Processor Level 2 Parameters MERIS CoastColour L2W v1 2 SNAPSHCAE lt parameters gt MERIS CoastColour L1P v1 3 SNAPSHO SEn Na Saat pan Baka TO sauna 9 z lt doCalibration gt true lt doCalibration gt MERIS CoastColour L2R v1 3 SNAPSHO lt Whether to perform MERIS Smile effect correction gt MERIS CoastColour L2W v1 3 SNAPSHC lt doSmile gt true lt doSmile gt MERIS QAA IOP v1 1 SNAPSHOT lt Perform removal of detector to detector systematic radiometric differences in MERIS Lib data products gt i The arithmetic expression used for land detection gt Durchsuchen Figure 7 Level 2 processor Figure 8 Level 2 parameters The text area Level 2
37. efined file sets select MERIS RR L1b 2008 3 Inthe component Temporal Filter select By date list Enter following entries 2008 06 01 2008 06 02 2008 06 03 space separated 4 Inthe component Spatial Filter select By region Select region northsea In the component Level 2 Processor select MERIS CoastColour L2W v1 3 6 Inthe component Level 2 Parameters leave the defaults 14 Brockmann Consult GmbH Calvalus Acceptance Test Plan 7 In the component Output Parameters make sure Directory is empty File format is BEAM DIMAP and Perform staging step after successful production is on 8 Press Check Request The production request should be shown now If not please verify that the settings are correct If the settings are actually correct you shall see the production request parameters now 9 Press Order Production 10 After a few seconds the Manage Productions tab should be automatically selected If not Click it 11 Anew entry shall appear in the productions table 3 4 3 3 Expected Result The new entry in the production table under the Manage Productions tab shall look similar to the following screenshot L2 Processing Match up Analysis L3 Processing Trend Analysis Manage Regions ge P Production User sie ng d N Staging Result Status Time Status 20110916191816 L2_271ab00ad0861b Level 2 production using input path eodata MER_RR__1P RUNNING r03 2008 MM ddV N1 and L2 processor norman 0 00 24 UNKNOWN Cancel
38. ence variables e g CHL in situ measurements If a region of interest ROI is given and the current record point does not intersect the ROI s geometry skip record Compute the EO data product s pixel location at the record s point Compute the EO data product s pixel time If max time difference gt 0 and delta of pixel time and record time is gt max time difference skip record For all variables in the EO data product Extract n x n macro pixel area for all variables of the EO data product or if a Level 2 processor is specified for all output variables of the data product process to Level 2 For all pixel positions x y in the macro pixel collect the variable s value list If a variable value at x y is NaN skip pixel If a good pixel expression is given and it evaluates to FALSE skip pixel Collect value of variable Compute intermediate statistics from value list mean sigma If filtered mean coefficient FMC is given and gt 0 For all collected variable values x If mean FMC sigma lt x lt mean FMC sigma remove value from list Compute final statistics from value list min max mean sigma CV Create a new record by concatenating to the in situ reference data record the pixel location the pixel time and all data product s final variable statistics from value list min max mean sigma CV If a good record expression is given and it evaluates to FALSE for the new concatenated record skip re
39. eptance Test Plan e Filtered mean coefficient FMC e Good pixel expression and good record expression The good pixel expression is evaluated for each single pixel and is used to decide if the pixel contributes to the macro pixel It is evaluated in the context of the current L2 product So it can comprise any band and flag names that are contained in the generated L2 output The good record expression is evaluated in the context of the current record comprising all input fields read from the in situ data source and the extracted and aggregated macro pixels Thus the good record expression can contain references to all extracted variables and all their statistical parameters If the variable is lt name gt then lt name gt min lt name gt max lt name gt mean lt name gt sigma lt name gt cv lt name gt n are all valid references In addition to the standard BEAM maths expressions the good record expression also may contain the median function which has a variable arguments list median x1 x2 x3 3 5 1 1 Match up Algorithm In order to understand the effect of the various MA parameters the MA procedure is described here For each EO data product If a region geometry is given and it does not intersect with the data products swath coverage boundary skip EO data product For all records in the in situ reference data file a record contains geo lat lon point time and any number of other attributes including refer
40. evel 3 processing If left empty a temporary output directory will be used Finally users can select whether they want to perform a staging immediately after the processing has finished Staging is a process of copying and reformatting the output files to a user accessible location 11 Calvalus Acceptance Test Plan B Brockmann Consult GmbH 3 3 1 5 Check Request and Order Production All four production scenarios have two common buttons namely Check Request and Order Production located at bottom of each production tab Check Request Order Production Check Request is used to validate the production request and if it is valid display the production request that will used to order a given production Order Production will first validate the production request and if it is valid submit the order to the server 3 3 1 6 Production Manager As its name suggests Pacang Processing Staging Ki Producti Lier a Status Time Status the production RE EET Level 2 production using input path mMm a n age r IS u sed to eodala MER_RA_ IPMN 200E MMS d noTan COMPLETED 004 3 COMPLETED Restan Danan lasa JN and L2 processor Coast oiar Lowe manage scheduled me ed A Level 3 production input path N se running and completed eodataMER_RR_1P 03 201008 MMy S od ae Ai ai COMPLETED ajana Poena ANT and L2 processor aas okur LEW productions Once a ede Ee Lewel 2 production using input path 1 i eodata MER_RA_ 1PMOUVSO
41. ew browser window should open displaying some general project information By clicking on the tabs of the main menu bar the different portal pages shall appear o i ag _ 0 x http www brockmann consult de calvalus tr CBB preceding PA E Meistbesuchte Seiten 4 gt Geek And Poke BC Calvalus L SST_cci L OC_cci CC _ UML L CMS L SOCIS gt E Lesezeichen Calvalus About Calvalus Portal for Earth Observation Cal Val and User Services Input File Set Predefined file sets MERIS RR L1b 2004 MERIS RR L1b 2005 MERIS RR L1b 2006 MERIS RR Lib 2007 MERIS RR L1b 2008 KI Use alternate input path Path pattern eodata MER_RR__1P r03 S yyyy S MM S dd N1 The common use is to specify the path of an output of a previous production here The path patti expression any may contain and references to the settings in the temporal and spatial filters the and dey tyvyy SIMM dd and the name of the selected region using region MERIS FSG L1b CoastColour Dataset MERIS FSG L1b acadia Temporal Filter No filter Start date 2004 01 01 End date 2008 12 31 By date range Start date 2008 05 01 End date 2008 07 31 Spatial Filter No filter global By region acadia CosstColour amazondelta ConstCotour antaresubatuba CoastColour balticsea CoastColour beibu bay CoastColour benguela CosstColour caneverde Co niar Figure 3 Calvalus portal in Firefox 6 3 3 Test Group 2 Common Component
42. he Add button below the table of variables The list of Level 3 Parameters available variables is specific to the selected L2 processor a stan mmn conc_chi kal AMG ML 7 0 5 MaN The pixels used for the L3 Kd 490 ANG 0 5 NAN products must pass a test given Add Remove by th e goo d pix e expres si on Thi S Good pixel expression aw flags INVALID Stepping period 30 days Spatial resolution 9 277 kmipixel expression IS a BEAM ba nd maths Compositing period 10 days Supersampling 3 pixels expression that may contain all Number of periods la days Target width 4 320 pixels Target height 2 160 pixels the bands and flags contained in the L2 output products The time range used for generating Figure 16 Level 3 parameters the L3 output products is given by the Temporal Filter selection see chapter Common Components The frequency L3 output files are produced is determined by the parameter stepping period e g every 30 days The resulting number of L3 products is the number days of the total time range divided by the number of days given by the stepping period The actual number of input product days that are used to produce each L3 output file is given by the parameter compositing period which must equal to or less than the stepping period e g 4 days 8 days monthlies The default spatial resolution is 9 28 km per output pixel resulting in a grid resolution of 4319 x 2190 pixels for global co
43. in a Level 2 output product will create match up pairs for which scatter plots and regression parameters are computed in the MA Such columns are optional If no such pair exists the MA degenerates to a simple point data extraction process e All subsequent non empty and non comment lines are data records comprising values for each column e Values can be missing in this case two separators follow each other A missing numerical value can also be indicated using the token NaN not a number again letter case insensitive 3 5 2 Realisation in Hadoop Similar to the L2 i 1 Proc 8 N production type the Ela p OER YA OutnRecs L MA production L1 Fila_ gt gt gt prac g PA OupoRecs ki MA Output Gen scenario creates a pila m ER aprang ig Recs Reducer Task L1 Filo HA p OutnRecs mapper task for each L2 Proc amp re L1 File gt gt Matcher OutpRecs file in the possibly AE filtered input product N l file set on a dedicated es MA Report _Inp Recs node in the cs Hadoop cluster Figure 15 MA production type in Hadoop The mapper task reads in the in situ point data records and creates output records using the L1 input file processed to L2 All output records are passed to a single Reducer task which aggregates the records computes statistics and generates the plots Again the Hadoop processing engine will try to select the node according to the location of the data in the
44. me Size Last Modified L3 2010 08 01 2010 08 10 zi6 10335 53 kb sun 18 Sep Z011 16 22 13 GMI 12 minutes After pressing the request html 10 9 kb Sun 18 Sep 2011 16 22 14 GMT download button you shall see the request xml 5 0 kb Sun 18 Sep 2011 16 22 14 GMT following results in a new browser window The file on the list should have about 10 MB after downloading and unpacking and it shall look the following after opening in BEAM VISAT 25 Calvalus Acceptance Test Plan B Brockmann Consult GmbH beam 1 con Cc Ch _mea an MIS o AA P File Edit View Analysis Tools Window Help RaAls Bl8 GBB On EUW WS EZZ OAL a 1 conc_chl_mean 1 conc_chl_sigma 1 num_obs 1 num_passes x bs o Es B 1 conc_chl_sigma PM EL PRA 1 L3_2008 08 01_2008 08 10 ag Metadata E 1 conc chl mean E num obs E num passes ec E conc_chi_sigma Name conc_chl_mean T Unit null Min 0 01 Max 45 075 Eed ries A 4 More Options B 1 num passes amp Products m Pixel Info L TES AP Pp 93 61 1 A oe HE 1 conc dil mean WGS84 DD No pos 08 35 15 205M of 254M AO 3 6 4 Test 5 2 3 6 4 1 Aim The goal of the second L3 production type test is the production of a single global 10 day Level 3 product Around 150 full orbit MERIS RR products 82 GB will be used and they will be processed to ocean colour Le
45. rection tool developed by ACRI and childgen a MERIS AATSR subsetting tool developed by BC There is currently no user interface for the distributed processing of Unix executables other than via the command line Thus there are no user performed acceptance test for this type of tools other than a remote desktop demonstration on how these tools are operated Calvalus Acceptance Test Plan B Brockmann Consult GmbH 3 Acceptance Tests 3 1 Acceptance Test Plan The acceptance of the Calvalus system is based on the acceptance of the following test groups performed in the order they are given here e Test Group 1 Web Application Accessibility Test Group 2 Common Components e Test Group 3 Level 2 Processing L2 e Test Group 4 Match up Analysis MA e Test Group 5 Level 3 Processing L3 e Test Group 6 Trend Analysis TA e Test Group 7 Production Management Each test group is introduced by a feature description For the test groups that cover the production types L2 L3 MA TA a brief explanation is given for the production scenario realisation using the Hadoop Map Reduce programming model A test group comprises one or more test cases For each test its aim and purpose are provided the procedure to be followed by the user and a description of the expected behaviour and outputs In the descriptions of the procedure and the expectations the words shall and should are underlined in order to easily capture the condition
46. results of their productions At the time of this writing the Calvalus Portal is compatible with most internet browsers It has been developed using the Google Web Toolkit 2 3 and has been tested with Firefox 6 Chrome 13 and Internet Explorer 9 The version of the Calvalus Portal used for the tests in this document is 1 1 released as of 27 10 2011 2 2 Calvalus Command line Tools The Calvalus software distribution package comprises a number of command line tools which have been developed in order to maintain and operate the Hadoop processing system They are currently not accessible by end users because they need to operate on a Unix computer with direct access to the Hadoop cluster and HDFS file system E Brockmann Consult GmbH Calvalus Acceptance Test Plan The most important tool is cpt the Calvalus production tool A usage dump is provided in the Annex It is used for submitting production requests in form of XML files OGC WPS Execute operation request from the command line and for installing uninstalling versioned software bundles such as the Calvalus processing system BEAM and BEAM processors in the HDFS Other command line tools are used to install and distribute the Hadoop configuration and software to the cluster nodes and also to execute any Unix programs on the Hadoop cluster The Unix executables that have been used so far are l2gen the OC Level 2 processor from NASA s SeaDAS software package AMORGOS MERIS geo cor
47. rra N Brockmann C and Regner P BEAM the ENVISAT MERIS and AATSR Toolbox in Proceedings of the MERIS AATSR Workshop 2005 ESRIN Frascati Italy Sanjay Ghemawat Howard Gobioff and Shun Tak Leung The Google File System in 19th ACM Symposium on Operating Systems Principles Lake George NY http labs google com papers gfs html Jeffrey Dean and Sanjay Ghemawat MapReduce Simplified Data Processing on Large Clusters OSDI 04 Sixth Symposium on Operating System Design and Implementation San Francisco CA http labs google com papers mapreduce html Ariel Cary Zhengguo Sun Vagelis Hristidis Naphtali Rishe Experiences on Processing Spatial Data with MapReduce Lecture Notes In Computer Science Vol 5566 Proceedings of the 21st International Conference on Scientific and Statistical Database Calvalus Acceptance Test Plan B Brockmann Consult GmbH RD 7 RD 8 RD 10 RD 11 RD 12 RD 13 RD 14 RD 15 RD 16 RD 17 RD 18 RD 19 RD 20 RD 21 Management New Orleans LA USA http users cis fiu edu vagelis publications Spatial MapReduce SSDBM2009 pdf R Doerffer H Schiller MERIS Lake Water Algorithm for BEAM and MERIS Regional Coastal and Lake Case 2 Water Project Atmospheric Correction ATBD ESRIN Contract No 20436 June 2008 Zhong Ping Lee Kendall L Carder and Robert A Arnone Deriving 20 09 2002 inherent optical properties from water color
48. ry data 7 SUNGLINT High sun glint retrieved 8 INVALID _ Invalid pixels The following table lists the flag coding for the l2w flags dataset Bit Name Description 1 WLR OOR Water leaving reflectance out of range 2 CONC OOR Water constituents out of range 3 OOTR Spectrum out of training range 4 WHITECAPS Risk for white caps 8 INVALID Invalid pixels CoastColour L2W Parameters The following table lists the available processor parameters of the L2W processor Parameter Default Value Description GG Me a eTO true Whether to perform the calibration doSmile true Whether to perform MERIS Smile effect correction doEqualization true Perform removal of detector to detector systematic radiometric differences in MERIS Lib data products brightTestWavelength 8605 Wavelength of the band used by the brightness test in the CoastColour cloud screening brightTestThreshold 0 03 Threshold used by the brightness test in the CoastColour cloud 32 A Brockmann Consult GmbH Calvalus Acceptance Test Plan averageSalinity ave cere Em EE e cloudiceExpression landExpression outputReflec landExpression outputReflec iLO ya lucha xe Li xpices Ss Om useQaaForlops gaa aLowerBound qaa aUpperBound qaa bbLowerBound gqaa bbUpperBound gaa aphLowerBound gaa aphUpperBound gqaa adgUpperBound gaa divideBYyPI 3 ds Hip Plags cC CLOUD Lip Tlaga tk SNOW _ ICE Ie DE hacie Coen aii fals
49. s 3 3 1 Feature Description The aim of this group of tests is to demonstrate that common user interface components are used in the Calvalus portal work as expected The common components are shared among four production scenarios and they work in the same way There are seven common components which are described in the following tests Calvalus Acceptance Test Plan id Brockmann Consult GmbH 3 3 1 1 Input File Set An input file set comprises a list of EO data product files that are identified by a file path that may contain regular expressions wildcards A production scenario can only have a single file set as input The current file sets comprise MERIS RR for the years 2002 to 2010 and regional subsets for the South Pacific Gyre SPG and North Atlantic NA as used by the ESA Ocean Colour CCI project Each file set also knows the date range of its contained data Input File Set Predefined file sets LI Use alternate input path Path pattern eodata MER_RR__1P r03 S yyyy S MMV S ddy N1 MERIS RR Lib 2004 The common use is to specify the path of an output of a previous production here The path pattern is a regular MERIS RR L1b 2002 MERIS RR Lib 2006 MERIS RR Lib 2007 MERIS RR L1b 2008 MERIS FSG L1b CoastColour Dataset MERIS FSG L1b acadia expression any may contain and references to the settings in the temporal and spatial filters the year month and day Siwi MMM Sidd and the name of the sele
50. s for a successful test If a shall condition is not fulfilled during the test the test is considered unsuccessful Note At the time of this writing the web application performs an automatic logout after 120 minutes An immediate login afterwards is possible 3 2 Test Group 1 Web Application Accessibility 3 2 1 Feature Description The aim of this test group is simply verifying that the application is accessible using an Internet browser and that users can log in It is the precondition for all subsequent tests 3 2 2 Test1 1 3 2 2 1 Aim Test that Calvalus portal web application is up and running at a dedicated publicly accessible location and that users can sign in 3 2 2 2 Set up Use Firefox 6 Chrome 13 or IE 9 with JavaScript enabled A Brockmann Consult GmbH 3 2 2 3 Procedure 1 Point your browser to http www brockmann consult de calvalus The Calvalus log in screen shall be shown in the browser 2 Type in your user credentials given before Lan the tests are performed 3 2 2 4 Expected Results Calvalus Acceptance Test Plan Calvalus aaa Portal for Earth Observation Cal Val and User Services Username Noman Password sessss Figure 2 Login screen You shall now be signed into the portal and facing the user interface of the Calvalus portal similar to the screenshot below Your user name should appear in the upper right most corner By clicking on the About Calvalus link a n
51. ted Results The MA should not take longer than 6 minutes and shall not exceed 12 minutes The output of the MA is a report comprising the following elements 21 Calvalus Acceptance Test Plan B Brockmann Consult GmbH 1 CSV file comprising of measurement from all pixel within the macro pixels 2 CSV file comprising aggregated values mean and sigma for each macro pixels 3 XML report with attached XSD file allowing for viewing the XML in a browser 4 scatter plot of reference versus satellite measurements for each matched variable conc_chl Kd 490 The scatter plots shall look similar to the ones shown below conc_chl NOMAD Kd_490 NOMAD coastcolour processing 1 3 SNAPSHOT CoastColour L2W coastcolour processing 1 3 SNAPSHOT CoastColour L2W n 15 a 14 05290 b 0 45860 n 5 a 0 01540 b 0 60901 45 0 5 w 40 i p D a3 wuih 0 4 w 30 sot v L 26 prai a 0 3 5 tt oO a 20 wa D Id Le 5 0 2 a 10 Z ad w 0 1 Wah 0 of 0 0 a i 0 5 10 15 20 25 30 35 40 45 50 00 01 02 03 04 05 Og CONC_CHL in situ KD 490 in situ 3 5 4 Test 4 2 3 5 4 1 Aim Similar to Test 4 1 except that the MA parameters are changed in order to constrain the generated match up records 3 5 4 2 Procedure Same as in Test 1 but in Step 8 set the Good record expression to the value CONC_CHL lt 40 This should filter out all reference records whose in situ reference value for chlorophyll A is below 40 mg m
52. that the Check Reguest action validates the current production settings and if the settings are valid display the request parameters 3 3 6 2 Procedure L 2 Click on tab Level 2 Processing or any other production tab Use the browser to scroll down to the end of the current page and click Check Request Depending on the current setting the action shall either show an error message or show the parameters of the current production You can easily cause an error if you select a time range that is out of the time range of the selected product set Check Request shall show an error message 13 Calvalus Acceptance Test Plan Brockmann Consult GmbH 4 Fix the time range Check Request shall now show the parameters of the current production 3 4 Test Group 3 Level 2 Processing L2 3 4 1 Feature Description The production type Level 2 Processing L2 allows user to process a filtered input file set using a selected processor to an output product set If a spatial region filter is applied the input scenes are first tailored to the given region geometry thus the outputted product files is also a subset The result of the production is a zipped set of output files in a user selected EO data format currently BEAM DIMAP NetCDF GeoTIFF that can finally be downloaded by the user For the demonstration of the Calvalus system the CoastColour L2W processor will be used A description of the L2W output product and its processing
53. to the Calvalus system It is a simple JavaScript based web application that lets users order and manage Calvalus productions and finally download the produced results The name portal is justified by the fact that it provides users a portal to the actual processing system the Hadoop cluster comprising 20 Unix machines quad core 8 GB and 120 TB of data storage The Calvalus system currently hosts MERIS RR Level 1b data from 2002 to 2010 Using this data set users can order four different productions types 1 L1 to L2 processing 2 L2 match up analysis or point data extraction 3 L1 to L3 or L2 to L3 processing 4 L3 trend analysis The following snapshot shows the main menu of the portal MUN A WN aa Ca va U S About Calvalus Portal for Earth Observation Cal Val and User Services W SS EET L2 Processing Match up Analysis L3 Processing Trend Analysis Manage Regions Figure 1 Main menu of the Calvalus portal The input dataset is organised into product file sets which can be temporally and spatially filtered for all types of productions before they are passed as input to the processing system Temporal filters are realised as date range or data list while the spatial filter is based on geographical regions A dedicated region manager is used manage build in and also user defined regions After a production request has been submitted users can observe cancel stage productions and finally download the
54. vel 2 and to Level 3 binned files 3 6 4 2 Procedure 1 Click on the tab Level 3 Processing Select the Input file set MERIS RR L1b 2002 2010 In the Temporal Filter component select By range enter 2008 05 01 to 2008 05 10 2 3 4 Select No filter global in the Spatial Filter component 5 Inthe component Level 2 Processor select MERIS CoastColour L2W v1 3 6 Inthe component Level 2 Parameters leave defaults 7 In the component Level 3 Parameters enter the same values as for Test 5 1 but change spatial resolution to 9 28 km pixel target size shall now be 4320 x 2160 pixels Ee Press the Check Request button You should see the production request now 9 Now press the Order Production button to submit the production request It will now take a while before you are redirected to the Manage Productions tab of the portal but it should not take longer than a minute 26 E Brockmann Consult GmbH Calvalus Acceptance Test Plan 3 6 4 3 Expected Results The Level 2 processor L2W usually requires 20 minutes to process a full orbit of MERIS RR L1b product to L2 Ignoring computation time used for binning negligible compared to L2 processing the 150 L1b files would take 50 h when processed in sequence Calvalus should take no longer than 1 5 h and shall not exceed 3 h The output of the production shall be similar to the one of Test 5 1 The L3 output file is now 10 times larger 92 MB Calvalus staging area staging norm
55. verage Level 3 products Finally the supersampling parameter can be used to 75 Calvalus Acceptance Test Plan E Brockmann Consult GmbH reduce or avoid the Moir effect which occurs in output images if the binning grid is only sparsely filled by input pixels This situation usually occurs if the spatial resolution used for the binning is similar to the input pixel resolution The supersampling subdivides every input Level 2 pixel in n x n super sub pixels which all have the same values but different and unique geographical coordinates This way an input pixel may be distributed to more than one adjacent bin cell The binning algorithm implemented in Calvalus is the same that is used by the NASA OBPG for creating the SeaWiFS and MODIS ocean colour Level 3 products RD 11 3 6 2 Realisation in Hadoop As for L2 and MA the L3 production scenario creates a mapper task for each file in the possibly filtered input product file set on a dedicated node in the Hadoop cluster The mapper task reads in the input product processes it to Level 2 data and according to the binning parameters performs a spatial binning of the data The output of the mapper are spatially aggregated bin cells A number of reducer tasks are then responsible for the temporal binning at the individual bin latitude ranges They get the spatially binned outputs from all the mappers perform the temporal binning and output bin cells for each bin latitude range A special form
56. vice Specification 1 0 08 06 2007 http www opengeospatial org standards wps Case2R source code repository at https github com bcdev beam 1 4 3 ongoing meris case2 QAA source code repository at https github com bcdev beam 1 0 2 ongoing meris gaa BEAM user manual http www brockmann consult de beam 48 ongoing O 06 03 2009 E Brockmann Consult GmbH Calvalus Acceptance Test Plan RD 22 Sean W Bailey P Jeremy Werdell A multi sensor approach for the on orbit validation of ocean color satellite data products Remote Sensing of Environment 102 2006 12 23 RD 23 DUE CoastColour Product User Guide http www coastcolour org documents Coastcolour PUG v2 1 pdf 1 5 Document Overview After this formal introduction Chapter 2 introduces the user interfaces of the Calvalus software to be tested Chapter provides a detailed description of the tests to be performed AnnexA isa description of the output product and processing parameters of the Ocean Colour CoastColour L2W Level 2 processor Annex B is a usage dump of the Calvalus command line tool cpt Calvalus Acceptance Test Plan B Brockmann Consult GmbH 2 Overview This chapter introduces the user interfaces of the Calvalus software that are going to be tested and demonstrated namely the Calvalus Portal and the a set of tool invoked from a Unix command line 2 1 Calvalus Portal The Calvalus portal is the main user interface

Download Pdf Manuals

image

Related Search

Related Contents

Démocratie participative, démocratie délibérative  Uniden BC248CLT User's Manual    Philips MiniVac Handheld vacuum cleaner FC6142  SS-CHSW-DS9P-2 取扱説明書 V2.0  INSTALLATION AND OPERATING INSTRUCTIONS  Logic User's Guide Contents  Elinchrom FX 100 Camera Accessories User Manual  CONTENTS - Sab Satellite  Textil - Clinex Spain  

Copyright © All rights reserved.
Failed to retrieve file