SpectraProc User Guide
Contents
1. Before invoking the abundance settings the endmembers should be selected as described in 4 8 1 To display the abundance settings dialog select Spectral Mixing gt Abundance Settings All mixtures are listed in the Classes excl endmembers list box The abundances of the end members are set for each of these mixtures by 1 Selecting the mixture 2 Select the endmembers that are contained in the mixture one after another from the Available endmembers list box and shift them down into the Selected endmembers list box 3 For every selected endmember do select the endmember by clicking on it then enter the abundance in the provided box Click Save The total abundance is updated When defining the abundance of the last endmember click on Complement This automatically assigns the abundance to the last endmember by summing the total abundance to 1 Abundance Settings Classes excl endmembers Available endmemberz Selected 0 667 Abundance of endmemberz 360 OP selected endmermber Save Complement 1 Total abundance Figure 35 Abundance Settings dialog A Hueni UserGuide_V02 doc Version 0 2 16 06 2006 SpectraProc Page 27 User Guide of 43 4 9 Library Building A library is a collection of statistics data built for a certain configuration of the processing chain The Library status field in the main window indicates if a library has been built for the current process ing setting2. ber spectra vectors the unconstrained least squares solution for the abundances of the endmembers in the spec trum vector x To start the unmixing select Spectral Mixing gt Unmix For each mixture the unmixing results are displayed in the message window first stating the calculated abundances for each endmember then the abundances stored in the database and finally the absolute errors for each endmember abundance see example of such an output below Spectral Unmixing O69 OVA SCOP 0397 300V U UF simulated mixture abundances from database 0 67 0V_360P 0 33 360V_OP absolute errors 0 03 OVA 36 0P 0 02 30O0V 02 4 11 Adding new Sensors New sensors can be added to the database if their properties are known The sensors details must be entered in a tabulator formatted file as described below Two classes of sensors are supported Gaussian and Ratio see also 4 6 3 The header structure is identical for both sensor classes lt Sensor name gt lt Sensor description gt lt Column headers gt The sensor name should not include any of the following characters lt gt The column header line is not interpreted by the program and exists only for the convenience of the user preparing the sensor description file For Gaussian sensors the file body contains one line per sensor band lt Band gt TAB lt Average Wavelength nm gt TAB lt FWHM nm gt TAB lt Calibrated Y X gt For Ratio
3. Keshava N amp Mustard J F 2002 Spectral Unmixing IEEE Signal Processing Magazine 19 1 44 57 Landgrebe D 1997 On Information Extraction Principles for Hyperspectral Data Purdue Univer sity Papula L 1994 Mathematik fuer Ingenieure und Naturwissenschaftler Viewegs Richards J A 1993 Remote Sensing Digital Image Analysis Berlin Springer Verlag Savitzky A amp Golay M J E 1964 Smoothing and Differentiation of Data by Simplified Least Squares Procedures Analytical Chemistry 36 8 1627 1639 Shaw G amp Manolakis D 2002 Signal Processing for Hyperspectral Image Exploitation IEEE Signal Processing Magazine 19 1 12 16 Thenkabail P S Enclona E A amp Ashton M S 2004 Accuracy assessment of hyperspectral waveband performance for vegetation analysis applications Remote Sensing of Environment 91 354 376 University of Waikato 2005 WEKA http www cs waikato ac nz ml weka Vane G amp Goetz A F H 1988 Terrestrial Imaging Spectroscopy Remote Sensing of Envi ronment 24 1 29 A Hueni UserGuide_V02 doc Version 0 2 16 06 2006
4. e a Location Database Connections SpectralDB ode o a Stylesheet FGDCESA y x Contents Preview Metadata Name Type E band_range MySQL Table gt 16 Database Connections El derivative MySQL Table Add OLE DB Connection E derivative_calc_method MySQL Table Add Spatial Database Conr ES feature_space MySQL Table 9 E feature_space_type MySQL Table Y Address Locators library MySQL Table Ga GIS Servers 22 mixture MySQL Table A Search Results 3 pca_data MySQL Table 22 sensor MySQL Table 22 sensor_element MySQL Table 22 sensor_response_type MySQL Table 22 site MySQL Table 22 smoothing_filter MySQL Table 22 smoothing_filter_type MySQL Table species MySQL Table 22 spectrum MySQL Table 22 statistic MySQL Table 22 study MySQL Table 22 waveband_filter MySQL Table E wavweband_filter_range MySQL Table OLE DB Connection selected Figure 45 SpectralDB and tables in ArcCatalog 4 12 3 Adding Table Data to a Map To add table data to a map select Add Data in ArcMap and choose the SpectralDB under Data base Connections Figure 46 Then select the site table as the data source Figure 47 The sampling sites can then be displayed on the map by selecting Display XY Data and using the longitude and latitude fields of the site table as X and Y coordinates Add OLE DB Connection Add Spatial Database Connection Le SpectralDB odc Name SpectralD B odc
5. pb P b pb NTBI 4 6 5 3 PCT Principal components transformation requires as input the eigenvectors of a given dataset A princi pal component analysis PCA must be carried out before PCT can be applied to the data PCA results in eigenvalues and eigenvectors The eigenvectors are used in the transformation matrix G If only the first n components are to be used the transformation matrix is a sub matrix of G consist ing of the first n column vectors The dimension of the resulting feature space is therefore n y G x nx nxm mxl where m original size of data space n new size of data space equal to the number of selected components 4 6 5 4 Creation Modification and Deletion of Feature Space Definitions Based on the three basic feature space types new feature spaces can be created Existing feature spaces can be edited or deleted For any of these functions navigate to the Chain Settings gt Feature Space submenu Figure 27 A Hueni UserGuide_V02 doc Version 0 2 16 06 2006 SpectraProc Page 22 User Guide of 43 aa Statistics Spectral Mixing Help Waveband Filter Setup Feature Space Edit Loaded Spectr Copy and Edit El a y Delete Figure 27 Feature Space Submenu To create a new feature space select Copy and Edit or New in the Feature Space submenu New opens up an empty dialog for the definition of a new feature space Copy and Edit copies
6. 1 3 Setting up the Spectral Database Schema Tables and Users O Start the Query Browser enter localhost as Server Host root for User Name root for Password or your own password if you have chosen so spectral_db for Default Schema Click Yes when asked if you want to create this new schema O Run MySQL Administrator O Goto the User Administration tab and create a new user User name and password are both SpectraProc Edit the Schema privileges by adding SELECT INSERT UPDATE AND DELETE to the assigned privileges Assign Resources max_questions 0 max_updates 0 max_connections 0 This allows unlimited access If you want to limit the access select positive values O For the new user add localhost to the list of hosts from which the user SpectraProc can connect click on the icon under User Accounts to do this O Go to Restore and load the supplied Spectral DB_ XXXXXXX SQL file Select spec tral_db as the target schema This will create all tables and populate them 2 1 4 Network Access The SpectraProc application can access the database over the network if the following configura tions are performed O The IP addresses of all machines from which the user SpectraProc can connect are added to the SpectraProc privileges Alternatively a range of IP addresses can be con figured using a netmask see MySQL online documentation for details or a value of will allow acces
7. 12 Configure Data Source Name Connector ODBC Connector ODBC Configuration Data Source Name Spectral DB This dialog is used to edit a Data Source Name DSN Description Server 1022005 massey ac n2 User SpectraProc Password Database spectral_db v g Figure 41 MySQL ODBC Configuration dialog 4 12 2 Establishing a Database Connection in ArcCatalog Start ArcCatalog and select Add OLE DB Connection Figure 42 A Hueni UserGuide_V02 doc Version 0 2 16 06 2006 SpectraProc User Guide A ArcCatalog ArcInfo Database Connections File Edit View Go Tools Window Help Location Database Connection E O O a a E U a E E e F FGDC ESRI Stylesheet Contents Preview Metadata Name lt Add OLE OB Connection Add Spatial Database Connection Database Connections E Add OLE DB Connection i i 3 Add Spatial Database Conr Address Locators BACA GIS Servers i E Search Results Figure 42 Adding an OLE DB Connection in ArcCatalog In the Data Link Properties select Microsoft OLE DB Provider for ODBC Drivers click Next gt gt Data Link Properties Provider Connection Advanced All Select the data you want to connect to OLE DB Provider s MediaCatalogDB OLE DB Provider MediaCatalogMergedDB OLE DB Provider MediaCatalogWebDB OLE
8. 1300 1500 1700 1900 2100 2300 2500 Reflectance Reflectance Wavelength nm Wavelength nm Figure 14 An example of pre and post filtering of water noise bands The waveband filter regions are freely configurable for each study To modify the current filter set tings select Chain Settings gt Waveband Filter Setup Figure 15 Statistics Spr Waveband Filter Setup Feature Space Figure 15 Waveband Filter Setup menu entry Waveband Filter Setup Wiaveband filter nm Modify Delete Add default ranges Figure 16 Waveband Filter Setup dialog no regions defined The Waveband Filter Setup dialog lists the waveband filter ranges in the listbox on the left side Figure 16 To add a new filter region click New In the pop up window enter the lower and upper wavelengths in nanometres of the region Figure 17 After clicking on OK in this dialog the newly created filter range is shown in the Waveband Filter Setup dialog Figure 18 A Hueni UserGuide_V02 doc Version 0 2 16 06 2006 SpectraProc Page 16 User Guide of 43 Filter band definition Lower wavelength 1350 Cancel Upper wavelength 1470 MES Figure 17 Filter band definition dialog Waveband Filter Setup Wiaveband filter nm 350 470 nm Modify Delete Add default ranges Figure 18 A new waveband filter range added to the list of filter ranges To modify a
9. 46154 The error matrix written to a file contains the errors of omission and commission the total accuracy the producer and user accuracy and the minimum maximum and mean for both producer and user accuracy Table 2 Error matrix of DA using a 3 dimensional NTBI feature space Blackfern Cabbage tree Lemonwood _ Row Total ee Blackfern 11 00 0 00 0 00 11 00 100 00 Cabbage_tree 7 00 19 00 6 00 32 00 59 38 Lemonwood 0 00 1 00 21 00 22 00 95 45 Column Total 18 00 20 00 27 00 65 00 Producer Accuracy 61 11 95 00 77 78 78 46 Prod Acc Min 61 11 Prod Acc Max 95 00 Prod Acc Mean 77 96 User Acc Min 59 38 User Acc Max 100 00 User Acc Mean 84 94 A Hueni UserGuide_V02 doc Version 0 2 16 06 2006 SpectraProc Page 29 User Guide of 43 To run the DA with an independent data set choose Statistics gt Discr Analysis using indep set A dialog is displayed to select the study that contains the independent set Figure 38 Note that in order to work correctly the folder names of corresponding species in the training and in the independent study must be identical Cancel PosDep_ Mat hal anama hi arase Figure 38 Selection dialog for independent data sets 4 10 3 Principal Component Analysis Principal components analysis PCA is applied after the derivative calculation step l e the PCA is carried out for the current settings of waveband filtering sensor synthesizing and derivative calcula tion To run
10. As study name type your name_example e g Mikes_example The Species directory path is in this case the path pointing to the folder named Vegetation_example Set the number of samples per species for library inclusion to 10 Once the new study is created it will be automatically selected as the current study otherwise select it see 4 4 and import the spectra see 4 5 5 4 Get to Know Your Data Check the settings of the processing chain O Smoothing filter none O Current Sensor Hyperion O Derivative 0 Use the Export function see 4 7 to export the Hyperion synthesized spectra to a CSV file Processing Stage Synthesized Data Details All Spectra Species Restriction Include all species Names Folder Transpose Ticked Format csv aa Maa a This will create a file in the SpectraProc output directory The output file name is shown in the mes sage window Open this file by double clicking which should load it directly into Excel The columns displayed should be similar to the example shown in Figure 48 Column A contains the average wavelengths of the Hyperion bands The following columns contain the spectral data for each spec trum The column headers are the species folder names A Hueni UserGuide_V02 doc Version 0 2 16 06 2006 Page 38 of 43 SpectraProc User Guide Seles amp Xx ES Microsoft Excel Mikes_example_Hyperion_synth_all_species csv e File Edit View Tools Data 5 PLUS Sl
11. DB Provider Microsoft Jet 4 0 OLE DB Provider Microsoft OLE DB Provider For Data Mining Services Microsoft OLE DB Provider for Internet Publishing Microsoft OLE DB Provider for OLAP Services 8 0 Microsoft OLE DB Provider for Oracle Microsoft OLE DB Provider for Outlook Search Microsoft OLE DB Provider for SOL Server Microsoft OLE DB Simple Provider MSDataShape OLE DB Provider for Microsoft Directory Services Figure 43 Data Link Properties Provider tab Page 34 of 43 Figure 43 and In the Connection tab select the data source from the dropdown list and enter again the username and Data Link Properties Provider Connection Advanced Al Specify the following to connect to ODBC data 1 Specify the source of data Use data source name Spectral DB v Refresh C Use connection string Linn Bld 2 Enter information to log on to the server User name SpectraProc Password T Blank password Allow saving password 3 Enter the initial catalog to use OK Cancel Help Figure 44 Data Link Properties Connection tab A Hueni UserGuide_V02 doc password SpectraProc Figure 44 Click OK Version 0 2 16 06 2006 SpectraProc Page 35 User Guide of 43 The database is now listed in ArcCatalog and the tables are displayed when selecting the database Figure 45 1 ArcCatalog ArcInfo Database Connections SpectralDB odc File Edit view Go Tools Window Help amp
12. E 2 DAL lil 100 ce A Insert Format Window Help A IBlackfern Blackfern Blackfern Blackfern Blackfern Blackfern Blackfern 3 54E 02 3 10E 02 2 64E 02 2 69E 02 2 60E 02 3 83E 02 3 87E 0 Moa gt 1 Mikes_example_Hyperion_synth_al 3 70E 02 4 07 E 02 4 O5E 0 2 4 16E 02 4 20E 02 4 39E 02 4 72E O2 5 64E 02 7 SDE 02 9 94E 02 1 19E 01 1 338 01 1 30 01 1 32E 01 1 21E 01 1 14E 01 4 24E 02 3 42E 02 3 61 E 02 3 7 4E 02 3 56 E 02 3 90E 02 4 29E 02 5 16E 02 6 56E 02 9 08E 02 1 10E 01 1 23E 01 1 29E 01 1 248 01 1 14E 01 1 06 01 3 00E 02 4 16E 02 3 36E O2 3 50E 02 3 62E 02 3 7 4E 02 4 11E 02 5 132 02 r 15E 02 9 7 4E 02 1 19E 01 1 338 01 1 39 01 1 33E 01 22E 01 1 14E 01 2 02E 02 3 00E 02 3 19E 02 3 33E 02 3 45E 02 3 57 E 02 3 95E 02 4 56 E 02 6 32E 02 9 3BE 0 2 1 14 01 1 27 E 01 1 32E 01 1 27 E 01 1 16E 01 1 10E 01 2 FAE 02 2 ADE 02 3 05E 02 3 17 E 02 3 20E 02 3 37 E 02 3 656 02 4 ADE 02 5 90E 02 r ObE 0 2 9 51E 02 1 07 E 01 1 11E 01 1 07 E 01 9 01E 02 9 24 02 lt 110 3 98E 02 4 22E 02 4 50E 02 4 6E 02 4 BDE 02 4 95E 02 5 47 E 02 6 67 E 02 9 61E 02 1 31E 01 1 60E 01 1 80E 01 1 05 01 1 71E 01 1 52E 01 Draw gt Lg AutoShapes Y mM JOA al dd aL AY Ready Figure 48 Example of a SpectraProc CSV output 4 02E 02 4 25E 02 4 51E O2 4 bbE U 4 50E 02 4 94E 02 5 4bE Ul 6 56E 0 4 61E 02 1 31E 01 1 59E 01 1 70E
13. MEW SENSO So lod 30 AT E SY SUSI set rs chee a ac cetse a ae dk ease toed a ek 33 4 12 1 Setting up the Spectral DB as a Data SourC8 ooocccccccccconiccnncnnenccconnnnnnnnnnncnnonnnnnnnnnennnnos 33 4 12 2 Establishing a Database Connection in ArcCatalog ooccccccocccncnncccnonncnnncononnnononnnnns 33 4 12 3 Adding Table Data to a Map ooccccccccnnnonnccccnoconcnncnancnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnns 35 Be MU lcd ecc 37 5 1 A ties ate acted a tumciosmae sea ecncmebuomectaetemmtaaaten mente tawsceuduen eetacchacucesumncaide 37 5 2 Examine the Folder and Ple Structure sii aoe N 37 5 3 Creating a new Study and Loading the Spectra occcccccoonnccncccnnncconccononnconcnonannconcnonanncnncnnnnos 37 5 4 GettokKknow YOUR ID ala a o 37 5 5 Discriminant Analysis in PCT Feature Space oocccccccnonccnncccnonncnnononanccnncnnnnnnnnnnnnnnconnnnnnanennnoss 41 O RelGrencES sise 43 A Hueni UserGuide_V02 doc Version 0 2 16 06 2006 SpectraProc Page 5 User Guide of 43 1 Introduction Field spectroradiometry has become increasingly popular in the last few years The technology has advantages over conventional techniques allowing the non destructive sampling of objects and possibly enabling the user to gain critical information more quickly and cheaply As a result many scientists are now actively researching applications of hyperspectral sensing The operation of the instruments tends to be relatively easy and data are colle
14. Page 33 User Guide of 43 4 12 Linking to a GIS System If a GPS unit is used to record the spatial position of the sampling sites during the sampling process every site in the database contains the latitude longitude and altitude in WGS84 format The database tables can be accessed by a GIS system if ODBC connections are supported The connection is shown here on the example of ArcGIS 4 12 1 Setting up the Spectral DB as a Data Source Note that before the Spectral DB can be configured as a data source the MySQL ODBC driver available from the MySQL website must be installed on the system Start Data Sources ODBC from the Windows administrative tools In the New Data Source dialog Figure 40 select the MySQL ODBC Driver and click on Finish Create New Data Source Microsoft ODBC for Oracle Microsoft Paradox Driver db Microsoft Paradox Treiber db Microsoft Text Driver txt csv Microsoft Text Treiber txt cs Microsoft Visual FoxPro Driver Microsoft Visual FoxPro Treiber MySQL ODBC 3 51 Driver SOL Server lt Morte fa fe fe fe Po v core Figure 40 New Data Source dialog This brings up the ODBC configuration dialog Figure 41 Type in a name for the Data Source Name Set the Server to the address of the database server Enter the User name and Password SpectraProc Select the database schema containing the spectral data from the Database drop down list N Connector ODBC 3 51
15. Show of type Datasets and Layers lpr X Cancel Figure 46 Selecting the SpectralDB as data source A Hueni UserGuide_V02 doc Version 0 2 16 06 2006 SpectraProc Page 36 User Guide of 43 Look in SpectralDB odc al ZE band_range E sensor_element waweband_filter ZE derivative E sensor_response_type Wwaveband_filter_range ZE derivative_calc_method 22 feature_space El smoothing_filter 22 feature_space_type E smoothing_filter_type E species ZE mixture E spectrum E pca_data E3 statistic El study Name l site Show of type Datasets and Layers lyr Cancel Figure 47 Selecting the site table as data source A Hueni UserGuide_V02 doc Version 0 2 16 06 2006 SpectraProc Page 37 User Guide of 43 5 Tutorial 5 1 Overview The tutorial is based on an example dataset comprised of three plant species The folder containing the species folders is called Vegetation_ example 5 2 Examine the Folder and File Structure Open the folder Vegetation_example It contains three species folders Blackfern Cabbage tree and Lemonwood Open each of these species folders and examine the contents of the site directo ries contained in them Blackfern has only one sample site while Lemonwood and Cabbage tree have two Also browse inside the site directories to find the ASD binary files 5 3 Creating a new Study and Loading the Spectra Create a new study as detailed in 4 2
16. Table 1 Example of a JM and B output using a 10 dimensional DI feature space Blackfern Cabbage _tree Lemonwood Blackfern 1 93 1 90 Cabbage_tree 3 41 1 76 Lemonwood 2 99 2 11 A Hueni UserGuide_V02 doc Version 0 2 16 06 2006 SpectraProc Page 28 User Guide of 43 4 10 2 Discriminant Analysis Discriminant analysis DA uses a discriminant function to assign a class to samples In the case of the calibration set being subjected to DA the result is an assessment of the separability of the classes The DA is carried out in the currently selected feature space The DA result is thus also an assessment of the discriminating power of the feature space Note that the discriminant function used is the one currently selected in the main window Before running DA the library for the current pre processing settings must exist See 4 9 on how to build libraries To run the DA select Statistics gt Discriminant Analysis A dialog will be displayed to select the names to be used in the output Figure 37 Discriminant Analysis Select name type Hames C Common C Maori C Latin f Folder Figure 37 Discriminant Analysis dialog A message stating the file name of the error matrix and the overall accuracy is reported in the mes sage window similar to the example shown below Discriminant Analysis Error matrix written to file C Data MPhil Remote Sensing SpectraProc_output Error Matrix csv Overall Accuracy 78
17. endmember in the mixture For the process of unmixing refer to 4 10 4 The unmixing process needs to have a set of endmembers and a set of mixtures to be unmixed using the endmember information For this purpose species can be designated as endmembers If the spectral mixtures are the result of a controlled mixing experiment the abundances of the end members are usually know These known endmember fractions can be entered into the database for each mixture thus allowing the automatic calculation of the error involved in the estimation by the unmixing procedure 4 8 1 Endmember Designation To display the endmember designation dialog choose Spectral Mixing gt Endmember Selection Figure 33 a Helo Test Endmember Selection Abundance Settings Unmix Figure 33 Endmember Selection menu entry Initially all species will be listed in the Classes list while the Endmembers list is empty To desig nate a species as an endmember select the class by clicking on it then click on the arrow pointing A Hueni UserGuide_V02 doc Version 0 2 16 06 2006 SpectraProc Page 26 User Guide of 43 towards the right to shift this class into the Endmember list Similarly to move an endmember back to the Classes list select the endmember and click the arrow pointing towards the left Endmember Selection Classes Endmemberz ff S60P 360 _0P Figure 34 Endmember Selection dialog 4 8 2 Abundance Settings
18. file output if Folder name is specified as desired name type see 4 7 for more information on file export The species directories contain the site directories These can be named freely and the sites will be automatically numbered when read into the database Thus naming a site 1 or s1 is perfectly acceptable A Hueni UserGuide_V02 doc Version 0 2 16 06 2006 SpectraProc User Guide Page 9 of 43 The site directories contain all spectral files collected at this site They are auto numbered by the ASD capturing software Folders E 5 vegetation example A E 5 Blackfern E sitel E Cabbage_tree gt sitet E site E 5 Lemonwood 5 sitel 5 site 5 sites e El cabbage 002 cabbage 005 Ej cabbage 004 Ej cabbage 005 Ej cabbage 006 E cabbage 007 cabbage 008 Ej cabbage 009 Size JKB JKB JKB JKB JKB J KE JKB JKB JKB JKB Type 000 File 001 File 002 File 003 File 004 File 005 File 006 File 007 File 008 File 009 File Figure 2 Example of a directory structures holding spectral files A Hueni UserGuide_V02 doc Version 0 2 16 06 2006 SpectraProc Page 10 User Guide of 43 4 Operation 4 1 Overview 4 1 1 Dataflow A typical dataflow is illustrated in Figure 3 An ASD FieldspecPro spectroradiometer is used to cap ture the radiance of field objects A GPS connected to the field laptop records the spatial position of the field object Refle
19. is practically not usable for all bands because some wavelengths may have been filtered previously segmenting the spectrum In these situations the range is symmetrically reduced to avoid filtered regions 4 6 3 3 Supplied Sensors The sensors currently supplied with the software are Name Type Description ________ _ _ _____ Sensor as flown on EO 1 Effects a 1 1 copy of ASD data e no data reduction Opt_Hyperion Gaussian The Hyperion sensor according to the blueprint i e all sen sor elements are operating To change the current sensor select a sensor from the sensor drop down list box Figure 24 Current Sensor Figure 24 Selection of current sensor A Hueni UserGuide_V02 doc Version 0 2 16 06 2006 SpectraProc Page 20 User Guide of 43 It may be advisable to apply the downsampling only to smoothed data in order to avoid aliasing The combination of smoothing and downsampling is called decimation Fliege 1994 4 6 4 Derivative Calculation The derivative order is set to zero by default i e no derivative is calculated To change the order use the up and down arrows next to the box showing the derivative order Figure 25 If the order is changed to a non zero value the calculation method can be selected lterative calculation is based on the gradient of the linear curve segment between two wavebands b pb pb yA 121 pl i A b Ab where p b reflectance of band i A b wave
20. number of correctly classified spectra is contained in the diagonal elements The errors of omission are written in the off diagonal elements E g 6 Blackfern spectra were wrongly classified as Cabbage tree Close the DA output file select PCT_10 as feature space and repeat the process of library building and DA and inspect the DA output file once more The classification accuracy should have risen to 100 A Hueni UserGuide_V02 doc Version 0 2 16 06 2006 SpectraProc Page 42 User Guide of 43 Scree Plot 4 Eigenvalue Figure 56 Scree plot of the eigenvalues of the first 10 components Table 6 Error Matrix for DA on PCT_5 feature space Blackfern Cabbage tree Lemonwood _ Row Total rit racy Blackfern 12 0 0 12 1 00E 02 Cabbage_tree 6 20 0 26 7 69E 01 Lemonwood 0 0 27 27 1 00E 02 Column Total 18 20 27 65 Producer Accuracy 6 67E 01 1 00E 02 1 00E 02 9 08E 01 A Hueni UserGuide_V02 doc Version 0 2 16 06 2006 SpectraProc Page 43 User Guide of 43 6 References Bojinski S Schaepman M Schlaepfer D Itten K 2003 SPECCHIO a spectrum database for remote sensing applications Computers amp Geosciences 29 27 38 Elvidge C D amp Chen Z 1995 Comparison of broadband and narrowband red and near infrared vegetation indices Remote Sensing of Environment 54 38 48 Fliege N J 1994 Multirate Digital Signal Processing Chichester John Wiley amp Sons
21. of the smoothing effect of the synthesizing operation e g by plotting a noise spectrum smoothed minus interpolated synthesized data O Sort species numerically Sorts the species by interpreting the species names as num bers A Hueni UserGuide_V02 doc Version 0 2 16 06 2006 SpectraProc Page 25 User Guide of 43 Species restriction Name type selection selection File Export Xx Processing Stage Species Restiction Names C Raw Include all species C Common File level selection f Al spectra m Data averaging A teeta El Interpolabe Synth stage only L options Transpose E Mean per site Sort species numerically observations C Filtered Include only library relevant species Maori Processing stage Smoothed rambo C Lain for file export E Smhesized F Do rehas Folder C Derived I Auto numbe Transpose Observations as columns C Feature Space Include filtered bard Lere Data Details E Offset by E fio Study Include the filtered Formal band regions C Mean per species Offset each species by a given percentage Interpolate output by File format linear segments selection Sort the species names numerically Figure 32 File Export dialog 4 8 Spectral Mixtures Spectral mixtures occur when more than one endmember is present in the field of view If the end members are known the mixtures can be unmixed to give estimates of the abundance of each
22. sensors the file body contains 1 to many lines per band to model the response see also Figure 22 lt Band gt TAB lt Wavelength nm gt TAB lt Ratio gt A Hueni UserGuide_V02 doc Version 0 2 16 06 2006 SpectraProc Page 31 User Guide of 43 Examples of Gaussian and Ratio sensor description files are given in tables 4 and 5 A Hueni UserGuide_V02 doc Version 0 2 16 06 2006 SpectraProc Page 32 User Guide of 43 Table 4 Example of a Gaussian sensor Hyperion ETE O E S y o Hyperion Sensor as flown on BOL o Ao A asmasmfx OC 86576 a apo E A a y y oaen masaja O IS E o e aowa onaj a ez amsempro A poo _ _ segs amsmfro Table 5 Example of a Ratio sensor Landsat7 Landsat Landsat 7 Thematic Mapper Sensor Band Wavelength nm Ratio 1 435 O OLG 1 436 0 027 1 4377 0 048 1 438 0 094 1 439 0 167 1 440 0 297 1 441 0 459 1 442 0 605 1 443 Os 728 1 444 Us 109 To import a new sensor select Database gt Import Sensor In the Import Sensor dialog select the sensor response type and the input file by clicking on Browse Then click OK and the sensor data will be imported An according message will be displayed in the message window when the sensor is loaded Import Sensor Input file Cancel Browse Sensor Element Response Type fe Gaussian i Ratio Figure 39 Import Sensor dialog A Hueni UserGuide_V02 doc Version 0 2 16 06 2006 SpectraProc
23. y Blackfern 6 00E 01 Blackfern Blackfern 5 00E 01 Blackfern 4 00E 01 AAA IAN Blackfern y NY Blackfern 3 00E 01 yY dE 4 A Blackfern Blackfern 2 00E 01 Blackfern 1 00E 01 Blackfern A Blackfern 0 00E 00 P Cabbage_tree 0 00E 00 5 00E 02 1 00E 03 1 50E 03 2 00E 03 2 50E 03 3 00E 03 Cabbage_tree Cahbhane tree Figure 50 The same plot as above but rescaled to a reflectance of 1 0 Once the waveband filter is setup repeat the export function close the previously written file first in Excel and load it again into Excel and plot it The plot will look similar to the one shown in Figure 51 Note that now the noise regions are plotted as straight line segments For visualization pur poses this should be avoided as it suggests that data exists in the filtered regions Blackf 1 20E 00 An Blackfern Blackfern Blackfern 1 00E 00 Blackfern Blackfern Blackfern 8 00E 01 Blackfern Blackfern Blackfern 6 00E 01 Blackfern Blackfern Blackfern 4 00E 01 Blackfern Blackfern Blackfern 2 00E 01 Blackfern Blackfern Cabbage_tree 0 00E 00 0 00E 00 5 00E 02 1 00E 03 1 50E 03 2 00E 03 2 50E 03 Cabbage_tree Cabbage_tree Figure 51 Spectra after setting up the waveband filter regions To plot the filtered regions as gaps select the option Include filtered bands in the file export dialog I
24. 00 specification of file output drive and directory output_drive C output_dir Data MPhil Remote Sensing SpectraProc_output Hint access to SpectraProc is made easier by placing a shortcut to the executable on the desktop A Hueni UserGuide_V02 doc Version 0 2 16 06 2006 SpectraProc Page 8 User Guide of 43 3 Design of Sampling Experiments 3 1 Overview The data collected during sampling campaigns must be organised in a structured way in order to allow the automated import into the spectral database Section 3 2 explains the background of the structure used and section 3 3 gives a practical example of a directory and file structure that should be adopted for data collection Preferably sampling experiments should be designed to reflect the structure used by SpectraProc Alternatively existing data can be arranged to meet the requirements 3 2 Hierarchical Structure A hierarchical data structure that reflects the real world and the setup of sampling campaigns for vegetation is used This structure is derived from the following conditions 1 Reflectances of several different species are captured 2 In order to describe the in species variation several specimens of a species are sampled 3 The variability of the specimens is described by several measurements per specimen The spatial extent where a specimen is sampled is termed a sample site thus a species contains a number of sample sites The sites are numbered in the order o
25. 01 1 82E 01 1 60 01 Create an X Y plot The result should be similar to the one shown in Figure 49 Maybe surprisingly the plot shows not the expected spectra but some noise that reaches values of up to 140 To see the vegetation spectra rescale the Y axis to a maximum of 1 0 see Figure 50 The noise is caused by atmospheric absorption Thenkabail et al 2004 As first step this water band noise should be removed from the data before any further processing or analysis is carried out To remove the noise affected regions from the spectra open the waveband filter dialog see 4 6 1 and click on Add default ranges This adds three wavelength regions that are known to contain waterband noise 1 60E 02 1 40E 02 1 20E 02 1 00E 02 8 00E 01 6 00E 01 4 00E 01 2 00E 01 Blackfern Blackfern Blackfern Blackfern Blackfern Blackfern Blackfern Blackfern Blackfern Blackfern Blackfern Blackfern Blackfern Blackfern Blackfern Blackfern Blackfern Blackfern 0 00E 00 Cabbage _tree 0 00E 00 5 00E 02 a 1 50E 03 2 00E 03 2 50E 03 3 00E 03 Cabbage_tree 1 00E 03 Cahbbaae tree Figure 49 A plot showing mainly noise A Hueni UserGuide_V02 doc Version 0 2 16 06 2006 SpectraProc Page 39 User Guide of 43 Blackfern 1 00E 00 Blackfern 9 00E 01 f A Blackfern IN Blackfern 8 00E 01 FASANAN Blackfern ANVAN Blackfern 7 00E 01 AOIN Y Blackfern Nef
26. A Roba Nei ado a 10 4 1 3 Spectra OC Operation aa o a 11 4 1 4 Spectral Proc ssmg CONCEDI eset arith Steed kee tas 11 4 1 5 MSV MUST AG a dln a tet aN 12 4 2 Creating anew Study ins A 13 4 3 Deleting AM CXISUNG Studya a aa a aain 14 4 4 SEMA UE el oa rd 14 4 5 Male PP A 14 4 6 Processing Chain cts iris 15 4 6 1 Waveband Pine td a e a 15 4 6 2 TV OUI MING ica isis Bair ici 16 4 6 3 Sensor Synthesizing DOWNSAMpliNQ ooccccccccooncnncocnnoncnnnnnnnncnnnononnnnnnnnnonancnnncnonancnns 17 AOL Br Rao SENO dd adi 17 AOS EI A e UU 0 18 A A O A st dulciensaeeeetiies 19 4 6 4 DEnvalvo Caca o aL aaa e DO 20 4 6 5 Feature Space TransforMatiON ccccccoonccnncccnnnncnncnononcnnnnnonancnnnnnnnncnnnnonnnnrnnnnnnannrnnnnnnnns 21 O A A E O UME A 21 MOS A o II 21 As POr E N E a tua coatuotod aueesaden 21 4 6 5 4 Creation Modification and Deletion of Feature Space Definitions cccccccncnccnnnnnnnnnnnnnnnnnnoss 21 AOD Peatlire Space Selecion eree N dl 23 4 7 E ea a x OO Mtl AIE E A E A E EE ES EE AERA histatins basta atm iisiatee atm 23 4 8 SS A oe Ome ene N 25 4 8 1 Endmenbe Desi e o ados 25 4 8 2 Abundance Settings setos tia 26 4 9 ADAN A a o o Ls ol Stee 27 A Hueni UserGuide_V02 doc Version 0 2 16 06 2006 SpectraProc Page 4 User Guide of 43 AOL ANAIS SS tados 27 AO Separabllissc ie ne ta ra er e o co do 27 4 10 2 DISCATIN AAA ASIS A ci 28 410 3 Principal COMPONEN IS Sii od 29 ANO AMINO ai o ida 30 AWA Adang
27. PCA select Statistics gt PCA A message stating the output file similar to the one be low is displayed in the message window Principal Component Analysis PCA output written to file C Data MPhil Remote Sensing SpectraProc_output pca_output csv The two first components will be plotted as a scatterplot in a graphics window this plot is also known as scoreplot The pca_output file contains the eigenvalues proportions and cumulative proportions see Table 3 for an example The eigenvalues and eigenvectors are stored in the database Table 3 First ten eigenvalues and proportions of a PCA PC Eigenvalue Proportion Cumulative 1 2 211 0 944 0 944 2 0 097 0 042 0 986 3 0 019 0 008 0 994 4 0 007 0 003 0 997 5 0 005 0 002 0 999 6 0 001 0 001 0 999 7 0 000 0 000 0 999 8 0 000 0 000 1 000 9 0 000 0 000 1 000 10 0 000 0 000 1 000 A Hueni UserGuide_V02 doc Version 0 2 16 06 2006 SpectraProc Page 30 User Guide of 43 4 10 4 Unmixing The unmixing procedure implemented is experimental at this stage and works on Landsat7 data only Before starting the unmixing make sure that the current sensor is Landsat7 the study has the mixtures and endmembers defined and the abundances are entered for each mixture The model used is unconstrained Keshava and Mustard 2002 s sj s x where Xx spectrum vector to be unmixed L x 1 S endmember matrix L x M consisting of M endmembers with the columns being the endmem
28. Savitzky Golay coefficients for quadratic and cubic functions are the same and the same applies to orders 4 and 5 Savitzky and Golay 1964 Smoothing Filter Details Figure 20 Smoothing Filter Details dialog A Hueni UserGuide_V02 doc Version 0 2 16 06 2006 SpectraProc Page 17 User Guide of 43 The result is automatically filtered to remove artefacts that appear at the start and end of every valid waveband segment Figure 21 The new valid segment sizes are calculated by AU ome AU pos _ filter _ size Al omo Al neg _ filter _ size where Al Au lower and upper segment wavelengths Thus every segment looses information of filter_size 1 Smoothed Reflectance Curve of Pittosporum eugenioides showing smoothing artefacts Smoothed Reflectance Curve of Pittosporum eugenioides after the removal of smoothing artefacts an a va 1500 7 7 Wavelength nm 1500 Wavelength nm Figure 21 A smoothed signature of Pittosporum eugenoides before and after the removal of smoothing artefacts Generally bigger filter sizes remove more noise while higher polynomial orders fit the original data values better 4 6 3 Sensor Synthesizing Downsampling The sensor synthesizing downsampling is effectively a data reduction operation The response of the sensor selected as the current senso
29. Spectral Research Solutions SpectraProc User Guide Version 0 2 Date 16 06 2006 Status Draft Author A Hueni File UserGuide_V02 doc Pages 43 Classification Distribution SpectraProc Page 2 User Guide of 43 History Version Date Jauho Rema o oa 03 04 2006 06 04 2006 Modified MySQL installation 16 06 2006 Modified Tutorial for new example set A Hueni UserGuide_V02 doc Version 0 2 16 06 2006 SpectraProc Page 3 User Guide of 43 Table of Contents e MNO CUIG WIN io ici 5 2 Installation and Configuration ccooncccnnncicncncccocnncnnnnncnnnnncnannncnnnnrnnannrenananenans 6 2 1 DAD o de 6 2 1 1 malling MyS E o A 6 2 1 2 Installing MySQL Administrator and Query BrOWSE cccccccccccccncncnnnncnnnnnnnnnnnnnnnininnnnnos 6 2 1 3 Setting up the Spectral Database Schema Tables and Users oocoooocccnccconocccnncnnnancnnnnos 6 2 1 4 NetWOIKACCOS S samicantin roce dcs 6 2 2 SPCCIar MOG A PPICAIO Nalda 6 3 Design of Sampling Experiments occconncccnnncccncncccnnnccnnnnccnnnnnnnannnenannnenanenenans 8 3 1 A A eee aa TE Cee APT 8 3 2 Hierarchical Structure en rie ee to a oe eee ee 8 3 3 Directory Structure and Spectral FilesS oocccccccocnncnncccconnconnnonnncnnnonnnnncnnnnnnnancnnnononanennnnss 8 4 ODA OM asirio 10 4 1 Ne seme ete ee ane ane ae Se eR te a eS ee 10 4 1 1 DAAMOWI da A A AI ten ae nes reen Onn ae ne eee nee mene ere 10 4 1 2 File Oyster cei siren Cc i tO da daca th cath
30. a new Study To create a new study select Database gt Create New Study Figure 7 This brings up the new study dialog Figure 8 Database Chain Settings Statistics Sp Create New Study Import Spectra into current Study Import Sensor Figure 7 Create New Study menu entry Enter a study name of 45 character maximum and not containing special characters that can not be part of any Windows file name like lt gt The study description is optional and can contain any string up to 200 characters long The species directory path is a Windows pathname pointing to the directory that contains all spe cies folders of this study To set the path either type it in manually or select the Browse button that brings up a directory tree Figure 9 and select the appropriate directory The number of samples per species for library inclusion can be set to any positive number If a spe cies has less than the specified number of samples it will not be included in the statistics calcula tions when building libraries Once a new study is created it will automatically be selected as the current study Create a new study Study name My new study Study description i example Species directory path listratorDesktopWWegetation_examplel Browse Number of samples per 11 5 species for library inclusion Cancel Figure 8 New study dialog Browse for Folder Current Selection Ci Documents and Set
31. a study into the database make sure the correct study is selected Please refer to 4 3 on the process of selecting the current study Then select Database gt Import Spectra into current Study Figure 13 A popup window will appear indicating the loading progress Addi tionally the loaded species are listed in the message window Database Chain Settings Statistics Sp Create New Study Import Spectra into current Study Import Sensor Figure 13 Import Spectra menu entry A Hueni UserGuide_V02 doc Version 0 2 16 06 2006 SpectraProc Page 15 User Guide of 43 4 6 Processing Chain Settings 4 6 1 Waveband Filtering Waveband filtering is the process of removing unwanted regions of the spectrum All subsequent processing operations will then ignore these regions A typical example is the removal of water bands These water band regions are usually found around 1350 1440 nm 1790 1980 nm and 2360 2500 nm Figure 14 If contact probes are used the noise can be minimal because the atmosphere is effectively non existent Raw Reflectance Curve of Pittosporum eugenioides Filtered Reflectance Curve of Pittosporum eugenioides 1 00E 00 1 00E 00 9 00E 01 9 00E 01 8 00E 01 8 00E 01 7 00E 01 A 7 00E 01 6 00E 01 6 00E 01 5 00E 01 5 00E 01 4 00E 01 4 00E 01 y 3 00E 01 3 00E 01 2 00E 01 2 00E 01 HS 00E hi 1 00E 01 0 00E 00 300 500 700 900 1100 1300 1500 1700 1900 2100 2300 2500 300 500 700 900 1100
32. bers only species that have been designated as endmembers are written Only mixtures only species that are not designated endmembers are written oO The name that is used as species name can be one out of five choices Common common name is defined in database Maori Maori name as defined in database Latin Latin name as defined in database Folder name of the folder that contains the site directories of the species Auto number incrementing numbers are assigned to the species according to their or der in the database OOOO The current version of the software supports only one file level setting study l e all species of the current study are written to one single file Four checkboxes allow for the following operations O Transpose the observations are written as columns instead of rows This option can make the import of data into other programs easier O Include filtered bands this is for visualisation purposes only The regions that were re moved in the waveband filtering stage are written as empty cells i e when plotted the filtered regions will appear as gaps O Offset each species will be set off by a freely choose able percentage of 100 reflec tance This is to be used for visualisation purposes only O Interpolate this option is only available when the Synthesized stage is selected as output step The synthesized data Is interpolated to ASD data using straight line seg ments This is useful for the assessment
33. ctance and metadata are automatically saved on the field laptop in a binary file for every reading taken These binary files are transferred to a laboratory computer where they are read by SpectraProc and stored in the relevant tables in the spectral database GPS E Spatial data Reflectance oa _Reflectance _Binary file file 8 metadata Ene Co gt A Field ASD Field laptop Lab computer Spectral object Spectroradiometer database Figure 3 Dataflow and involved hardware 4 1 2 File System Interfaces SpectraProc provides input and output interfaces to the file system see Figure 4 Input file formats are ASD binary file as produced by the ASD FieldSpecPro Spectroradiometer ENVI Z Profiles that are signatures extracted from hyperspectral imagery in ENVI and sensor specifications in a proprie tary tabulator separated format ASD files can be imported into the database as part of a study or loaded into memory for classification against a study dataset ENVI Z Profiles can be loaded for classification only Sensor specification files are a way of defining new sensors in the database Output can be written in three data formats 1 CSV Comma Separated Values for import into various 3 party applications like spreadsheets or statistic packages 2 ENVI Spectral Library for import into ENVI and subsequent use for e g signature matching and 3 ARFF which is a special format used by WEKA WEKA is a collection of machine learning algorith
34. cted quickly However the interpretation of these data is not so simple The main issue when dealing with hyperspectral data is their dimen sionality which is the result of sampling a wide spectral range in very narrow bands This is in itself a problem because the influence of noise on narrow channels is much higher than on traditional broadband channels Hyperspectral data are more complex than previous multispectral data and different approaches for data handling and information extraction are needed Vane and Goetz 1988 Landgrebe 1997 Hyperspectral data are essentially multivariate data consisting of hundreds or even thousands of variables lt has been shown that more bands do not automatically imply better results Although the separability of classes does increase with growing dimensionality the classification accuracy does not follow this trend endlessly but will decrease at a certain point This is called the Hughes Phe nomenon and is caused by the ever increasing number of samples needed to build sound statistics if the dimensionality grows Landgrebe 1997 In practice this means that more samples must be collected to ensure successful statistical analyses It is necessary therefore to collect a large num ber of spectral data files each containing a hyperspectral spectrum The sheer number of files and variables can become overwhelming Interestingly very few studies concerned with hyperspectral data have ever mentioned how the data
35. dancy is created by oversampling i e the spectral signal is sampled at small enough steps to describe very narrow features that could be discriminating Shaw and Manolakis 2002 The redundancy and general noisiness of the data usually mean that certain pre processing must be carried out before any useful analysis can be performed SpectraProc bundled with the associated spectral database is a solution for the efficient storage and pre processing of field spectroradiometer data The system has been successfully used in studies concerned with New Zealand native vegetation soil properties and pastures A Hueni UserGuide_V02 doc Version 0 2 16 06 2006 SpectraProc Page 6 User Guide of 43 2 Installation and Configuration 2 1 Database 2 1 1 Installing MySQL O Download the installation wizard ZIP file for Windows from www mysal com or use the ZIP file mysql 4 1 14 win32 zip supplied on the installation CD Install the above ZIP file by running it Use standard configuration In the configuration section select o Install as windows service o Root password root or any other password but make sure you can remember it 00d 2 1 2 Installing MySQL Administrator and Query Browser O Download the windows install MSI files from www mysql com or use the ZIP files mysql administrator 1 1 2 win msi and mysql query browser 1 1 14 win msi supplied on the installation CD O Install both applications by running the above MSI files 2
36. dialog on the example of a NTBI feature space Waveband Definition Wavelength 1 550 Cancel Wavelength 2 468 Saes Figure 29 Waveband Definition Dialog A Hueni UserGuide_V02 doc Version 0 2 16 06 2006 SpectraProc Page 23 User Guide of 43 Feature Space Mame PET D Cancel Description E 10 principal components Dimension 10 HA Figure 30 PCT Feature Space dialog To delete an existing feature space make sure the currently selected feature space is the one to be deleted Then select Delete from the Feature Space submenu The current feature space is re moved from the database 4 6 5 5 Feature Space Selection The feature space selection is comprised of two panels a a selection of the feature space type offering DI NTBI and PCT and b a drop down list box that contains all feature spaces of the cur rent type see Figure 31 Feature Space Typez Sc Feature Spaces DG Figure 31 Feature space settings 4 7 File Export SpectraProc features a powerful file export that facilitates the analysis and visualisation of the data in third party products All output files are written to the directory The output is controlled by a single dialog see Figure 32 The file export dialog is displayed by selecting File gt Export Data can be exported at any processing stage Raw Filtered removal of noisy band regions Smoothed Synthesized Derived Feature Space transfor
37. f sampling At each site several readings are taken to capture the variation exhibited by the specimen in question A site therefore contains a number of spectra This leads to a hierarchical directory structure Figure 1 Spectrum 1 Spectrum 1 Spectrum 1 Spectrum 2 Spectrum 2 Spectrum 2 Spectrum n Spectrum n Spectrum n Figure 1 Hierarchical directory structure Although the term species is used it essentially represents the different classes found in a study These classes can either be assigned due to already existing classification systems for e g plants or minerals In other cases a hypothesis might exist that a group of objects can be separated into classes If so the setup of the experiment should mirror this hypothesis If no such assumption ex ists all objects can be put into the same class i e species and the identification of classes could then be carried out by e g cluster analysis 3 3 Directory Structure and Spectral Files The hierarchical structure is used for the directory structure that holds the spectral files Ideally this directory structure is setup when designing the experiment Figure 2 shows an example of a directory structure The main directory Vegetation example holds all species directories of the study This main directory is the folder that needs to be specified when creating a study in the database see 4 2 The names given to the species directories are stored in the database and are included in
38. f the data is plotted again the result will be similar to the one shown in Figure 52 Two other points are noteworthy a some reflectance values are higher than one and b one Cab bage tree spectrum seems to be an outlier The reflectances higher than one could be the effect of changing illumination conditions during the sampling or a result of the BRDF The outlier problem can currently not be solved automatically Two options exist a remove the concerned spectrum from the file system and then delete recreate and reload the study or b re A Hueni UserGuide_V02 doc Version 0 2 16 06 2006 SpectraProc Page 40 User Guide of 43 move the concerned spectrum from the database using SQL commands Future versions of the software may feature automatic outlier detection and elimination Blackfern 1 20E 00 Blackfern Blackfern Blackfern 1 00E 00 Blackfern Blackfern Blackfern 8 00E 01 Blackfern Blackfern Blackfern 6 00E 01 Blackfern Blackfern Blackfern 4 00E 01 Blackfern Blackfern Blackfern 2 00E 01 Blackfern Blackfern 0 00E 00 Cabbage_tree 0 00E 00 5 00E 02 1 00E 03 1 50E 03 2 00E 03 2 50E 03 3 00E 03 Cabbage_tree Cabbage_tree Figure 52 Waveband filtered data export with the include filtered bands option To generate stacked plots select the option Offset by in the file export dialog and set the per centage to 100 Then plot the data agai
39. for mu 0 and sigma 2 T T T i T T 10 8 6 4 2 0 2 4 6 8 10 Figure 23 Gaussian curve illustrating the FWHM measure A Hueni UserGuide_V02 doc Version 0 2 16 06 2006 SpectraProc Page 19 User Guide of 43 The coefficients used for the convolution operation are given by the Gaussian function 1 1 wavelength _ band _ i wavelenth _ band _ i 2 O c f wavelength _ band _1 y2 T O where wavelength _ band _i wavelength of the i th ASD band wavelength _ band _ j wavelength of the j th band of the sensor to be synthesized c the i th coefficient for the convolution operation The band convolution is calculated by u range GOR H range r u 8 J H range Ci U range where r the synthesized reflectance value of the j th synthesized band c the coefficient determined by the Gaussian function for the wavelength of the i th ASD band r reflectance value of i th ASD band range defines the range of values to be used for the band convolution symmetrically to the middle ASD waveband The middle ASD waveband is the one closest to the average wavelength of the j th synthesized band The nature of the Gaussian function effectively carries out a smoothing of the data The range of values used for the convolution is set to 3 times the standard deviation range 3 0O i e 99 74 of all contributing values are used Papula 1994 This range
40. had been organised and stored A further issue that is rarely addressed is the reusability of the data Reference data is usually com piled in so called spectral libraries The majority of the publicly available spectral libraries are dis tributed as physical files This has drawbacks such as low flexibility and low query performance Bojinski et al 2003 Another drawback of most libraries is their restriction in the number of spec tra per class In many cases only one reference spectrum is supplied This reduces any statistical analysis to first order statistics The use of average values may be useful in some circumstances however Landgrebe 1997 noted that the reduction of data to mean values results in a loss of in formation Second order statistics contain vital information about the distribution of data in spectral or feature space and should therefore be included in spectral data collections The time and effort that is spent in collecting spectral data combined with the characteristically large number of files makes it clear that spectral data should be well organised Otherwise valuable data can be lost or loses its value because of missing metadata Considering the above it seems logical to employ a database to store spectral data in a suitable form A further characteristic of hyperspectral data is the data redundancy It has been shown that neighbouring wavebands have a high degree of correlation Thenkabail et al 2004 This redun
41. length of band i p b b first derivative of linear curve segment between reflectances of band i and band i 1 The n th derivative is thus calculated by applying the above formula n times The SavGol method uses Savitzky Golay coefficients to calculate the derivatives In this case the derivative is actually the derivative of the approximating function at the wavelength of interest Thus the SavGol method automatically performs a smoothing of the data This can be useful when deal ing with noisy data where a single smoothing operation is not removing enough noise Both methods loose N data points per valid segment where N is the derivative order Derivative Derivative Dernyative Calculation method Calculation method Calculation method o 3 il Iterative e C Iterative z C SavGol EN T EEEa Figure 25 Derivative calculation settings zero derivative left first derivative iterative middle and first derivative SavGol method right For the SavGol method the filter size and polynomial order can be modified by clicking on the Edit button This brings up the Derivative Filter Details dialog Figure 26 The filter size and order are automatically restricted to meet the following minimal conditions for the calculation of the derivative polynomial _ order derivative _ order filter _ size gt max polynomial _ order 1 derivative _ order 1 Thus the minimal filter size depends on both the polynomial and the deri
42. med O O O Oa E A Hueni UserGuide_V02 doc Version 0 2 16 06 2006 SpectraProc Page 24 User Guide of 43 The processing for these stages depends upon the parameters that are defined at that point in time for the processing cascade If e g Synthesized is selected every spectrum is put through the cas cade of filtering smoothing and synthesizing and is then written to the file l e if a certain stage is selected all the preceding stages are automatically performed The data can be averaged either on site level or on species level If the site mean is chosen a col umn containing the site number will be added to the output file Selecting the All spectra option performs no averaging but writes every spectrum to the file The averaging is the last step of the processing e g if the Synthesized and Mean per species is selected then all spectra of each species are processed till the synthesizing stage and then averaged and written to the file Three file formats are currently supported O CSV Comma Separated Values O ENVI spectral library file O ARFF WEKA file format The species that are written to the file can be controlled by selecting on of four options O Include all species all species are written O Include only library relevant species only species that have equal or more than the minimum number of spectra are written The minimum number of spectra is set when defining a new study see 4 2 Only endmem
43. ms for data mining tasks University of Waikato 2005 CSV File 7 S is SpectraProc ENVI D T ENVI Z E Spectral Profile qn 5 Library File a fo ile 3 _ O Sensor ARFF Speci File fica ytions A Spectral DB Figure 4 File System Interfaces A Hueni UserGuide_V02 doc Version 0 2 16 06 2006 SpectraProc Page 11 User Guide of 43 4 1 3 SpectraProc Operation SpectraProc is study based e spectral data is grouped into studies Advisably a new study is created to hold data for every new sampling experiment The spectral data files of studies must be structured according to the hierarchical structure introduced in 3 This structuring allows the auto mated loading of spectra into the database Typically the operations carried out during the lifetime of a study are 1 Creation of a new study 2 Loading of spectra 3 Repeated pre processing and analysis Optionally in ongoing studies new data can be added to the database by simply selecting the load ing operation again Only new spectra will be added to the database in this case 4 1 4 Spectral Processing Concept The spectral database only stores the raw spectral data Further processing of the spec tra is performed at runtime and the results are held in memory Once a spectrum is loaded from the database it is put through a cascade of operations as shown in Figure 5 The result of every stage is saved in a sepa rate data struc
44. n The spectra of each species are offset by the same amount see Figure 53 If only the species mean is selected the resulting plot is similar to the ones shown in diverse journal articles see Figure 54 Note that when creating graphs in these cases the Y axis scale should not displayed as it makes physically not much sense reflectances bigger than 1 y J Reflectance offset for clarity 1350 1550 1750 1950 2150 2350 Wavelength nm Figure 53 Stacked plots offset by 100 showing all spectra per species A Hueni UserGuide_V02 doc Version 0 2 16 06 2006 SpectraProc Page 41 User Guide of 43 Blackfern Cabbage _tree Lemonwood gt gt 5 O _ O 5 z Y lt S O Cc iy 3 O o 250 450 650 850 1050 1250 1450 1650 1850 2050 2250 2450 Wavelength nm Figure 54 Stacked plot showing the mean per species 5 5 Discriminant Analysis in PCT Feature Space Discriminant Analysis DA is carried out in feature spaces As an example the data will be trans formed by a principal component transformation PCT The number of components used in the transformation has a direct impact on the classification accuracy achieved by DA as will be demon strated hereafter Select PCT as the current feature space type Note that the Feature Spaces dropdown listbox states that there exist no feature spaces of this t
45. n existing range select the range to be modified in the listbox and click Modify To delete an existing range select the range and click Delete To add the default ranges as defined in the configuration file click on Add default ranges see 2 2 on how to configure the default filter ranges 4 6 2 Smoothing Smoothing applies a smoothing function to the spectra The default smoothing filter type is None meaning no smoothing is carried out The smoothing type is selected using the drop down list box in the main window Figure 19 Two different Savitzky Golay implementations are offered in this list Moving Window MW and Fast Fourier Transformation FFT The output is exactly the same however due to internal overhead the MW method is currently the faster of the two Smoothing Filter Y none Savitzky Golay M Savitzku Golay FFT Figure 19 Smoothing filter tyoe drop down list box The processing details of the smoothing filter are set by clicking on the Edit button below the list box This brings up a dialog where the polynomial order and the filter window size can be changed Figure 20 The filter size is automatically incremented resp decremented in steps of two to keep the filter size and uneven number i e the number of points on the left and right of the point to be smoothed are equal The polynomial order sets the order of the curve that is used in the least squares approximation Note that the
46. ow is used to display processing and error information The listbox on top of the text output panel is used to display spectra files that are loaded directly into memory The Indiv Classify button under it classifies the selected individually loaded spectra against the current library The library status box on the top right of the screen indicates whether statistical information has been compiled for the current pre processing settings Study selection List of directly Classification button for directly Library status field loaded spectra loaded spectra Smoothing filter PP PS Connected to specteal_db on localhost SpectraProc mix File Library Database ChainNWettings Statistics Spectral Mixing Help Test Smoothing filter gt Y detail settings 7 button Current Study vaded Spectra NZ Native Plants y Native Plants MA i Library Status Mot ReBdy Smoothing Filter Sensor Indiv Classify none ha z OR Derivative order Current Sensor Hiypen on Derivative Nerves calculation method __ Calculation method Processing message window ie F R cr Derivative calculation detail settings button Feature Space Types Feature space C NTBI type C PCT Feature Spaces Feature space DGVI Classaher Classifier Mahalanobis vw discriminant function Ready Figure 6 Main window elements A Hueni UserGuide_V02 doc Version 0 2 16 06 2006 SpectraProc Page 13 User Guide of 43 4 2 Creating
47. r is calculated from ASD data The synthesizing of other sensor responses using ASD data is useful due to several reasons 1 Reduction of dimensionality 2 Direct comparison of airborne spaceborne sensor and ground data 3 Implicit smoothing of the data 4 Prediction and assessment of the usefulness of a certain sensor Two classes of sensors are supported Ratio and Gaussian 4 6 3 1 4 5 3 1 Ratio Sensors The sensor element function of these sensors is modelled by a number of known coefficients thus the synthesizing operation is simply a convolution of a defined wavelength region using these coef ficients An example of such ratios is shown for Landsat7 TM band 1 Figure 22 A Hueni UserGuide_V02 doc Version 0 2 16 06 2006 SpectraProc Page 18 User Guide of 43 Ratios for Landsat Band 1 Figure 22 Ratios for Landsat7 TM band 1 The convolution is calculated by uw_ C07 _ i lw_j q uw_j C i lw_ j where r the synthesized reflectance value of the j th synthesized band C the coefficient for wavelength r reflectance value of i th ASD band lw _ J lower wavelength of the j th band uw _ J upper wavelength of the j th band 4 6 3 2 Gaussian Sensors The sensor element response function of these sensors is best approximated by a Gaussian func tion The sensor elements are technically defined by the middle wavelength and the full width at half the maximum FWHM see Figure 23 Gaussian function
48. s To build a library select Library gt Build 4 10 Analysis 4 10 1 Separability Before running the separability analysis the library for the current pre processing settings must be built See 4 9 on how to build libraries The JM and B distance analyses are carried out in the cur rently selected feature space l e the discriminating power of feature spaces can be assessed by the separability analysis To run the separability analysis select Statistics gt Separability Report In the displayed JM B Dis tance dialog select the name and distance type and press OK Figure 36 A short message will be displayed in the message window looking similar to Separability Analysis Using JM and B distance JM and B output written to file C Data MPhil Remote Sensing SpectraProc_output JM and B output csv The output filename is shown in this message Note that only library relevant species are included in this report JME Distance Mamez Distance Type C Common C JM EA C Maori ee C Latin JM andE f Folder Figure 36 JM B Distance dialog The JM distance is listed in the upper triangular matrix the B distance in the lower triangular matrix see Table 1 The JM distance is asymptotic to 2 0 A JM distance of 2 0 indicates a complete separability and a distance of 1 9 a good separability Richards 1993 The B distance is not as ymptotic and can get infinite Inf due to limits in numerical precision
49. s from any possible machine O The port of the MySQL server 3306 by default is not blocked by Microsoft Windows 2 2 SpectraProc Application Copy the SpectraProc directory supplied on the installation CD to C Program Files Open the di rectory after copying it and open spectra_config txt Edit the configuration file as described below A Hueni UserGuide_V02 doc Version 0 2 16 06 2006 SpectraProc Page 7 User Guide of 43 O Server if the database is running on a machine different to the machine this copy of SpectraProc is installed on change localhost to the IP address of the database server If the database is access over the network also see 2 1 4 O Database if the database has been given a different name during the restore operation change the name of the database O Default filter settings these are the waveband filter ranges that are used as default val ues in the waveband filter setup see 4 6 1 More filter ranges can be added to the configuration file by adding a new line starting with filter_br for each range O Output drive and directory specify the output drive and the output directory All file out put is written to this directory This must be a valid i e existing pathname Configuration settings for spectraProc database settings server localhost user SpectraProc password SpectraProc database spectral_db default filter settings filter_br 1350 1440 filter_br 1790 1980 TLICeEr Dr 2500 29
50. the settings of the currently selected feature space into the dialog The settings can subsequently be changed before creating the new feature space in the database To modify an existing feature space select Edit All the above three commands invoke the same dialog shown here on the example of a NTBI fea ture space consisting of three individual NTBls Figure 28 The name of the feature space can be any alohanumeric string of maximal 45 characters the description can be any alphanumeric string of maximal 250 characters The bands listbox lists the band combinations used to calculate the indices E g index 1 uses bands 550nm and 468nm New band combinations are entered by click ing on New existing band combinations are modified by clicking modify Both commands bring up a dialog box to define the two bands Figure 29 To delete an existing band combination select the combination in the list and click Delete The dimension field in the Feature Space dialog is non editable for Dis and NTBls where it reflects the number of band combinations For PCT feature spaces the dimension field specifies how many components are to be used for the transformation In the given example the first 10 components will be used Figure 30 Feature Space Mame NDVIT 3 Description Pee Thenkabail et al 000 Dimension Bards nm 550 468 nm 550 682 nm 320 696 nm Hew Modify Delete Figure 28 Feature Space edit
51. tings Administr Desktop B My Documents E e My Computer A 31 Floppy 4 07 See Local Disk ic EL DYDICD Rw Drive Ds EF Control Panel H My Network Places al Recycle Bin LE Figure 9 Directory tree dialog A Hueni UserGuide_V02 doc Version 0 2 16 06 2006 SpectraProc Page 14 User Guide of 43 4 3 Deleting an existing Study To remove a study and all associated data from the database select this study as the current study then select Database gt Delete Current Study Figure 10 Before any data is deleted a warning will be displayed Figure 11 To delete all data of the current study click OK to abort the operation click Cancel The message window will list the involved operations of the deletion REA Chain Settings Statistics 5 Create New Study Delete Current Study Import Spectra into current Study Import Sensor Figure 10 Delete Current Study menu entry SpectraProc Warning The study Random Materials including all spectral data will be deleted Click OK iF you want to delete this study cana Figure 11 Deletion warning box 4 4 Selecting Studies To select a study to work with select the appropriate study from the drop down list in main window Figure 12 Current Study Random Materials Pos_dep_ outdoor Probe Rotation Dervative es eee ed Figure 12 Selection of current study 4 5 Loading Spectra To load the spectra of
52. ture in memory This allows easy file export of spectral data at any proc essing step Consequently the structure of this processing chain implicates that a change of processing parameters of any stage will influence the results of the subsequent stages A complete Spectral DB Load spectrum NS Raw data Wave band filtering Waveband set of pre processing parameters thus de oroc Gata scribes the processing from raw to feature space transformed data All processing settings are stored in the data N base Repeated processing of the data with ee the same settings will always result in the data same final data set NL Synthesized Derivative data calculation paa Derived data Feature Space Trans formation Data in Fea ture Space Figure 5 Spectral data processing cascade A Hueni UserGuide_V02 doc Version 0 2 16 06 2006 SpectraProc Page 12 User Guide of 43 4 1 5 User Interface The graphical user interface GUI is based on the structure of the processing cascade see Figure 6 The left side of the main window consists of controls for the selection of the study and the main settings for smoothing filter synthesizing derivative calculation feature space transformation and classifier discriminant function Processing details are entered in pop up windows which are ex plained in the relevant sections The text output panel in the middle of the main wind
53. vative order A minimal filter size of derivative_order 1 will result in the removal of n derivative_order number of points Derivative Filter Details Filter size Cancel Polpnormial order Figure 26 Savitzky Golay settings for derivative calculation A Hueni UserGuide_V02 doc Version 0 2 16 06 2006 SpectraProc Page 21 User Guide of 43 Note that too big filter sizes remove relatively lot of data because the derivative calculation takes place after the data reduction step sensor synthesizing downsampling E g a filter size of 5 ap plied to data downsampled by factor ten will remove 4 data points i e 40nm per segment 4 6 5 Feature Space Transformation Three types of feature spaces are implemented O Derivative Indices DI Oo Normalized Two Band Indices NTBI O Principal Component Transformation PCT 4 6 5 1 DI DGVI DGVIs Derivative Greenness Vegetation Indices are an example of a DI These indices are trying to describe the shape of the reflectance curve The DGVI calculation is based on the equations used by Thenkabail et al Thenkabail et al 2004 and Elvidge and Chen Elvidge and Chen 1995 DGVI 5 Abbi pba i m Ab where 0 b 6 first derivative of reflectance curve between b and b m n start and end band number of DGVI area b centre wavelength of band i band number Ab step width Di bj 4 4 6 5 2 NTBI Normalized two band indices are calculated by _ plb
54. ype see Figure 55 A PCT is only possible if a Principal Components Analysis has been carried out Perform PCA see 4 10 3 for details Feature Space Types Dl C HTEI 0 PET Feature Spaces No feature spaces of this wpe Figure 55 PCT feature spaces Open the PCA output file in Excel and plot the first 10 eigenvalues against the component number The result should be similar than the one showed in Figure 56 This graph is called screeplot The sharper the drop off in eigenvalues the better are the results that can be expected from the follow ing PCT Note that once the PCA has been performed the Feature Spaces dropdown listbox contains a selection of feature spaces The numbers at the end of the names are equal to the number of com ponents used in the feature space E g PCT_10 will effect a PCT using the first 10 components of the transformation matrix i e the data will be reduced to 10 variables dimensions Select PCT_5 as feature space The discriminating power of feature spaces can be assessed by carrying out a discriminant analy sis Before the DA can be executed the library for the current processing chain settings must be built see 4 9 how to build libraries Then subject the data to DA see 4 10 2 for details The result ing overall accuracy should be around 90 7 The DA results are detailed in the DA output file Er ror_Matrix csv see Table 6 The columns are the species that were classified The
Download Pdf Manuals
Related Search