Home

MUSE Pipeline User Manual

1. Doc VLT MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ISSUE Toue Date Date 2015 04 28 Page 65 of 132 continued from previous page Parameter Type Values Description default other ovscreject string der This influences how values are rejected when computing overscan statistics Either no rejec tion at all none rejection using the DCR algo rithm dcr or rejection using an iterative con stant fit fit ovscsigma double 30 If the deviation of mean overscan levels between a raw input image and the reference image is higher than ovscsigma x stdev stop the process ing If overscan vpoly this is used as sigma rejection level for the iterative polynomial fit the level comparison is then done afterwards with 100 x stdev to guard against incompatible set tings Has no effect for overscan offset ovscignore int 3 The number of pixels of the overscan adjacent to the data region of the CCD that are ignored when computing statistics or fits combine string sigclip average Type of image combination to use median minmax sigclip nlow int 1 Number of minimum pixels to reject with min max nhigh int 1 Number of maximum pixels to reject with min max nkeep int 1 Number of pixels to keep with minmax Isigma double 3 Low sigma for pixel rejection with sigclip hsigma double 3 High sigma for pixel rejection with sigclip losigmabadpix double 30 Low sigma t
2. Category Type Constraint min ARC raw required 1 MASTER_BIAS calib required 1 MASTER_DARK calib optional usually not used MASTER_FLAT calib optional TRACE_TABLE calib required 1 WAVECAL_TABLE calib required 1 BADPIX_TABLE calib optional usually not used LINE_CATALOG calib required 1 9 5 3 Recipe parameters Parameter Type Values Description default other nifu int 0 IFU to handle If set to O all IFUs are processed serially If set to 1 all IFUs are processed in parallel overscan string vpoly If this is none stop when detecting discrepant overscan levels see ovscsigma for offset it assumes that the mean overscan level represents the real offset in the bias levels of the exposures involved and adjusts the data accordingly for vpoly a polynomial is fit to the vertical over scan and subtracted from the whole quadrant ovscreject string der This influences how values are rejected when computing overscan statistics Either no rejec tion at all none rejection using the DCR algo rithm der or rejection using an iterative con stant fit fit Continued on next page Doc VLT MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ISSUE Toue Date Date 2015 04 28 Page 79 of 132 continued from previous page Parameter Type Values Description default other ovscsigma double 30 If the deviation of mean
3. 2 gt OMP_NUM_THREADS 24 MUSE_XCOMBINE_RA_OFFSETS 0 0000000 5 4920943e 05 0 00031381379 MUSE_XCOMBINE_DEC_OFFSETS 0 0000000 8 3338105e 05 7 2223397e 05 sorex log file scipost_combine log muse_scipost lambdamin 6900 lambdamax 7100 scipost_combine sof The product files created by this command are 1 gt ls 1 fits DATACUBE_FINAL fits IMAGE_FOV_0001 fits The final data cube covers the combined field of view of the individual input exposures but only the small wavelength band defined by the recipe parameters By default the recipe muse_scipost creates a white light field of view image which in the above case covers only the limited wavelength range 6 6 4 Combining Exposures using muse_exp_combine With the previous method one has to re run the full science post processing in order to get the data cubes for a different wavelength band or combination of input fields A slightly more flexible approach is shown here Assuming that the computer used to run muse_scipost is capable of processing a single exposure without re stricting the wavelength range In this case one can create a fully reduced pixel table for the first exposure using the following EsoRex command 1 gt cat scipost_l sof PIXTABLE_OBJECT_0001 01 fits PIXTABLE_OBJEC PIXTABLE_OBJECT_0001 24 fits PIXTABLE_OBJECT SMUSE_CAL LSF_PROFILE fits LSF_P
4. Category Type Constraint min OBJECT raw one type of raw file required STD raw one type of raw file required SKY raw one type of raw file required ASTROMETRY raw one type of raw file required ILLUM raw optional 1 MASTER_BIAS calib required 1 MASTER_DARK calib optional usually not used Continued on next page ESO MUSE Pipeline User Manual Doc VLI MAN ESO 14670 6186 Issue Issue 9 Date Date 2015 04 28 Page 88 of 132 continued from previous page Category Type Constraint min MASTER_FLAT calib required 1 TRACE_TABLE calib required 1 WAVECAL_TABLE calib required 1 GEOMETRY_TABLE calib required 1 TWILIGHT_CUBE calib optional BADPIX_TABLE calib optional 9 8 3 Recipe parameters Parameter Type Values Description default other nifu int 0 IFU to handle If set to O all IFUs are processed serially If set to 1 all IFUs are processed in parallel overscan string vpoly If this is none stop when detecting discrepant overscan levels see ovscsigma for offset it assumes that the mean overscan level represents the real offset in the bias levels of the exposures involved and adjusts the data accordingly for vpoly a polynomial is fit to the vertical over scan and subtracted from the whole quadrant ovscreject string der This influences how values are rejected when computing overscan statistics Either no
5. Parameter Type Values Description default other lambdaref double 7000 Reference wavelength used for correction of dif ferential atmospheric refraction The R band peak wavelength 7000 Angstrom that is usu ally used for guiding is close to the central wave length of MUSE so a value of 7000 0 Angstrom should be used if nothing else is known A value less than zero switches DAR correction off 9 10 4 Product frames The following product frames are created by the recipe SKY_CONTINUUM Default file name Description SKY_MASK Created sky mask IMAGE_FOV Whitelight image used to create the sky mask SKY_SPECTRUM Sky spectrum within the sky mask SKY_LINES Estimated sky line flux table Estimated continuum flux spectrum 9 10 5 Quality control parameters The following quality control parameters are available for the muse_create_sky products QC SKY THRI QC SKY LINE ES HOLD NAME OC SKY LINE QOC SKY LINE a FI OC SKY CONT FLUX Threshold in the white light considered as sky used to create this mask Name of the strongest line in group k AWAV Wavelength air of the strongest line of group 1 UX Flux of the strongest line of group 1 Total flux of the continuum QC SKY CONT MAXDEV Maximum absolute value of the derivative of the continuum spectrum Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline Use
6. Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU alae Date Date 2015 04 28 Page 118 of 132 Frame tags e GEOMETRY_CUBE muse_geometry Cube of the field of view to check the geometry calibration It is restricted to the wavelength range given in the parameters and contains an integrated image white over this range e DATACUBE_ STD muse_standard Reduced standard star field exposure e DATACUBE_ASTROMETRY muse_astrometry Reduced astrometry field exposure e DATACUBE_FINAL muse_scipost Output datacube muse_exp_combine Output datacube if save contains cube muse_scipost_make_cube Output datacube A 34 EURO3DCUBE Description Euro3D format See Format Definition Document RD6 for details Contrary to the examples in the Euro3D specs we use floats instead of doubles for the entries in the group table This is because the E3D tool is otherwise not able to read the values correctly This data format may be written alternatively to the common DATACUBE format if the parameter format is set to Euro3D or xEuro3D FITS extensions e E3D_DATA FITS table Column name Type Description SPEC_ID int Spectrum identifier SELECTED int Selection flag NSPAX int Number of instrument spaxels composing the spectrum SPEC_LEN int Useful number of spectral elements pixel SPEC_STA int Starting wavelength of spectrum pixel XPOS double Horizontal position pix YPO
7. Column name Type Description ifu int IFU number slice int Slice number within the IFU sensitivity double array Detector sensitivity relative to the reference unused should be 1 offset double Wavelength calibration offset refraction double Relative refraction index slit_width double Slit width bin_width double Bin width lsf_width double array LSF gauss hermitean width hermit3 double array 3th order hermitean coefficient hermit4 double array 4th order hermitean coefficient hermit5 double array Sth order hermitean coefficient hermit6 double array 6th order hermitean coefficient Frame tags e LSF_PROFILE muse_Isf Slice specific LSF parameters A 3 10 GEOMETRY_TABLE Description This file provides the relative location of each slice in the MUSE field of view It contains one table of 24x48 1152 rows one for each slice Other columns e g columns containing errors estimates of the slice properties xerr yerr may be present in this table but are ignored by the MUSE pipeline FITS extensions e FITS table Column name Type Description SubField int sub field IFU channel number SliceCCD int Slice number on the CCD counted from left to right SliceSky int Slice number on the sky x double X position within field of view pix y double y position within field of view pix angle double Rotation angle of slice deg width double Width of slice within field of view pix
8. Frame tags Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU isle Date Date 2015 04 28 Page 123 of 132 e GEOMETRY_TABLE muse_geometry Relative positions of the slices in the field of view A 3 11 SPOTS_TABLE Description This file lists all detections and properties of all spots the image of a pinhole at one arc line during geometrical calibration It is thought to be used for debugging of the geometrical calibration FITS extensions e FITS table Column name Type Description filename string Raw filename from which this measurement origi nates image int Number of the image in the series POSENC2 int X position of the mask in encoder steps POSPOS2 double X position of the mask mm POSENC3 int Y position of the mask in encoder steps POSPOS3 double Y position of the mask mm POSENC4 int Z position of the mask in encoder steps POSPOS4 double Z position of the mask mm VPOS double Real vertical position of the mask mm ScaleFOV double Focus scale in VLT focal plane from the FITS header arcsec mm SubField int Sub field number SliceCCD int Slice number as counted on the CCD lambda double Wavelength Angstrom SpotNo int Number of this spot within the slice 1 is left 2 is the central one 3 is right within the slice xc double x center of this spot on the CCD pix yc double y center of this spot on the CCD pix xfwhm double
9. Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU alae Date Date 2015 04 28 Page 111 of 132 continued from previous page Column name Type Description DEC double Declination deg filter string Filter name used for column mag mag double Object Vega magnitude mag Frame tags e ASTROMETRY_REFERENCE PRO CATG ASTROMETRY_REFERENCE Catalog of astrometry reference stars A 2 4 EXTINCT_TABLE Description This is a simple binary FITS table with the dependency of the extinction on wavelength The wavelengths should cover at least the MUSE wavelength range The atmospheric extinction values should be applicable for Paranal ideally for the night of observations FITS extensions e FITS table may appear more than once Column name Type Description lambda double Wavelength Angstrom extinction double Extinction mag airmass Frame tags e EXTINCT_TABLE PRO CATG EXTINCT_TABLE Atmospheric extinction table A 2 5 BADPIX_TABLE Description This is a FITS table with 24 extensions This is used in the low level recipes working on raw data Each extension lists known bad pixels of one CCD FITS extensions e FITS table may appear 24 times Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ae ue Date Date 2015 04 28 Page 112 of 132 Colu
10. continued from previous page Parameter Type Values default other Description crtype crsigma rc pixfrac Id format weight filter string double double double int string string string median iraf mean median 15 1 25 0 8 Cube Cube Euro3D xCube xEuro3D exptime exptime fwhm none white Type of statistics used for detection of cos mic rays during final resampling iraf uses the variance information mean uses standard mean stdev statistics median uses median and the median median of the absolute median deviation Sigma rejection factor to use for cosmic ray re jection during final resampling A zero or nega tive value switches cosmic ray rejection off Critical radius for the renka resampling method Pixel down scaling factor for the drizzle re sampling method Number of adjacent pixels to take into account during resampling in all three directions loop distance this affects all resampling methods ex cept nearest Type of output file format Cube is a stan dard FITS cube with NAXIS 3 and multiple ex tensions for data and variance The extended x formats include the reconstructed image s in FITS image extensions within the same file Type of weighting scheme to use when combin ing multiple exposures exptime just uses the exposure time to weight the exposures fwhm uses the DIMM information in the header
11. 9 13 3 Recipe parameters Values default other cube Parameter Description Type Select output product s to save Can contain one or more of cube output cube and associ ated images if this is not given no resampling is done at all or combined fully reduced and combined pixel table for the full set of exposures this is useful if the final resampling step is to be done again separately If several options are given they have to be comma separated save string resample dx dy dlambda string double double double drizzle nearest linear quadratic renka drizzle lanczos 0 0 0 0 0 0 The resampling technique to use for the final out put cube Horizontal step size for resampling in arcsec or pixel The following defaults are taken when this value is set to 0 0 0 2 for WFM 0 075 for NFM 1 0 if data is in pixel units Vertical step size for resampling in arcsec or pixel The following defaults are taken when this value is set to 0 0 0 2 for WFM 0 075 for NFM 1 0 if data is in pixel units Wavelength step size in Angstrom Natural in strument sampling is used if this is 0 0 Continued on next page Doc VLT MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ISSUE Toue Date Date 2015 04 28 Page 105 of 132 continued from previous page Parameter Type Values Description default othe
12. VLT MAN ESO 14670 6186 Issue Issue 9 Date Date 2015 04 28 Page 75 of 132 9 4 3 Recipe parameters Parameter Type Values default other Description nifu overscan ovscreject ovscsigma ovscignore combine lampwise sigma dres tolerance int string string double int string boolean double double double 0 vpoly dcr 30 sigclip average median minmax sigclip true 1 0 0 05 0 1 IFU to handle If set to 0 all IFUs are processed serially If set to 1 all IFUs are processed in parallel If this is none stop when detecting discrepant overscan levels see ovscsigma for offset it assumes that the mean overscan level represents the real offset in the bias levels of the exposures involved and adjusts the data accordingly for vpoly a polynomial is fit to the vertical over scan and subtracted from the whole quadrant This influences how values are rejected when computing overscan statistics Either no rejec tion at all none rejection using the DCR algo rithm dcr or rejection using an iterative con stant fit fit If the deviation of mean overscan levels between a raw input image and the reference image is higher than ovscsigma x stdev stop the process ing If overscan vpoly this is used as sigma rejection level for the iterative polynomial fit the level comparison is
13. multiple IMAGE_FOV pipeline recipe IMAGE_FOV recipe processing order association mandatory optional input raw data input external static data master calibration output product PIXTABLE_SKY intermediate reduced data output product D CUBE final output data Figure 2 3 The second stage of the MUSE data reduction cascade These recipes start from the output of the pre processing recipe muse_scibasic and combine the data from all 24 IFUs into a single fully reduced and calibrated data cube Again the Association map indicates the required and the optional input for each of the recipes Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU alae Date Date 2015 04 28 Page 20 of 132 3 Installation 3 1 System Requirements Please Read Carefully The processing of MUSE data is very demanding in terms of computing resources In particular it requires a machine with sufficient memory installed Less critical but still important is the number of available CPU cores and the amount of available disk space Because of the memory constraints the MUSE DRS is only supported on 64 bit platforms The recommended platform is a powerful workstation with a recent 64 bit Linux system The minimum system configuration is e 32 GB of memory e 4CPU cores physical cores e 1 TB of free disk space e GCC 4 4 6 or newer The recommended configuration of the target machin
14. Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ae Sue Date Date 2015 04 28 Page 55 of 132 slices 1 48 iterations 2 3 2D polynomial fit residuals limits 0 063554 0 089181 1000 EE ETT ett N m ie he ia 5 Bg Ry A Ee o E T LALA 500 1000 1500 2000 2500 3000 3500 4000 Figure 7 3 The graphical window showing the output of the muse_wave_plot_residual s tool plotting all slices of IFU 10 using the wavelength calibration residuals table see text for details to wavelength using the 1 parameter 1 gt muse_wave_plot_residuals 1 WAVECAL_RESIDUALS 10 fits In case one wants to look at only one slice one can use the s parameter with a slice number color cuts are adjustable using the c parameter with two numbers and one can study a different iteration by default the final iteration of the fit in each slice is selected using 1 and a positive integer For a more in detail inspection of the solution of a single slice one can use the muse_wave_plot_column tool This needs both the wavelength calibration table and the table with the residuals make sure to use the tables of the same recipe run and IFU It can be used on the data of a single slice parameter s or on a single CCD column c It is most useful when displaying the vertical axis as residuals using r Figure 7 4 shows the output of the command 1 gt muse_wave_plot_column s 12 r WAVECAL_TABLE 10 fi
15. thres double 5 8 Threshold for detection gap in factors of standard deviation Only used if er dcr combine string none none Type of combination to use Note that in most average median cases science exposures cannot easily be com minmax sigclip bined on the CCD level so this should usually be kept as none This does not pay attention about the type of input data and will combine all raw inputs nlow int 1 Number of minimum pixels to reject with min max nhigh int 1 Number of maximum pixels to reject with min max nkeep int 1 Number of pixels to keep with minmax Isigma double 3 Low sigma for pixel rejection with sigclip hsigma double 3 High sigma for pixel rejection with sigclip scale boolean true Scale the individual images to a common expo sure time before combining them saveimage boolean true Save the pre processed CCD based image of each input exposure before it is transformed into a pixel table skylines string 5577 339 6300 304 List of wavelengths of sky emission lines in Angstrom to use as reference for wavelength offset correction using a Gaussian fit It can contain multiple isolated lines as comma separated list individual shifts are then combined into one weighted average offset Set to none to deactivate skyhalfwidth double 5 Half width of the extraction box in Angstrom around each sky emission line skybinsize double 0 1 Size of the bins in Angstrom per pixel for the intermediate spectrum to d
16. EUROPEAN SOUTHERN OBSERVATORY ES Organisation Europ enne pour des Recherches Astronomiques dans 1 H misphere Austral 0 Europ ische Organisation fiir astronomische Forschung in der s dlichen Hemisph re VERY LARGE TELESCOPE MUSE Pipeline User Manual VLI MAN ESO 14670 6186 Issue 9 Date 2015 04 28 Es J Prepared MUSE Pipeline Team 2015 04 28 Name Date Signature Approved J Vernet R Bacon Name Date Signature Released J Vernet Name Date Signature This page was intentionally left blank Doc VLT MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ISSUE Toue Date Date 2015 04 28 Page 3 of 132 Change record Issue Rev Date Section Parag affected Reason Initiation Documents Remarks 1 2014 05 30 All First version 2 2014 07 01 1 6 3 3 References to Reflex documentation added 7 3 New pixel table handling tool added 3 2014 07 08 4 8 2 Added failed tracing recovery procedure for SV data 4 2014 11 21 6 6 3 6 6 4 Correct typing error in environment variable name 5 2014 11 27 Updated to version 1 0 of the MUSE pipeline 6 2015 01 23 2 6 3 Comments incorporated Section on legacy calibrations added 7 2015 03 11 All Software release number updated 8 2015 04 07 9 4 9 12 Updated to version 1 0 3 of the MUSE pipeline 7 Typing error fixed All Software release number updated 9 2015 04 28 All Software release number updated This page was intentionally
17. TWILIGHTm INPUTi TWILIGHTm INPUTi TWILIGHTm INPUTi TWILIGHTm INPUTi input list TWILIGHTm MASTE MEDIAN Median value of the combined exposures in IFU m TWILIGHTm MASTE TWILIGHTm MASTE TWILIGHTm MASTE TWILIGHTm MASTE TWILIGHTm MASTE IFU m the exposure Di dW mm MEAN Mean value of the combined exposures in IFU m STDEV Standard deviation of the combined exposures in FU m MIN Minimum value of the combined exposures in IFU m MAX Maximum value of the combined exposures in IFU m INTFLUX Flux integrated over the whole CCD of the combined exposures of TWILIGHTm INTFLUX Flux integrated over all slices of IFU m Computed using the pixel table of Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU Isley Date Date 2015 04 28 Page 87 of 132 9 8 muse_scibasic Remove the instrumental signature from the data of each CCD and convert them from an image into a pixel table 9 8 1 Description Processing handles each raw input image separately it trims the raw data and records the overscan statistics subtracts the bias taking account of the overscan if overscan is not none optionally detects cosmic rays note that by default cosmic ray rejection is handled in the post processing recipes while the data is reformatted into a datacube so that the default setting is cr none here converts the images from a
18. The first valid FITS extension with a FITS WCS so with either NAXIS or WCSAXES is used set override parameters for the output cube There are however several restrictions ESO MUSE Pipeline User Manual Doc VLI MAN ESO 14670 6186 Issue Issue 9 Date Date 2015 04 28 Page 127 of 132 e The axes have to be in the order RA 1 DEC 2 wavelength 3 e Only support gnomonic projection spatially TAN and linear or log air wavelength sampling are sup ported e A tilted 3rd axis is rejected e Only the primary WCS description is evaluated e The WCS transformation matrix has to be in CDi_j form PCi_j is not accepted e Floating point WCS parameters without dots in the FITS are not recognized Output cubes FITS NAXIS 3 written by the MUSE pipeline can be used as OUTPUT_WCS Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU fue Date Date 2015 04 28 Page 128 of 132 B Benchmarks B 1 The Reference System The specifications of the reference system used to obtain the benchmark results are summarized in the following table CPU 4 x Intel R Xeon R CPU E5 4620 0 2 20GHz 8 cores each CPU Cores 64 32 physical 32 logical System Memory 128GB Storage System 5 TB transfer rate 80 MB s B 2 Benchmark results The benchmark results were obtained running the MUSE DRS recipes with a maximum number of threads as shown in the tabl
19. a recovery procedure is described in Section 8 2 Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ae Isle Date Date 2015 04 28 Page 25 of 132 5 Data Description 5 1 Raw Data The MUSE raw data of all 24 IFUs is stored in a single FITS file as individual FITS extensions When MUSE raw data is retrieved from the ESO archive they will have the standard ESO archive file names which are made up of instrument identifier followed by a time stamp The time stamp corresponds to the contents of the FITS header keywords MJD OBS and DATE OBS respectively i e to the date and time when the exposure has been taken a difference of 1 ms between the file name and the contents of the keywords may be present In the case of MUSE the files returned by the archive are tile compressed which is indicated by the file name suffix fits fz instead of the regular fits suffix so that the file name of a MUSE raw data file will look like MUSE 2014 02 20T23 31 38 542 fits fz These tile compressed files may be unpacked using the funpack tool which is distributed as part of the CFIT SIO package see Section E for the web site If the MUSE DRS recipes are executed directly from the command line using EsoRex there is no need to uncompress the MUSE raw data files since the MUSE DRS will do this on the fly This is however not true for Reflex which works only on the uncompressed files The MUSE raw data fi
20. obtained directly on the observation itself Otherwise if the science exposure contains enough sky the sky subtraction can be done directly when the science exposure is processed The recipe requires that exposure s of a mostly empty sky field are available and have been pre processed as shown in Section 6 4 It also recommended use an Isf profile which has been created during the basic reduction or is taken from the static calibrations The Isf profile contains a model of the slice and wavelength dependent MUSE line spread function which is used to create a smooth sky continuum model by subtracting the fitted sky lines from the full sky spectrum The required calibrations are the list of sky lines the extinction table and the response curve and optionally the telluric correction The input set of frames and the EsoRex command line to run muse_create_sky using default settings are shown 1 gt cat sky sof PIXTABLE_SKY_0001 01 fits PIXTABLE_SKY PIXTABLE_SKY_0001 24 fits PIXTABLE_SKY SMUSE_CAL LSF_PROFILE fits LSF_PROFILE SMUSE_CAL STD_RESPONSE_0001 fits STD_RESPONSE SMUSE_CAL STD_TELLURIC_0001 fits STD_TELLURIC SMUSE_CAL extinct_table fits EXTINCT_TABLE SMUSE_CAL sky_lines fits SKY_LINES 2 gt OMP_NUM_THREADS 24 esorex log file sky log muse_create_sky sky sof The result of the above command are
21. A GR we a de eA ee 27 5 34 Combined Product Data ss er es esoe ma RR ER Ew ee 28 6 Data Reduction Cookbook 29 Doc VLT MAN ESO 14670 6186 ESO MUSE Pipeline User Manual pete Toue Date Date 2015 04 28 Page 6 of 132 6 1 Getting Started with EsORez co ss cana Ge ea edd ge d OSPR Oe Sawa Sake baad 29 62 Dalat orp aniv anon ok o ede ad ee ee Se A de Adc 30 6 2 1 Useful Header Keywords o ooe cao 6 444 Ba DR ta Grea a Chia 30 6 2 2 Data Classification and Association o e o 2006 ee be a Ee ee a es 31 03 Basic Reduction s sa aoe a ew ae a a ee ea Sa Qe da we a 33 G l Bab sgait erre ee aa eee aed oe oid a at oh ee eo 33 62 AGG 2 kok o ee Mad Pe ee oe AI Ee oes 34 033 Pit Pielde occiso ES A e er dS 36 63 4 Wavelength Calibration sc eo Oe ke ee a ee 36 6 3 5 Line Spread FUNCOM o 1464 4 ba bee ee a be we a 38 6 3 0 Instrument Geomel ac escocia ad a OG doe Bad a Harned Dasa bared 40 637 Ulumindtion Correction s s soss ace bk woe AG ee td ee ae we A a e 40 64 Observation Pre processing e ca ca co coda eRe ea a ba ea ba 41 65 Observation Post Processing a ec c 25 Fae ee Re aaa ee ee eS 42 63 Flux Calibration so s 24 6 eb owed es ON Pe a ea he ee ebay 42 2 Gky Riedie gs jagd ta bende aaa de wba da 44 G53 ASTOMGWY o a eae be ee eS ee 45 64 Science Observations a 2a o ee ee eS PREG a Re eee eS 46 66 Combining Exposures 2 se hone oe ee ee a 47 6 6 1 Correcting Coordinate Offsets 2 ooa ee
22. Description This recipe combines several separate flat field images into one master flat field file and traces the location of the slices on the CCD The master flat contains the combined pixel values of the raw flat exposures with respect to the image combination method used normalized to the mean flux The trace table contains polynomials defining the location of the slices on the CCD Processing trims the raw data and records the overscan statistics subtracts the bias taking account of the over scan if overscan is not none and optionally the dark from each raw input image converts them from adu to count scales them according to their exposure time and combines the exposures using input parameters To trace the position of the slices on the CCD their edges are located using a threshold method The edge detection is repeated at given intervals thereby tracing the central position the mean of both edges and width of each slit vertically across the CCD Deviant positions of detections on CCD rows can be detected and excluded before fitting a polynomial to all positions measured for one slice The polynomial parameters for each slice are saved in the output trace table Finally the area between the now known slice edges is searched for dark and bright pixels using statistics in each row of the master flat 9 3 2 Input frames Category Type Constraint min FLAT raw required 3 MASTER_BIAS calib required 1 MASTER_DA
23. FWHM in x direction on the CCD pix yfwhm double FWHM in y direction on the CCD pix flux double Flux of the spot as integrated on the CCD image bg double Background level around the spot dxcen double distance to center of slice at vertical position yc posi tive right of center pix twidth double trace width of the slice at the vertical CCD position of the spot pix Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU alae Date Date 2015 04 28 Page 124 of 132 Frame tags e SPOTS_TABLE muse_geometry Measurements of all detected spots on all input images A 3 12 FLUX_TABLE Description This is a simple binary FITS table with the dependency of the flux on wavelength FITS extensions e FITS table Column name Type Description lambda double Wavelength Angstrom Flux double Flux erg s cm 2 arcsec 2 fluxerr double Error of the flux optional column erg s cm 2 arcsec 2 Frame tags e SKY SPECTRUM muse_create_sky Sky spectrum within the sky mask muse_scipost Sky spectrum within the sky mask 1f skymethod model and save contains sky model e SKY_CONTINUUM muse_create_sky Estimated continuum flux spectrum muse_scipost Estimated continuum flux spectrum if skymethod model and save contains sky model A 3 13 STD_RESPONSE Description MUSE flux response table FITS extensions e FITS table Colu
24. Likwid Lightweight Performance Tools which are available at https code google com p 1ikwid Linux only Neither setting the environment variable OMP_NUM_THREADS nor using any thread pinning tool is necessary as long as one gets a good performance To allow for a comparison of the performance the benchmarks for the ESO baseline system are given in Appendix B The standard setup at ESO are 24 threads i e 1 thread per IFU OMP_NUM_THREADS 24 with the threads pinned to the machines physical CPU cores for all recipes with the exception of the geometric calibration recipe which can be run only with a reduced number of threads A short overview on how to use the thread pinning tools can be found in Appendix C A Note for Mac OS X Users On Mac OS X the MUSE DRS is always built without support for running multi threaded This is due to the lack of essential features in the OpenMP implementation of Mac OS X As a consequence on Mac OS X the MUSE DRS will always run in single threaded mode Turning on the multi threaded mode for the calibration recipes and the preprocessing recipe has no effect and the data would be processed sequentially one IFU after the other 3 5 Hints on Using 3rd Party Tools Any 3rd party tool which is used to visualize or inspect the product files created by the MUSE DRS should be 64 bit applications since the products from the MUSE DRS can be larger than 2GB Files larger than 2 GB l Actually for some i
25. Manual ae alate Date Date 2015 04 28 Page 21 of 132 However since the calibration recipes can work on a single IFU at a time the memory requirements can be relaxed for these recipes at the price of reduced performance For the creation of the final data cube the only possibility to reduce the required memory is to limit the wavelength range of the output data cube to an appro priate size cf Section 7 1 3 2 Installing the Software The MUSE Data Reduction Software is distributed as a standard pipeline kit package and can be obtained from the ESO web pages at http www eso org sci software pipelines Apart from the MUSE DRS itself the distributed package contains all dependencies needed for the installation the tools to run the MUSE DRS recipes this cookbook and the installer utility for the kit Using the installer the MUSE DRS can be installed on recent versions of any major Linux distribution as well as Mac OS X However the recommended target platform for using the MUSE DRS is a 64 bit Linux system since Mac OS X imposes certain restrictions when it comes to running the MUSE DRS To install the MUSE DRS unpack the kit in a temporary location go to the top level directory of the unpacked distribution package and execute the installer script as shown in the following example Note The installation script uses the compiler which is found first in the path If more than one compiler are installed on the system one
26. Mean width of slices QC TRACE WIDTHS STDEV Standard deviation of widths of slices QC TRACE WIDTHS MIN Minimum width of slices QC TRACE WIDTHS MAX Maximum width of slices QC TRACE GAPS MEDIAN Median of gaps between slices OC TRACE GAPS MEAN Mean of gaps between slices QC TRACE GAPS STDEV Standard deviation of gaps between slices QC TRACE GAPS MIN Minimum of gap between slices QC TRACE GAPS MAX Maximum gap between slices Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU alate Date Date 2015 04 28 Page 74 of 132 9 4 muse_wavecal Detect arc emission lines and determine the wavelength solution for each slice 9 4 1 Description This recipe detects arc emission lines and fits a wavelength solution to each slice of the instrument The wave length calibration table contains polynomials defining the wavelength solution of the slices on the CCD Processing trims the raw data and records the overscan statistics subtracts the bias taking account of the over scan if overscan is not none and converts them from adu to count Optionally the dark can be subtracted and the data can be divided by the flat field but this is not recommended The data is then combined using input parameters first into separate images for each lamp If lampwise is not given or if resample is given these lamp separate exposures are summed to create a single combined master arc To compute the wavelength solution arc
27. Slice number wlclJ double Polynomial coefficients for the wavelength solution MSE double Mean squared error of fit Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU alae Date Date 2015 04 28 Page 121 of 132 Frame tags e WAVECAL TABLE muse_wavecal Wavelength calibration table A 3 8 WAVECAL_RESIDUALS Description This is an optional FITS table output on request by the muse_wavecal recipe It can be used to verify the quality of the wavelength solution FITS extensions e FITS table Column name Type Description slice int Slice number iteration int Iteration x int X position on the CCD pix y int y position on the CCD pix lambda float Wavelength Angstrom residual double Residual at this point Angstrom rejlimit double Rejection limit for this iteration Angstrom Frame tags e WAVECAL_RESIDUALS muse_wavecal Fit residuals of all arc lines if residuals true A 3 9 LSF_PROFILE Description This file gives a parametrization of the line spread function for each slice It can contain data from one or more IFUs It is a FITS table with at least 48 rows one for each slice In the future a file of this type might contain a different representation of the LSF FITS extensions e FITS table Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU ald Date Date 2015 04 28 Page 122 of 132
28. The reason for this is that the astrometric solution may compensate for small inaccuracies in the geometry table so that a specific astrometric solution should always be used in pairs with its matching geometry table The pipeline distribution comes with a carefully verified WCS solution as a master calibration which matches the geometry table that comes with the pipeline distribution Also since the WCS solution was found to be stable when analyzing the available commissioning data there is no need to recreate the astrometric calibration and this step may safely be skipped For the full description of the muse_astrometry recipe please refer to Section 9 Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ae alae Date Date 2015 04 28 Page 46 of 132 6 5 4 Science Observations At this point all calibrations are in place and one can start with the post processing of the science observations using muse_scipost The recipes applies the previously created on sky calibrations and creates the final data cube from the pre processed pixel tables of scientific observations In the following example a full data cube is created from a single exposure the sky background is removed and the flux calibration and the astrometric calibration created in the preceding processing steps are applied Finally four field of view images are created from the data cube each of them covering the wavelength band of one of the
29. and SKY_CONTINUUM An LSF_PROFILE is nec essary for the two model based methods Cut off the data below this wavelength after load ing the pixel table s Cut off the data above this wavelength after load ing the pixel table s Reference wavelength used for correction of dif ferential atmospheric refraction The R band peak wavelength 7000 Angstrom that is usu ally used for guiding is close to the central wave length of MUSE so a value of 7000 0 Angstrom should be used if nothing else is known A value less than zero switches DAR correction off Carry out a check of the theoretical DAR correc tion using source centroiding If correct it will also apply an empirical correction Fraction of the image to be considered as sky If an input sky mask is provided the fraction is ap plied to the regions within the mask If the whole sky mask should be used set this parameter to 1 Spectral sampling of the sky spectrum Angstrom Spectral sampling of the continuum spectrum Angstrom Correct the radial velocity of the telescope with reference to either the barycenter of the Solar System bary the center of the Sun helio or to the center of the Earth geo Continued on next page one was passed in the input set of files This causes creation of an output cube with a linear WCS and may result in errors If you want to use a sensible default leave this true but do not
30. arc lamp shutter open closed INS LAMP5 ST HgCd arc lamp on off INS SHUT5 ST HgCd arc lamp shutter open closed INS TEMP4 VAL Ambient temperature C DET BINX Detector binning along X axis rows DET BINY Detector binning along Y axis columns DET READ CURNAME Detector readout mode name DET CHIP NAME Detector chip instrument channel name DET CHIP LIVE Detector alive Other keywords EXTNAME Name of the FITS extension Almost all product file which are created by the MUSE pipeline recipes also contain a number of Quality Control Parameters QC parameters These QC parameters are values which are computed by the MUSE recipes as indicators of the quality of the raw data and the reduction process They are available from the FITS header of the pipeline products as hierarchical keywords starting with the leading group component QC 6 2 2 Data Classification and Association In order to create the set of frames for a particular MUSE recipe both the raw input data files and the calibra tion data files have to classified i e the frame tag which indicates the type of data the file contains has to be determined The result of this i e the frame tag has to be given in the second column of the set of frames cf section 6 1 The type of a MUSE raw data files is fully determined by a unique combination of the header keywords DPR CATG DPR TYPE and DPR TECH The type of a MUSE pipeline product is comp
31. as well none preserves an existing weight column in the input pixel tables without changes The filter name s to be used for the output field of view image Each name has to correspond to an EXTNAME in an extension of the FIL TER_LIST file If an unsupported filter name is given creation of the respective image is omit ted If multiple filter names are given they have to be comma separated Continued on next page ESO MUSE Pipeline User Manual Doc VLT MAN ESO 14670 6186 Issue Issue 9 Date Date 2015 04 28 Page 101 of 132 continued from previous page Parameter Type Values default other Description skymethod lambdamin lambdamax lambdaref darcheck skymodel_fraction skymodel_sampling rvcorr string double double double string double double skymodel_csampling double string model none subtract model model 4000 10000 7000 none none check correct 0 05 1 25 1 25 bary bary helio geo none The method used to subtract the sky back ground model should work in all cases it uses a global sky spectrum model with a local LSF If model is selected calibration frames for SKY_LINES and LSF_PROFILE must be set SKY CONTINUUM and SKY MASK are optional If subtract model is selected precalculated sky lines and continuum are subtracted if specified by SKY_LINES
32. bad column search in the master bias NSATURATED Number of saturated pixels in output data MEAN Average of the raw median values of all input files in quadrant n STDEV Standard deviation of the raw median values of all input files in quadrant n QC BIAS QC BIAS MASTERn QC BIAS MASTERn QC BIAS MASTERn QC BIAS MASTERn QC BIAS MASTERn QC BIAS MASTERn cent pair of bi QC BIAS MASTERn each adjacent QC BIAS MASTERn OC BIAS MASTERnN OC BIAS MASTER OC BIAS MASTER OC BIAS LEVELnN QC BIAS LEVELn QC BIAS LEVELn MEDIAN Median of the raw median values of all input files in quadrant n ESO MUSE Pipeline User Manual Doc VLI MAN ESO 14670 6186 Issue Issue 9 Date Date 2015 04 28 Page 67 of 132 9 2 muse_dark Combine several separate dark images into one master dark file and locate hot pixels 9 2 1 Description This recipe combines several separate dark images into one master dark file The master dark contains the combined pixel values of the raw dark exposures with respect to the image combination method used and normalization time specified Processing trims the raw data and records the overscan statistics subtracts the bias taking account of the over scan if overscan is not none from each raw input image converts them from adu to count scales them according to their exposure time and combines them using input parameters Hot pixels are then identi
33. col 1019 x col 1035 col 1051 a col 988 v col 1004 col 1020 gt col 1036 o col 1052 a col 989 col 1005 col 1021 o col 1037 col 1053 v col 990 o col 1006 x col 1022 G col 1038 a col 1054 col 991 col 1007 gt col 1023 o col 1039 a col 1055 col 992 col 1008 o col 1024 G col 1040 v col 1056 col 993 x col 1009 s col 1025 a col 1041 col 1057 col 994 col 1010 o col 1026 a col 1042 o col 995 o col 1011 2 col 1027 v col 1043 Figure 7 4 The graphical window showing the output of the muse_wave_plot_column tool plotting slice 12 of IFU 10 using the wavelength calibration residuals and wavelength calibration tables see text for details 7 3 Miscellaneous Tools 7 3 1 Handling of MUSE pixel tables MUSE pixel tables are heavily used as intermediate data products and they have one special column that is not easy to interpret the origin column Because of that a tool muse_pixtable_dump was added which decodes these values and shows them in a readable form One should always use the c parameter to limit the number of rows that are displayed otherwise it might take very long to complete One should also give the starting row of the region that one is interested in using s T 1 gt muse_pixtable_dump s 100000 c 10 PIXTABLE_OBJECT_0001 01 fits This command results in the output MUSE pixel table PIXTABLE_OBJECT_0001 01 fits showing 10 rows starting at index 100000 of 13514329 xpos ypos la
34. environment variable OMP_NUM_THREADS is explicitly set for each EsoRex command although it would be sufficient to do it once for the whole session before the first recipe is run Note that all following examples use bash shell syntax If a shell other than bash is used the command lines may have to be adapted In particular this may be true for setting environment variables on the same line prior to the command to be executed 6 3 1 Bias In the first processing step the raw bias frames are combined into a master bias frame using the muse_bias recipe The created master bias will then be used in the subsequent reduction steps First the location and the raw frame tag of at least 3 raw bias frames is put into the set of frames Note that currently neither the narrow field mode nor is the adaptive optics available Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ae alae Date Date 2015 04 28 Page 34 of 132 1 gt cat bias sof data muse raw bias MUS data muse raw bias MUS data muse raw bias MUS data muse raw bias MUS data muse raw bias MUS 2014 02 21T09 48 53 153 fits BIAS 2014 02 21T09 50 36 645 fits BIAS 2014 02 21T09 52 16 513 fits BIAS 2014 02 21T09 53 47 996 fits BIAS 2014 02 21T09 55 04 515 fits BIAS AA e To process the data in parallel the number of threads is limited to 24 and all 24 IFUs are then processed with the shown EsoRex command 1 g
35. environment variables MUSE_XCOMBINE_RA_OFFSETS and MUSE_XCOMBINE_DEC_OFFSETS Each of these variables has to contain as many floating point numbers separated by commas as exposures to combine are involved U Each number is the direct difference of the measured position to the reference position no cos 6 the values are interpreted in units of degrees RA_OFFSET RAmeasured RAreference DEC_OFFSET DECmeasured DECreference It is important that the offsets are given in the order of increasing DATE OBS of the exposures involved If one of the exposures has been chosen as the reference the offsets of O for this exposure have to be explicitly given when setting the environment variables 6 6 2 Limiting Wavelength Ranges All recipes reading pixel tables support the lambdamin and lambdamax recipe parameters to restrict the wavelength range they are working on These recipe parameters expect a wavelength given in Angstrom and this can be used to limit the amount of memory required by the recipes see Section 7 1 for details For the combination of exposures the limiting factor in terms of memory is the size of the output data cube which has to be constructed in memory Thus in limiting the wavelength range the output data cube can cover a larger area on the sky 6 6 3 Combining Exposures using muse_scipost The following example shows how three exposures are combined using muse_scipost alone The
36. files follow the ESO specification RD5 for FITS files with data bad pixel maps and variance The units of the extensions are given by the standard BUNIT keyword If the DQ extension is missing the bad pixel status is then encoded as NaN values in the data and variance extensions FITS extensions e DATA 2D FITS image float Data values e DQ 2D FITS image int optional Euro3D data quality This information is used to propagate information about bad pixels found e g in the processing of dark and flat field exposures e STAT 2D FITS image float Data variance Frame tags e MASTER BIAS muse_bias Master bias e MASTER DARK muse_dark Master dark e MASTER FLAT muse_flat Master flat e ARC_RED_LAMP muse_wavecal Reduced ARC image per lamp e ARC_RED muse_wavecal Reduced combined master ARC image if lampwise false or resample true e MASK_REDUCED muse_geometry Reduced pinhole mask images e MASK_COMBINED muse_geometry Combined pinhole mask image Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU alae Date Date 2015 04 28 Page 115 of 132 IMAGE _FOV muse_create_sky Whitelight image used to create the sky mask muse_scipost Field of view images corresponding to the filter parameter muse_exp_combine Field of view images corresponding to the filter parameter if save contains cube muse_scipost_make_cube Field of vi
37. find bright pixels in the master flat samples boolean false Create a table containing all tracing sample points merge boolean false Merge output products from different IFUs into a common file 9 3 4 Product frames The following product frames are created by the recipe Default file name Description MASTER_FLAT TRACE_TABLE TRACE_SAMPLES Master flat Trace table Table containing all tracing sample points if samples true 9 3 5 Quality control parameters The following quality control parameters are available for the muse_flat products Qc Qc Qc Qc QC QC QC QC QC QC Qc Qc QC QC QC QC LAT I LAT IN LAT 1I zZ UTi MEDIAN Median value of raw flat i in input list UTi MEAN Mean value of raw flat i in input list UTi STD EV Standard deviation of raw flat i in input list Z LAT I LAT I LAT I Z Z W tg W U U U Z UTi LAT MASTEI LAT MASTE LAT MASTE LAT MASTE LAT MASTEI tion rmyaayyA Amaia mA yA yA yA YY R R LAT MASTER STD R R R UTi MIN Minimum value of raw flat i in input list UTi MAX Maximum value of raw flat i in input list NSATURATED Number of saturated pixels in raw flat i in input list MEDIAN Median value of the master flat before normalization MEAN Mean value of the master flat before normalization EV Standard deviation of the master flat before norma
38. input set of frames now contains the pixel tables of the three exposures note the 4 digit index in the names of the pixel tables Apart from the additional pixel tables the set of frames is identical to the one shown previously for processing a single exposure The EsoRex command line shows the three coordinate offsets for the 3 input exposures where the first exposure is used as the coordinate reference The wavelength range is restricted to the intervall 6900 A to 7100 A 1 gt cat scipost_combine sof PIXTABLE_OBJECT_0001 01 fits PIXTABLE_OBJECT Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ae alae Date Date 2015 04 28 Page 49 of 132 PIXTABLE_OBJECT_0001 24 fits PIXTABLE_OBJEC PIXTABLE_OBJECT_0002 01 fits PIXTABLE_OBJECT PIXTABLE_OBJECT_0002 24 fits PIXTABLE_OBJECT PIXTABLE_OBJECT_0003 01 fits PIXTABLE_OBJECT H x lt D w na E_OBJECT_0003 24 fits PIXTABLE_OBJEC E_CAL LSF_PROFILE fits LSF_PROFIL E_CAL STD_RESPONSE_0001 fits STD_RESPONSE E_CAL STD_TELLURIC_0001 fits STD_TELLURIC _CAL ASTROMETRY_WCS_0001 fits ASTROMETRY_WCS E_CAL sky_lines fits SKY_LINES E_CAL extinct_table fits EXTINCT_TABL E_CAL filter_list fits FILTER_LIST AJ FI E WOM MM e U C nana an LU Q gt
39. lines are detected at the center of each slice using threshold detection ona S N image and subsequently assigned wavelengths using pattern matching to identify lines from the input line catalog Each line is then traced to the edges of the slice using Gaussian centering in each CCD column The Gaussians not only yield center but also centering error and line properties e g FWHM Deviant fits are detected using polynomial fits to each arc line using the xorder parameter and rejected If lampwise is switched on these analysis and measuring steps are carried out separately on images exposed by the different arc lamps reducing the amount of blending that can otherwise influence line identification and Gaussian centering The final two dimensional fit uses all positions of all lamps their wavelengths and the given polynomial orders to compute the final wavelength solution for each slice iteratively rejecting outliers This final fit can be either unweighted fitweighting uniform for fastest processing or weighted other values of fitweighting for higher accuracy 9 4 2 Input frames Category Type Constraint min ARC raw required 1 MASTER_BIAS calib required 1 MASTER_DARK calib optional usually not used MASTER_FLAT calib optional usually not used TRACE_TABLE calib required 1 LINE_CATALOG calib required 1 BADPIX_TABLE calib optional usually not used ESO MUSE Pipeline User Manual Doc
40. method or using a Gaussian fit lambdamin double 6800 When passing any MASK_CHECK frames in the input use this lower wavelength cut before re constructing the image lambdamax double 7200 When passing any MASK_CHECK frames in the input use this upper wavelength cut before re constructing the image 9 6 4 Product frames The following product frames are created by the recipe Default file name Description MASK_REDUCED MASK_COMBINED SPOTS_TABLE GEOMETRY_TABLE GEOMETRY_CUBE GEOMETRY_CHECK Reduced pinhole mask images Combined pinhole mask image Measurements of all detected spots on all input images Relative positions of the slices in the field of view Cube of the field of view to check the geometry calibration It is restricted to the wavelength range given in the param eters and contains an integrated image white over this range Optional field of view image to check the geometry cali bration integrated over the wavelength range given in the parameters 9 6 5 Quality control parameters The following quality control parameters are available for the muse_geometry products GEO GEO GEO GEO QC QC QC QC IFUm ANGLE EXPi FWHM MEAN Average FWHM of all bright spots in exposure k EXPi FWHM MEDIAN Median FWHM of all bright spots in exposure k EXPi FWHM STDEV Standard deviation of FWHM of all bright spots in exposure k Angle of the mask with respect to the
41. pass Doc VLT MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ISSUE Toue Date Date 2015 04 28 Page 102 of 132 continued from previous page Parameter Type Values Description default other astrometry boolean true If false skip any astrometric calibration even if an ASTROMETRY_WCS 9 12 4 Product frames The following product frames are created by the recipe 9 12 5 Quality control parameters Default file name Description DATACUBE_FINAL IMAGE_FOV OBJECT_RESAMPLED PIXTABLE_REDUCED PIXTABLE_POSITIONED PIXTABLE_COMBINED SKY_MASK SKY_SPECTRUM SKY_LINES SKY_CONTINUUM Output datacube Field of view images corresponding to the filter parame ter Stacked image if save contains stacked Fully reduced pixel tables for each exposure if save con tains individual Fully reduced and positioned pixel table for each individual exposure if save contains positioned Fully reduced and combined pixel table for the full set of exposures if save contains combined Created sky mask if skymethod model and save con tains skymodel Sky spectrum within the sky mask if skymethod model and save contains skymodel Estimated sky line flux table if skymethod model and save contains skymodel Estimated continuum flux spectrum if skymethod model and save contains skymodel The following quality control parameters are available for the
42. should make sure that an appropriate 64 bit compiler will be found first when the installation script is executed 1 gt tar zxf muse kit X Y Z tar gz 2 gt cd muse kit X Y Z tar gz 3 gt install pipeline Then follow the instructions on the screen Once the script finishes successfully and the path variables have been set the installation of the MUSE DRS is complete 3 3 Toolchain Support ESO offers three different tools to process data obtained with one of the VLT instruments One command line tool EsoRex and two GUI based tools Gasgano and Reflex respectively This manual will focus on the EsoRex command line tool which offers a manual control on the reduction process We recommend the user to use Reflex and the MUSE workflow to process MUSE observations as this offers a convenient way to execute a fixed setup of the reduction chain and it includes automatic data organisation as well as the alignment and co adding of exposures See RD8 and RD9 for details on how to use the MUSE Reflex workflow and Reflex itself While Gasgano is useful as file browser for exploring MUSE data sets it is not recommended to use it to run the MUSE DRS recipes Using Gasgano to run MUSE recipes will therefore not be discussed further in this manual Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU ald Date Date 2015 04 28 Page 22 of 132 3 4 Hints on Running the MUSE Pipeline Recipes In ord
43. strength of the line if known The line fluxes may be used in the data reduction software as a first guess to the expected flux the actual fluxes will be determined using line fitting Additionally to identify the lines and associate them with an arc lamp a column ion with element and ionization status and a quality flag are needed Optionally a comment column might be useful FITS extensions FITS table Column name Type Description lambda float Wavelength Angstrom flux float Relative flux ion string Ion from which the line originates quality int Quality flag 0 undetected line 1 line used for pattern matching 2 line that is part of a multiplet 3 good line fully used 5 bright and isolated line use as FWHM reference comment string Optional comment optional column Frame tags e LINE_CATALOG PRO CATG LINE_CATALOG List of arc lines A 2 2 SKY_LINES Description This type of file contains one or more binary tables with the relative fluxes on the sky emission lines If both tables are present they are merged so that lines should not appear in both tables FITS extensions e LINES FITS table Doc VLT MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ISSUE Pose Date Date 2015 04 28 Page 110 of 132 Column name Type Description name string Line name group int Line group id lambda double Air wavelength Angstrom flux d
44. the night sky from the data in a pixel table s belonging to one exposure of mostly empty sky 9 10 2 Input frames Category Type Constraint min PIXTABLE_SKY raw required 1 EXTINCT_TABLE calib required 1 STD_RESPONSE calib required 1 STD_TELLURIC calib optional SKY_LINES calib required 1 SKY_CONTINUUM calib optional LSF_PROFILE calib optional SKY_MASK calib optional 9 10 3 Recipe parameters Parameter Type Values Description default other fraction double 0 75 Fraction of the image to be considered as sky If an input sky mask is provided the fraction is ap plied to the regions within the mask If the whole sky mask should be used set this parameter to 1 sampling double 1 25 Spectral sampling of the sky spectrum Angstrom csampling double 1 25 Spectral sampling of the continuum spectrum Angstrom cr string cube none cube Type of cosmic ray cleaning to use Cube is spectrum the standard CR cleaning which works on a dat acube spectrum uses sigma clipping on the spectrum lambdamin double 4000 Cut off the data below this wavelength after load ing the pixel table s lambdamax double 10000 Cut off the data above this wavelength after load ing the pixel table s Continued on next page ESO Doc VLI MAN ESO 14670 6186 MUSE Pipeline User Manual TSU ald Date Date 2015 04 28 Page 95 of 132 continued from previous page
45. the previous example the images are created as separate files They are saved as FITS files named according to the filter name e g DATACUBE_Cousins_R fits Instead of creating new files the images are appended to the input file as FITS image extensions if the option x is used 1 gt muse_cube filter x f Johnson_V Cousins_R Cousins_I DATACUBE fits filter_list fits Note that no backup of the original file is created Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ae alae Date Date 2015 04 28 Page 61 of 132 8 Troubleshooting 8 1 Typical Problems In many cases the ERROR and WARN NG messages of the MUSE pipeline alert the user of a problem with the data or the reduction In the following a few typical cases and solutions for them are described 8 2 Failed Tracing In some cases vignetting of the slices on the edge of an IFU is severe enough to cause the tracing of the slices to fail when running the muse_flat recipe If this happens a typical error message would be ERROR muse_flat muse_trace tid 005 The trace fit in slice 10 of IFU 6 failed and likely followed by more warnings and errors The output TRACE_TABLE then contains invalid elements in the row for that slice This causes subsequent problems for the wavelength calibration muse_wavecal and skyflat handling muse_skyflat detecting such a broken trace table 4000 3500 3000 250
46. thread configuration cache sizes and a lot more To get these information 1ikwid topology is simply executed 1 gt likwid topology Building the access demon always requires root privileges however this is not needed and the tools are fully usable without it Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU ale Date Date 2015 04 28 Page 130 of 132 this produces quite some output on the terminal The information which is useful in this case is the number of threads per CPU core and the list of CPU cores In this list the physical cores appear first The actual thread pinning tool is 1ikwid pin which is called again using the muse_bias example as shown here 1 gt likwid pin c N 0 5 esorex muse_bias nifu 1 bias sof where the numbering of the CPUs refers to the numbering show in the output of Likwid topology One advantage of 1ikwid pin is that it automatically sets the environment variable OMP_NUM_THREADS ac cording to the list of CPUs specified on the command line if OMP_NUM_THREADS was not already defined before For detailed information on the two commands please refer to their man page Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU alae Date Date 2015 04 28 Page 131 of 132 D Calibrations for Commisioning and Science Verification Data When processing observations which were taken during one of the MU
47. wel GE we a E a EE we Ge Ws ee 91 DAT DESCUPUSO o e eea oe ee we ee ee ee h Set HE ew 91 So DAPUETTAMES oir A Be he Ea Eee ae eee 91 Doc VLT MAN ESO 14670 6186 ESO MUSE Pipeline User Manual pete toue Date Date 2015 04 28 Page 9 of 132 203 Recipe param eS ec se es dri Ge bal aa Be Oe Pe Barwa Packs i d 91 JA Product Tames coe cosce ii a e ee a eS we ede Adc 92 995 Quality control parameters 0 2 66 ea eae 93 IY muse Creasy o f SB at Wa uae ave ae el So eB ae A a amp alee ae Oe oe ee 94 DIO Descupladc cia ara a Aes ota we ea eh See we aat 94 910 2 Input HUMES 2 044 See acteacegakeaagd bo aid a gag ate eh ee as 94 9 10 3 Recipe parameters o lt ege ak e a Eee AE e ee 94 9 1074 Product frames oa ed ei ee he OR eS we eh ee ae we dbs 95 9 10 5 Quality control parameters s omnes gr a a aa 95 SUL WWE AGOST suas aah a a ee a a ds eS 96 LII Bescon are dota era da do BG a rd 96 DU mpu ames oe oi wm avd ee ee Ay ld re a ae Dc eer ee 96 O11 3 Recipe parameters oc ec ca ca ca sioka ebb eae hee ed eh ba ed bd BA bo 96 Olle Prodmcttrames lt 0 s irede i ed BE ee A a ea a ee BE A 97 0 11 5 Quality control parameters cold a a a ee ee ed es 97 AZ WS SCIPO ac ta tahoe dha eR Ree ae hehe bebe ebb age a 98 9121 DESCUPUSO oe s a ke ee a a ee Rw ee ee E 98 ala mpu Pamies oso ee eae ee ee RRS Se SR a 98 9 12 3 Recipe parameters 2 coso eccone ee kG ee ee a E 99 9 124 Product frames 2 a a a a de baa es 102 012
48. with the processed data from the beginning to the very end of the data reduction sequence The errors and data quality information is stored in the MUSE DRS product files following the convention described in RD5 In the following the general layout of the products carrying this information is summarized A detailed description of the format of all products can be found in Appendix A 5 3 1 MUSE Images A MUSE image i e the reduced image of a single CCD as it is or can be created by almost every MUSE recipe has three 2D FITS image extensions DATA The reduced image DO The data quality flags encoded in an integer value according to the Euro3D standard cf RD6 STAT The variance of the reduced data The labels DATA DO and STAT refer to the name of the respective FITS extension i e the EXTNAME keyword 5 3 2 MUSE Pixel Tables The pixel table format is used to store the intermediate products of the pre processing recipe muse_scibasic and a few post processing recipes For each CCD pixel of a single IFU the pixel table records the value of each pixel together with its coordinates position and wavelength This format has been introduced to avoid intermediate re sampling of the data The pixel table keeps the un resampled pixel value until the very end of the data reduction What is changed instead in the different processing steps are only the coordinates which are assigned to each pixel In addition to the value of ea
49. 0 2000 1500 1000 500 750 1 2 1 4 Y f 0 8 a 1 0 6 0 4 0 2 Most critically the science reduction muse_scibasic will stop when trace result slices 9 to 11 0 800 850 300 950 Figure 8 1 The graphical window showing the output of the muse_trace_plot_samples tool see text for details If this is the case the most likely fix is from the default value downwards to to carefully adjust the edgefrac parameter of the muse_flat recipe e g 0 4 or 0 3 Note that for too low values the pipeline might not be able to tell the slices apart any more It is likely that warnings continue to appear for the darkest slices so it is advisable to run the recipe with the samples parameter so that the TRACE_SAMPLES output is created This can then be used with the comma 1 gt muse_trace_plot_samples TRACE_TABLE 06 fits MASTE nd sl 9 s2 11 TRACE_SAMPLES 06 fits R_FLAT 06 fits Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ae alae Date Date 2015 04 28 Page 62 of 132 see Figure 8 1 to visually verify that the trace table contains a good description of the slice location see Section 7 2 for details Specifically for some of the data sets taken during the first MUSE Science Verification run the tracing of a few slices of IFU 6 may fail if the flat fields have been taken at temperatures lower than approximately 7 C cf temper
50. 1 resampl residuals merge wavecal sof The primary product created by this command is the wavelength solution In addition to the primary product the shown EsoRex command also creates for each IFU a product file with the fit residuals of the arc lines reduced arc lamp images for each lamp separately and a combined one and a resampled arc lamp image which can be used for a visual inspection of the wavelength solution An example is shown in Figure 6 3 For the full description of the muse_wavecal recipe please refer to Section 9 6 3 5 Line Spread Function For the subtraction of the sky background in the later steps of the processing of MUSE observations a represen tation of the MUSE line spread function LSF is needed The line spread function can be calculated using the recipe muse_Isf which computes the slice and wavelength dependent LSF from a set of arc lamp spectra The recipe takes the same set of arc lamp exposures as it is used with the muse_wavecal recipe But in order to measure the faint wings of the line spread function with a reasonable signal to noise ratio at least 10 arc lamp exposures for each arc lamp should be used From the previous processing steps the recipe needs as input the wavelength solution the trace table and the master bias and from the static calibration the line catalog is needed Optionally the bad pixel table the master dark and the master flat field may be added to the input set of frames Th
51. 47 6 6 2 Limutme Wavelength Rapes co cocer de eed ek ee Re ee ee 48 6 6 3 Combining Exposures using muse_scipost 2 2 o e 48 6 6 4 Combining Exposures using muse_exp_combine 0 49 Tips amp Tricks 52 71 Restrictins wavelength ranges cies bs bd de Sa ORR Ee eda eR a aaka 52 we Veriieanot TORS o e caw a bobo Paw ede ee ee ee eee oe OE il 52 Gel Verification of the tracing s lution gt o o coasa eo e400 ba ea Sa ke be ee 52 7 2 2 Verification of the wavelength solution s sooo ee ek Pew aa a ee 54 da Miscellaneous Tools ios ads SES Ree eee Eee bad as 56 73 1 Handlme ot MUSE pixel tables ere ec erma a eee ree ee 56 Doc VLT MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ISSUE Toue Date Date 2015 04 28 Page 7 of 132 1 3 2 Handling of MUSE bad pixel maps 22 4545 2485408444 Se he be ee 57 43 3 Working with Data Cubes 2 0 0 5 24 24 cesena na a a a de 58 8 Troubleshooting 61 Sl Typical Problems i ek dba Dob a eA bP AOE we Re Pa be ha eh hs 61 S2 Failed TERE e o a ack oe ee ee ey we a we de ek PE eo 61 Bo Uhelgemle s z eho Shed ep ee ave Se ee ee eh Soe ee IAE da 62 Sd Debugging Options i os do tenere rer a wea ee Oe oo de bawa ea ke Hares 63 9 Recipe Reference 64 Ol amuse Dias gee ec ee weed ee a eno ee ad we hh Werte ek wat ew a a a ae es a ee oh 64 Ol Desplome aaa de pact ra a ala wee gaete Ghee ad 64 91 2 IMpUETTAMES 2 22564 b4 244 ei b
52. 5 Quality Control paraieltls csoc cr ea REE SEED OD EERE Ree ee He 102 9 13 WISE exp COmDINE k e aiaa Se de a a Ee al he ah he ec Sa A ew ad 104 O13 DESCAPU N sa Baek eee ta ae de Aa oe a ee dale ba ee 104 013 2 NEMS ir a ee ee da AA eee oe ee eee 104 O 13 3 Recipe parameters lo corea a So we baw ew ed ES 104 9 T34 Product Tames ane soi See ete Se as eR aa ek ey ee ea ee eae 106 9 13 5 Quality control parameters s s sos sose s eatea we bade ee ede ee ee ee 106 A Data Formats 107 Acl Raw Data Piles 2 442485 24 bo dor DELS A OE Ae Ee ChE bade 107 Ali RAW IMAGE vcd a ee Baka ed Eee ree eee 107 Doc VLT MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ISSUE Toue Date Date 2015 04 28 Page 10 of 132 A2 State Calibration Fil s o eec 4a es creg ba ealeda ha GPE eoa Sake E de ee ea 109 Aal LINE CATALOG oe host a Sd ke BOR we ar et ae 109 A22 SRY LINES 22 6 6s 4 bb ba Ma Ee eee ee hha eh eee das 109 A2 3 ASTROMETRY_ REFERENCE 2 ereed ee ee ee ees 110 Ala EXTINCT TABLE 2 6 44 04 2 o 4 Sod Bath eh dk Oly oe Be A Rw eed 111 A23 BADPIX TABLE 22 2224484246 a ee eae Hh ee Ga 111 320 SID FLUX TABLE ede ad be ee bo ee oh hace Be ae ae e 112 Ali PILIER LST ooo a A aS ew bee de oe 112 AB TELLURIC REGIONS oere ecane ee Rw dk ok ge ecg a se aa ete nes 113 A3 Recipe Product Piles o se 4544 604 oe Da ee be ee De ea o eee 114 A31 MUSE_IMAGE 24 2224 eee conga ede OG Ge A Gam 114 Peo PINE TABLE a des a se
53. 77 of 132 9 4 5 Quality control parameters The following quality control parameters are available for the muse_wavecal products QC WAVECAL SLICEJ LINES NDET Number of detected arc lines in slice j QC WAVECAL SLICEJ LINES NID Number of identified arc lines in slice j OC WAVECAL SLICEJ LINES PEAK MEAN Mean peak count level above background of detected arc lines in slice j OC WAVECAL SLICEJ LINES PEAK STDEV Standard deviation of peak count level above background of detected arc lines in slice j QC WAVECAL SLICEJ LINES PEAK MIN Peak count level above background of the faintest line in slice j OC WAVECAL SLICEJ LINES PEAK MAX Peak count level above background of the brightest line in slice j QC WAVECAL SLICEJ LAMP1 LINES PEAK MEAN Mean peak count level of lines of lamp above background of detected arc lines in slice j Not produced with lampwise FALSE QC WAVECAL SLICEJ LAMP1 LINES PEAK STDEV Standard deviation of peak count level of lines of lamp above background of detected arc lines in slice j Not produced with lampwise FALSE OC WAVECAL SLICEJ LAMP1 LINES PEAK MAX Peak count level above background of the bright est line of lamp in slice j Not produced with lampwise FALSE QC WAVECAL SLICEJ LINES FWHM MEAN Mean FWHM of detected arc lines in slice j QC WAVECAL SLICE J LINES FWHM STDEV Standard deviation
54. FITS header keywords are useful for that what are the valid frame tags and which header keywords should be checked in order to find matching calibration frames 6 2 1 Useful Header Keywords The following table summarizes FITS header keywords which provide useful information about the observation instrument configuration and status The keywords are grouped by their context and intended use respectively Keyword Name Description Keywords for frame classification INSTRUME Name of the instrument DPR CATG Raw data frame product category DPR TYPE Raw data frame product type i e the type of observation DPR TECH Raw data frame observation technique TPL ID Name of the template used to create the exposure PRO CATG Pipeline product category i e the type of the product Keywords describing the observation OBJECT Target name RA Telescope pointing RA J2000 deg DEC Telescope pointing DEC J2000 deg MJD OBS Modified Julien Date at the start of the exposure DATE OBS Human readable version of MJD OBS OBS NAME Name of the observation block OBS STAR Start time of the observation block OBS TARG NAME Target name TPL STAR Start time of the template within the observation block TPL EXPNO Exposure sequence number within the template TPL NEXP Number of exposures within the template TEL AIRM START Airmass at e
55. IN Minimum FWHM of the LSF in slice j ICEJ FWEM MAX Maximum FWHM of the LSF in slice j Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU alae Date Date 2015 04 28 Page 80 of 132 9 6 muse_geometry Compute relative location of the slices within the field of view and measure the instrumental PSF on the detec tors 9 6 1 Description Processing first works separately on each IFU of the raw input data in parallel it trims the raw data and records the overscan statistics subtracts the bias and converts them from adu to count Optionally the dark can be subtracted and the data can be divided by the flat field The data of all input mask exposures is then averaged The averaged image together with the trace mask and wavelength calibration as well as the line catalog are used to detect spots The detection windows are used to measure the spots on all images of the sequence the result is saved with information on the measured PSF in the spots tables Then properties of all slices are computed first separately on each IFU to determine the peak position of the mask for each slice and its angle subsequently the width and horizontal position Then the result of all IFUs is analyzed together to produce a refined horizontal position applying global shifts to each IFU as needed The vertical position is then determined using the known slice ordering on the sky the relative peak positions are put into se
56. ME keyword the unit in the standard BUNIT keyword The units evolve through the processing In case CUNITi are available in the primary FITS header they are used to track a spatial WCS for the construc tion of the reconstructed datacube and are not to be used to interpret the data in the pixel table Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU alae Date Date 2015 04 28 Page 116 of 132 Formerly pixel tables were written as binary FITS tables and the MUSE pipeline can still read and write them for backward compatibility In that format the standard FITS table keywords in the table extension header are used to track column names TTYPEi and units TUNITi Reading and writing the binary table format is typically slower than the image format Pixel tables can be stored in binary table format using the environment variable MUSE_PIXTABLI E_SAVE_AS_TABLE FITS extensions The default table format MUSE_PIXTABLE_SAVE_AS_TIMAGE is not set is e xpos 1D FITS image float x position of a pixel within the field of view pix rad deg e ypos 1D FITS image float y position of a pixel within the field of view pix rad deg e lambda 1D FITS image float Wavelength assigned to the pixel Angstrom e data 1D FITS image float Data value count 10 20 erg s cm 2 Angstrom e dq 1D FITS image int 32bit bad pixel status
57. MP_NUM_THREADS 24 esorex log file flat log muse_flat nifu 1 A merge flat sof The products of this EsoRex command are the master flat frame and the trace tables each of them stored as single FITS files containing the master flat field and the tracing solution for all IFUs respectively 1 gt ls 1 fits MASTER_FLAT fits TRACE_TABLE fits An example of a MUSE master flat is shown in Figure 6 2 For the full description of the muse_flat recipe please refer to Section 9 6 3 4 Wavelength Calibration In this processing step the wavelength solution for each IFU is created using the recipe muse_wavecal To create the dispersion solution the recipe needs at least three raw arc lamp frame as input One raw frame for each available arc lamp It is possible to use more than one of these sets as input These sets of three arc lamp frames are created by the MUSE calibration template and therefore the a complete set should always be available In addition this can be verified by looking at the FITS header keywords in dicating the lamp shutter status INS SHUT3 ST INS SHUT4 ST and INS SHUT5 ST Only one of the To verify that the lamps a actually active the keywords INS LAMP3 ST INS LAMP4 ST and INS LAMP5 ST should be checked Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ae alae Date Date 2015 04 28 Page 37 of 132 0 19 0 38 0 56 0 75 0 94 1 1 1 3 1 5 1 7 Figure 6 2 Ex
58. RK calib optional usually not used BADPIX_TABLE calib optional usually not used 9 3 3 Recipe parameters Parameter Type Values Description default other nifu int 0 IFU to handle If set to 0 all IFUs are processed serially If set to 1 all IFUs are processed in parallel Continued on next page Doc VLT MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ISSUE Toue Date Date 2015 04 28 Page 71 of 132 continued from previous page Parameter Type Values Description default other overscan string vpoly If this is none stop when detecting discrepant overscan levels see ovscsigma for offset it assumes that the mean overscan level represents the real offset in the bias levels of the exposures involved and adjusts the data accordingly for ypoly a polynomial is fit to the vertical over scan and subtracted from the whole quadrant ovscreject string der This influences how values are rejected when computing overscan statistics Either no rejec tion at all none rejection using the DCR algo rithm der or rejection using an iterative con stant fit fit ovscsigma double 30 If the deviation of mean overscan levels between a raw input image and the reference image is higher than ovscsigma x stdev stop the process ing If overscan vpoly this is used as sigma rejection level for the iterative polynomial fit the level
59. ROFIL SMUSE_CAL STD_RESPONSE_0001 fits STD_RESPONSE SMUSE_CAL STD_TELLURIC_0001 fits STD_TELLURIC SMUSE_CAL ASTROMETRY_WCS_0001 fits ASTROMETRY_WCS x FI Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ae ald Date Date 2015 04 28 Page 50 of 132 SMUSE_CAL sky_lines fits SKY_LINES SMUSE_CAL extinct_table fits EXTINCT_TABLI SMUSE_CAL filter_list fits FILTER_LIST Pl 2 gt OMP_NUM_THREADS 24 esorex log file scipost_1 log muse_scipost save individual scipost_l sof The product files created by this commands are 1 gt ls 1 x fits DATACUBE_FINAL_0001 fits IMAGE_FOV_0001 fits PIXTABLE_REDUCED_0001 fits which are then renamed to prevent overwriting them with the EsoRex calls needed to process the remaining exposures 1 gt mv DATACUBE_FINAL fits DATACUBE_FINAL_EXP01 fits 2 gt mv IMAGE_FOV_0001 fits IMAGE_FOV_0001_EXP01 fits 3 gt mv PIXTABLE_REDUCED_0001 fits PIXTABLE_REDUCED_0001_EXP01 fits T T By using the recipe parameter save_individual a fully reduced pixel table is created as an additional product Fully reduced means that all on sky calibrations have been applied and the only processing step missing is the resampling into the final data cube Proceeding in the same way with t
60. RV corrected pixel table e PIXTABLE_POSITIONED muse_scipost Fully reduced and positioned pixel table for each individual exposure if save contains positioned e PIXTABLE_COMBINED muse_scipost Fully reduced and combined pixel table for the full set of exposures if save contains combined muse_exp_combine Combined pixel table if save contains combined muse_scipost_combine_pixtables Combined pixel table A 3 3 DATACUBE Description Three FITS NAXIS 3 cubes in two extensions for data values and variance A bad pixel is represented by a NaN value in the data and variance extensions Such datacubes follow the ESO specification RD5 for FITS files with data bad pixel maps and variance The units of the extensions are given by the standard BUNIT keyword They can have two dimensional image extensions of the same type as IMAGE_FOV For these the EXTNAME will be called the same as the filter function that was used to create it Depending on recipe parameters additional filtername_STAT extensions may be present to represent the variance of the images These images then follow the ESO specification RD5 FITS extensions e DATA 3D FITS image float Data values e STAT 3D FITS image float Data variance e 2D FITS image float optional may appear more than once Data values of a filtered image e 2D FITS image float optional may appear more than once Data variance of a filtered image
61. S double Vertical position pix GROUP_N int Group number SPAX_ID string Spaxel identifier DATA_SPE float array Data spectrum QUAL_SPE int array Data quality spectrum STAT_SPE float array Associated statistical error spectrum Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU alate Date Date 2015 04 28 Page 119 of 132 e E3D_GRP FITS table Column name Type Description GROUP_N int Group number G_SHAPE string Spaxel shape keyword G_SIZE float Horizontal size per spaxel arcsec G_ANGLE float Angle of spaxel on the sky deg G_SIZE2 float Vertical size per spaxel arcsec G_POSWAV float Wavelength for which the WCS is valid Angstrom G_AIRMAS float Airmass G_PARANG float Parallactic angle deg G_PRESSU float Pressure hPa G_TEMPER float Temperature K G_HUMID float Humidity Frame tags e DATACUBE_FINAL muse_scipost Output datacube muse_exp_combine Output datacube if save contains cube muse_scipost_make_cube Output datacube A3 5 TRACE TABLE Description This file gives the trace solution for each slice in the form of a polynomial It is a FITS table with 48 rows one for each slice FITS extensions e FITS table Column name Type Description SliceNo int Slice number Width float Average slice width tc0_i3 double polynomial coefficients for the central trace solution MSEO double me
62. SE Currently the distributed catalog includes 11 fields of the 6 globular clusters Optionally the extinction table a response curve and a telluric correction may be present in the input set of frames If the optional calibrations are present the corresponding corrections are performed before the reference objects are identified and the WCS solution is computed However experience shows that the improvement that can be achieved by using the optional calibrations is very limited if it is present at all The following shows the input set of frames and the EsoRex command to create the WCS solution 1 gt cat astrometry sof PIXTABLE_ASTROMETRY_0001 01 fits PIXTABLE_ASTROM RY E PIXTABLE_ASTROMETRY_0001 24 fits PIXTABLE_ASTROMETRY SMUSE_CAL STD_RESPONSE_0001 fits STD_RESPONSE SMUSE_CAL STD_TELLURIC_0001 fits STD_TELLURIC SMUSE_CAL extinct_table fits EXTINCT_TABLE SMUSE_CAL astrometry_reference fits ASTROMETRY_REFERENCE 2 gt OMP_NUM_THREADS 24 esorex log file astrometry log muse_astrometry astrometry sof The results are the data cube and the computes WCS solution 1 gt ls 1 x fits ASTROMETRY_WCS_0001 fits DATACUBE_ASTROMETRY_0001 fits It is necessary to always use an astrometric solution together with the geometry table that has been used to compute it
63. SE commissioning or the Science Ver ification runs the geometric calibration and the astrometric solution geometry_table_wfm fits and astrometry_wcs_wfm fits which are included in the MUSE pipeline distribution are not applicable In stead the calibrations prepared for these runs should be used These legacy calibration files are available from the MUSE pipeline download page as a tar archive It includes the pairs of geometric calibration and astrometric solution suitable for data taken during the three Commissioning runs COMM1 COMM2a and COMM2b and the two Science Verification runs SV1 and SV2 The following table summarizes which calibrations should be used for which observations The filename extension fits has been omitted in the table The distribution package providing these files is available in the MUSE Additional Dataset section of the ESO pipeline download page http www eso org sci software pipelines as Legacy Static Calibra tions Run Expiration Date Geometry Table Astrometric Solution COMM1 2014 02 22 geometry_table_wfm_comm1 astrometry_wcs_wfm_comml COMM2a SV1 2014 07 15 geometry_table_wfm_comm2a astrometry_wcs_wfm_comm2 COMM2b SV2 2014 12 01 geometry_table_wfm_comm2b astrometry_wcs_wfm_comm2b Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ae alae Date Date 2015 04 28 Page 132 of 132 E Useful links This section compiles some useful links ESO MUSE web page
64. SE oaee ae e ance hae a Ae aie ded a eea O ae a Bone a Ee a a a La 78 gSA DESEMPU N osa a ae a e a e a ad aa a aa ee dae dopa gle aa 78 95 2 Input am s aada alent a a rs aa edhe dh 78 95 3 Recipe parameters 2 240645 oea 04 bk ea eA ea e a E eoa a 78 95A Product irames o s e sacia Gee eee a A e E e 79 9 5 5 Quality control parameters s s 25 da a a ee ON ee ae EE a 79 9G a OA 80 OO DESCUOPR N es 6s 42 dar eed da do Dade we e eed 80 96 2 Inpitirames 2 256464 eu en a ek OS ee ew de ee 80 OOo AOCIPE PATAS fe cece e Se ho ed ee OO ee A we A 80 964A Prodmet Tames oers os e dees A spar ee a A we Rhee a eds eee 81 9 6 5 Quality control parameters o s e s eeue oe a e ew ee a ee a 81 OT gme TIME c a a a a a a ee ba ee ee a Bale ed he ele 83 OTN ESCAPA caco dd OR Re ee ee ba Ee ES 83 DN IMPUETEAMES esos we we ew dee ee Oe ee 83 9 73 Reape Parameters os sg as ae Ot ek See Bele eee eee 84 O04 Product rames lt ros ok a Eo dd a A AR Re eee eee OA 86 9 7 3 Quality control paratict rs e css ee ew siaa E ORE Oe ED eS Be ke ee HS 86 9S IMUSE SCIDASIO A A wd me ON hme 87 DUA E e 2 e a Sok tea ads Be ot oe ee ed ws ede ee 87 Og TAPETES e pease a O A O Se SS OS OW ed ode 4 de 87 9 8 3 Recipe parameters e ona ce ar ae oe ek WG ao ew hw eh ees eee 88 O84 Product frames 4 64n402 20 a ba bbe e ee eae eed dee e eae be 90 98 5 Quality control parameters lt sc ea scoe a aote er a ba eas 90 99 pse standard o Tee see ce Rae a Gwe enw a oe a aD a
65. _pixtable_erase_slice PIXTABLE_OBJECT_0005 14 fits 14 10 PIXTABLE_OBJECT_0005 14_el0 fits it removes slice 10 numbering on the CCD from a pixel table of IFU 14 as produced by muse_scibasic The IFU number is always required on the command line so that when given a pixel table with multiple IFUs in it the tool knows for which one to erase the slice 1 gt muse_pixtable_erase_slice PIXTABLE_REDUCED_0001 fits 14 10 PIXTABLE_REDUCED_0001_e1410 fits If a pixel table contains even multiple exposures then it erases the given slice of the given IFU of all exposures 7 3 2 Handling of MUSE bad pixel maps Three tools exist to create or supplement MUSE bad pixel tables the BADPIX_TABLE files that are optionally read by every recipe that starts from raw data see Section 6 4 If one wants to create such a bad pixel table from scratch one can start with one of the image based master cal ibrations These contain a DO extension that is a bad pixel map While this is automatically used by subsequent recipes one can transform it into a bad pixel table with the muse_badpix_from_dq tool Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ae alae Date Date 2015 04 28 Page 58 of 132 1 gt muse_badpix_from_dgq MASTER_FLAT 10 fits BADPIX_TABLE 10 fits This would create a new table containing all bad pixels that were detected by the muse_flat
66. a data cube World Coordinate System Wide Field Mode Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU alae Date Date 2015 04 28 Page 16 of 132 2 Overview 2 1 The MUSE Instrument MUSE is an optical wide field integral field spectrograph that uses the image slicing technique to cover a field of view FOV of 1 x 1 in wide field mode WFM with a sampling of 0 2 x 0 2 spaxels The full field is split up into 24 sub fields each 2 5 x 60 in WFM which are fed into one of the 24 integral field units IFUs of the instrument Each IFU illuminates a 4k x 4k CCD by separating the incoming light into 48 slices This is illustrated in Figure 2 1 On the CCD the slices appear as approximately 76 5 pixel wide vertical bands separated by gaps of about 6 pixel At the edges of the detector the slices are slightly curved outwards up to 2 pixel The slices are offset vertically i e along the wavelength axis forming a repetitive horizontal pattern of three steps This pattern is affected by curvature across the CCD which results in a different wavelength coverage for each slice For performance reasons the MUSE detectors are always read in 4 port mode All four quadrants have equal size and are visible in the raw data images separated horizontally and vertically by pre scan and over scan regions On the detectors and the raw images the dispersion axis is oriented along the pixel col
67. a e led RY pl ae ca to s 115 AS DATACUBE 2 24 62444 6444 dos eh bw ae be ES bed bees 117 Add EUROSDCUBE ro hada A Sa We A a eae Be be ae 118 092 TRACE TABLE 2 ce dos ee oie as A a hw ede eae ee de 119 A36 TRACE SAMPLES coca 4232234 44a Ge Rb OER aS Ea ES ER RS ES 120 AG WAVECAL TABLE o 2 040 464 4468 4 44 49 4 GRA eae Rad eee ee eS 120 A 3 8 WAVECAL_RESIDUALS 2 22 2 e e 121 ASS LSP PROFILE oe co o eo A ea a ee e ge 121 A 3 10 GEOMETRY TABLE o occore a e ai ee ao 122 ASI SPOTS TABLE 2 4 re 448426 4 ra cat BS 123 Asi PLUS TABLE aria a A A id Gee OA ew Bees 124 Add STO_RESPONSE 2364442 2206 ma diera MO a a a ER AR 124 laa CT BLUR oe ee ata is di NE bee eee 125 AS SID FLUXES ec eo do oe ee rd Ye ww Ss ed BR OK ee OS 125 A 3 16 AMPL_CONVOLVED e 126 BA URGE ALA MES sa a ew Gd ee he ee we a te e aes 126 AAT OUIPUT WGS oio 642 444 46 based BGO EGR RGR eR aE eR ERS 126 B Benchmarks 128 Biol The Reference system ooe hele ae eR Oe eS ea a See A alee 128 Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ISSUE Tue Date Date 2015 04 28 Page 11 of 132 B 2 Benchmark results 2 4 246400 64 Paha ba baa ha Le beds Gage Pacha be wa 128 C Performance Tools 129 Lol Umetako cec dedu Mae ee be ae eA A eR Ee ee See BE es 129 C 2 Using the Likwid Lightweight Performance Tools o o ooo 129 D Calibrations for Commisioning and Science Verification D
68. a eee ee hee ea bad bd Bas 64 91 3 Recipe POTAMEIES sors A A E O aE 64 O14 Product iames o ed e is Oe E eee ae Se 65 O15 Quality control parameters o coses eda de ge y Ge Bawa wake baa 66 OD MUSE Mar i 6 cs he ed ek eh ce ee A ae Bk BE owe ad ed AAA A A 67 D2 Deu cr ae Be A eat oe eae ia A ee Re 67 922 Taput hames so inm beds ace e sar Be Re oe ok Ge ww ee a eds ace 67 92 3 Recipe PANIES au dR Ane teh b a whee aa wed be 67 O24 Producttram s lt cs oc b4 40 2444444085 bbe See ed bee ba eds 68 O25 Quality control parameters lt se sse mos deo ae ee we ae ee ee ee we 69 Oo ISE a A che eRe ee ee edi ee he Je Gk ae Oe ee ee ee as a 70 Oo DESCNPR N 2 add Bae Se eae ee ew Sd Oe Bara Park ee a 70 Of Input fames 22 ee hee he ee ee a ae we ed eS 70 933 Recipe PATAS 2 ee kok es ard ed we a i a et en wee we eae a we 70 Woe Product Mames 0 ood ae a oe Bee OR a Pe ee we A eee 72 935 Quality control patamieters cs lt s cascia 40 0 8 awed ba eae ee ee ee 72 OA muse Waveddl aori ok a a di OR aa Pe bE da 74 DAT DESCUPUON o e s eca saa ek a eee ee Oe 74 Ong DAPUETTAMeS oir a Ae EA E ore eke 74 Doc VLT MAN ESO 14670 6186 ESO MUSE Pipeline User Manual pete teous Date Date 2015 04 28 Page 8 of 132 94 3 Recipe paramers one Ge a edd i 2 e Oe Barwa a ad 75 94A Producttrames 2 02 24 esaa e ado ee aa a d eh ee a ew ed 76 DAS Quality control parameters lt s sredni e wo aaa ar te wet ee E ee a 77 O Ise
69. ample of a MUSE master flat field of a single IFU channel 1 as it is created by the recipe muse_flat keywords should indicate an open lamp shutter for a single arc lamp frame and for a complete input set one needs 3 such arc lamp frames one for each shutter Dark correction and flat field correction will be applied if the master dark frame and the master flat field frame are present in the set of frames respectrively However applying the two corrections is not needed for the arc lamp frames and thus the input data for these correction are omitted in the set of frames shown in the following What is needed from the previous processing steps are the trace tables and the master bias frame and from the static calibrations the line catalog is required The set of frames and the EsoRex command line to create the wavelength solution are shown here 1 gt cat wavecal sof data muse raw wave MUS data muse raw wave MUS 2014 02 19T10 58 29 838 fits ARC 2014 02 19T11 00 03 593 fits ARC Pl El Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ae alae Date Date 2015 04 28 Page 38 of 132 data muse raw wave MUSE 2014 02 19T11 02 10 205 fits ARC SMUSE_CAL MASTER_BIAS fits MASTER_BIAS SMUSE_CAL TRACE_TABLE fits TRACE TABLE SMUSE_CAL line_catalog fits LINE_CATALOG 2 gt OMP_NUM_THREADS 24 esorex log file wavecal log muse_wavecal nifu
70. an squared error of fit central solution tcl_ij double polynomial coefficients for the left edge trace solution MSE1 double mean squared error of fit left edge solution t 2_ij double polynomial coefficients for the right edge trace solution MSE2 double mean squared error of fit right edge solution Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU alae Date Date 2015 04 28 Page 120 of 132 Frame tags e TRACE TABLE muse_flat Trace table A 3 6 TRACE SAMPLES Description This is an optional FITS table output on request by the muse_flat recipe It can be used to verify the quality of the tracing i e find out how accurate the pipeline was able to determine the location and boundary of the slices on the CCD FITS extensions e FITS table Column name Type Description slice int Slice number y float y position on the CCD pix mid float Midpoint of the slice at this y position pix left float Left edge of the slice at this y position pix right float Right edge of the slice at this y position pix Frame tags e TRACE SAMPLES muse_flat Table containing all tracing sample points if samples true A 3 7 WAVECAL_TABLE Description This file gives the dispersion solution for each slice in one IFU It is a FITS table with 48 rows one for each slice FITS extensions e FITS table Column name Type Description SliceNo int
71. and bottom of the CCD and between the slices near the edges and in the center of the CCD The tool muse_trace_plot_widths was written to help to assess that there are no sudden jumps in the tracing When called with a tracing samples table the samples of all slices are shown as displayed in Figure 7 2 A color gradient from green on the left of the CCD to red on the right plus different symbols are used to make the slices distinguishable It is apparent that the slices on the edges of the CCD are the widest above 78 pix while those near the center of the CCD are narrow below 76 pixels 7 2 2 Verification of the wavelength solution The tool muse_wave_plot_residuals can be used to verify the two dimensional wavelength solution of each slice or of all slices of one IFU To use it one needs to run the muse_wavecal recipe with the residuals option so that the extra product WAVECAL_RESIDUALS is created Then one can run e g 1 gt muse_wave_plot_residuals WAVECAL RESIDUALS 10 fits and get a 2D map in CCD coordinates of the residuals of all the computed arc line centers with respect to the final solution This is displayed in Figure 7 3 There one can see regions on the CCD that are not covered by arc lines as white patches and the points with the strongest blue and red colors give the strongest deviations from the final solution One can use the same command to change the vertical axis of the plot from CCD pixels
72. as defined by the Euro3D specification e stat 1D FITS image float The data variance count 2 10 20 erg s cm 2 Angstrom 2 e origin 1D FITS image int Encoded value of IFU and slice number as well as x and y position in the raw trimmed data e weight 1D FITS image float The optional relative weight of this pixel SAV If MUSE_PIXTABLE_ KE AS_ TAI BLE 1 each of the image extensions is stored as a table column where the name of the column corresponds to the extension name of the respective image extension of the default format Frame tags e PIXTABLE_OBJECT muse_scibasic Output pixel table for OBJECT input muse_scipost_correct_dar DAR corrected pixel table muse_scipost_calibrate_flux Flux calibrated pixel table muse_scipost_apply_astrometry Pixel table with astrometric calibration e PIXTABLE_STD muse_scibasic Output pixel table for STD input Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU alate Date Date 2015 04 28 Page 117 of 132 e PIXTABLE_SKY muse_scibasic Output pixel table for SKY input e PIXTABLE_ASTROMETRY muse_scibasic Output pixel table for ASTROMETRY input e PIXTABLE_REDUCED muse_scibasic Output pixel table muse_scipost Fully reduced pixel tables for each exposure if save contains individual muse_scipost_subtract_sky Output pixel table s for sky subtraction muse_scipost_correct_rv
73. ata 131 E Useful links ESO MUSE Pipeline User Manual Doc VLI MAN ESO 14670 6186 Issue Issue 9 Date Date 2015 04 28 Page 12 of 132 Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU aed Date Date 2015 04 28 Page 13 of 132 1 Introduction 1 1 Scope This document is a quick start guide to the processing of MUSE observations using the MUSE Instrument Pipeline Recipes also known as the MUSE Data Reduction Software or MUSE DRS While it is not as detailed as the full MUSE Pipeline Manual when it comes to how individual processing recipes work it should guide the users through the installation of the software and bring them quickly to the point where they can use the software tools to create science ready datacubes from raw MUSE data sets The processing of MUSE data is extremly demanding with respect to the computing environment Therefore the document describes the system requirements the installation and the setup of the environment in some detail in order to avoid the most common pitfalls and it is strongly recommended that the users reads these sections before they start working on MUSE data This manual refers to the MUSE Data Reduction Software version 1 0 4 The current release has a few known issues which are listed in Section 4 1 2 Acknowledgements This cookbook is to a large extent based on the MUSE Data Reduction Software Manual prepar
74. ature given in the FITS keyword INS TEMP 4 VAL of the raw flat field frame In order to recover from that and finish the processing of the SV data sets one can follow the recovery procedure shown here 1 Run muse_flat on a set of flat fields taken at an ambient temperature of approximately 8 C and create the master flat field and the trace tables Data sets of flat fields taken at this temperature are available from the ESO archive As an alternative a set of verified trace tables for IFU 6 is available from the pipeline download page at http www eso org sci software pipelines The set contains the trace table for both the nominal and the extended wavelength range 2 Run muse_flat on the set of flat fields where the recipe fails to create all tracing solutions which in general should be the flat fields closest in time and temperature with respect to the science observation This will create 47 product files instead of the expected 48 since the trace table for IFU 6 has not been created 3 Then when running muse_wavecal on a set of arc lamp frames and muse_scibasic on an on sky obser vation science standard star etc use the 47 products from the previous step low temperature flat fields i e all 24 master flat fields and the 23 trace tables To compensate for the missing trace table of IFU 6 one should use the trace table of IFU 6 created from the high temperature flat fields in the first step 4 Since the the flux on the s
75. bine usually do a good job to automatically recover the relative offsets from the information in the FITS header of each exposure provided that two conditions are met 1 the same VLT guide star was used to observe all exposures 2 the exposures were all taken at the same position angle Since MUSE exposures are often observed using a dither pattern that involves 90 rotations condition 2 is often not true and the user has to provide offset corrections to the pipeline The reference may be some external data or one of the exposures itself For instance the reconstructed field of view images may be used for measuring the coordinate offsets creating these images can be done quickly with muse_scipost using a vary narrow range in wavelength 12This is due to a slight decentering of the axis of the derotator leading to a derotator wobble Future versions of the pipeline should be able to automatically correct this effect Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ae alae Date Date 2015 04 28 Page 48 of 132 The offset corrections can then be applied in two ways 1 Edit the FITS headers Before starting muse_scipost or muse_exp_combine to create a combined cube edit the main FITS headers of the input pixel tables and replace the RA and DEC header keywords with corrected values that are measured externally 2 Provide offsets The coordinate offsets are given by setting the
76. can be done by setting the environment variable MUSE_PIXTABLE_SAVE_AS_TABLE 1 In the example the usage of muse_scibasic is shown for a scientific exposure as input i e for the raw frame tag OBJECT But it has to be applied in the same way to the observations of standard stars astrometric fields and empty sky fields i e the raw frame tags STD ASTROMETRY and SKY For each of these four types of observations the recipe takes one or more raw exposures as input The required calibrations are master bias master flat trace table wavelength solution and the geometry table Optionally a master sky flat and a bad pixel table may be used The resulting set of frames and the EsoRex command to run muse_scibasic are shown here 1 gt cat object sof data muse raw object MUSE 2014 08 04T09 34 48 604 fits OBJECT data muse raw object MUSE 2014 08 04T09 32 59 174 fits ILLUM SMUSE_CAL MASTER_BIAS fits MASTER_BIAS E_CAL MASTER_FLAT fits MASTER_FLAT E_CAL TWILIGHT_CUBE fits TWILIGHT_CUBE E_CAL TRACE_TABLE fits TRACE_TABLE E_CAL WAVECAL_TABLE fits WAVECAL_TABLI E_CAL badpix_table fits BADPIX_TABLE E_CAL geometry_table fits GEOMETRY_TABLE mi hh E E UY UY Ur 1N 1 Ur C v UN y UF U U 2 gt OMP_NUM_THREADS 24 esorex log file object log muse_scibasic nifu 1 merge object sof Optionally the recipe muse_scibasic
77. cated sky observations should be combined In this case each of science observations has to be calibrated separately using the corresponding sky observation After this is done the fully reduced science observations can be combined with muse_exp_combine Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ae alae Date Date 2015 04 28 Page 52 of 132 7 Tips amp Tricks 7 1 Restricting wavelength ranges All post processing recipes read pixel tables When testing such steps of the data reduction it can be benificial to work on only a subset of the data like a small wavelength range All relevant recipes therefore support the lambdamin and lambdamax recipe parameters This causes the code to still read the full pixel tables but then the pixels with wavelengths not in the given range are discarded This can be used to speed up processing and constrain memory consumption e g to test parameter ranges that affect the cube reconstruction Note however that they may have unforeseen consequences if e g the data are truncated right on a bright sky line 7 2 Verification Tools Usually the QC parameters as documented in Section 9 as well as the messages written to the terminal and the log file by the recipe should give a good basis to verify that the processing worked as expected But in some cases a visual verification is necessary and tools which help with the visualization and the verificat
78. cause gnuplot to write to a file instead of displaying a window ESO MUSE Pipeline User Manual Doc VLI MAN ESO 14670 6186 Issue Issue 9 Date Date 2015 04 28 Page 53 of 132 Yv CHASS SF 4000 3500 3000 2500 2000 1500 1000 500 trace result slices 12 to 20 1000 1100 1200 1300 1400 1500 1600 1700 1090 26 4254 90 4 Figure 7 1 The graphical window showing the output of the muse_trace_plot_samples tool then plotting slices 12 to 20 in IFU 6 using the trace samples table the trace table and the master flat field image see text for details Tip when the default gnuplot setup is used with the x11 wxt or qt terminals one can use the right mouse button on the plot window to zoom the display to a rectangular region When also passing the tracing table on the command line the tool plots the polynomial solutions for both edges and the center ove 1 gt muse_trace r the crosses that mark the sampling points plot_samples s1 1 s2 48 TRACE_SAMPLES 06 fits TRACI E_TABL E 06 fits Here one has to be careful to select files that belong to the same IFU Then one can visually verify that the polynomial solution matches the individual traced points Finally one can also use the master flat field product as background of the plot so that one can actually check that the tracing po 1 gt muse_trace ints
79. ch CCD pixel and its position the data quality and the statistical error are also recorded Since version 1 0 of the MUSE DRS the default file format of the pixel tables 1s based on images in contrast to the table format used in previous releases This change has been introduced for performance reasons gt 5 3 3 MUSE Data Cubes The final output of the MUSE pipeline are the MUSE data cubes Similar to the MUSE image format the data cubes consist of up to three 3D FITS cube NAXIS 3 volume data extensions The FITS cube extensions are In case it should be necessary the old table format is still available and can be restored by setting the environment variable MUSE_PIXTABLE SAVE_AS_TABLE 1 Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ae alae Date Date 2015 04 28 Page 28 of 132 also named DATA DQ and STAT However by default the data quality extension is not present instead pixels which do not have a clean data quality status are directly encoded as Not a Number NaN values in the DATA extension itself In an extended form the data cube may also contain one or more one for each selected filter 2D FITS image extensions with the reconstructed FOV images and the corresponding variance images However by default the FOV images are saved as individual files in the standard MUSE image format 5 3 4 Combined Product Data By default the MUSE calibration recipes write product files w
80. ch name has to correspond to an EXTNAME in an extension of the FIL TER_LIST file If an unsupported filter name is given creation of the respective image is omit ted If multiple filter names are given they have to be comma separated lambdamin double 4000 Cut off the data below this wavelength after load ing the pixel table s lambdamax double 10000 Cut off the data above this wavelength after load ing the pixel table s ESO Doc VLI MAN ESO 14670 6186 MUSE Pipeline User Manual TSU alate Date Date 2015 04 28 Page 106 of 132 9 13 4 Product frames The following product frames are created by the recipe Default file name Description DATACUBE_FINAL Output datacube if save contains cube IMAGE_FOV Field of view images corresponding to the filter parame PIXTABLE_COMBINED Combined pixel table if save contains combined ter if save contains cube 9 13 5 Quality control parameters The following quality control parameters are available for the muse_exp_combine products OC EXPCOMB POSk X Position of source k in x direction in combined frame QC EXPCOMB POSk Y Position of source k in y direction in combined frame QC EXPCOMB FWHMk X FWHM of source k in x direction in combined frame QC EXPCOMB FWHMk Y FWHM of source k in y direction in combined frame Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU Isic Dat
81. comparison is then done afterwards with 100 x stdev to guard against incompatible set tings Has no effect for overscan offset ovscignore int 3 The number of pixels of the overscan adjacent to the data region of the CCD that are ignored when computing statistics or fits combine string sigclip average Type of combination to use median minmax sigclip nlow int 1 Number of minimum pixels to reject with min max nhigh int 1 Number of maximum pixels to reject with min max nkeep int 1 Number of pixels to keep with minmax Isigma double 3 Low sigma for pixel rejection with sigclip hsigma double 3 High sigma for pixel rejection with sigclip scale boolean true Scale the individual images to a common expo sure time before combining them normalize boolean true Normalize the master flat to the average flux trace boolean true Trace the position of the slices on the master flat nsum int 32 Number of lines over which to average when tracing order int 5 Order of polynomial fit to the trace Continued on next page ESO Doc VLI MAN ESO 14670 6186 MUSE Pipeline User Manual TSU ale Date Date 2015 04 28 Page 72 of 132 continued from previous page Parameter Type Values Description default other edgefrac double 0 5 Fractional change required to identify edge when tracing losigmabadpix double 5 Low sigma to find dark pixels in the master flat hisigmabadpix double 5 High sigma to
82. du to count subtracts the dark divides by the flat field and optionally propagates the integrated flux value from the twilight sky cube For special cases users can choose to combine all input files at the image level so that the pixel table is only created once for the combined data This is not recommended for science data where the combination should take place after correcting for atmospheric effects before the creation of the final cube The reduced image is then saved if saveimage true The input calibrations geometry table trace table and wavelength calibration table are used to assign 3D coor dinates to each CCD based pixel thereby creating a pixel table for each exposure If skylines contains one or more wavelengths for bright and isolated sky emission lines these lines are used to correct the wavelength calibration using an offset The data is then cut to a useful wavelength range if crop true If an ILLUM exposure was given as input it is then used to correct the relative illumination between all slices of one IFU For this the data of each slice is multiplied by the normalized median flux of that slice in the ILLUM exposure As last step the data is divided by the normalized twilight cube if given using the 3D coordinate of each pixel in the pixel table to interpolate the twilight correction onto the data This pre reduced pixel table for each exposure is then saved to disk 9 8 2 Input frames
83. e Date 2015 04 28 Page 107 of 132 A Data Formats The MUSE pipeline uses and produces a number of files in different formats which are described in this section For each data format the structure of the FITS extensions is described and the tags of all frames are listed that use this format A 1 Raw Data Files A 1 1 RAW_IMAGE Description Raw CCD images taken with the MUSE instrument Files coming from the instrument usually contain all 24 images from the IFUs in a single file Science exposures may have 3 additional extensions if the MUSE SGS is used FITS extensions e Primary extension Extension 0 header only contains the keywords specific to the exposure and common to all channels such as information about the observing program telescope overall instrument and weather conditions etc e Extensions 1 to 24 contains for each channel a short header with information specific to the detector and the image frame corresponding to this particular channel The order of the extensions in the FITS file does not follow the order of the channel numbers so they should be addressed with their extension name CHANO1 CHANO2 CHANDOS CHAN22 CHAN23 CHAN24 corresponding to the channel number The size of the image frame in the absence of binning is 4224 x 4240 including the overscan regions e Extension 25 to 27 for science exposures contain information from the SGS taken in parallel to the science exposures w
84. e and with the threads pinned to the physical cores using the Likwid tools The results are summarized in the following table Note that these numbers are rough numbers and should just give an indication of what can be expected Recipe Number of input OMP_NUM_THREADS Execution Time Peak Memory Usage frames s GB typical muse_bias 5 24 90 26 muse_flat 5 24 100 27 muse_wavecal 3 24 220 15 muse_lsf 45 24 1300 75 muse_twilight 8 24 500 45 muse_scibasic 1 24 140 15 muse_standard 1 24 290 17 muse_create_sky 1 24 420 17 muse_astrometry 1 24 220 14 muse_scipost 1 24 380 18 25 Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU alae Date Date 2015 04 28 Page 129 of 132 C Performance Tools This section summarizes the usage of two thread pinning tools which can be used on Linux systems to set the CPU affinity of the threaded MUSE recipes This means that the threads which run when a MUSE recipe is executed stay on the processor where they where started in first place This is done for two reasons i to make sure that the threads are running on the physical cores only since the logical cores are less performant and ii to avoid that the threads stick to a processor to avoid that data has to be transferred through the machines memory subsustem which in general has a quite limited bandwidth The first tool taskset is usually already available on recent Linux system or the if it is miss
85. e the originating CCD pixel slice IFU and exposure number of each entry in the table If the exposure column in the output displays zeros then the pixel table only contains one exposure The two CCD columns in the output give the coordinates on the trimmed image while the Raw columns use the un trimmed coordinates found in the unprocessed raw data from the instrument Another tool that might be useful is muse_pixtable_crop to extract part of a pixel table into another file The crop regions can be one of the two spatial axes or the wavelength axis The following command cuts the input table simultaneously in the x direction and in wavelength 1 gt muse_pixtable_crop x1 30 x2 30 11 5570 12 5583 PIXTABLE_OBJECT_0001 12 fits pt1 12_small fits Since the spatial pixel table columns change depending of the stage of the processing care must by taken to use values for the correct units The command therefore echos the range specified with the units expected for a given pixel table The command above outputs MUSE pixel table PIXTABLE_OBJECT_0001 12 fits 13485859 rows cropping to lambda 5570 00 5583 00 Angstrom xpos 3 000e 01 3 000e 01 pix ypos 1 288e 01 4 175e 01 pix MUSE pixel table pt1 12_small fits 7383 rows saved The tool muse_pixtable_erase_slice can be used to remove the data of a complete slice of one IFU from a pixel table When run like this 1 gt muse
86. e following example shows the minimal contents of the set of frames and the EsoRex command to compute the line spread function 1 gt cat l1sf sof data muse raw wave MUSE 2014 02 19T10 58 29 838 fits ARC data muse raw wave MUSE 2014 02 19T11 00 03 593 fits ARC data muse raw wave MUSE 2014 02 19T11 02 10 205 fits ARC SMUSE_CAL MASTER_BIAS fits MASTER_BIAS SMUSE_CAL TRACE_TABLE fits TRACE_TABLE SMUSE_CAL WAVECAL TABLE fits WAVECAL TABLE SMUSE_CAL line_catalog fits LINE_CATALOG 14 al 2 gt OMP_NUM_THREADS 24 esorex log file lsf log muse_lsf nifu 1 A merge lsf sof The primary product of this EsoRex command is the Isf profile which is a stack of slice specific images of the line spread function for each IFU Generating an Isf profile is computational heavy Using a standard set of 5 exposures per arc lamp the EsoRex command shown before uses approximately 80 GB of memory and takes about 30 minutes As an alternative to creating the Isf profile with muse_lsf the Isf profiles that come as part of the MUSE pipeline package could be Doc VLT MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ae alate Date Date 2015 04 28 Page 39 of 132 JETA MAN Mi MIME id MEE 5 7 0 0086 5 8 11 17 23 Figure 6 3 Example of a resampled arc lamp frame of a single IFU channel 1 as it is created by the recipe muse_wav
87. e for creating the final data cube from a single MUSE observation and the required set of calibrations is e 64 GB of memory e 24 CPU cores physical cores e 4 TB of free disk space e GCC 4 8 2 or newer The peak memory consumption depends on the number of input exposures used and the size of the final data cube When a data cube is created from a single MUSE exposure the peak memory usage may vary between approximately 18GB and 25 GB depending on the orientation of the field of view Peak memory consumption is reached at a rotation angle of 45 and 135 with respect to North On disk the size of the final data cube file may vary between 4GB and 5 3 GB for a single observation If itis forseen to combine several MUSE observations the memory needed depends on the maximum number of observations to combine In general the memory consumption grows linearly with the number of observations Finally when it comes to creating MUSE mosaics one should be aware that the size of the data cube may become really huge and the required memory grows accordingly For details on combining MUSE exposures refer to Section 6 6 The same memory requirements apply to the MUSE calibration recipes The minimum memory estimate of 32 GB given before is suitable for processing standard calibration sets of 5 exposures Adding exposures to the input of the recipes requires additional memory Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User
88. e product files are named in the same way but with the OBJECT part replaced by the corresponding frame tag The 4 digit number indicates the index of the input exposure which in this case is always 0001 since there was only a single raw science exposure present in the SOF If more than one raw exposure is present the index will run from 1 to the number of exposures The full description of the muse_scibasic recipe can be found in Section 9 6 5 Observation Post Processing This section covers the second stage of the MUSE data reduction cascade shown in Figure 2 3 Again the pro cessing steps are executed from left to right although only the response computation is mandatory for creating the final data cube 6 5 1 Flux Calibration In this first processing step the response curve for the flux calibration of the science observations is created from a pre processed standard star observation The raw standard star observation has already been processed with muse_scibasic and resulting 24 pixel tables are now located in the current working directory In addition to the set of 24 pixel tables the recipe requires an exintion table and a standard flux table Optionally a list of predefined telluric regions may be present in the input SOF which is omitted in the following The standard flux table is a catalog of reference spectra of spectrophotometric standard stars The current version of this catalog which is included in the MUSE pi
89. e_exp_combine lambdamin 6282 5 lambdamax 7817 5 ec sof 4 gt mv DATACUBE_FINA fits CUBE_green fits 5 gt esorex muse_exp_combine lambdamin 7815 ec sof 6 gt mv DATACUBE_FINA 1 fits CUBE_red fits and the following set of frames 1 gt cat ec sof PIXTABLE_REDUCED_0001_EXP01 fits PIXTABLE_REDUCED PIXTABLE_REDUCED_0001_EXP02 fits PIXTABLE_REDUCED PIXTABLE_REDUCED_0001_EXP03 fits PIXTABLE_REDUCED PIXTABLE_REDUCED_0001_EXP04 fits PIXTABLE_REDUCED PIXTABLE_REDUCED_0001_EXP05 fits PIXTABLE_REDUCED PIXTABLE_REDUCED_0001_EXP06 fits PIXTABLE_REDUCED PIXTABLE_REDUCED_0001_EXP07 fits PIXTABLE_REDUCED PIXTABLE_REDUCED_0001_EXP08 fits PIXTABLE_REDUCED PIXTABLE_REDUCED_0001_EXP09 fits PIXTABLE_REDUCED cube_wcs_header fits OUTPUT_WCS The FITS header of the reference file cube_wcs_header fits contains the world coordinate system cov ering the full axis ranges of the final data set The WCS information must be given using the CDi _3 notation Then to combine the three partial data sets one can run muse_cube_combine as shown in the following 1 gt muse_cube_combine CUBE_COMBINED fits CUBE_blue fits CUBE_green fits CUBE_red fits This will automatically analyze the PRO REC1 PARAMi NAME and PRO REC1 PARAMi VALUE ke
90. ecal This image is the result of combining the three individual arc lamp images and applying the determined wavelength solution As it is expected from a good wavelength solution the arc lines appear as straight horizontal lines i e they are now on the same wavelength used in the following reduction steps The analysis of MUSE commissioning data show that the parameters of the line spread function are stable so that a frequent update of the Isf profile is not necessary For the full description of the muse_lsf recipe please refer to Section 9 Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ae alte Date Date 2015 04 28 Page 40 of 132 6 3 6 Instrument Geometry The instrument geometry or geometrical calibration determines where in the field of view each slice of each IFU is located The result is the geometry table which can be created using the muse_geometry recipe However this recipe requires special input data and some expert knowledge to get to a good result For this reason geometry tables are carefully prepared in advance They are considered as a static calibration which is distributed as part of the MUSE pipeline release package Therefore an example and a description of the recipe muse_geometry is not shown here 6 3 7 Illumination Correction Twilight flats are part of the regular calibrations taken for MUSE In this processing step the raw sky flat frames are combined into a
91. ed by T Urrutia O Streicher and P Weilbacher for use within the MUSE consortium We also would like to thank J Richard P Weilbacher and B Husemann for providing valuable comments 1 3 Stylistic Conventions Throughout this document the following stylistic conventions are used bold in text sections for commands and other user input which has to be typed as shown italics in the text and example sections for parts of the user input which have to be replaced with real contents teletype in the text for FITS keywords program names file paths and terminal output and as the general style for examples commands code etc In example sections expected user input is indicated by a leading shell prompt In the text bold and italics may also be used to highlight words 1 4 Notational Conventions Hierarchical FITS keyword names appearing in the document are given using the dot notation to improve readability This means that the prefix HIERARCH ESO is left out and the spaces separating the keyword name constituents in the actual FITS header are replaced by a single dot ESO MUSE Pipeline User Manual Doc VLI MAN ESO 14670 6186 Issue Issue 9 Date Date 2015 04 28 Page 14 of 132 15 Reference Documents RD1 RD2 RD3 RD4 RD5 RD6 RD7 RD8 RD9 MUSE Pipeline Manual MUSE User Manual MUSE Calibration Plan Gasgano User s Manual FITS format description for pipe
92. ed in the following with short description of the contents of each of the files For a detailed description of the data layout please refer to Appendix A 2Due to the format of the MUSE raw data files it is actually possible to use the compressed files if the suffix fz is removed However in general other instruments this may not work 3The term channel is equivalent to IFU and the two terms may be used interchangably Here static means that these files are not created by a recipe but prepared manually by other means They may however change from one pipeline release to the next ESO Doc VLI MAN ESO 14670 6186 MUSE Pipeline User Manual TSU alae Date Date 2015 04 28 Page 26 of 132 astrometry_reference fits badpix_table fits extinct_table fits filter_list fits line_catalog fits geometry_table_wfm fits sky_ std_ lines fits flux_table fits vignetting_mask fits The catalog positions of the astrometric reference object for all the fields used according to the MUSE calibration plan Each field is stored in a separate extension The detector positions of well known bad pixels The data for each channel is stored in a separate extension The positions listed here will be propagated as part of the data quality extension of the product files Atmospheric extinction at the Paranal observatory as a function of wavelength A in units of mag a
93. er calibrations are created from raw calibration data and they are applied in the pre processing step The different steps of the basic reduction are shown in Figure 2 2 and will be executed from left to right As already mentioned in Section 2 2 the recipes of this first processing stage are working on the data of a single CCD and there are basically 3 ways to run the recipes e manual mode e serial mode e parallel mode using threads where the different modes are selected with the recipe option nifu For the manual mode one has to provide the number of the IFU to process 1 e a number in the range from 1 to 24 With this mode one can also process several IFUs in parallel by launching more than one EsoRex command The serial mode is selected with nifu 0 In this case all IFUs are processed one after the other with a single EsoRex command Parallel processing of all IFUs is selected with nifu 1 All three option can use the same set of frames provided that it contains the entries for all 24 IFUs In the following examples the data is processed in parallel and the recipe products are written using the combined format i e the recipe options ni fu 1 and merge are used in the examples whenever possible The prefix command for pinning the threads is omitted in order to keep the command line examples easy to read How to call EsoRex with pinning the threads is shown in Section C For reasons of clarity the
94. er to have a good performance when processing MUSE data the MUSE DRS recipes are multi threaded using the OpenMP thread model and thus may use all the CPU cores that are available to speed up the process ing While this mode has to be explicitly enabled using a recipe parameter for the MUSE calibration recipes and the preprocessing recipe this is always used in the second stage recipes of the MUSE DRS It is assumed that this is the standard way to run the MUSE DRS However it is not always the best solution to simply try to use all available CPUs on the machine For instance reducing the number of allowed threads can be used to reduce the memory consumption of the first stage recipes of course this increases the recipes execution time The maximum number of threads the MUSE DRS recipes actually any application using OpenMP may can be set using the environment variable OMP_NUM_THREADS In addition modern computer architectures have certain features which may make it necessary to control the way high performance applications are executed on a particular machine in order to get to a good performance This applies also to the MUSE DRS What may be necessary is to make sure that the individual threads the application runs stick to the CPU where they were started in first place There are several so called thread pinning tools available which can be used for this For example Linux distributions come with the taskset utility or one can use the
95. eral separate bias images into one master bias file The master bias contains the combined pixel values in adu of the raw bias exposures with respect to the image combination method used Processing trims the raw data and records the overscan statistics corrects the data levels using the overscan 1f overscan is not none and combines the exposures using input parameters The read out noise is computed for each quadrant of the raw input images and stored as QC parameter The variance extension is filled with an initial value accordingly before image combination Further QC statistics are computed on the output master bias Additionally bad columns are searched for and marked in the data quality extension 9 1 2 Input frames Category Type Constraint min BIAS raw required 3 BADPIX_TABLE calib optional usually not used 9 1 3 Recipe parameters Parameter Type Values Description default other nifu int 0 IFU to handle If set to O all IFUs are processed serially If set to 1 all IFUs are processed in parallel overscan string vpoly If this is none stop when detecting discrepant overscan levels see ovscsigma for offset it assumes that the mean overscan level represents the real offset in the bias levels of the exposures involved and adjusts the data accordingly for vpoly a polynomial is fit to the vertical over scan and subtracted from the whole quadrant Continued on next page
96. ers The input calibrations geometry table trace table and wavelength calibration table are used to assign 3D coor dinates to each CCD based pixel thereby creating a pixel table from the master sky flat These pixel tables are then cut in wavelength using the lambdamin and lambdamax parameters The integrated flux in each IFU is computed as the sum of the data in the pixel table and saved in the header to be used later as estimate for the relative throughput of each IFU The pixel tables of all IFUs are then merged using the integrated fluxes as inverse scaling factors and a cube is reconstructed from the merged dataset using given parameters A white light image is created from the cube This skyflat cube is then saved to disk with the white light image as one extension To construct a smooth 3D illumination correction the cube is post processed in the following way the white light image is used to create a mask of the illuminated area From this area the optional vignetting mask is removed The smoothing is then computed for each plane of the cube the illuminated area is smoothed by a 5x7 median filter normalized fit with a 2D polynomial of given polynomial orders and normalized again A smooth white image is then created by collapsing the smooth cube If a vignetting mask was given the corner area given by the mask is used to compute a 2D correction for the vignetted area the original unsmoothed white light image is correc
97. es are created by the recipe Default file name Description MASTER_DARK Master dark ESO Doc VLI MAN ESO 14670 6186 MUSE Pipeline User Manual TSU alae Date Date 2015 04 28 Page 69 of 132 9 2 5 Quality control parameters The following quality control parameters are available for the muse_dark products Qc Qc QC QC Qc Qc Qc Qc QC Qc DAR DAR DAR DAR DAR DAR DAR DAR DAR DAR K K K K K K K K K K INPUTi NSATURATED MASTE MASTE MASTE MASTE MASTE MASTE MASTE MASTE MASTE N NN NNNUNN NSATURATED Number of saturated pixels in raw dark i in input list NBADPIX Number of bad pixels determined from master dark MEDIAN Median value of the master dark MEAN Mean value of the master dark STDEV Standard deviation of the master dark MIN Minimum value of the master dark MAX Maximum value of the master dark DC Dark current measured on master dark in randomly placed windows DCERR Dark current error measured on master dark in randomly placed windows Number of saturated pixels in output data Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU alae Date Date 2015 04 28 Page 70 of 132 9 3 muse flat Combine several separate flat images into one master flat file trace slice locations and locate dark pixels 9 3 1
98. ever it is possible to limit the wavelength range to process using the recipe options lambdamin and lambdamax which will reduce the memory usage cf Section 7 1 The recipe muse_scipost accepts the pixel tables of more than one exposure as input In this case the exposures will be combined into a single data cube by default However in order to obtain the expected results when combining exposures read Section 6 6 first As long as single science exposures are concerned the data processing has finished at this point and one can start with the scientific analysis of the data For the full description of the muse_scipost recipe please refer to Section 9 6 6 Combining Exposures This section shows how multiple exposures are combined efficiently and correctly and should help to bypass some pitfalls The combination of exposures i e the combination of the pre processed pixel tables of a scien tific exposure can be done with two different methods recipes The exposures can be combined directly in muse_scipost or one can use the utility recipe muse_exp_combine The latter uses a fully reduced pixel table which is an intermediate product of muse_scipost There are two issues which have to be considered when multiple exposures are combined the correction of coordinate offsets and limiting the wavelength range 6 6 1 Correcting Coordinate Offsets When combining multiple exposures the pipeline recipes muse_scipost and muse_exp_com
99. ew images corresponding to the filter parameter OBJECT_RED muse_scibasic Pre processed CCD based images for OBJECT input if saveimage true OBJECT_RESAMPLED muse_scibasic Resampled 2D image for OBJECT input if resample true muse_scipost Stacked image if save contains stacked muse_scipost_make_cube Stacked image if stacked true STD_RED muse_scibasic Pre processed CCD based images for STD input if saveimage true STD_RESAMPLED muse_scibasic Resampled 2D image for STD input if resample true SKY_RED muse_scibasic Pre processed CCD based images for SKY input if saveimage true SKY_RESAMPLED muse_scibasic Resampled 2D image for SKY input if resample true ASTROMETRY_RED muse_scibasic Pre processed CCD based images for ASTROMETRY input if saveimage true ASTROMETRY_RESAMPLED muse_scibasic Resampled 2D image for ASTROMETRY input if resample true REDUCED_RESAMPLED muse_scibasic Resampled 2D image if resample true A 3 2 PIXEL_TABLE Description In the reduction approach of the MUSE pipeline data need to be kept un resampled until the very last step The pixel tables used for this purpose can be saved at each intermediate reduction step and hence contain lists of pixels together with output coordinates and values By default they are saved as multi extension FITS images where each extension corresponds to one table column The name of the column is saved in the EXTNA
100. fied us ing image statistics and marked in the data quality extension The combined image is normalized to the specified exposure time QC statistics are computed on the output master dark 9 2 2 Input frames Category Type Constraint min DARK raw required 3 MASTER_BIAS calib required 1 BADPIX_TABLE calib optional usually not used 9 2 3 Recipe parameters Parameter Type Values Description default other nifu int 0 IFU to handle If set to 0 all IFUs are processed serially If set to 1 all IFUs are processed in parallel overscan string vpoly If this is none stop when detecting discrepant overscan levels see ovscsigma for offset it assumes that the mean overscan level represents the real offset in the bias levels of the exposures involved and adjusts the data accordingly for vpoly a polynomial is fit to the vertical over scan and subtracted from the whole quadrant ovscreject string der This influences how values are rejected when computing overscan statistics Either no rejec tion at all none rejection using the DCR algo rithm der or rejection using an iterative con stant fit fit Continued on next page Doc VLT MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ISSUE teous Date Date 2015 04 28 Page 68 of 132 continued from previous page Parameter Type Values Description default other
101. filters given on the command line Thus the input set of frames will contain the 24 pre processed pixel tables of the exposure of the science target the estimated fluxes of the sky lines the response curve the telluric correction the extinction table the astrometric calibration and the list of filter transmission curves Note The astrometric calibration to be used here should be created from the the same geometry table as the input pixel tables see Section 6 5 3 for details The input set of frames and the EsoRex command to run muse_scipost are show in the following Note that de pending on the orientation of the original scientific exposure cf Section 3 1 this command will need between approximately 18 GB and 25 GB of RAM in order finish sucessfully 1 gt cat scipost sof PIXTABLE_OBJECT_0001 01 fits PIXTABLE_OBJEC E_OBJECT_0001 24 fits PIXTABLE_OBJEC E_CAL LSF_PROFILE fits LSF_PROFIL E_CAL STD_RESPONSE_0001 fits STD_RESPONSE E_CAL STD_TELLURIC_0001 fits STD_TELLURIC CAL ASTROMETRY_WCS_0001 fits ASTROMETRY_WCS E_CAL sky_lines fits SKY_LINES E_CAL extinct_table fits EXTINCT_TABLE E_CAL filter_list fits FILTER_LIST x FI H GCG GOO CQ O C W2 D tu H DANNNANAN Y 1 UY UY 41 Y y y YU 2 gt O P_NUM_THREADS 24 esorex log file scipos
102. finding matching calibrations is just made of a few key words Since there is only a single detector configuration available for MUSE there is no real need to verify that the detector parameters of the raw data frames and the calibrations match in principle the header keywords DET BINX DET BINY and DET READ CURID or DET READ CURNAME could be used To find a calibrations with a matching instrument setup one can use one of the header keywords INS MODE and INS OPTI2 NAME depending whether the whole instrument configuration should match or only the field mode The keyword INS MODE provides information on the field mode wide or narrow field mode whether the AO was used and whether the nominal or the extended wavelength range 1 e with or without the Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ae alae Date Date 2015 04 28 Page 33 of 132 blue cutoff filter in place was selected The keyword INS OPTI2 NAME contains only the setup of the field mode It may be used in cases where the other parameters of the instrument setup do not matter Finally if one obtains calibrations as part of an archive request from the ESO archive then the ESO archive took already care that they match the raw data of the request Thus there may be no need to redo the data association 6 3 Basic Reduction Now it is time to start processing the data During the basic reduction the mast
103. fits Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ae alae Date Date 2015 04 28 Page 36 of 132 In the following processing steps the master dark will not be used For the full description of the muse_dark recipe please refer to Section 9 6 3 3 Flat Field In this processing step the recipe muse_flat is used to combine raw lamp flat field frames into a master flat frame In addition to that the slices are located on the image and their edges are traced along the wavelength axis vertical axis Finally bright and dark pixels are located The set of frames must contain at least 3 raw flat field frames and the master bias frame Optionally the master dark frame may added too but it has been omitted in the following The optional bad pixel table has been used in the following example However note that the use of a bad pixel table may actually degrade the tracing solution if it contains bad columns in particular if they are located near the edge of a slice 1 gt cat flat sof data muse raw flat MUSE 2014 02 21T12 14 59 316 fits FLAT data muse raw flat MUSE 2014 02 21T12 16 25 212 fits FLAT data muse raw flat MUSE 2014 02 21T12 18 10 540 fits FLAT data muse raw flat MUSE 2014 02 21T12 19 34 837 fits FLAT data muse raw flat MUSE 2014 02 21T12 20 48 296 fits FLAT SMUSE_CAL MASTER_BIAS fits MASTER_BIAS SMUSE_CAL badpix_table fits BADPIX_TABLE 2 gt O
104. he other exposures one finally ends up with one fully reduced pixel table for each exposure These can now be used as input for the recipe muse_exp_combine to create a final combined data cube The advantage is that creating cubes for different wavelength bands or combination of the exposures now involves only the resampling step To get to the same final data cube as in the previous section the command to be issued is 1 gt cat exp_combine sof PIXTABLE_REDUCED_0001_EXP01 fits PIXTABLE_REDUCED PIXTABLE_REDUCED_0001_EXP02 fits PIXTABLE_REDUCED PIXTABLE_REDUCED_0001_EXP03 fits PIXTABLE_REDUCED SMUSE_CAL filter_list fits FILTER_LIST 2 gt OMP_NUM_THREADS 24 MUSE_XCOMBINE_RA_OFFSETS 0 0000000 5 4920943e 05 0 00031381379 MUSE_XCOMBINE_DEC_OFFSETS 0 0000000 8 3338105e 05 7 2223397e 05 sorex log fil xp_combine log muse_exp_combine lambdamin 6900 lambdamax 7100 exp_combine sof The product files created by this command are I gt ls 1 fits DATACUBE_FINAL fits IMAGE FOV_0001 fits Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU ald Date Date 2015 04 28 Page 51 of 132 And finally there are also other scenarios where combining the fully reduced pixel tables using the recipe muse_exp_combine is useful For instance when science observations which are accompanied by dedi
105. hen SGS is activated The SGS will record images with the TCCD and produce stack median images every approx 2 min These median images can be average over the entire science exposure to give a deeper image of the region surrounding the MUSE FOV Extension 25 SGS_IMG an image of size 1024 x 1024 contains the average of all the stacked median images taken during the science exposure Extension 26 SGS_CUBE a cube of 1024 x 1024 x N pixels containing all N stacked median images taken during the science exposures Extension 27 SGS_DATA a FITS table containing information from the SGS system in the form of 4 NSTARS x 10 x N entries for the N measurements done using NSTARS NSTARS lt 10 stars detected in the SGS For Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU aed Date Date 2015 04 28 Page 108 of 132 each measurement the 4 first columns give general information about the time and the offsets sent to the telescope while the last 10 x NSTARS entries give information on each star Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU alae Date Date 2015 04 28 Page 109 of 132 A 2 Static Calibration Files A 2 1 LINE_CATALOG Description This is a list of arc lines to be used for wavelength calibration It is a FITS table with one row for each line which contains central wavelength of the line in question and a relative
106. hese are environment variables and not recipe options they are set outside of the recipe call In the bash shell one can simply add this as a prefix to the EsoRex command like 1 gt MUSE_DEBUG_WAVECAL 1 esorex muse_wavecal wavecal sof which defines the environment variable only for this execution of EsoRex To use it across several EsoRex executions or your bash script one can use the following prior to running the command 1 gt export MUSE J EBUG_WAVECAL 1 A complete list of the environment variables can be found at the end of the README file which by default can be found in lt installation prefix gt share doc esopipes muse X Y Z Remember that these variables are only exist to support recipe debugging They should be used with caution since they might generate a large number of files output that one may be unfamiliar with or cause unexpected side effects Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU alae Date Date 2015 04 28 Page 64 of 132 9 Recipe Reference This section is essentially the reference manual of the MUSE pipeline recipes In the following the documen tation of the invididual pipeline recipes is provided in terms of input data recipe parameters output products and QC parameters created 9 1 muse_bias Combine several separate bias images into one master bias file 9 1 1 Description This recipe combines sev
107. hich contain the data belonging to a single IFU and therefore write 24 files for each kind of product For example the recipe muse_bias creates 24 master bias products on disk This keeps the file size small and the product files can be easily handled by standard tools visualization tools FITS header viewers etc In addition to that there is a combined format for the product files where the results for the individual IFUs are stored in a single FITS file similar to the MUSE raw data files In this format the extensions are named as in the individual products with a prefix identifying the IFU from which it originates For example a combined master bias file contains 72 FITS extensions data error and data quality extensions for 24 IFUs which are called CHAN lt nifu gt DATA CHAN lt nifu gt DO and CHAN lt nifu gt STAT with nifu 01 24 With the exception of the pixel tables the combined format is available for all pipeline products which would oth erwise be saved to disk as 24 individual files The MUSE recipes can read combined products as well as products stored as individual files For different calibrations the two product formats may even be mixed For instance the master calibration files lsf_profile_slow_wfm n fits and lsf_profile_slow_wfm e fits which are distributed as part of the MUSE pipeline kit are stored as combined products As of version 1 0 of the MUSE DRS the recipe option merge can be used to enable writing rec
108. ilight sky As an additional product the data cube created from the combined twilight images is available It is the data cube of the twilight sky before the smoothing is applied The recipe works on the data of all 24 IFUs simultaneously in the same way as the post processing recipes do Thus the recipe options ni fu and merge are not applicable to this recipe For the full description of the muse_twilight recipe please refer to Section 9 lOSince version 1 0 the recipe muse_twilight replaces the recipe muse_skyflat Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ae alae Date Date 2015 04 28 Page 41 of 132 6 4 Observation Pre processing At this point all necessary calibrations have been created to remove the instrument signature from the on sky exposures using the pre processing recipe muse_scibasic This also converts the observations to the pixel table format which is the input format for the post processing recipes During this conversion sky lines are used to correct for offsets in the wavelength solution and optionally the data is corrected for the relative illumination of the slices for individual IFUs Pixel tables can be saved as FITS images or as a true FITS binary tables To benefit from the more efficient way to read and write FITS images the results are by default saved using the FITS image based format If the pixel tables should be stored a FITS binary tables then this
109. ing it can be obtained by installing the package ut il linux using the system package manager The second tool Likwid is actually a tool suite and more flexible and convenient to use but requires a manual installation Likwid is what is used at ESO C 1 Using taskset This command is usually available on modern Linux distribution For example to run the muse_bias recipe using taskset one has to execute the following command line 1 gt taskset c 0 5 esorex muse_bias nifu 1 bias sof This will run muse_bias on the first 6 CPUs of the machine For a detailed description of the taskset command try 1 gt man taskset C 2 Using the Likwid Lightweight Performance Tools This tool suite has been developed to support programmers in writing high performance multi threaded code As such it provides a larger set of tools but as a user only two are really of interest As mentioned before the tools have to be installed manually This also requires editing the file config mk if one does not have root privileges to do a system wide installation However this is straight forward by following the provided instructions in the file INSTALL Once the tool suite has been installed the usage is very similar to taskset However in addition to the actual thread pinning tool it provides a second tool which can be used to query the topology of the computer This first tool 1ikwid topology provides information on the machines architechture
110. ing smoothing x order for polyfit default recommended 4 pa rameter that influences the FWHM for the gaus sian recommended 10 or x dimension of me dian filter recommended 5 vignypar int 4 Parameter used by the vignetting smoothing y order for polyfit default recommended 4 pa rameter that influences the FWHM for the gaus sian recommended 10 or y dimension of me dian filter recommended 5 ESO Doc VLI MAN ESO 14670 6186 MUSE Pipeline User Manual TSU alae Date Date 2015 04 28 Page 86 of 132 9 7 4 Product frames The following product frames are created by the recipe Default file name Description DATACUBE_SKYFLAT Cube of combined twilight skyflat exposures TWILIGHT_CUBE Smoothed cube of twilight sky 9 7 5 Quality control parameters The following quality control parameters are available for the muse_twilight products QC QC Qc Qc Qc QC Qc Qc QC QC QC QC QC MEDIAN Median value of raw exposure i of IFU m in input list MEAN Mean value of raw exposure i of IFU m in input list STDEV Standard deviation of raw exposure i of IFU m in input list MIN Minimum value of raw exposure i of IFU m in input list MAX Maximum value of raw exposure i of IFU m in input list NSATURATED Number of saturated pixels in raw exposure i of IFU m in TWILIGHTm INPUTi TWILIGHTm INPUTi
111. ion of the results may be needed A few such tools are shipped with the MUSE pipeline and get installed into lt installation prefix gt bin They are described in this section 14 The visual tools use gnuplot for plotting gt All tools mentioned here give a usage hint when called without parameters 7 2 1 Verification of the tracing solution When one has doubts about the validity of the tracing solution computed by the muse_flat recipe one can specify the samples parameter so that the extra output product TRACE_SAMPLES is written one file per IFU This file contains all tracing samples computed by the recipe i e left and right edge as well as the slice center at many vertical positions These can be plotted using the tool muse_trace_plot_samples If just using this file only the central two slices are plotted 1 gt muse_trace_plot_samples TRACE_SAMPLES 06 fits If one also passes the number of the slices to show one can e g plot all slices 1 gt muse_trace_plot_samples s1 1 s2 48 TRACE_SAMPLES 06 fits 3 Since the pixel tables are not and cannot be sorted reading the full tables is necessary It causes a temporary peak in memory usage to at least the size of the pixel table 14 Available from http www gnuplot info The plots can hence be customized in the same way as other gnuplot based scripts One can use e g using the file SHOME gnuplot to set up the preferred terminal type or
112. ipe products using the combined format Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ae alae Date Date 2015 04 28 Page 29 of 132 6 Data Reduction Cookbook 6 1 Getting Started with EsoRex EsoRex is a command line tool which can be used to execute not only the MUSE DRS recipes but the recipes of all standard VLT VLTI instrument pipelines With EsoRex in your path the general structure of an EsoRex command line is 1 gt esorex esorex options recipe recipe options sof sof where options appearing before the recipe name are options for EsoRex itself and options given after the recipe name are options which affect the recipe All available EsoRex options can be listed with the command 1 gt esorex help and the full list of available parameters of a specific recipe can be obtained with the command 1 gt esorex help lt recipe name gt The output of this command shows as parameter values the current setting 1 e all modifications from a config uration file or the command line are already applied The listing of all recipes known to EsoRex can be obtained with the command 1 gt esorex recipes The last arguments of an EsoRex command are the so called set of frames A set of frames is a simple text file which contains a list of input data files for the recipe Each input file is followed by an unique identifier frame classification or frame tag indicati
113. irmass The filter transmission as a function of wavelength A for various filters which are used to define the wavelength band for the recon struction of field of view images from the observed spectra Each transmission curve is stored in a separate extension A list of air wavelengths A of arc lamp lines Additional informa tion on relative flux originating ion and line quality flags are also present and may be used by the DRS The wide field mode geometrical calibration i e the mapping of the individual slices from the CCD of each IFU into the MUSE field of view A list of OH line transitions and other sky lines used for modelling and correction of the sky background The spectra of the spectrophotometric standard stars used according to the MUSE calibration plan The spectra of the each standard star are stored in a separate extension For each standard star the air wavelength A the flux ergs cm A and its error are given It is a mask to correct for the vignetting which usually is present in the lower right corner of the MUSE field of view For calibration files like the astrometric catalog the filter list or the standard flux table the name of the field the filter or the object is the name of the extension and can be obtained by looking up the EXTNAME header keyword In addition to this set of static calibrations the distribution kit will also contain a few master calibration files which in princi
114. left blank Doc VLT MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ISSUE teous Date Date 2015 04 28 Page 5 of 132 Contents 1 Introduction 13 Ll ICM ch ee ogee ee oe Gees eee Ee eM E E a Gea er ee da ed i 13 12 Acknowledgements coi ip ora o A A OR Re ee be bE ba A a 13 L3 Styliste Conventions scs e a ee we we dee o ed RE a G 13 1 4 Notational Conventions se ss cawr be ee eR Eh a a ee ES 13 1 3 Reference Documents sido a KR we Ww ae er GES 14 16 Abbreviations atid ACRONYMS s o a eg A a Rae a ee et ee 15 2 Overview 16 21 Whe MUSE Ts iment kw Gk as eRe a ee ae ek ws ie a eS eS ek ae 16 22 The MUSE Data Reduction Pipeline s2 0 004 cc a ed ea ae ea eS 16 3 Installation 20 3 1 System Requirements Please Read Carefully o o oo o 20 z2 Installing the Sofware s oioi a e A A AE eee e e 21 33 Toolchain Support esoo 26 be aa eK a A te 21 3 4 Hints on Running the MUSE Pipeline Recipes 0 o 22 35 Hints on Using 3rd Party Tools u ooo er RO q cd a k ee 22 4 Known Issues 24 5 Data Description 25 2L RAR Daa see Bead eh ee NA a a A ee a 25 32 Statie Calibration Data sco i e aeie e ai a a Ae ee E hae bee eS 25 5 5 Pipeline Products s e cos ewe o A a A ee 27 Sal MUSE IMOLE ond 2 ias eek ee wed wd add De Sedo weed Sake Zand 27 3 42 MUSE Pixel Tables dos fc et wre he Ws We es ar doe e dd 27 339 MUSE Data CUDES o copio a don doe eA Bee Be
115. les are composed of 24 FITS extensions one for each IFU which contain the detector data and the IFU detector specific keywords and a primary FITS HDU which contains only keywords The keywords in this primary HDU contain almost all the information which is needed to correctly identify and process the individual files In addition science exposures will contain an extra three FITS extensions which contain information from the MUSE Slow Guiding System SGS if the SGS is activated see Section A for details The 24 extensions containing the detector data of the IFUs can be identified using the contents of the EXTNAME FITS keyword The extensions are called CHANO 13 CHANO2 etc For technical reasons the numbering of the FITS extensions does not correspond to the numbering of the MUSE channels so that individual channels should be looked up by name However the sequence of the channels as they are stored in the FITS file is fixed 5 2 Static Calibration Data In addition to the calibration data which are created by the MUSE calibration recipes the MUSE DRS requires a number of so called static calibrations which are prepared manually and which are part of the MUSE DRS distribution package After the installation procedure is complete the static calibrations are located in lt installation prefix gt calib muse X Y Z cal unless a different place has been specified dur ing the installation The set of static calibrations is summariz
116. letely determined by the header keyword PRO CATG and the keyword value is also the frame tag to be used in the set of frames The latter also applies to the static calibration files Table 6 1 summarizes the the valid raw data frame tags which are defined for the MUSE together with the corresponding combination of DPR keywords and the name of the MUSE recipe used to process them To classify MUSE raw data files it is almost always sufficient to just look at the keyword DPR TYPE In particular the keyword DPR TECH is always equal to IFU for all MUSE raw data files Although the input files to the second stage recipes are strictly speaking pipeline product files and therefore Doc VLTI MAN ESO 14670 6186 Issue Issue 9 ESO MUSE Pipeline User Manual Date Date 2015 04 28 Page 32 of 132 Frame tag Recipe DPR CATG DPR TYPE DPR TECH BIAS muse_bias CALIB BIAS IFU DARK muse_dark CALIB DARK IFU FLAT muse_flat CALIB FLAT LAMP IFU SKYFLAT muse_twilight CALIB FLAT SKY IFU ARC muse_wavecal CALIB WAVE IFU MASK muse_geometry CALIB WAVE MASK IFU STD muse_scibasic CALIB STD IFU ASTROMETRY muse_scibasic CALIB ASTROMETRY IFU SKY muse_scibasic SCIENCE SKY IFU OBJECT muse_scibasic SCIENCE OBJECT IFU ILLUM muse_scibasic CALIB FLAT LAMP ILLUM IFU Table 6 1 Raw data frame tags of the first stage recipes their frame tag is given b
117. lices that are affected is very low one may choose to remove them from pixel tables created by muse_scibasic and thus from the final combined cube To do so the tool muse_pixtable_erase_slice cf Section 7 2 is provided The general syntax of the command is muse_pixtable_erase_slice lt pixtable in gt lt ifu gt lt slice gt lt pixtable out gt For instance to remove the most problematic slice of IFU 6 which is slice 10 one should run 1 gt muse_pixtable_erase_slice PIXTABLE_OBJECT_0001 06 fits 6 10 PIXTABLE_OBJECT_0001 06_ERASED fits 5 At this point one can continue as usual with running muse_scipost 8 3 The Logfile The logfile contains a lot of information that is related to the data reduction Especially if you encounter a problem reading the logfile is likely to give you an idea at which point in the process the problem occurred The logfile displays the following messages preceded by a time stamp Doc VLT MAN ESO 14670 6186 ESO MUSE Pipeline User Manual a alate Date Date 2015 04 28 Page 63 of 132 INFO These lines tell the user what processing the pipeline is doing at which point and with which files DEBUG Here more technical details and information are given e g the number of pixels rejected in a cosmic ray rejection Usually one needs to change settings to see them i e use msg level debug or log level deb
118. line products with data error and data quality information The Euro3D Data Format Kissler Patig et al Issue 1 2 May 2003 3D Visualization Tool Manual Reflex MUSE Tutorial Reflex User Manual TBD VLI MAN ESO 14670 1477 VLI MAN ESO 14670 0500 VLTI PRO ESO 19000 1932 VLI MAN ESO 19500 5667 VLI MAN ESO 19500 565 1 VLTI MAN ESO 19540 6195 VLI MAN ESO 19000 5037 ESO Doc VLI MAN ESO 14670 6186 MUSE Pipeline User Manual TSU alae Date Date 2015 04 28 Page 15 of 132 1 6 Abbreviations and Acronyms AO CCD CPL CPU DFS DRS ESO EsoRex FITS FOV GCC GUI HDU IFU LSF MUSE NaN NFM OpenMP PAF pixel PSF QC SGS SOF SV spaxel TBC TBD VLT voxel WCS WFM Adaptive Optics Charge Coupled Device Common Pipeline Library Central Processing Unit Data Flow System Data Reduction System European Southern Observatory ESO Recipe Execution Tool Flexible Image Transport System Field of View GNU Compiler Collection Graphical User Interface Header Data Unit Integral Field Unit Line Spread Function Multi Unit Spectroscopic Explorer Not a Number Narrow Field Mode Open Multi Processing VLT parameter file format picture element of a raster image Point Spread Function Quality Control Slow Guilding System Set Of Frames Science Verification spatial element of a data cube To be confirmed To be defined Very Large Telescope volume element of
119. lization MIN Minimum value of the master flat before normalization MAX Maximum value of the master flat before normalization INTFLUX Flux value integrated over the whole master flat field before normaliza FLAT MASTER NSATURATED Number of saturated pixels in output data malization FLAT MASTER SLICEJ MEAN Mean value around the vertical center of slice j before normalization FLAT MASTER SLICEJ STDEV Standard deviation around the vertical center of slice j before nor TRACE SLICE_L XPOS Location of midpoint of leftmost slice Doc VLT MAN ESO 14670 6186 ESO MUSE Pipeline User Manual pete isle Date Date 2015 04 28 Page 73 of 132 QC TRACE SLICE_L TILT Tilt of leftmost slice measured as angle from vertical direction OC TRACE SLICE_R XPOS Location of midpoint of rightmost slice OC TRACE SLICE_R TILT Tilt of rightmost slice measured as angle from vertical direction OC TRACE SLICEj MAXSLOPE The maximum slope of the derived tracing functions of slice j within the CCD QC TRACE SLICE10 WIDTH Width of top left slice in the IFU 10 on CCD OC TRACE SLICE46 WIDTH Width of top right slice in the IFU 46 on CCD QC TRACE SLICE3 WIDTH Width of bottom left slice in the IFU 3 on CCD QC TRACE SLICE39 WIDTH Width of bottom right slice in the IFU 39 on CCD OC TRACE WIDTHS MEDIAN Median width of slices QC TRACE WIDTHS MEAN
120. llowing the same procedure as before the set of frames is prepared containing at least 3 raw dark frames and the 24 master bias files from the previous processing step which are the required calibrations Assuming the master bias files are located in a directory SMUSE_CAL the SOF and the EsoRex command will look as shown here 7Note that in parallel mode the messages written to the terminal and or the log file may not appear in order but interlaced depending on the execution order of the different threads which in general is not predictable 8i e if EsoRex is executed with the option suppress prefix true which is the built in default setting Doc VLT MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ae Isle Date Date 2015 04 28 Page 35 of 132 1197 1197 1198 1199 1200 1200 1201 1202 1203 Figure 6 1 Example of a MUSE master bias of a single IFU channel 1 as it is created by the recipe muse_bias 1 gt cat dark sof data muse raw dark MUSE 2014 02 11T20 32 24 014 fits DARK data muse raw dark MUSE 2014 02 11T20 33 31 876 fits DARK data muse raw dark MUSE 2014 02 11T20 34 31 876 fits DARK SMUSE_CAL MASTER_BIAS fits MASTER_BIAS 2 gt OMP_NUM_THREADS 24 esorex log file dark log muse_dark nifu 1 merge dark sof The result of this EsoRex command is the master dark frame stored as a single FITS file with 72 FITS extensions FITS files 1 gt ls 1 fits MASTER_DARK
121. mbda data dq stat weight exposure IFU xCCD yCCD xRaw yRaw slice pix pix Angstrom flux flag flux 2 No No pix pix pix pix No flux in count flux 2 in countx x 2 78 72976685 141 46130371 6346 222 1 35237e 01 0x00000000 1 62160e 01 0 0000e 00 0 1 47 1438 79 1470 1 79 66138458 141 48033142 6346 263 8 97605e 00 0x00000000 1 28346e 01 0 0000e 00 0 1 48 1438 80 1470 1 80 59300232 141 49934387 6346 304 1 71657e 01 0x00000000 1 90208e 01 0 0000e 00 0 1 49 1438 81 1470 1 81 52462769 141 51837158 6346 346 1 49865e 01 0x00000000 1 73776e 01 0 0000e 00 0 1 50 1438 82 1470 1 82 45624542 141 53739929 6346 388 1 54953e 01 0x00000000 1 70677e 01 0 0000e 00 0 1 51 1438 83 1470 1 83 38786316 141 55642700 6346 430 1 67681e 01 0x00000000 1 85787e 01 0 0000e 00 0 1 52 1438 84 1470 i3 84 31948090 141 57545471 6346 472 7 34210e 00 0x00000000 1 16375e 01 0 0000e 00 0 1 53 1438 85 1470 1 85 25110626 141 59446716 6346 515 1 18395e 01 0x00000000 1 47541e 01 0 0000e 00 0 1 54 1438 86 1470 1 86 18272400 141 61349487 6346 558 1 56828e 01 0x00000000 1 79335e 01 0 0000e 00 0 1 55 1438 87 1470 di 87 11434174 141 63252258 6346 601 1 29650e 01 0x00000000 1 54910e 01 0 0000e 00 0 1 56 1438 88 1470 1 Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ae aed Date Date 2015 04 28 Page 57 of 132 Unlike other FITS table related tools it interprets the origin column and all special FITS headers to resolv
122. mbdamax double 10000 lambdaref double 7000 darcheck string none none check correct filter string white Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ISSUE Teue Date Date 2015 04 28 Page 92 of 132 continued from previous page Parameter Type Values Description default other smooth string ppoly none How to smooth the response curve before writ ing it to disk none does not do any kind of smoothing such a response curve is only useful if smoothed externally median does a median filter of 15 Angstrom half width ppoly fits piecewise cubic polynomials postprocessed by a sliding average filter of 15 Angstrom half width Cut off the data below this wavelength after load ing the pixel table s Cut off the data above this wavelength after load ing the pixel table s Reference wavelength used for correction of dif ferential atmospheric refraction The R band peak wavelength 7000 Angstrom that is usu ally used for guiding is close to the central wave length of MUSE so a value of 7000 0 Angstrom should be used if nothing else is known A value less than zero switches DAR correction off Carry out a check of the theoretical DAR correc tion using source centroiding If correct it will also apply an empirical correction The filter name s to be used for the output field of view image Each name has to correspond to an EXTNAME in an extension of the FIL TER_LIST file If an u
123. ment The example shown is for wide field mode narrow field mode operates in the same way with a scaled down field size Note that the sizes given are approximate the real data does not exactly cover a square region on the sky Doc VLT MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ae alae Date Date 2015 04 28 Page 18 of 132 DARK UN E FLAT ARC MASK SKYFLAT lt 0 6 lt lt lt O lt Q lt 0 lt 0Q lt 0 lt lt lt 0 lt 0 lt 0O0 lt 0 lt lt 0D lt 0 lt 0dD0 lt e f muse _lsf pipeline recipe y recipe processing order association o mandatory optional E AT input raw data input external static data master calibration output product PIXTABLE_SKY intermediate reduced data output product DATA_CUBE final output data Figure 2 2 The first stage of the MUSE data reduction cascade It shows the basic calibration recipes and the pre processing recipe muse_scibasic together with the Association map indicating the required and the optional input for each of the recipes PIXTABLE_OBJECT PIXTABLE_STD PIXTABLE_SKY PIXTABLE_ASTROMETRY Doc VLT MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ae alate Date Date 2015 04 28 Page 19 of 132 4 PIXTABLE_ PIXTABLE_ ASTROMETRY OBJECT PIXTABLE_REDUCED aap DATACUBE_FINAL DATACUBE_FINAL
124. mn name Type Description lambda double wavelength Angstrom Continued on next page Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU Tele Date Date 2015 04 28 Page 125 of 132 continued from previous page Column name Type Description response double instrument response derived from standard star 2 5 log10 count s Angstrom erg s cm 2 Angstrom resperr double instrument response error derived from standard star 2 5 log10 count s Angstrom erg s cm 2 Angstrom Frame tags e STD_RESPONSE muse_standard Response curve as derived from standard star s A 3 14 STD_TELLURIC Description MUSE telluric correction table FITS extensions e FITS table Column name Type Description lambda double wavelength Angstrom ftelluric double the telluric correction factor normalized to an airmass of 1 ftellerr double the error of the telluric correction factor Frame tags e STD_TELLURIC muse_standard Telluric absorption as derived from standard star s A 3 15 STD_FLUXES Description 2D Image containing measurements of flux integration of all stars detected in a standard star field This is mainly thought to be used for debugging The image contains a spectral axis axis 1 with corresponsing WCS informa tion Axis 2 is the arbitrary numbering of stars detected in the field Seve
125. mn name Type Description xpos int X position of a bad pixel on untrimmed raw data pix ypos int Y position of a bad pixel on untrimmed raw data pix status int 32bit bad pixel mask as defined by Euro3D value float Extra value e g depth for traps count Frame tags e BADPIX_TABLE PRO CATG BADPIX_TABLE This file can be used to list known bad pixels that cannot be found by automated test on dark or flat field frames A 2 6 STD_FLUX_TABLE Description This is a binary FITS table with the dependency of the flux on wavelength and an optional column containing the error of the flux The wavelengths should cover at least the MUSE wavelength range The pipeline expects several such tables in multiple binary table extensions of a single FITS file It then loads the one nearest to the observed sky position using the RA and DEC keywords present in each FITS extension FITS extensions e FITS table may appear more than once Column name Type Description lambda double Wavelength Angstrom Flux double The standard star flux erg s cm 2 Angstrom fluxerr double Error of the standard star flux optional optional col umn erg s cm 2 Angstrom Frame tags e STD_FLUX_TABLE PRO CATG STD_FLUX_TABLE Reference flux distribution of a standard star Such a table has to exist for each observed standard star A 2 7 FILTER_LIST Description This FITS table contains all filte
126. mplementations the default maximum number of threads is 1 or 2 In this case OMP_NUM_THREADS must be used to enlarge the number of allowed threads Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual Issue isle Date Date 2015 04 28 Page 23 of 132 cannot be handled by 32 bit applications unless they have been specifically build for that This applies in particular to the visualization tool used to inspect the final data cube product with file sizes of 4GB or more Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ae alae Date Date 2015 04 28 Page 24 of 132 4 Known Issues The MUSE Pipeline is continuously improved The release notes including the latest improvments and changes is always available at http www eso org sci software pipelines muse In addition this sec tions summarizes the most critical issues which are known at the time the release has been made Users processing MUSE data with the MUSE DRS version 1 0 4 should be aware of the following known issues e The sky subtraction is currently not optimal and may leave artifacts due to the current parametrization of the line spread function An alternative method is being worked on but it still needs to be fully validated This issue has the highest priority and an update will be made available as soon as possible The tools described in section 7 2 and section 7 3 do not yet work with pipeline
127. muse_scipost products Qc QC Qc QC QC QC SCIPOST SCIPOST SCIPOST SCIPOST FWHMk Y SCIPOST THRESHOLD SCIPOST LINE1 NAME POSk X POSk Y Position of source k in x direction in combined frame Position of source k in y direction in combined frame FWHMk X FWHM of source k in x direction in combined frame FWHM of source k in y direction in combined frame Threshold in the white light considered as sky used to create this mask Name of the strongest line in group 1 Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ka pee Date Date 2015 04 28 Page 103 of 132 QC SCIPOST LINE1L AWAV Wavelength air of the strongest line of group QC SCIPOST LINE1 FLUX Flux of the strongest line of group QC SCIPOST CONT FLUX Total flux of the continuum QC SCIPOST CONT MAXDEV Maximum absolute value of the derivative of the continuum spectrum Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU alae Date Date 2015 04 28 Page 104 of 132 9 13 muse_exp_combine Combine several exposures into one datacube 9 13 1 Description Sort reduced pixel tables one per exposure by exposure and combine them with applied weights into one final datacube 9 13 2 Input frames Constraint min required 2 optional optional Category Type PIXTABLE_REDUCED raw FILTER_LIST calib OUTPUT_WCS calib
128. ng the contents of this file The input files have to be given as an absolute path however EsoRex allows the use of environment variables so that a common directory prefix can be abreviated Individual lines may be commented out by putting the hash character in the first column An example of a set of frames is shown in the following 1 gt cat bias sof data muse raw MUSE 2014 02 21T09 48 53 153 fits BIAS SRAW_DATA MUSE 2014 02 21T09 50 36 645 fits BIAS SRAW_DATA MUSE 2014 02 21T09 52 16 513 fits BIAS SRAW_DATA MUSE 2014 02 21T09 53 47 996 fits BIAS SRAW_DATA MUSE 2014 02 21T09 55 04 515 fits BIAS These set of frames files will have to be created by the user using a text editor for instance Which classification has to be used with which recipe will be shown in section 6 2 2 Finally if more than one set of frames is given on the command line EsoRex concatenates them into a single set of frames Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU ald Date Date 2015 04 28 Page 30 of 132 6 2 Data Organization Running the MUSE pipeline recipes using EsoRex requires that the user prepares the set of frames file To be able to do this one has to find the correct input files for each recipe and assigns the correct frame tag and one has to find calibration file which were taken with a matching instrument configuration The following sections will summarize which
129. nsupported filter name is given creation of the respective image is omit ted If multiple filter names are given they have to be comma separated 9 9 4 Product frames The following product frames are created by the recipe Default file name Description DATACUBE_STD Reduced standard star field exposure STD_FLUXES The integrated flux per wavelength of all detected sources STD_RESPONSE Response curve as derived from standard star s STD_TELLURIC Telluric absorption as derived from standard star s ESO Doc VLI MAN ESO 14670 6186 MUSE Pipeline User Manual TSU alae Date Date 2015 04 28 Page 93 of 132 9 9 5 Quality control parameters The following quality control parameters are available for the muse_standard products QC STANDAR QC STANDAR QC STANDAR QC STANDAR D D D D POSk X POSk Y FWHMk X FWHMk Y Position of source k in x direction in combined frame Position of source k in y direction in combined frame FWHM of source k in x direction in combined frame FWHM of source k in y direction in combined frame Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU alate Date Date 2015 04 28 Page 94 of 132 9 10 muse_create_sky Create night sky model from selected pixels of an exposure of empty sky 9 10 1 Description This recipe creates the continuum and the atmospheric transition line spectra of
130. o find dark columns in the combined bias hisigmabadpix double 3 High sigma to find bright columns in the com bined bias merge boolean false Merge output products from different IFUs into a common file 9 1 4 Product frames The following product frames are created by the recipe Default file name Description MASTER_BIAS Master bias ESO Doc VLI MAN ESO 14670 6186 MUSE Pipeline User Manual TSU alae Date Date 2015 04 28 Page 66 of 132 9 1 5 Quality control parameters The following quality control parameters are available for the muse_bias products INPUTi NSATURATED Number of saturated pixels in raw bias i in input list MEDIAN Median value of master bias in quadrant n MEAN Mean value of master bias in quadrant n STDEV Standard deviation value of master bias in quadrant n MIN Minimum value of master bias in quadrant n MAX Maximum value of master bias in quadrant n RON Read out noise in quadrant n determined from difference images of each adja ases in the input dataset in randomly placed windows RONERR Read out noise error in quadrant n determined from difference images of pair of biases in the input dataset in randomly placed windows LOPE X Average horizontal slope of master bias in quadrant n LOPE Y Average vertical slope of master bias in quadrant n NBADPIX Bad pixels found as part of the
131. o the Gaussian fit to each sky emission line Continued on next page ESO MUSE Pipeline User Manual Doc VLT MAN ESO 14670 6186 Issue Issue 9 Date Date 2015 04 28 Page 90 of 132 continued from previous page Parameter Type Values default other Description resample dlambda merge boolean double boolean false 1 25 false Resample the input science data into 2D spectral images using tracing and wavelength calibration solutions for a visual check Note that the image produced will show small wiggles even when the input calibrations are good and were applied suc cessfully Wavelength step in Angstrom per pixel to use for resampling Merge output products from different IFUs into a common file 9 8 4 Product frames The following product frames are created by the recipe Default file name Description OBJECT_RED OBJECT_RESAMPLED PIXTABLE_OBJECT STD_RED STD_RESAMPLED PIXTABLE_STD SKY_RED SKY_RESAMPLED PIXTABLE_SKY ASTROMETRY_RED PIXTABLE_ASTROMETRY ASTROMETRY_RESAMPLED Pre processed CCD based images for OBJECT input if saveimage true Resampled 2D resample true Output pixel table for OBJECT input Pre processed CCD based images for STD input if saveimage true Resampled 2D image for STD input if resample true Output pixel table for STD input Pre p
132. of FWHM of detected arc lines in slice j QC WAVECAL SLICEJ LINES FWHM MIN Minimum FWHM of detected arc lines in slice j OC WAVECAL SLICE3J LINES FWHM MAX Maximum FWHM of detected arc lines in slice j OC WAVECAL SLICEJ RESOL Mean spectral resolution R determined in slice j QC WAVECAL SLICEJ FIT NLINES Number of arc lines used in calibration solution fit in slice j OC WAVECAL SLICEJ FIT RMS RMS of the wavelength calibration fit in slice j OC WAVECAL SLICEJ DWLEN BOTTOM Difference in wavelength computed for the bottom left and bottom right corners of the slice on the CCD OC WAVECAL SLICEJ DWLEN TOP Difference in wavelength computed for the top left and top right corners of the slice on the CCD OC WAVECAL SLICEJ WLPOS Position of wavelength given in WLEN in slice j OC WAVECAL SLICEJ WLEN Wavelength associated to WLPOS in slice j QC WAVECAL INPUTi NSATURATED Number of saturated pixels in raw arc i in input list QC WAVECAL NSATURATED Number of saturated pixels in output data OC WAVECAL LAMP1 INPUTi NSATURATED Number of saturated pixels in raw arc i of lamp in input list ESO MUSE Pipeline User Manual Doc VLI MAN ESO 14670 6186 Issue Issue 9 Date Date 2015 04 28 Page 78 of 132 9 5 muse _Isf Compute the LSF 9 5 1 Description Compute the slice and wavelength dependent LSF from a lines spectrum ARC lamp 9 5 2 Input frames
133. on default other resample string drizzle nearest The resampling technique to use for the final out linear quadratic put cube renka drizzle lanczos crtype string median iraf Type of statistics used for detection of cos mean median mic rays during final resampling iraf uses the variance information mean uses standard mean stdev statistics median uses median and the median median of the absolute median deviation crsigma double 50 Sigma rejection factor to use for cosmic ray re jection during final resampling A zero or nega tive value switches cosmic ray rejection off lambdamin double 5000 Minimum wavelength for twilight reconstruc tion lambdamax double 9000 Maximum wavelength for twilight reconstruc tion dlambda double 250 Sampling for twilight reconstruction this should result in planes of equal wavelength coverage xorder int 2 Polynomial order to use in x direction to fit the full field of view yorder int 2 Polynomial order to use in y direction to fit the full field of view vignmaskedges double 0 02 Pixels on edges stronger than this fraction in the normalized image are excluded from the fit to the vignetted area Set to non positive number to in clude them in the fit vignsmooth string polyfit polyfit Type of smoothing to use for the vignetted re gaussian median gion given by the VIGNETTING_MASK gaus sian uses vignxpar vignypar 2 as FWHM vignxpar int 4 Parameter used by the vignett
134. ons and derive a telluric correction spectrum for them The final response cuve is then linearly extrapolated to the largest possible MUSE wavelength range and smoothed with the method given by smooth The derivation of the telluric correction spectrum assumes that the star has a smooth spectrum within the telluric regions If there are more than one exposure given in the input data the derivation of the flux response and telluric corrections are done separately for each exposure For each exposure the datacube used for flux integration is saved together with collapsed images for each given filter 9 9 2 Input frames Category Type Constraint min PIXTABLE_STD raw required 1 EXTINCT_TABLE calib required 1 STD_FLUX_TABLE calib required 1 TELLURIC_REGIONS calib optional FILTER_LIST calib optional 9 9 3 Recipe parameters Parameter Type Values Description default other profile string moffat gaussian Type of flux integration to use gaussian and moffat circle moffat use 2D profile fitting circle and square square are non optimal flux integrators select string flux flux How to select the star for flux integration flux distance uses the brightest star in the field distance uses the detection nearest to the approximate coordi nates of the reference source Continued on next page median ppoly lambdamin double 4000 la
135. orptin correction file was given Then the sky subtraction is carried out unless skymethod none either directly subtracting an input sky continuum and an input sky emission lines for skymethod subtract model or skymethod model create a sky spectrum from the darkest fraction skymodel_fraction of the field of view then fitting and subtracting sky emission lines using an initial estimate of the input sky lines then the continuum residuals after subtracting the sky lines from the sky spectrum is subtracted as well If save contains skymodel all sky related products are saved for each exposure Afterwards the data is corrected for the radial velocity of the observer rvcorr before the input or a default astrometric solution is applied Now each individual exposure is fully reduced the pixel tables at this stage can be saved by setting individual in save If multiple exposures were given they are then combined If save contains combined this final merged pixel table is saved Finally if save contains cube the data is resampled into a datacube using all parameters given to the recipe The extent and orientation of the cube is normally computed from the data itself but this can be overridden by passing a file with the output world coordinate system OUTPUT_WCS for example a MUSE cube This can also be used to sample the wavelength axis logorithmically in that file set CTYPE3 AWAV LOG As a last
136. ouble Line flux 10 20 erg s cm 2 arcsec 2 dq int Quality of the entry gt 0 dont use e OH_TRANSITIONS FITS table optional Column name Type Description name string Transition name like OH 8 3 Ple 22 5 2 lambda double Air wavelength Angstrom v_u int Upper transition level vil int Lower transition level nu int Vibrational momentum E_u double Upper energy J J_u double Upper momentum A double Transition probability Frame tags e SKY_LINES PRO CATG SKY_LINES Catalog of OH transitions and other sky lines muse_create_sky Estimated sky line flux table muse_scipost Estimated sky line flux table if skymethod model and save contains skymodel A 2 3 ASTROMETRY_REFERENCE Description This FITS file lists astrometric sources in fields to be observed with MUSE as astrometric calibrators It is used by the muse_astrometry recipe One such table exists per field the tables contains a list of point sources Each row contains information about one object in the field The pipeline expects several such tables in multiple binary table extensions of a single FITS file It then loads the one nearest to the observed sky position using the RA and DEC keywords present in each FITS extension FITS extensions e FITS table Column name Type Description SourcelID string Source identification RA double Right ascension deg Continued on next page
137. overscan levels between a raw input image and the reference image is higher than ovscsigma x stdev stop the process ing If overscan vpoly this is used as sigma rejection level for the iterative polynomial fit the level comparison is then done afterwards with 100 x stdev to guard against incompatible set tings Has no effect for overscan offset ovscignore int 3 The number of pixels of the overscan adjacent to the data region of the CCD that are ignored when computing statistics or fits combine string sigclip average Type of lampwise image combination to use median minmax sigclip save_subtracted boolean false Save the pixel table after the LSF subtraction line_quality int 3 Minimal quality flag in line catalog for selection merge boolean false Merge output products from different IFUs into a common file 9 5 4 Product frames The following product frames are created by the recipe 9 5 5 Quality control parameters Default file name Description LSF_PROFILE PIXTABLE_SUBTRACTED bugging Slice specific LSF parameters Subtracted pixel table if save_subtracted true for de The following quality control parameters are available for the muse_Isf products Qc Qc Qc QC LSF LSF LSF LSF IFUm IFUm IFUm IFUm SLICE SLICE SLICES SL FWHM M EAN Mean FWHM of the LSF slice j FWHM STDEV Standard deviation of the LSF in slice j FWHM M
138. ovscsigma double 30 If the deviation of mean overscan levels between a raw input image and the reference image is higher than ovscsigma x stdev stop the process ing If overscan vpoly this is used as sigma rejection level for the iterative polynomial fit the level comparison is then done afterwards with 100 x stdev to guard against incompatible set tings Has no effect for overscan offset ovscignore int 3 The number of pixels of the overscan adjacent to the data region of the CCD that are ignored when computing statistics or fits combine string sigclip average Type of image combination to use median minmax sigclip nlow int 1 Number of minimum pixels to reject with min max nhigh int 1 Number of maximum pixels to reject with min max nkeep int 1 Number of pixels to keep with minmax Isigma double 3 Low sigma for pixel rejection with sigclip hsigma double 3 High sigma for pixel rejection with sigclip scale boolean true Scale the individual images to a common expo sure time before combining them normalize double 3600 Normalize the master dark to this exposure time in seconds To disable normalization set this to a negative value hotsigma double 5 Sigma level in terms of median deviation above the median dark level above which a pixel is de tected and marked as hot merge boolean false Merge output products from different IFUs into a common file 9 2 4 Product frames The following product fram
139. peline distribution as st d_flux_table fits will eventually contain spectra of all spectrophotometric standard stars which are regularly observed with MUSE The standard stars in the standard flux table are looked up by their coordinates Currently reference spectra for the following standard stars are available e GD71 GD108 GD153 e HD49798 CD 32 9927 bad resolution should not be used Doc VLT MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ae alae Date Date 2015 04 28 Page 43 of 132 e GJ 754 1 A e LDS 749B e Feige 110 e LIT 7987 e EG 274 e LTT 3218 The complete set of frames for computing the response curve is shown here followed by the EsoRex command line to launch the recipe 1 gt cat std sof PIXTABLE_STD_0001 01 fits PIXTABLE_STD PIXTABLE_STD_0001 24 fits PIXTABLE_STD SMUSE_CAL extinct_table fits EXTINCT_TABLE SMUSE_CAL std_flux_table fits STD_FLUX_TABLE 2 gt OMP_NUM_THREADS 24 esorex log file std log muse_standard filter white std sof The recipe creates a data cube of the standard star observation and the response curve and the telluric correction spectrum 1 gt ls 1 x fits DATACUBE_STD_0001 fits STD_RESPONSE_0001 fits STD_TELLURIC_0001 fits Again the sequence number is an exposure index which runs from 1 to the number of exposures present in the SOF The data c
140. ple can be created using the provided MUSE DRS recipes These calibration files the Isf profiles 1sf_pro file_slow_wfm n fits and 1sf_ profile _slow_wfm e fits the astrometric cali bration astrometry_wcs_wfm fits and the two response curves std_response_wfm e fits std_response_wfm n fits contain the model of the MUSE line spread function the WFM astrometric calibration and the WFM response curves for the extended and nominal wavelength range respectively These files are provided as a convenience since their creation is either very time consuming Isf profiles where found to be stable in the analysis of the available commissioning data and or may be used as a fallback solution in view of the mentioned limitations of the current pipeline version cf Section 1 1 Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU alae Date Date 2015 04 28 Page 27 of 132 Strictly speaking also the geometry table could be created using the MUSE DRS However this requires a large specific data set and expert knowledge to get to a good result Thus the geometry table is and will be distributed by ESO as a static calibration For static calibrations geometric calibration and astrometric solution needed to process observations taken before December Ist 2014 please refer to Appendix D 5 3 Pipeline Products The MUSE DRS is designed to propagate errors and data quality information together
141. pplement the information in existing bad pixel tables these can be passed in with the i parameter 1 gt muse_badpix_from_ascii i BADPIX_TABLE_existing fits bad_pixels ascii 12 BADPIX_TABLE_12 fits 2 gt muse_badpix_from_region i BADPIX_TABLE_existing fits 10 12 100 2000 256 12 BADPIX_TABLE_12 fits 7 3 3 Working with Data Cubes The tool muse_cube_combine is useful if one has to deal with a large dataset which cannot be fully com bined with the muse_scipost or muse_exp_combine recipes In that case one can run muse_exp_combine several times setting the wavelength limits see Sect 7 1 such that they overlap in only about 2 wavelength planes in the output cubes One then has to take care to resample all sub cubes to the same output grid de fined using the OUTPUT_WCS input see Sect 9 13 and Sect A They will then only be filled at the relevant wavelength ranges the rest will contain NaN Then this tool can be used to combine them into a fully populated cube As an example a dataset is too large to fit into memory at once but does fit if split into three sections Then one can run the recipe muse_exp_combine with the parameters Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual pete Toed Date Date 2015 04 28 Page 59 of 132 1 gt esorex muse_exp_combine lambdamax 6285 ec sof 2 gt mv DATACUBE_FINA 1 fits CUBE_blue fits 3 gt esorex mus
142. products saved using the combined data format i e to use them the recipes have to be run without the option merge In general the recipe muse_astrometry works without problems but it may occasionally fail to identify enough stars depending on the observing conditions Using a smaller value for the recipe parameter detsigma may solve this Once muse_astrometry finishes sucessfully it is recommended to verify that the derived pixel scale is close to the nominal value of 0 2 spaxel The measured pixel scale is stored as the FITS keywords QC ASTRO SCALE X and QC ASTRO SCALE Y of the astrometric solution In any case it is always a safe alternative to use the astrometric solution shipped with the pipeline distri bution kit The cosmic ray rejection has been improved substantially However the default settings are chosen to be conservative so that no real data gets rejected However more aggressive settings may be used to optimize the results of the cosmic ray rejection for individual exposures Another issue specifically affects data taken during the first MUSE Science Verification run For flat fields which are taken at low ambient temperature temperatures lower than 7 C the tracing of the edges of the slices may fail Data taken after this first SV run should not be affected since the issue will be fixed at the instrument level In order to be able to complete the processing of the SV data if such a failure happens if muse_flat
143. quence taking into account the vertical offsets of the pinholes in the mask Finally the geometry table is cleaned up from intermediate debug data and saved As a last optional step additional raw input data is reduced using the newly geometry to produce an image of the field of view If these exposures contain smooth features they can be used as a visual check of the quality of the geometrical calibration 9 6 2 Input frames Category Type Constraint min MASK raw required 50 MASTER_BIAS calib required 1 MASTER_DARK calib optional usually not used MASTER_FLAT calib optional TRACE_TABLE calib required 1 WAVECAL_TABLE calib required 1 LINE_CATALOG calib required 1 BADPIX_TABLE calib optional usually not used MASK_CHECK calib optional 9 6 3 Recipe parameters Parameter Type Values Description default other iful int 1 First IFU to analyze ifu2 int 24 Last IFU to analyze Continued on next page Doc VLT MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ISSUE koue Date Date 2015 04 28 Page 81 of 132 continued from previous page Parameter Type Values Description default other sigma double 2 2 Sigma detection level for spot detection in terms of median deviation above the median centroid string gaussian Type of centroiding and FWHM determination to barycenter use for all spot measurements simple barycenter gaussian
144. r crtype string median iraf Type of statistics used for detection of cos mean median mic rays during final resampling iraf uses the variance information mean uses standard mean stdev statistics median uses median and the median median of the absolute median deviation crsigma double 10 Sigma rejection factor to use for cosmic ray re jection during final resampling A zero or nega tive value switches cosmic ray rejection off rc double 1 25 Critical radius for the renka resampling method pixfrac double 0 6 Pixel down scaling factor for the drizzle re sampling method Id int 1 Number of adjacent pixels to take into account during resampling in all three directions loop distance this affects all resampling methods ex cept nearest format string Cube Cube Type of output file format Cube is a stan Euro3D xCube dard FITS cube with NAXIS 3 and multiple ex xEuro3D tensions for data and variance The extended x formats include the reconstructed image s in FITS image extensions within the same file weight string exptime Type of weighting scheme to use when combin exptime fwhm ing multiple exposures exptime just uses the none exposure time to weight the exposures fwhm uses the DIMM information in the header as well none preserves an existing weight column in the input pixel tables without changes filter string white The filter name s to be used for the output field of view image Ea
145. r Manual TSU Tue Date Date 2015 04 28 Page 96 of 132 9 11 muse_astrometry Compute an astrometric solution 9 11 1 Description Merge pixel tables from all IFUs apply correction for differential atmospheric refraction optionally apply flux calibration and telluric correction if the necessary input data was given and resample the data from all exposures into a datacube Use the cube to detect objects which are then matched to their reference positions from which a two dimensional WCS solution is computed 9 11 2 Input frames Category Type Constraint min PIXTABLE_ASTROMETRY raw required 1 ASTROMETRY_REFERENCE calib required 1 EXTINCT_TABLE calib optional STD_RESPONSE calib optional STD_TELLURIC calib optional 9 11 3 Recipe parameters Parameter Type Values Description default other centroid string moffat gaussian Centroiding method to use for objects in the field moffat box of view gaussian and moffat use 2D fits to derive the centroid box is a simple centroid in a square box detsigma double 3 Source detection sigma level to use radius double 5 Initial radius in pixels for pattern matching iden tification in the astrometric field faccuracy double 5 Factor of initial accuracy relative to mean posi tional accuracy of the measured positions to use for pattern matching niter int 2 Number of iterations of the astrometric fit rejsigma double 3 Rejection
146. r each individual exposure positioned fully re duced and positioned pixel table for each individ ual exposure the difference to individual is that here the output pixel tables have coordinates in RA and DEC this is only useful if both the rela tive exposure weighting and the final resampling are to be done externally combined fully re duced and combined pixel table for the full set of exposures the difference to positioned is that all pixel tables are combined into one with an added weight column this is useful if only the final resampling step is to be done separately stacked an additional output file in form of a 2D column stacked image i e x direction is pseudo spatial y direction is wavelength The resampling technique to use for the final out put cube Horizontal step size for resampling in arcsec or pixel The following defaults are taken when this value is set to 0 0 0 2 for WFM 0 075 for NFM 1 0 if data is in pixel units Vertical step size for resampling in arcsec or pixel The following defaults are taken when this value is set to 0 0 0 2 for WFM 0 075 for NFM 1 0 if data is in pixel units Wavelength step size in Angstrom Natural in strument sampling is used if this is 0 0 Continued on next page ESO MUSE Pipeline User Manual Doc VLT MAN ESO 14670 6186 Issue Issue 9 Date Date 2015 04 28 Page 100 of 132
147. r functions that can be used for image reconstruction Each filter curve is contained within one sub table Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU alae Date Date 2015 04 28 Page 113 of 132 FITS extensions e FITS table Column name Type Description lambda double Wavelength Angstrom throughput double Filter throughput in fractions of 1 Frame tags e FILTER_LIST PRO CATG FILTER_LIST File to be used to create field of view images A 2 8 TELLURIC_REGIONS Description This FITS table regions of telluric absorption lines It can be used to override the internal telluric bands used in the muse_standard recipe FITS extensions e FITS table Column name Type Description lmin double Lower limit of the telluric region Angstrom lmax double Upper limit of the telluric region Angstrom bgmin double Lower limit of the background region Angstrom bgmax double Upper limit of the background region Angstrom Frame tags e TELLURIC_REGIONS PRO CATG TELLURIC_REGIONS File to be used to override the internal telluric bands Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU alae Date Date 2015 04 28 Page 114 of 132 A 3 Recipe Product Files A 3 1 MUSE_IMAGE Description A reduced CCD image of one IFU accompanied with quality and statistics information These
148. ral ESO DRS MUSE keywords in the output header contain information regarding each source x and y position pix in the corresponding data cube approximate celestial position deg and integrated flux their numbering corresponds to the axis 2 coordinate The unit of the integrated flux is given by the standard BUNIT keyword Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU alae Date Date 2015 04 28 Page 126 of 132 FITS extensions e DATA 2D FITS image float Integrated fluxes per wavelength bin e DQ 2D FITS image int optional Corresponsing Euro3D data quality per wavelength bin e STAT 2D FITS image float Corresponsing data variance per wavelength bin Frame tags e STD_FLUXES muse_standard The integrated flux per wavelength of all detected sources A 3 16 AMPL_CONVOLVED Description This FITS image contains two extensions PHOTONS and ENERGY showing filter convolved values of the convolved flat fields FITS extensions e PHOTONS 2D FITS image float Photon counts ph e ENERGY 2D FITS image float Per pixel energy u Frame tags e AMPL_CONVOLVED muse_ampl Combined and convolved master AMPL image A 4 Other data files A 4 1 OUTPUT_WCS Description Normally the MUSE pipeline automatically adapts the output cube dimensions and sky location depending on the data This type of file can be used to override this automatism
149. re rejected when computing overscan statistics Either no rejec tion at all none rejection using the DCR algo rithm der or rejection using an iterative con stant fit fit ovscsigma double 30 If the deviation of mean overscan levels between a raw input image and the reference image is higher than ovscsigma x stdev stop the process ing If overscan vpoly this is used as sigma rejection level for the iterative polynomial fit the level comparison is then done afterwards with 100 x stdev to guard against incompatible set tings Has no effect for overscan offset ovscignore int 3 The number of pixels of the overscan adjacent to the data region of the CCD that are ignored when computing statistics or fits combine string sigclip average Type of combination to use median minmax sigclip nlow int 1 Number of minimum pixels to reject with min max nhigh int 1 Number of maximum pixels to reject with min max nkeep int 1 Number of pixels to keep with minmax Isigma double 3 Low sigma for pixel rejection with sigclip hsigma double 3 High sigma for pixel rejection with sigclip scale boolean false Scale the individual images to a common expo sure time before combining them Continued on next page Doc VLT MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ISSUE Toue Date Date 2015 04 28 Page 85 of 132 continued from previous page Parameter Type Values Descripti
150. recipe including those that were present in all inputs used in that run of muse_flat Since the tool gets the full FITS file including all headers of the output product it can set up the correct FITS headers for the bad pixel table This tool can also be used to merge flagged pixels from the DO extension into an existing table 1 gt muse_badpix_from_dq i BADPIX_TABLE_in fits MASTER_FLAT 10 fits BADPIX_TABLE_out fits If one has manually recorded single bad pixels in an ASCII file or measured regions of bad pixels one can use muse_badpix_from_ascii ormuse_badpix_from_region Here one needs to specify the IFU that contains the bad pixels to store since no FITS header with the information is available 1 gt muse_badpix_from_ascii bad_pixels ascii 12 BADPIX_TABLE_12 fits 2 gt muse_badpix_from_region 10 12 100 2000 256 12 BADPIX_TABLE_12 fits muse_badpix_from_region requires the region to be in the format x1 x2 y1 y2 and also needs a Euro3D like flag value as 2nd argument The ASCII table has to contain three values per row x position y position and flag value By default both tools expect the coordinates to be measured on the raw image if they were determined on trimmed data instead the t argument has to be set 1 gt muse_badpix_from_ascii t bad _pixels ascii 12 BADPIX_TABLE_12 fits 2 gt muse_badpix_from_region t 10 12 100 2000 256 12 BADPIX_TABLE_12 fits Again these tools can be used to su
151. rejec tion at all none rejection using the DCR algo rithm dcr or rejection using an iterative con stant fit fit ovscsigma double 30 If the deviation of mean overscan levels between a raw input image and the reference image is higher than ovscsigma x stdev stop the process ing If overscan vpoly this is used as sigma rejection level for the iterative polynomial fit the level comparison is then done afterwards with 100 x stdev to guard against incompatible set tings Has no effect for overscan offset ovscignore int 3 The number of pixels of the overscan adjacent to the data region of the CCD that are ignored when computing statistics or fits crop boolean true Automatically crop the output pixel tables in wavelength depending on the expected useful wavelength range of the observation mode used 4750 9350 Angstrom for nominal mode 4600 9350 Angstrom for extended mode Continued on next page Doc VLT MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ISSUE Toue Date Date 2015 04 28 Page 89 of 132 continued from previous page Parameter Type Values Description default other cr string none none dcr Type of cosmic ray cleaning to use for quick look data processing xbox int 15 Search box size in x Only used if cr dcr ybox int 40 Search box size in y Only used if cr dcr passes int 2 Maximum number of cleaning passes Only used if cr dcr
152. rocessed CCD based images for SKY input if saveimage true Resampled 2D image for SKY input if resample true Output pixel table for SKY input Pre processed CCD based images for ASTROMETRY in put if saveimage true Resampled 2D image for ASTROMETRY input if resample true Output pixel table for ASTROMETRY input image for OBJECT input if 9 8 5 Quality control parameters The following quality control parameters are available for the muse_scibasic products QC SCIBASIC NSATURATED QC SCIBASIC LAMBDA SHIFT Number of saturated pixels in output data Shift in wavelength applied to the data using sky emission line s ESO MUSE Pipeline User Manual Doc VLI MAN ESO 14670 6186 Issue Issue 9 Date Date 2015 04 28 Page 91 of 132 9 9 muse_standard Create a flux response curve from a standard star exposure 9 9 1 Description Merge pixel tables from all IFUs and correct for differential atmospheric refraction To derive the flux response curve integrate the flux of all objects detected within the field of view using the given profile Select one object as the standard star either the brightest or the one nearest one depending on select and compare its measured fluxes to tabulated fluxes to derive the sensitivity over wavelength Postprocess this sensitivity curve to mark wavelength ranges affected by telluric absorption Interpolate over the telluric regi
153. s ESO Instrument Pipeline page ESO 3D Viewer Likwid Lightweight Performance Tools FITS image compression utilities struments muse html tp www eso org sci facilities paranal tp www eso org sci software pipelines tp casa nrao edu casa_obtaining shtml tps code google com p likwid tp heasarc nasa gov fitsio fpack
154. sigma level of the astrometric fit rotcenter string 0 01 1 20 Center of rotation of the instrument given as two comma separated floating point values in pixels lambdamin double 4000 Cut off the data below this wavelength after load ing the pixel table s lambdamax double 10000 Cut off the data above this wavelength after load ing the pixel table s Continued on next page ESO Doc VLI MAN ESO 14670 6186 MUSE Pipeline User Manual TSU Isley Date Date 2015 04 28 Page 97 of 132 continued from previous page Parameter Type Values Description default other lambdaref double 7000 Reference wavelength used for correction of dif ferential atmospheric refraction The R band peak wavelength 7000 Angstrom that is usu ally used for guiding is close to the central wave length of MUSE so a value of 7000 0 Angstrom should be used if nothing else is known A value less than zero switches DAR correction off darcheck string none none Carry out a check of the theoretical DAR correc check correct tion using source centroiding If correct it will also apply an empirical correction 9 11 4 Product frames The following product frames are created by the recipe Default file name Description DATACUBE_ASTROMETRY Reduced astrometry field exposure ASTROMETRY_WCS Astrometric solution 9 11 5 Quality control parameters The following quality con
155. slicer system computed as median angle of all slices of this IFU for which the measurement could be made QC GEO IFUm WLEN QC GEO IFUm WLEN mj LUX M OC GEO IFUm WLEN nj LUX M Nominal wavelength of arc line 1 EAN Average integrated flux in all spots at reference wavelength 1 EDIAN Median integrated flux in all spots at reference wavelength 1 Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU alae Date Date 2015 04 28 Page 82 of 132 QOC GEO IFUm WLEN1L FLUX STDEV Standard deviation of integrated flux in all spots at reference wavelength 1 QC GEO MASK ANGLE Angle of the mask with respect to the slicer system computed as median angle of all slices of all IFUs for which the measurement could be made Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU ale Date Date 2015 04 28 Page 83 of 132 9 7 muse_twilight Combine several twilight skyflats into one cube compute correction factors for each IFU and create a smooth 3D illumination correction 9 7 1 Description Processing first handles each raw input image separately it trims the raw data and records the overscan statistics subtracts the bias taking account of the overscan if overscan is not none converts the images from adu to count subtracts the dark divides by the flat field and combines all the exposures using input paramet
156. step the computed cube is integrated over all filter functions given filter that are also present in the input filter list table 9 12 2 Input frames Category Type Constraint min PIXTABLE_OBJECT raw required 1 EXTINCT_TABLE calib required 1 STD_RESPONSE calib required 1 STD_TELLURIC calib optional ASTROMETRY_WCS calib optional FILTER_LIST calib optional OUTPUT_WCS calib optional SKY_LINES calib optional SKY_CONTINUUM calib optional LSF_PROFILE calib optional SKY_MASK calib optional ESO MUSE Pipeline User Manual Doc VLT MAN ESO 14670 6186 Issue Issue 9 Date Date 2015 04 28 Page 99 of 132 9 12 3 Recipe parameters Parameter Type Values default other Description Save resample dx dy dlambda string string double double double cube skymodel drizzle nearest linear quadratic renka drizzle lanczos 0 0 0 0 0 0 Select output product s to save Can contain one or more of cube skymodel individual positioned combined and stacked If sev eral options are given they have to be comma separated cube output cube and associated images if this is not given no final resampling is done at all skymodel up to four additional output products about the effectively used sky that was subtracted with the model method individual fully reduced pixel table fo
157. t OMP_NUM_THREADS 24 esorex log file bias log muse_bias nifu 1 merge bias sof Once the EsoRex command has finished the master bias has been created in the current working directory together with the log file bias 109 of the EsoRex run By default the name of the product files created by the MUSE recipes are derived from the value of the PRO CATG header keyword i e their frame tag with suffixes and sequence numbers added as needed The last 2 digit sequence number refers to the MUSE channel number Thus after the running muse_bias the current working directory contains now the master bias product stored in a single FITS file with 72 FITS extensions 1 gt ls 1 fits MASTER_BIAS fits Without the option merge the recipe would have produced 24 FITS files with 3 FITS extensions each where the IFU from which it has been created is indicated by the 2 digit sequence number at the end of the file name e g the MASTER_BIAS of IFU 1 would be stored in the file MASTER_BIAS 01 f1its An example of a MUSE master bias is shown in Figure 6 1 For the full description of the muse_bias recipe please refer to Section 9 6 3 2 Dark In this next step a set of raw dark frames is combined into a master dark frame using the recipe muse_dark Since the dark current of the MUSE CCDs is small it is unlikely that the master dark frame will be needed in the following processing steps Thus this step is optional Fo
158. t log muse_scipost filter white Johnson_V Cousins_R Cousins_I scipost sof The product files created by this command are 1 gt ls 1 x fits DATACUBE_FINAL fits IMAGE_FOV_0001 fits IMAGE_FOV_0002 fits IMAGE_FOV_0003 fits IMAGE_FOV_0004 fits The 4 digit sequence number of the created field of view images refers to the different filters The ordering of the field of view images corresponds to the order of the filters on the command line By default the recipe performs a model based sky subtraction i e initially a model of the sky lines and the sky continuum are computed using the darkest regions in the field of view and then these models are subtracted Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU aed Date Date 2015 04 28 Page 47 of 132 from the data To use the models obtained from a previous run of muse_create_sky one can change the sky subtraction method using the option skymethod If this option is set to subtract model the input models of the sky lines and the sky continuum are subtracted from the observation data as they are As mentioned before running muse_scipost to process a single exposure requires already a quite capable com puter in terms of memory And contrary to the basic calibration recipe reducing the number of threads will not reduce the memory needed It is also not possible to process an observation by splitting it into smaller tiles How
159. takes an illumination flat field frame tag IT LLUM as shown in the example This flat field is a special raw continuum lamp exposure illuminating the whole field of view It is taken regularly during the night at a rate of 1 exposure every 1 2 hours as part of the instrument calibration plan It can be added to the set of frames as shown in the example If it is present the recipe muse_scibasic corrects for the relative illumination of the slices on a per IFU basis temperature dependent illumination changes The command shown before creates the pixel table stored as 24 FITS files for each input exposure and one reduced image I gt Ls I ofirtS OBJECT_RED_0001 fits l With version 1 0 the default format has been changed to FITS images Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ae alae Date Date 2015 04 28 Page 42 of 132 PIXTABLE_OBJECT_0001 01 fits PIXTABLE_OBJECT_0001 24 fits Currently the option merge affects only the products which are not pixel tables i e the reduced images which may be used for a visual check of the applied corrections On the other hand these reduced images are usually not needed and using the recipe option saveimage FALSE they are not created at all The product file names are derived from the frame tag of the raw input exposure which in this case was OBJECT For the other kind of on sky exposures th
160. ted for large scale gradients by dividing it with the smooth white image The residuals in the corner area then smoothed using input parameters This smoothed vignetting correction is the multiplied onto each plane of the smooth cube normalizing each plane again This twilight cube is then saved to disk 9 7 2 Input frames Category Type Constraint min SKYFLAT raw required 3 MASTER_BIAS calib required 1 MASTER_DARK calib optional usually not used MASTER_FLAT calib required 1 BADPIX_TABLE calib optional usually not used TRACE_TABLE calib required 1 WAVECAL_TABLE calib required 1 Continued on next page ESO MUSE Pipeline User Manual Doc VLI MAN ESO 14670 6186 Issue Issue 9 Date Date 2015 04 28 Page 84 of 132 continued from previous page Category Type Constraint min GEOMETRY_TABLE calib required 1 VIGNETTING_MASK calib optional 9 7 3 Recipe parameters Parameter Type Values Description default other overscan string vpoly If this is none stop when detecting discrepant overscan levels see ovscsigma for offset it assumes that the mean overscan level represents the real offset in the bias levels of the exposures involved and adjusts the data accordingly for vpoly a polynomial is fit to the vertical over scan and subtracted from the whole quadrant ovscreject string der This influences how values a
161. the products i gt is 1 fits IMAGE_FOV fits SKY_CONTINUUM fits SKY_LINES fits SKY_MASK fits SKY_SPECTRUM fits These are the reconstructed whitelight field of view image the estimated sky continuum spectrum estimated fluxes of the sky lines image mask of the sky regions derived from the white light image and the sky spectrum obtained from the sky regions defined by the mask Note that the sky continuum is always created even if no Isf profile is present in the input SOF However a reasonably smooth version of the sky continuum will only be obtained if an Isf profile is used For the full description of the muse_create_sky recipe please refer to Section 9 Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ae alae Date Date 2015 04 28 Page 45 of 132 6 5 3 Astrometry This processing step uses muse_astrometry to create the astrometric calibration from the pre processed pixel tables of an exposure of an astrometric field The obtained astrometric calibration is needed to create the world coordinate system for the science exposure and assign right ascension and declination to the spaxels of the data cube In addition to the input pixel tables the recipe needs an astrometric reference catalog The astrometric catalog that comes with the MUSE pipeline distribution will eventually contain the positions of the reference stars in all astrometric fields which are regularly observed with MU
162. then done afterwards with 100 x stdev to guard against incompatible set tings Has no effect for overscan offset The number of pixels of the overscan adjacent to the data region of the CCD that are ignored when computing statistics or fits Type of lampwise image combination to use Identify and measure the arc emission lines on images separately for each lamp setup Sigma level used to detect arc emission lines above the median background level in the S N image of the central column of each slice The allowed range of resolutions for pattern matching of detected arc lines to line list in fractions relative to the expected value Tolerance for pattern matching of detected arc lines to line list Continued on next page Doc VLT MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ISSUE Toue Date Date 2015 04 28 Page 76 of 132 continued from previous page Parameter Type Values Description default other xorder int 2 Order of the polynomial for the horizontal curva ture within each slice yorder int 6 Order of the polynomial used to fit the dispersion relation linesigma double 1 0 Sigma level for iterative rejection of deviant fits for each arc line within each slice a negative value means to use the default 2 5 residuals boolean false Create a table containing residuals of the fits to the data of all arc lines This is useful to assess the quality of the wa
163. three dimensional illumination correction using the recipe muse_twilight In addition the created twilight cube carries the integrated flux value on to the further processing steps as an estimate of the relative throughput of the IFUs The recipe needs as input at least 3 raw sky flat frames the master bias frame the master flat field the trace table the wavelength solution and the geometry table The set of frames and the EsoRex command for creating the master sky flat are shown 1 gt cat twilight sof data muse raw MUSE 2014 02 20T23 31 38 542 fits SKYFLAT data muse raw MUSE 2014 02 20T23 33 49 258 fits SKYFLAT data muse raw MUSE 2014 02 20T23 36 21 511 fits SKYFLAT SMUSE_CAL MASTER_BIAS fits MASTER_BIAS SMUSE_CAL MASTER_FLAT fits MASTER_FLAT SMUSE_CAL TRACE_TABLE fits TRACE TABLE SMUSE_CAL WAVECAL TABLE fits WAVECAL TABLE SMUSE_CAL geometry_table fits GEOMETRY_TABLE SMUSE_CAL vignetting_mask fits VIGNETTING_MASK 2 gt OMP_NUM_THREADS 24 esorex log file twilight log muse_twilight twilight sof The vignetting mask is an optional input to the recipe and if provided is used to correct vignetted regions in the MUSE field of view Usually vignetting is present in the lower right corner of the MUSE field of view The primary product of this processing step is the twilight cube which is a data cube of the smoothed tw
164. trol parameters are available for the muse_astrometry products QC QC QC QC QC QC QC QC QC AST AST AST AST AST AST Qc Qc AST AST AST AST AST RO RO RO RO RO RO RO RO RO RO RO POSKk X Position of source k in x direction in combined frame POSk Y Position of source k in y direction in combined frame FWHMk X FWHM of source k in x direction in combined frame FWHMk Y FWHM of source k in y direction in combined frame NSTARS Number of stars identified for the astrometric solution SCALE X Computed scale in x direction SCALE Y Computed scale in y direction ANGLE X Computed angle in x direction ANGLE Y Computed angle in y direction MEDRES X Median residuals of astrometric fit in x direction MEDRES Y Median residuals of astrometric fit in y direction Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU alae Date Date 2015 04 28 Page 98 of 132 9 12 muse_scipost Prepare reduced and combined science products 9 12 1 Description Sort input pixel tables into lists of files per exposure merge pixel tables from all IFUs of each exposure Correct each exposure for differential atmopheric refraction unless lambdaraf is far outside the MUSE wave length range Then the flux calibration is carried out if a respnse curve was given in the input it includes a correction of telluric absorption if a telluric abs
165. ts WAVECAL RESIDUALS 10 fits This is an example of a good calibration with low residuals the final RMS for the solution in this slice was 0 030 A The tool has automatically selected all columns belonging to this slice and colored them accord ing to their horizontal position on the CCD green is left red is right and used different symbols As one can see the fainter arc lines like the Ne I line at 5400 6 A have typically a much larger spread of residu als than the bright lines e g Ne I at 6678 3 A With the default parameters of muse_wavecal i e option fitweighting cerrscatter the weak lines are hence weighted much less in the fit of the wave length solution than the bright lines Doc VLT MAN ESO 14670 6186 ESO MUSE Pipeline User Manual Issue Issue 9 Date Date 2015 04 28 Page 56 of 132 slice 12 iteration 2 column 980 1057 polynomial and residuals limit 0 0891 81 0 05 Residuals Angstrom 0 1 5000 5500 6000 6500 7000 7500 8000 8500 9000 Wavelength Angstrom col 980 x col 996 a col 1012 a col 1028 a col 1044 col 981 col 997 o col 1013 a col 1029 o col 1045 x col 982 o col 998 a col 1014 v col 1030 col 1046 col 983 m col 999 a col 1015 col 1031 col 1047 o col 984 o col 1000 a col 1016 o col 1032 x col 1048 col 985 col 1001 v col 1017 col 1033 col 1049 o col 986 a col 1002 v col 1018 col 1034 o col 1050 col 987 col 1003 o
166. ube can be visualized using the ESO 3D Visualization tool cf RD 7 Section E or ds 9 for instance In addition to the data cube itself the data cube product file contains a reconstructed FOV image for each filter given on the command line option filter The default filter setting is white to produce a white light image All other allowed filters are the ones defined in the filter list calibration file filter_list fits the filter names are given by the keyword EXTNAME If a filter from the filter list is chosen the filter list must also be present in the SOF By default the recipe selects the brightest star in the field as the spectro photometric standard If this is inappro priate for the current observation this can be changed using the recipe option select distance which then takes the object closest to the expected position If more than one standard star observation is present in the input SOF the response computation is done sepa rately for each of the input exposures For the full description of the muse_standard recipe please refer to Section 9 Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ae alae Date Date 2015 04 28 Page 44 of 132 6 5 2 Sky Creation In this processing step a model of the sky spectrum is created using the recipe muse_create_sky It is only needed if the observed object fills the field of view to such an extent that a reasonable sky spectrum cannot
167. ug with esorex This should be done for all bug reports but should not be necessary for normal operations WARNING These messages warn about possible anomalies in the data They also point out non standard settings They do not cause the pipeline to fail but it is wise to check the data carefully afterwards ERROR These are lines where a process in the pipeline could not finish properly or where a sig nificant part of the process failed The error code and the corresponding line in the code is usually printed If possible an explanation is given of why the failure occurred Note that when the recipes runs with multiple threads the messages from a single thread on the screen or in the logfile may be interlaced with messages from other threads The messages from a single thread can however be identified by the thread index number e g t id 005 which is printed as part of the message The logfile should also be included in any bug report one is going to submit In this case if possible the failing recipe should be re run with the log level set to debug 8 4 Debugging Options Certain environment variables for testing and debugging were created while developing the pipeline Many of them might prove useful if some data cannot be reduced with the usual options that are exposed through the recipe interface Also one might want to dig deeper into the analysis of what is done to the data during the reduction process As t
168. umns vertical axis with the red end of the spectrum at the top and the blue end at the bottom The pixel rows horizontal axis correspond to the spatial axis 2 2 The MUSE Data Reduction Pipeline The MUSE pipeline is basically divided into two stages The first stage consists of the seven basic calibration recipes and a preprocessing recipe basic science reduction which work on the data of individual CCDs to determine and or remove the signature of each IFU The recipes of the second stage another three calibration recipes and the final science recipe use the pre processed data from the first stage and transform it into physical quantities which can be used for science These second stage recipes combine the data from all IFUs of one or more exposures into the final data cube The two stages of the reduction process are illustrated in Figure 2 2 and Figure 2 3 Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual Tau Date Date 2015 04 28 Page 17 of 132 24 IFUs 60 x2 5 each o ME BTE anamorphic magnification MUSE WFM a ae Y S S JO SMOAL ZT TOX S1 m 4 stacks of slices SOOTIS Sh yora TOX S slicing spectrograph CCD numbering left to right on FITS image 4240 pixels 76 5 pix slice 6 pix gap 4224 pixels detection Figure 2 1 Graphical representations of the splitting and slicing procedures in the MUSE instru
169. velength solution in detail fitsigma double 1 0 Sigma level for iterative rejection of deviant datapoints during the final polynomial wave length solution within each slice a negative value means to use the default 3 0 fitweighting string cerrscatter Type of weighting to use in the final polynomial uniform wavelength solution fit using centroiding error cerr scatter estimate and or scatter of each single line as esti cerrscatter mates of its accuracy resample boolean false Resample the input arc images onto 2D images for a visual check using tracing and wavelength calibration solutions Note that the image pro duced will show small wiggles even when the calibration was successful wavemap boolean false Create a wavelength map of the input images merge boolean false Merge output products from different IFUs into a common file 9 4 4 Product frames The following product frames are created by the recipe Default file name Description WAVECAL_TABLE WAVECAL_RESIDUALS ARC_RED_LAMP ARC_RED ARC_RESAMPLED WAVE_MAP Wavelength calibration table Fit residuals of all arc lines 1f residuals true Reduced ARC image per lamp Reduced combined master ARC lampwise false or resample true Resampled arc images if resample true Wavelength map of the input images image if Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ae alae Date Date 2015 04 28 Page
170. were correctly computed plot_samples s1 12 s2 20 TRACI TRACE_TABLI E 06 fits MASTER_FLAT 06 fits E_SAMPLES 06 fits Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual ae fue Date Date 2015 04 28 Page 54 of 132 trace slice widths slices 1 to 48 slice width at y position pix 0 500 1000 1500 2000 2500 3000 3500 4000 y position on CCD pix slice 01 slice 08 slice 15 slice 22 slice 29 slice 36 slice 43 slice 02 slice 09 slice 16 slice 23 slice 30 slice 37 slice 44 slice 03 slice 10 slice 17 slice 24 slice 31 slice 38 slice 45 slice 04 slice 11 slice 18 slice 25 slice 32 slice 39 slice 46 slice 05 slice 12 slice 19 slice 26 slice 33 slice 40 slice 47 slice 06 slice 13 slice 20 slice 27 slice 34 slice 41 slice 48 slice 07 slice 14 slice 21 slice 28 slice 35 slice 42 Figure 7 2 The graphical window showing the output of the muse_trace_plot_widths tool plotting slices 1 to 48 of IFU 6 using the trace samples table see text for details Plotting this may take a while so it s advisable to only use a subset of the slices The result of this command is shown in Figure 7 1 The widths of the slices on the CCD should be around 77 pixels but their actual widths may slowly vary between top
171. xposure start TEL AIRM END Airmass at the end of the exposure TEL PARANG START Parallactic angle at exposure start from site monitor TEL PARANG END Parallactic angle at exposure end from site monitor TEL AMBI FWHM START Observatory seeing at exposure start from site monitor TEL AMBI FWHM END Observatory seeing at exposure end from site monitor TEL AMBI RHU Observatory ambient relative humidity from site monitor TEL AMBI TEMP Observatory ambient temperature from site monitor Keywords describing the instrument detector setup EXPTIME Total integration time s Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual pete Toue Date Date 2015 04 28 Page 31 of 132 Keyword Name Description INS MODE Instrument mode field mode WFM NFM AO status nominal extended wave length range INS OPTI1 NAME Wavelength range cutoff filter setting Blue or Clear for nominal and ex tended wavelength range respectively INS OPTI2 NAME Field mode setting WFM or NFM INS LAMP1 ST Blue continuum lamp on off INS SHUT1 ST Blue continuum lamp shutter open closed INS LAMP2 ST Red continuum lamp on off INS SHUT2 ST Red continuum lamp shutter open closed INS LAMP3 ST Neon arc lamp on off INS SHUT3 ST Neon arc lamp shutter open closed INS LAMP4 ST Xenon arc lamp on off INS SHUT4 ST Xenon
172. y the value of the keyword PRO CATG they are listed in Table 6 2 for the sake of completeness Also listed here is a utility recipe which can be used to combine the fully reduced pixel tables which can be created by muse_scipost as intermediate products for details see Section 6 5 Frame tag Recipe PRO CATG PIXTABLE_STD muse_standard PIXTABLE_STD PIXTABLE_SKY muse_create_sky PIXTABLE_SKY PIXTABLE_ASTROMETRY muse_astrometry PIXTABLE_ASTROMETRY PIXTABLE_OBJECT muse_scipost PIXTABLE_OBJECT PIXTABLE_REDUCED muse_exp_combine PIXTABLE_REDUCED Table 6 2 Raw data frame tags of the second stage recipes Once one has accumulated a substatial number of MUSE files possibly even spread over several directories the Gasgano file browser may be a handy tool to not loose track of the data Gasgano is able to scan the files in some specified directories and applies the classification rules for MUSE to them The resulting frame tag will then be shown in the Gasgano GUI in the column Classification The last step in creating set of frames files as input for the MUSE recipes is to find the appropriate calibrations both static calibrations and master calibrations This means that one has to select calibrations which originate from the same or compatible instrument and or detector configuration The basic set of header keywords which is useful for
173. ywords in the headers of the CUBE_ lt color gt fits pipeline outputs to determine the wavelength range used throw away the small overlaps which are needed to guard against edge effects sort the exposures according to the wavelength the cover and then copy the relevant wavelength planes both DATA and STAT extensions into the single output cube The utility muse_cube_ filter can be used to integrate an existing cube in dispersion direction over a filter function While this is usually done by the recipes muse_scipost or muse_exp_combine the tool can be useful if a filter was not considered when the recipes were run in first place or when additional field of view images should be created after partial data cubes were combined using muse_cube_combine The typical usage of muse_cube_combine is shown in the following 1 gt muse_cube_filter filter_list fits This creates reconstructed f Johnson_V Cousins_R Cousins_I DATACUBE fits field of view images for the V R and filters from the given cube Only the filters which are present in the given filter list file can be used 16 As reference header one can use the header of one of the reduced pixel tables from the input set of frames whose WCS information is changed such that it covers the full final data set Doc VLI MAN ESO 14670 6186 ESO MUSE Pipeline User Manual TSU alate Date Date 2015 04 28 Page 60 of 132 In

MUSE Pipeline User Manual

Contents

Download Pdf Manuals

Related Search

Related Contents