SUMER Data Cookbook

Scope

This document gives a step-by-step procedure for the reduction of SUMER data produced as IDL-restore files from the SOHO CD-ROMs. A more generic description of the reduction process, which is applicable to any format of SUMER data files, is given in the document Reduction and analysis of SUMER data written by Luca Teriaca. A description of the reduction of SUMER data in FITS format (FITS and FTS) can be found at the MEDOC site. You can also obtain the FITS data from the SOHO archive at NASA/GSFC and follow the guide for data retrieval and reduction.

The data reduction process has evolved through the help of all users of SUMER data. This process is documented by the software notes that have been sent to the community during the years.

SUMER image data availability

There are two ways of retrieving SUMER data from the original telemetry: FITS and IDL-restore files. Both are produced from GSFC-generated CD-ROMs. The quality will ultimately be the same, but differences exist in the structure of files and the image header information. The IDL-restore files contain only single images and all of the housekeeping data which are in the original image header. The FITS data files may contain many images belonging to a study (data cubes), plus a FITS header which contains additional data about the observation, but a reduced set of the housekeeping data of the original image header. They are stored in the SOHO archive at GSFC and the remote sites of the archive (MEDOC site). The IDL-restore files can be retrieved from MPAe in Germany from the SUMER Image Database, where all SUMER-related CD-ROM data are stored on disk as binary files which are accessible by an online browser programme. This allows a quick inspection of a data set before converting the data into IDL restore files. Since May 2001, the MPAe computer centre has set up a web-based system which allows the user to browse through the data and - after a password has been given to the user - to download the SUMER images.

For each single image taken by SUMER an individual restore file is produced, containing the image header (HEADER_DATA) and the image array (IMAGE_DATA) which can be restored as IDL variables. (For further information see the page SUMER Data Availability).

Both types of data, FITS files and restore files, are not calibrated or converted to physical units nor are they corrected for shortcomings of the hardware of the SUMER instrument, but correction and calibration procedures are available, and these will be described below.

When restore files are used, - which is assumed in what follows - the images are contained in the variable “IMAGE_DATA”. The header information can be read from the “HEADER_DATA” variable by applying various header data functions (*.PRO) for each data item. A list of header functions is listed at the end of this document.

Data corrections and calibrations: General remarks

The application of the various correction and calibration procedures has to follow a specific order. The best way to see the reason for this and to understand the explicit sequence is to consider the process which leads from a given spectral radiance, L(lambda), of a specific solar area to the counts accumulated by a detector pixel illuminated by this area and the subsequent processing of the digitized image and its transfer to the ground. To correct as much as possible the shortcomings of the processing along this way, the corrections and calibrations have to be performed in the reverse order.

The first encounter of the solar radiation with the instrument having an effect on the final output occurs at the entrance aperture, followed by the reflectivity of the primary mirror, the slit, etc. until the overall responsivity of the detector and the average pixel size. The calibration programme, radiometry.pro, takes all of this into account.

Next, the inhomogeneities of the photocathode response and the microchannel plates come into play. Their correction will be done by the flat-field programme, sum_flatfield.pro.

High single pixel count rates cause a local depression of the microchannel plate gain, which can be corrected (to a certain extent) by the programme local_gain_corr.pro.

Field inhomogeneities between MCP and anode of the detector, and nonlinearity in the delay lines, cause a geometric deformation of the image. This can be corrected by destr_bilin.pro.

The position encoding electronics has a non-linearity of the ADCs which causes a difference in response between odd and even rows of the detectors. This can be corrected by ff_odd-even.pro.

The total detector count rate together with the deadtime of the amplifier will determine the deadtime effect, which can be corrected by deadtime_corr.pro. As mentioned above, there are non-linearities of the analogue amplifier, a 20 % variation from one spatial pixel row to the next. They are included in the flat-field correction. The output count rate is then integrated over a given sampling time.

Finally, the 16-bit words of the digital data are compressed into 8 bits in most of the cases (compression scheme 5). Decompression programmes are available to perform the inversion. Consequently, we have the following order of manipulations (not all of them are required in all cases):

1. Decompression
2. Deadtime correction
3. Odd-even pattern
4. Flat-field correction:
5. Local-gain correction
6. Geometric distortion correction
7. Radiometric calibration

In addition, it should be noted that the location of the slit image on the detector array is not at all constant. The size of the slit image varies with wavelength - an effect of the wavelength-dependent magnification. Furthermore, the whole image is offset by more than 20 pixels by the combined effects of residual alignment errors between the scan mirror rotation axis, the grating ruling direction, the direction of the grating focussing mechanism, and the direction of the detector rows. The effect can be evaluated for the destretched detector arrays (both A and B) by the function delta_pixel.pro, but it is not a correction procedure in the sense of the above list.

Data corrections and calibrations: Details

Decompression

The intensities of the image data array may have been compressed on-board (and usually are: with "quasi-logarithmic" Compression Method 5). A decompression is necessary (note: FITS and FTS file data are already decompressed and reversed during creation). You can use the routine DECOMP5.PRO to decompress IDL-restore files which had been compressed by Method 5. If a different compression scheme has been used, the program DECOMPRESSION.PRO can be used to determine all compression parameters and then decompress the images.

Reversion

The pixel addresses of the SUMER detectors are such that, the highest wavelength is on pixel 0, the lowest on pixel 1023; the wavelength stated in the header of the restore file (e.g. 609.7) is situated on the reference pixel PIXPOS(HEADER_DATA). Therefore, wavelengths are descending from left to right. For compatibility with the SOHO conventions and the following image correction routines, the spectral direction of the images must be reversed.
To prepare the data in the orientation used in the following routines, the IDL REVERSE routine will do: IMAGE_DATA=REVERSE(IMAGE_DATA)
The routine DECOMP_R.PRO will do decompression (Scheme 5) and reversal of a set of restore files in one directory if the path and file name string is adjusted accordingly.

Dead time

The total count rate of the detector during one exposure might be high, such that electronic deadtime correction factors must be applied. The deadtime error of the detector electronics is not negligible whenever the total count rate on the detector is above 50 000 counts/s. Note, the total count rate on the detector cannot be inferred from the total counts in an image if subformat images are used. Instead, the header data functions XEVENT and YEVENT give the count rate on the total detector array. The DEADTIME_CORR.PRO routine corrects the radiometric calibration using the total count rate as input.

Odd-even pattern

A difference in the sensitivity between odd and even rows of the image is the most obvious non-uniformity of the instruments response. It is caused by the non-linearity of the ADC in the image digitization circuits of the detectors. The odd-even pattern is always present along the slit direction. The normal flat-field correction routine takes this into account. Thus, do not apply this correction if the normal flat-field correction will be used (proceed to the next step). If only the odd-even correction shall be applied, or an advanced flat-field correction (using flat-fields without odd-even structure) shall be applied, then use this procedure first (Please read the note SUMER_flatfield_correction).

To apply the odd-even correction to the A-detector or B-detector data, find in the calibration directory [flight.ff] the files ODDEVEN_ARRAY_A or ODDEVEN_ARRAY_B, respectively, and use these as a flat-field array (“flatfile”) in the SUM_FLATFIELD.PRO function as usual. This will take care of your sub-frame image size, if you have not chosen full size images for your data.

A routine to apply only the odd-even correction to a set of IDL-files is FF_ODDEVEN.PRO. The routine calls the SUM_FLATFIELD.PRO-function and thus the “flatfile” must be defined as above.

Flat field

The detector response is not uniform over the field-of-view, and this can be corrected with a flat-field correction matrix. Several flat-field correction matrices are available at the calibration directory [flight.ff]. Use the files “FF_*D_R.RST”. These are IDL-restore files already reversed to be compatible with the orientation of FITS files (not FTS files!). The routine SUM_FLATFIELD.PRO can be used to apply the flat-field correction to IDL-restore files. It handles all SUMER image formats.

The flat-field pattern changes slightly with time due to change of the channel plate gain with usage, causing a small shift of the image. Therefore, in general the data closest to the time of observation shall be used. For an advanced flat-field correction, the odd-even pattern must be corrected first (see above), and then the shift can be taken into account, as described in the document SUMER_flatfield_correction. Apply the flatfield correction using SUM_FLATFIELD.PRO as usual, now making sure that “flatfile” is specified as one file of the type “FF_*D_R_NOE.RST”, which must be used when the odd-even correction has already been done. The files needed for these corrections are available for in the calibration directory [flight.ff], where all the flatfield correction data are located.

Local gain

The local count rate in a spectral line may be high, such that local gain depression of the detector channel plates reduces the efficiency. In this case a gain depression correction can be applied: LOCAL_GAIN_CORR.PRO

Distortion

The detector image is distorted and needs a geometrical correction such that the rest position of the line profile is situated on the correct spectral pixel and the curvature of lines is removed. In addition, due to a discrepancy in the orientation of the grating and the detector, the spectral lines are inclined with respect to the detector vertical lines. Use the DESTR_BILIN.PRO routine to correct geometric distortion of the image and inclination of lines. It needs the bilinear interpolation subroutine BILIN_INTERP.PRO.

The flat-field and the distortion correction can be applied at once to a set of restore files by calling the FF_GC.PRO procedure.

Displacement

Due to the discrepancy in the orientation of the grating and the detector, the spectrum is inclined with respect to the detector horizontal lines, which causes the spectral lines to be displaced higher or lower on the detector as the wavelength is scanned. In addition, the position of the slit image on the detector is also shifted due to the nonlinearity of the grating focus mechanism which is moved simultaneously with the wavelength scan. The vertical displacement of the slit image as a function of wavelength can be determined with DELTA_PIXEL.PRO. The correction is only needed for co-registration of images with different reference wavelengths.

Line broadening

The width of spectral lines is affected by a contribution of the instrumental broadening to the Doppler broadening. Using the function CON_WIDTH_FUNCT_3.PRO the instrumental width can be taken out by using a de-convolution matrix. For a small wavelength dependence of the B-Detector use CON_WIDTH_FUNCT_4.PRO.

Radiometry

The detected intensity is given in counts per pixel per sampling interval. A radiometric calibration converts these to physical units.
The vertical extension of the long slit image on the detector corresponds to 300 arcsec on the Sun but is generally not 300 pixels long. It rather is a function of wavelength given by the focal position of the grating which determines the magnification factor of the spectrometer. This is being considered by the radiometric calibration procedure.

It is up to the user to decide which corrections and calibrations are to be applied. Since all corrections are not ideal but rather only approximations to restore the image that was originally formed in the focal plane of the SUMER instrument, it is up to the user to decide if a correction is needed for his application. One way of finding out whether or not a specific correction is required, is to check if its application has an effect that is significant for the study under consideration. The routines can be considered as optional corrections to be made to the data, however, it is crucial that the routines are applied in the order given above (see "General remarks" above).

Header data functions

IDL restore files contain the image header of each image. All modules can be used as follows: result = module_name(HEADER_DATA)

HEADER_DATA can be a single byte array of at least 92 raw header bytes or a multidimensional byte array in form of HEADER_DATA=bytarr(92, n)

---------------------------- SOURCE MODULES -------------------------------

ACIMGC.PRO        | returns accumulative image counter

ADMIN.PRO         | returns administration counter

BCP.PRO           | returns header(48)*256 + header(49)

BINY.PRO          | returns spectral binning factor y_d

BINZ.PRO          | returns spatial binning factor z_d

BPCNT.PRO         | returns # of counts in brightest pixel

BRIGHTPY.PRO      | returns brightest pixel address in y

BRIGHTPZ.PRO      | returns brightest pixel address in z

CMP1.PRO          | returns compression parameter 1

CMP2.PRO          | returns compression parameter 2: brightest pixel / averaged width

CMP3.PRO          | returns compression parameter 3: minimum pix / averaged centroid / max li

COG.PRO           | -

COMPRM.PRO        | returns ID number of compression method

DELSTA.PRO        | returns detector status (cf. XDL manual)

DELSTP.PRO        | returns raster step size in elementary motor steps

DETECTOR.PRO      | returns number for which Detector is used (0:no, 1:A, 2:B, 3:RSC)

EXPSTA.PRO        | returns start of exposure giving year/month/day/hour/min/sec

EXPTIM.PRO        | returns length of exposure time in seconds

FLATF.PRO         | returns if flatfield is done (1) or not (0)

FLREQ.PRO         | returns flight operation request number

H_STATUS.PRO      | returns status (Header Byte 22)

IIDY.PRO          | returns inter instrument Sun coordinate Y

IIDZ.PRO          | returns inter instrument Sun coordinate Z

IMGCNT.PRO        | returns image counter

IMGFORM.PRO       | returns image format

IMGTOT.PRO        | returns total counts in image

LOCATION.PRO      | returns location/scientist

MC2POS.PRO        | returns step position of MC2

MC3POS.PRO        | returns step position of MC3

MC4POS.PRO        | returns step position of MC4

MC6POS.PRO        | returns step position of MC6

MC8POS.PRO        | returns step position of MC8

MCERR.PRO         | returns error of motor controller

MCPI.PRO          | returns value of MCP-current

MCPV.PRO          | returns MCP high voltage

OPCNT.PRO         | returns operations counter

PIXPOS.PRO        | returns reference pixel

POPUDP.PRO        | returns POP/UDP number

RASSTP.PRO        | returns number of raster steps

ROTCOMP.PRO       | returns rotation compensation time

SLITNUM.PRO       | returns slit number

SUNY.PRO          | returns SUMER coordinate Y

SUNZ.PRO          | returns SUMER coordinate Z

TARGET.PRO        | returns the target zone number

UTC_HEAD.PRO      | calculates the UTC_HEAD when the integration of a SUMER image started

WAVEL.PRO         | returns wavelength at reference pixel

XEVENT.PRO        | returns x detector events in cts/s including specific scale factor

YEVENT.PRO        | returns y detector events in cts/s including specific scale factor

Udo Schühle, schuehle@mps.mpg.de

U. S., 31. October 2014

SUMER Home Page