Back to MAXI Data Analysis

Data analysis

A quick instruction of FTOOLS-based standard analysis with MAXI archive data obtained from DARTS at ISAS. Users basically need to run only 2 scripts, mxdownload for data download, and mxproduct for data analysis. For more detailed information about analysis, please see the Tips, Tricks, and Traps page.

Contents


1. Setting up

First of all, make sure to set up the essential environmental variables in your (UNIX-)Shell environment for HEAsoft, CALDB, and MAXI FTOOLS, as explained in the Preparation page. In short, all you need to do is to source your set-up script (see the examples in the Preparation page).

2. Downloading the data

Here we are assuming you download a small part of the MAXI archive data. If you want a significant part of the entire archive, such as a half of it, it would be quicker and more efficient to download everything in one go as instructed in the Preparation page.

Change directory to your chosen working directory for your analysis; suppose it is $HOME/analysis/ (Note it should be different from the top directory where the MAXI FTOOLS is installed). Then, run the script mxdownload, specifying the coordinates (in J2000) of the center of your chosen region, radius (optional), period, and instruments/modes (optional). The MAXI archive data since 2009-08-15 (MJD=55058) is available. Note that Ruby 2 is required for mxdownload. To see the version of Ruby in your environment, run ruby -v.

The format of mxdownload is,

mxdownload [OPTIONS] RA Dec (degree) TSTART TSTOP (YYYY-MM-DD)

The following is an example; it downloads the MAXI data for the area centered at the Crab Nebula for the period between 2010-01-01 and 2010-01-31.

 cd $HOME/analysis/
 mxdownload 83.633083 22.0145 2010-01-01 2010-01-31

Note that the example above would download the data up to 2010-02-01, a day after the specified date of 2010-01-31, because of the requirement by the pipeline software mxproduct, which uses a (tiny amount of) calibration data up to the day after the end-date of the data to process. All the MAXI data that satisfy the specified conditions are downloaded from the MAXI archive data-server (or the server you specify) to the sub-directories of the current directory.

MAXI does not monitor all the 768 sky regions everyday, some data of some regions for some days are not available. Also, There is a lack of data for different reasons: MAXI shutdown, downlink problems, processing errors etc. The detailed information about those problems will be appeared on the MAXI home page.

A user can specify a radius and an instrument mode with options, -r (default: 8.0 deg) and -i (default: gsc_low), respectively. The following is an example of downloading the SSC data for the radius of 10 degrees. For the coordinates with a negative Dec, put '--' before the part.

 mxdownload -r 10 -i ssc_med -- 10.00 -20.000 2010-01-01 2010-01-31 

You can view a concise on-line help with --help option. The three options --planonly, --listonly and -n are for dry-running (simulated run) and can be useful to see how it works and how many data files will be (attempted to be) downloaded.

The following is the directory structure created by mxdownload

and the following is the file/directory structure after the pipe-line processing by mxproduct explained in detail in the next section:

where CWD refers to the current working-directory, (gsc_low, gsc_med, ssc_med) is the instrument_mode (data rate), [MMMMM] is the Modified Julian Day (MJD) in 5 digits, [C] is the counter-ID of the GSC (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, a, b), [D] is the detector-ID of the SSC (“h” or “z”), [NNN] is the “Healpix”-ID, as explained below, [Object] is the prefix specified by a user (--object option of mxproduct), and [Inst] is one of (g_low, g_med, s_med), corresponding to (gsc_low, gsc_med, ssc_med), respectively.

Healpix is the library to define the regions on the sky, dividing the entire sky into regions with a similar area-size to one another. The data in the MAXI archive make use of Healpix system with NSIDE=8, RING format, which divides the entire sky into 768 regions. The J2000 celestial coordinates (RA, DEC) that MAXI uses are related to the Healpix coordinates (θ, φ) for each pixel-ID with the following formula:

 (RA, DEC) = (φ, π/2 - θ) [radian] (J2000).

3. Analyzing the data

Now you have the raw MAXI data in your local disk, you are ready to make data reduction, using one of MAXI FTOOLS, mxproduct, to get the spectra and light-curve with your choice of filtering selections.

3.1. Data reduction by mxproduct

Introduction

The following is the simplest example of the data reduction of the GSC-low data of Crab to make the spectra and light-curves. In this example, mxproduct is run at the root data-directory, which contains the directory obs/ .

 mxproduct --object=crab  83.633083 22.014500 2010-01-01 2010-01-31

This will create the GSC science products of spectra for the default region, response file for them, and light-curves in the default energy band, all for the region around the coordinates of (RA, DEC) [J2000] for the period between TSTART and TSTOP. The science product files are stored in the directory ./products/ (or, user-specified name by the option, --outpath), which is created if it does not exist.

The briefest help is given by

 mxproduct --help

or read through this document for the complete reference.

Overview

The format of mxproduct is,

mxproduct [OPTIONS] RA Dec (degree) TSTART TSTOP (YYYY-MM-DD or  YYYY-MM-DDTHH:MM:SS)

mxproduct requires 4 mandatory parameters, that is, the first two for the center coordinates and latter two for the start and end dates. In addition it accepts several optional arguments, each of which is given in the format of --option or --option=PARAMETER. Combining these, a user can apply four types of filtering criteria:

  1. Instrument/Mode
  2. Time filter
  3. Spatial filter
  4. Energy-band filter.

Here is the detail of how to specify and combine them.

Instrument/Mode
The GSC has 2 data types: gsc_low and gsc_med. A user must choose either of them one at a time, and can not mix them in the data reduction with MAXI FTOOLS. Our recommendation is gsc_low, which is the default set. To choose the medium data, set the option --gscmed=1. GSC consists of 12 counters, with the counter ID = 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, a, b. The recommended energy ranges are 2-20 keV and 2-30 keV for gsc_low and gsc_med, respectively.
The SSC has only 1 data type: ssc_med. SSC has 2 cameras, SSC-h and SSC-z. We recommend to use the 0.7-7 keV energy band.
Time filter
The third and fourth (mandatory) main command-line arguments specify the start (TSTART) and end (TSTOP) dates, respectively, in UTC in the 8-digit ISO-8601 format (YYYY-MM-DD) for your chosen period of observations. The data are selected from 00:00:00 on the day of TSTART to 23:59:59 on the day of TSTOP in default. If you want to specify the start and stop time, give the parameters as YYYY-MM-DDTHH:MM:SS.
Spatial filter (Region files)
The first and second (mandatory) main command-line arguments specify the nominal center position of the source in Right Ascension (RA) and Declination (DEC) in J2000, respectively.
In addition, you can supply up to two region files, that is, for source and background regions. See the options --srcregfile-gsc,--bgdregfile-gsc,--srcregfile-ssc,--bgdregfile-ssc.
The region-file format is the ds9 format in plain text with the fk5 coordinate system in degrees. When saving a region file in ds9, select coordinates in degrees (sexagesimal is selected by default).
The acceptable shape of the source region is limited to a simple circle only so far. For the background region is limited to circle - circle only. If anything else is specified, mxproduct does not run.
If the background region is far away from the source, the background level may be estimated wrong.
When you exclude neighbouring sources from the source/background region, the effective area may not be calculated correctly.
 # Region file format: DS9 version 4.1
 global color=green dashlist=8 3 width=1 font="helvetica 10 normal roman" select=1 highlite=1 dash=0 fixed=0 edit=1 move=1 delete=1 include=1 source=1
 fk5
 circle(49.951,41.512,6000")     
Example for the background region:
 # Region file format: DS9 version 4.1
 global color=green dashlist=8 3 width=1 font="helvetica 10 normal roman" select=1 highlite=1 dash=0 fixed=0 edit=1 move=1 delete=1 include=1 source=1
 fk5
 circle(49.951,41.512,108000")     
 -circle(49.951,41.512,7200")     

Energy-band filter
You can supply a plain text file that describes your desired (multiple) energy bands for the filter. It is especially useful to make light-curves in specified energy bands. See the options --ebandfname-gsc,--ebandfname-ssc.
The default energy bands are 2.0–5.95 and 6.0–11.95 keV for the GSC, 0.7–1.997 and 2.000–6.997 keV for the SSC.
The generated light-curves in multiple energy bands are stored in multiple extensions in a single output FITS file.
Each line of the file consists of a pair (Minimum, Maximum) of the energy in keV, separated by a single space.
The following is an example for 3-band filter of 2–5.95, 6–11.95, and 12–20 keV.
Note that each channel is 50 eV wide for GSC, and 3.65 eV for SSC.
 2.0 5.95
 6.0 11.95
 12.0 20.0  
Output files

mxproduct generates the GSC and/or SSC science products (spectra, spectral response file, and light-curves) in the directory products/ (or, user can specify the name by the command-line option, "--outpath") , as well as some auxiliary files in the directory trend/, both in the current directory. The default energy bands are 2.0–5.95 and 6.0–11.95 keV for the GSC, 0.7–1.997 and 2.000–6.997 keV for the SSC. The following is a set of example product filenames for GSC_Low, where the command-line option of --object=crab is given. For GSC_Med and SSC_Med, replace g_low with g_med and s_med, respectively.

Event file
crab_g_low.evt
Images
crab_g_low.img
crab_g_low_wmap_src.img
crab_g_low_wmap_bgd.img
Spectra and responses
crab_g_low_src.pi
crab_g_low_bgd.pi
crab_g_low.rsp
Light-curves
crab_g_low.lc
Scancur file (observation condition for the target)
crab_g_low_scancur.fits
Command log
crab_g_low_commands.log

During the processing, many intermediate files (such as, GTI files, spectrum files for individual cameras) are created in the same products (or user-specified name) directory. They are automatically deleted after the processing in default, unless --no-cleanup=1 is specified. The diagram in the previous subsection summarizes the directory and file structure.

The following figures show examples of WMAP image of Crab for source and background regions without (default settings) and with the region files. Without region files, the default source and background data are selected by elongation angles from the target in a scan direction and in a direction perpendicular to the scan direction. The generated images are neither exposure-corrected nor background-subtracted.

From left to right, a source WMAP image and a background WMAP image without region files, a source WMAP image and a background image with region files.
The generated light-curves are not yet corrected for the effective area. The absolute count rate, especially in the low energy band, is underestimated by ~10%. The light-curves may show some jumps, which might be artificial, not real by the target. The light-curves for Crab are shown below, about 10% less compared to the result by MAXI on-demand process especially below 4 keV. 
The light-curves of Crab from 2010-01-01 (MJD=55197) to 2010-01-31 (MJD=55227) by ondemand (left) and by mxproduct (right).
Options
--object=NAME
Prefix of the filenames of science products. Avoid any of the characters of [-+*/()] and a space for NAME (especially, watch out for the hyphen “-”), which would cause Memory Error while calculating spectrum files, using mathpha in FTOOLS. Default is target.
--gscmed=(0 or 1)
Specify 1 to use the GSC med-data (Default: 0).
--skip-gsc=(0(Default) or 1)
If 0 (default), processes all the available GSC data of the relevant mode as specified in --gscmed. Otherwise, skips all of them.
--skip-ssc=(0 or 1(Default))
If 0, processes all the available SSC data. Otherwise (default), skips all of them.
--datapath=PATH
Directory for the locally mounted data files, that is, the path where obs/ exists. Default is ".".
--mountdata=(0 or 1)
(Internal use) Uses the locally-mounted whole archive if 1.
--outpath=PATH
Directory where output files will be put. Default is "products/".
--srcregfile-gsc=FILENAME
Filename of the region file for the source with the GSC in the ds9/fk5 format. Only a simple circle shape is accepted.
--bgdregfile-gsc=FILENAME
Same as --srcregfile-gsc, but for the GSC background region. Only a simple circle - circle shape is accepted.
--srcregfile-ssc=FILENAME
Same as --srcregfile-gsc, but for the SSC source region.
--bgdregfile-ssc=FILENAME
Same as --srcregfile-ssc but for the SSC background region.
--ebandfname-gsc=FILENAME
Filename of the energy band for GSC light curves. See the previous subsection Overview for detail of the format.
--ebandfname-ssc=FILENAME
Same as the option --ebandfname-gsc but for the SSC.
--no-cleanup=(0(Default) or 1)
mxproduct generates a number of intermediate files, including GTI files, event files, spectrum files in the directory ./products/ (or user-specified name) during processing. In default, all these files are deleted when mxproduct completes the task or is halted/interrupted (except for the case the process is killed by the KILL signal). They are not deleted if this option is specified to be 1.
--help
Displays the help.
Examples
  1. The simplest form for the source1 at (RA, DEC)=(350°, −10°).
     mxproduct --object=src1 350 -10 2010-01-01 2010-01-31
    
  2. Processing the data for the radius of 10 degrees of Crab Nebura, where the MAXI data directory obs/ is mounted in /My/Data/Maxi/.
     mxproduct --object=crab --mountdata=1 --datapath=/My/Data/Maxi -r 10 83.633083 22.014500 2010-01-01 2010-01-31
    
  3. Specifying region files for source and background.
     mxproduct --object=crab --srcregfile-gsc=src.reg --bgdregfile-gsc=bgd.reg 83.633083 22.014500 2010-01-01 2010-01-31
    
  4. Specifying energy bands for GSC light-curves in a text file, gsc_eband.list, a directory name for output files, test1/, and start/stop time with hh:mm:ss.
     mxproduct --object=crab --ebandfname-gsc=gsc_eband.list --outpath=test1 83.633083 22.014500 2010-01-01T08:00:00 2010-01-31T20:00:00
    
  5. Example of the SSC data processing.
     mxproduct --object=crab --skip-gsc=1 --skip-ssc=0 83.633083 22.014500 2010-01-01 2010-01-31
    

3.2. Data analysis examples

3.2.1. Spectral analysis

The spectra and response files generated by mxproduct are in the standard FITS format. You can use your favorite software to analyze them. The following is the standard entry-level procedure with the XANADU/HEAsoft spectral-analysis package XSPEC to view the background-subtracted spectrum.

 xspec

 cpd /xw
 data 1:1 "crab_gsum_src.pi"
 back 1   "crab_gsum_bgd.pi"
 resp 1   "crab_gsum.rsp"
 setplot energy
 ignore bad
  
 iplot ldata
  
 log x on 1
 log y on 1
 label t Crab
 lwidth 5
 rescale x 1 10
 plot
 

3.2.2. Time-series analysis

The generated FITS file of the light-curves (crab_gsum.lc in the example above) contains the same number of the FITS extensions as that of the energy bands specified in the input text file (gsc_eband.list in the example above) with the --ebandfname-gsc (or --ebandfname-ssc) option. Each FITS Extension is named LCDAT_PIBANDn for the (n+1)-th line of the energy band in the input gsc_eband.list. For example, the FITS Extension LCDAT_PIBAND0 contains the light-curve for the energy-band of the first line in gsc_eband.list. Each bin-width of a light-curve is 40~200 sec, which corresponds to the period of a single MAXI scan in which the source is in the field-of-view. The light-curve bins are separated by the intervals between the two successional scans, which are ~92 min (= ISS orbital period) in most cases but can be as short as ~15 min.

You can plot the light curve(s) with your favorite software, specifying the TIME as X-axis, and the RATE and RERR columns as Y-axis and its error, respectively. The following is the example with the general plotting tool LCURVE in HEAsoft:

 lcurve nser=1 cfile1="products/crab_gsum.lc" window="-" dtnb=INDEF nbint=INDEF outfile=" " plot=yes  plotdev="/xw"

Go to Tips, Tricks, and Traps for more details.


Acknowledgment

Rendering of this page utilizes the prettify package developed by Google, providing your browser decides to load it.


The MAXI team
Contact: maxihelp@ml.ac.jaxa.jp