NAME:
  reductor
 PURPOSE:
  Automated photometry reduction tool.
 DESCRIPTION:
 CATEGORY:
  Photometry
 CALLING SEQUENCE:
  reductor
 INPUTS:
 OPTIONAL INPUT PARAMETERS:
 KEYWORD INPUT PARAMETERS:
  CALIBPATH-   Directory path pointing to calibration information,
                  default=datadir+'calib/'  (datadir dymnamically determined
                  by program).
  DATAMNTS-    Special mount points to locate image data directories,
                  a special list is available by default per instrument.
                  DATAMNTS is an array of strings, leading with a slash.
  DATATABLE-   Name of table to record photometry- this is passed 
                  transparently to dbphot (which supplies the default)
                  via colorsol, ltcrv and ltcrv2.
                  It allows specification of database as well as table
                  through a 'ddd.ttt' notation.
  DVERBOSE-    Code, passed to gettran and other routine to print db queries.
  DEBUG-          Turn on debug information output, default=OFF
  DUMPOBJ-     Name of object to dump complete reduction information on.
                  If set to 'all', all objects are dumped.
  FLUSH_INFO-  Flush all cached information, force a clean start
  FORCE     -  Flag, if true, ignores "ok" in instructions
  FULL      -  Flag passed to colorsol that generates a more involved listing
                  of the reductions.
  INSTDB    -  List of data bases to probe to find instrument corresponding
                  to rundate. It is an array of strings. The default is
                  ['pccd2', 'nasacam', 'pccd'] and the data bases are probed 
                  in that order.
  LANDCAT    - name of Landolt standard catalog (default landphot.dat in your
                  current IDL_PATH)
  MAGRESID   - Edit bad values by mag residual, not sigma residuals
  NOEDIT     - Suppress all bad point editing.
  NOSAVE     - Flag, if true, suppresses saving results to data bases
                  and files. It does not affect editing, printing or plots.
                  Reduc.log entries are not generated. The flag is passed to 
                  the subsidiary programs that process rules.
                  Note that NOSAVE does not inhibit promotion of files
                  or the updating of the common block.
  PRINT      - Send all plot output to the default printer (PORTRAIT.PRO),
                  The display device is set to X (with a call to DISPLAY.PRO)
                  upon exit.  If FORCE is also set, then DATAMON.pro will be
                  called at the end to generate a plot from the .log1 file.
  RESETBAD   - Flag, if true, turns off all bad flags.
  SAVEALLPLOTS-Flag, if set causes a number of things to happen.  This
                  flag implies /FORCE, /NOEDIT, PRINT=0, and NOSAVE=0 and
                  all plots generated are saved to postscript files in same
                  directory as the reduc.inf file.  The names of the
                  postscript files are derived from the names of the rules
                  in reduc.inf.
 TRANSFTABLE - Name of table to record and retrieve transformation 
                  coefficients- this is passed 
                  transparently to gettran (which supplies the default)
                  directly and to puttran via transf.
                  It allows specification of database as well as table
                  through a 'ddd.ttt' notation.
 OUTPUTS:
 KEYWORD OUTPUT PARAMETERS:
  ERRSTR   -   string, if not '', indicates a 1 line reductor error status
 COMMON BLOCKS:
   COM_REDUCTOR - This is used to save information between multiple runs
                    of this program.  Considerable information is cached to
                    speed execution on the same night of data.
 SIDE EFFECTS:
 Reductor writes various files, some in the local reduction directory, and some
    in special repositories, according to the processing rules invoked.
    It will modify its own control file, reduc.inf, to mark when the night's
    data directory is relocated. 
    The programs called by reductor generate 
    plot output and also place reduced observations in the 'data' table of the
    phot data base. Reductor generates a 'RefID', which is a field in the
    data table, which is always the 6 digit rundate, a hypen, and the 
    instrument code, eg '020418-PCCD'. These RefID's are special in that
    corresponding entries are not generated in the reference table, unlike
    externally submitted photometry data. To avoid conflicts, it is 
    essential that RefID's for external data sets be forced to have a
    different format (such as three substrings separated by 2 hyphens).
     A time and user stamped log of reductor rules run is 
    generated, called reduc.log, in the reduction directory.
 RESTRICTIONS:
 Code to write initial reduc.inf still needs to be fixed to use wrreduc call.
 PROCEDURE:
  The operation of the program is controlled by the contents of a file
     named reduc.inf that is in the current directory.  If not found, this
     program will attempt to build a relevant file to get started.  In normal
     use you will run this program many times on the way to getting your
     photometry reduced.  What steps are done and how they are done are
     controlled by lines in the reduc.inf file.

  The contents of reduc.inf are as follows:
     Line 1 - REDUCTOR v1.1          File id line
     Line 2 - LCCD/OSU2              Instrument id (no spaces)
     Line 3 - /cdrom/lo_mwb_0022/    Directory where RAW image data reside
     Line 4 - 920611                 YYMMDD, UT date code
     Line 5 - 15.0 25.0 100.0 39.4 11.3   Photometry extraction parameters,
                                          Object radius, inner sky, outer sky, 
                                          gain and readout noise.
     Line 6 to end, these are "rule" lines that invoke different steps of
        reduction operations.  In general, these lines start with a two
        character operation code followed by the information needed for that
        step.

     The operators are:
          2c - Two color lightcurve against all sky standards
          dp - Differential photometric reduction (like Pluto)
          lc - Single color lightcurve against all sky standards.
          sl - Block for reducing Stars (in Landolt system) (color unknown).
          tr - Transformation against Landolt standard stars.

     In addition to the stated arguments, all operator lines can be terminated
        with the word "ok" (no quotes).  Any line marked in this way is
        ignored unless /FORCE is specified.

 ------------------------------------------------------------------------------
 2c - Two color lightcurve against all sky standards
 ------------------------------------------------------------------------------
 2c OBJECT SERNO FILCOD FILNAM STDFIL COLOR1 COLOR2
                 FILCOD FILNAM STDFIL COLOR1 COLOR2
 1  OBJECT - Standard object code name to reduce
 2  SERNO  - Serial number of ojbect to reduce (usually 0)
 3  FILCOD - filter code (string) as in photometry log file
 4  FILNAM - filter name (best if 1 character)
 5  STDFIL - color code for Landolt system 01234 = UBVRI
 6  COLOR1 - First color code for color to reduce against
 7  COLOR2 - Second color code (ex: B-V is color1=1, color2=2)
 8  FILCOD - filter code (string) as in photometry log file
 9  FILNAM - filter name (best if 1 character)
10  STDFIL - color code for Landolt system 01234 = UBVRI
11  COLOR1 - First color code for color to reduce against
12  COLOR2 - Second color code (ex: B-V is color1=1, color2=2)

 ------------------------------------------------------------------------------
 dp - Differential photometric reduction (like Pluto)
 ------------------------------------------------------------------------------
 dp FILCOD FILNAM COLNAM OBJECT SERNO COLOR COMP [k2 V1 E1] [ct V2 E2] [k V1 E1] [td DATE]
 1  FILCOD - filter code (string) as in photometry log file
 2  FILNAM - filter name (1 character)
 3  COLNAM - name of color (2 character, ie., BV or VR)
 4  OBJECT - Standard object code name to reduce
 5  SERNO  - Serial number of ojbect to reduce (usually 0)
 6  COLOR  - Standard color of object
 7  COMP   - Standard object name for comparison star (replace blanks with _)
  argument flags:
    k2     - override second order extinction coefficient.
    ct     - Override color term.
    k      - Override extinction.
    td     - Override transformation with values from another date.
  optional
    V1,E1  - Second order extinction and error
    V2,E2  - Color term
    DATE   - YYMMDD for date to use transformation from
 
 ------------------------------------------------------------------------------
 lc - Single color lightcurve against all sky standards.
 ------------------------------------------------------------------------------
 lc OBJECT SERNO FILCOD FILNAM STDFIL COLOR1 COLOR2 STDCOL COLERR
 1  OBJECT - Standard object code name to reduce
 2  SERNO  - Serial number of ojbect to reduce (usually 0)
 3  FILCOD - filter code (string) as in photometry log file
 4  FILNAM - filter name (best if 1 character)
 5  STDFIL - color code for Landolt system 01234 = UBVRI
 6  COLOR1 - First color code for color to reduce against
 7  COLOR2 - Second color code (ex: B-V is color1=1, color2=2)
 8  STDCOL - Standard color of object
 9  COLERR - Uncertainty of color

 ------------------------------------------------------------------------------
 sl - Block for reducing Stars (in Landolt system) (color unknown)
 ------------------------------------------------------------------------------
 sl FILCOD1 FILCOD2 FILNAM1 FILNAM2 STDFIL1 STDFIL2
 1  FILCOD1 - filter code (string) as in photometry log file for color 1
 2  FILCOD2 - filter code (string) as in photometry log file for color 2
 3  FILNAM1 - filter name (best if 1 character) for color 1
 4  FILNAM2 - filter name (bset if 1 character) for color 2
 5  STDFIL1 - color code for Landolt system 01234 = UBVRI for color 1
 6  STDFIL2 - color code for Landolt system 01234 = UBVRI for color 2

 ------------------------------------------------------------------------------
 tr - Transformation against Landolt standard stars.
 ------------------------------------------------------------------------------
 tr FILCOD FILNAM STDFIL COLOR1 COLOR2 [k2 V1 E1] [ct V2 E2] [kt]
 1  FILCOD - filter code (string) as in photometry log file
 2  FILNAM - filter name (best if 1 character)
 3  STDFIL - color codes for Landolt system 01234 = UBVRI
 4  COLOR1 - First color code for color to reduce against
 5  COLOR2 - Second color code (ex: B-V is color1=1, color2=2)

  argument flags:
    k2     - override second order extinction coefficient.
    ct     - Override color term.
    kt     - turn on time dependent extinction.
    c2     - second order color term
    x2     - second order airmass term
    rx     - optional reference airmass
    rc     - optional reference color
  optional
    V1,E1  - Second order extinction and error
    V2,E2  - Color term

 MODIFICATION HISTORY:
  Written by Marc W. Buie, 96/10/16, Lowell Observatory
  96/10/31, MWB, added "bad" flag support from rdphalt
  97/01/24, MWB, added MAGRESID flag
  92/02/27, MWB, added "dp" rule for differential photometry reductions.
  2000/01/25, John Mattox, Boston University, added missing dumpall clause.
  2000/07/06, MWB, added '9 = M = Methane' filter to "dp" rule.
  2000/07/14, MWB, fixed bug that was killing new sky signal in log file.
                 Also added printing of datamon.pro plot if force and print set.
  2000/10/16, MWB, fixed bug that was ignoring forced extintion in dp rule.
  2001/11/06, MWB, allowed for cached ephemeris calculations
  2003/10/01, MWB, converted my Delfile call to system file_delete routine
                   converted my Mkdir calls to IDL file_mkdir calls
  2004/02/09, MWB, changed fixed paths to data files and reduction areas

  2006/05/12  Peter L. Collins, Lowell Observatory
                add readout noise handling; reduc,inf file format is now 1.1
                although 1.0 accepted; 'self' added to badpar calls.
  2006/06/22  PLC, added explicit call to reducprmt to promote reduc.inf-
                   reductor prompts for read noise if inf file 1.0. Added
                   nominal instrument default for nasacam.
  2006/08/03, MWB, changed to use database for transformation information
  2006/08/15, PLC, changed to use findrawfits to locate image data resident
                   on cluster, and data bases to map rundate to instrument.
                   Reduc.inf is rewritten if data directory relocates.
  2006/08/24, PLC, enhancements for use of reductor with REDUC widget. 
                   1. ERRSTR output keyword
                   2. GUI flag
                   3. Introduced rdreduc and wrresduc routines.
  2006/08/29, PLC, more enhancements for REDUC and better instrument default.
  2006/09/01, PLC, replaced findrawfits with more general routine finddata.
  2006/09/08, PLC, removed /gw/data7 as a data mount pt and also
                   reunited a revision "branch" in reductor.pro.
  2006/09/26, PLC, pass info.inst to ltcrv, ltcrv2, and colorsol via the  
                   INSTRUMENT keyword (for use in dbphot).
  2006/10/02, PLC, add support for 'reduc.log' (timestamped rules run).
  2006/10/05, PLC, mods to reduc.log output.
  2006/10/15, PLC, add NOSAVE flag.
  2006/12/07, PLC, support new format of data table in phot database.
  2006/12/10, PLC, DATATABLE keyword.
  2006/12/18, MWB, added SAVEALLPLOTS keyword
  2006/12/27, PLC, add TRANSFTABLE and DVERBOSE keywords.
  2007/01/18, MWB, fixed problem with non-database supported instruments.
  2007/02/15, PLC, mods to 2c processing to add a color record to phot db.
  2008/06/11, MWB, added support for roboccd
  2012/10/22, MWB, changed to find landphot.dat in IDL_PATH
  2014/03/06, MWB, propagated changes in required by ltcrv* and colorsol
                      big changes here, little used options may not work
  2018/01/11, MWB, added path to coral
  2018/03/12, MWB, added support for reference airmass and color control
  2024/07/06, MWB, changed to use obsinfo