arguments missing for doDqicorr call from x1dcorr

[NCastro-Segura]

Just for the record,

I will be back to this.

~/miniconda3/lib/python3.6/site-packages/costools/x1dcorr.py in x1dcorr(input, outdir, update_input, find, cutoff, location, extrsize, verbosity)
    183                     location=locn, extrsize=extrsz,
    184                     find_target={"flag": find, "cutoff": cutoff},
--> 185                     verbosity=verbosity)
    186 
    187 

~/miniconda3/lib/python3.6/site-packages/calcos/x1d.py in extractSpec(inlist, outdir, update_input, location, extrsize, find_target, verbosity)
    205                            x1d_ab_list[i])
    206             is_wavecal = makeFltCounts(cal_ver, corrtag_list[i],
--> 207                                        flt_list[i], counts_list[i])
    208             extract.extract1D(flt_list[i], counts_list[i], x1d_ab_list[i],
    209                               location=location, extrsize=extrsize,

~/miniconda3/lib/python3.6/site-packages/calcos/x1d.py in makeFltCounts(cal_ver, corrtag, flt, counts)
    446                 reffiles["brftab"])
    447     dq_array = timetag.doDqicorr(events, corrtag, info, switches, reffiles,
--> 448                                  phdr, headers[1], minmax_shift_dict)
    449 
    450     timetag.writeImages(x, y, epsilon, dq,

TypeError: doDqicorr() missing 2 required positional arguments: 'traceprofile' and 'gti'

[NCastro-Segura]

I added gti = fits.open(corrtag)['GTI'].data to makeFltCounts function

and set the arguments as follows
dq_array = timetag.doDqicorr(events, corrtag, info, switches, reffiles, phdr, headers[1], minmax_shift_dict,traceprofile=None, gti=gti)

[NCastro-Segura]

This took me to another error in the latest version of the software. I downgraded to version 3.3.7, corrected for the issue mentioned above and now it works.

The problem in version 3.3.9 is with the header keywords and the output during the spectral extraction. Here is the last section after creating the flt and counts file:

Info:  find-target option = no
X1DCORR   PERFORM
Extraction algorithm = TWOZONE
TWOZXTAB= lref$2bj2256nl_2zx.fits
PROFTAB = lref$2bj2256ql_profile.fits
DISPTAB = lref$2bj2256ml_disp.fits
HELCORR   PERFORM
BACKCORR  PERFORM
STATFLAG  T
FLUXCORR  PERFORM
FLUXTAB = lref$2bj2256gl_phot.fits
TDSCORR   PERFORM
TDSTAB  = lref$29o1427cl_tds.fits
Two-zone extraction method
Using profile centroid of 434.552635
Calculating background for reference profile
Using extraction percentiles of:
0.500000 and 99.500000 (outer region)
and 10.000000 and 90.000000 (inner region)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
~/miniconda3/lib/python3.6/site-packages/astropy/io/fits/column.py in _get_index(names, key)
   2031         try:
-> 2032             indx = names.index(key.rstrip())
   2033         except ValueError:

ValueError: 'cenwave' is not in list

During handling of the above exception, another exception occurred:

KeyError                                  Traceback (most recent call last)
<ipython-input-5-7f61f4096713> in <module>
      5 x1dcorr.x1dcorr(corrtag_split, outdir="HST_out/splittag_out/ck_sel/on/",
      6                         update_input=False,
----> 7                         find=False, verbosity=2)

~/miniconda3/lib/python3.6/site-packages/costools/x1dcorr.py in x1dcorr(input, outdir, update_input, find, cutoff, location, extrsize, verbosity)
    183                     location=locn, extrsize=extrsz,
    184                     find_target={"flag": find, "cutoff": cutoff},
--> 185                     verbosity=verbosity)
    186 
    187 

~/miniconda3/lib/python3.6/site-packages/calcos/x1d.py in extractSpec(inlist, outdir, update_input, location, extrsize, find_target, verbosity)
    208             extract.extract1D(flt_list[i], counts_list[i], x1d_ab_list[i],
    209                               location=location, extrsize=extrsize,
--> 210                               find_target=find_target)
    211 
    212         # For FUV, merge the x1d_a.fits and x1d_b.fits files to x1d.fits.

~/miniconda3/lib/python3.6/site-packages/calcos/extract.py in extract1D(input, incounts, output, update_input, location, extrsize, find_target)
    214         if switches["fluxcorr"] == "PERFORM":
    215             # Convert net count rate to flux.
--> 216             doFluxCorr(ofd, info, reffiles, switches["tdscorr"])
    217     # Update nrows, in case rows were skipped during 1-D extraction.
    218     nrows = ofd[1].data.shape[0]

~/miniconda3/lib/python3.6/site-packages/calcos/extract.py in doFluxCorr(ofd, info, reffiles, tdscorr)
   2231             cosutil.comparePulseHeightRanges(pharange, ref_pharange, tdstab)
   2232             filter["segment"] = segment[row]
-> 2233             tds_info = cosutil.getTable(tdstab, filter, exactly_one=True)
   2234             names = []
   2235             for name in tds_info.names:

~/miniconda3/lib/python3.6/site-packages/calcos/cosutil.py in getTable(table, filter, extension, exactly_one, at_least_one)
    510         if filter[key] == STRING_WILDCARD or filter[key] == NOT_APPLICABLE:
    511             continue
--> 512         column = data.field(key)
    513         if isinstance(filter[key], tuple) or isinstance(filter[key], list):
    514             (relation_fcn, value) = filter[key]

~/miniconda3/lib/python3.6/site-packages/astropy/io/fits/fitsrec.py in field(self, key)
    692         # NOTE: The *column* index may not be the same as the field index in
    693         # the recarray, if the column is a phantom column
--> 694         column = self.columns[key]
    695         name = column.name
    696         format = column.format

~/miniconda3/lib/python3.6/site-packages/astropy/io/fits/column.py in __getitem__(self, key)
   1637     def __getitem__(self, key):
   1638         if isinstance(key, str):
-> 1639             key = _get_index(self.names, key)
   1640 
   1641         x = self.columns[key]

~/miniconda3/lib/python3.6/site-packages/astropy/io/fits/column.py in _get_index(names, key)
   2039                 indx = names.index(_key)
   2040             elif count == 0:
-> 2041                 raise KeyError(f"Key '{key}' does not exist.")
   2042             else:              # multiple match
   2043                 raise KeyError(f"Ambiguous key name '{key}'.")

KeyError: "Key 'cenwave' does not exist."

[Rplesha]

@NCastro-Segura Currently, we don't recommend use of the x1dcorr task, as it has not been updated since we enabled TWOZONE extraction in CalCOS version 3.0. Instead we recommend running your files through CalCOS again:
https://stsci.service-now.com/hst?id=kb_article&sys_id=feb1ccd9db24db80fb50f9baae9619d5

[NCastro-Segura]

Many thanks @Rplesha for the quick response. I will use calcos as you suggested.

I am doing time resolved spectroscopy and just followed the suggestions in the data handbook.

BTW, I have used splittag to divide the corrtag files, however the exposure time of the subsections of the exposures is rather low. I haven't found any function to combine different corrtag files, I did a routine of my own to do so but I think the FP-POS may play an important role here. I'd like to know if there is one implemented in calcos.

Thanks,
Noel.

[Rplesha]

@NCastro-Segura I don't believe there is any function to combine different corrtag files that I know of. The FP-POS are combined at the x1d (so extracted spectrum) level.