Error in Diffusive response step with enrico_sed

[abhradeep-1996]

Hi @davidsanchez ,
I am suddenly facing a KeyError in gtdiffrsp step while running enrico_sed.

(fermi) hegro@hegro:~/Desktop/TIFR_Research/S4_0954p65/fermi/MJD_59000_60000$ enrico_sed sed_MJD_59685_59755.conf

************************************************************

*** SUMMARY: FRONTBACK ***

************************************************************

Source = S4_0954
RA = 149.697 degrees
Dec = 65.5652 degrees
Start = 671760005.0 MET (s)
Stop = 677808005.0 MET (s)
ROI = 10.0 degrees
E min = 100.0 MeV
E max = 300000.0 MeV
E min ext = 100.0 MeV
E max ext = 300000.0 MeV
IRFs = P8R3_SOURCE_V3
evclass = 128
evtype = 3

************************************************************

*** 1 gtselect --- Select data from library, coarse cut

************************************************************

time -p gtselect infile=/home/hegro/Desktop/TIFR_Research/S4_0954p65/fermi/MJD_59000_60000/event.list outfile=/home/hegro/Desktop/TIFR_Research/S4_0954p65/fermi/MJD_59000_60000/MJD_59685_59755/S4_0954_LAT_Analysis_EvtCoarse.fits.tmpout ra=149.697 dec=65.5652 rad=10.0 tmin=671760005.0 tmax=677808005.0 emin=0.0 emax=10000000.0 zmin=0.0 zmax=90.0 evclass=128 evtype="INDEF" convtype=-1 phasemin=0.0 phasemax=1.0 evtable="EVENTS" chatter=2 clobber=yes debug=no gui=no mode="ql"
Done.
real 20.57
user 20.12
sys 0.83

************************************************************

*** 2 gtselect --- Select data from library, fine cut

************************************************************

time -p gtselect infile=/home/hegro/Desktop/TIFR_Research/S4_0954p65/fermi/MJD_59000_60000/MJD_59685_59755/S4_0954_LAT_Analysis_EvtCoarse.fits outfile=/home/hegro/Desktop/TIFR_Research/S4_0954p65/fermi/MJD_59000_60000/MJD_59685_59755/S4_0954_LAT_Analysis_FRONTBACK_Evt.fits.tmpout ra=0.0 dec=0.0 rad=180.0 tmin="INDEF" tmax="INDEF" emin=100.0 emax=300000.0 zmin=0.0 zmax=90.0 evclass=128 evtype=3 convtype=-1 phasemin=0.0 phasemax=1.0 evtable="EVENTS" chatter=2 clobber=yes debug=no gui=no mode="ql"
Done.
real 0.19
user 0.33
sys 0.24

************************************************************

*** 3 gtmktime --- Update the GTI and cut data based on ROI

************************************************************

time -p gtmktime scfile=/home/hegro/Desktop/TIFR_Research/S4_0954p65/fermi/MJD_59000_60000/L230309123610426FC86184_SC00.fits sctable="SC_DATA" filter="(DATA_QUAL>0)&&(LAT_CONFIG==1)" roicut=no evfile=/home/hegro/Desktop/TIFR_Research/S4_0954p65/fermi/MJD_59000_60000/MJD_59685_59755/S4_0954_LAT_Analysis_FRONTBACK_Evt.fits evtable="EVENTS" outfile="/home/hegro/Desktop/TIFR_Research/S4_0954p65/fermi/MJD_59000_60000/MJD_59685_59755/S4_0954_LAT_Analysis_FRONTBACK_MkTime.fits.tmpout" apply_filter=yes overwrite=no header_obstimes=yes tstart=671760005.0 tstop=677808005.0 gtifile="default" chatter=2 clobber=yes debug=no gui=no mode="ql"
real 1.77
user 1.66
sys 0.45

************************************************************

*** 4 gtdiffrsp --- Compute Diffuse response

************************************************************

Traceback (most recent call last):
File "/home/hegro/enrico/bin/enrico_sed", line 48, in
liste = np.genfromtxt(sys.argv[1],dtype="str",unpack=True)
File "/home/hegro/miniconda3/envs/fermi/lib/python3.9/site-packages/numpy/lib/npyio.py", line 2309, in genfromtxt
raise ValueError(errmsg)
ValueError: Some errors were detected !
Line #9 (got 1 columns instead of 3)
Line #20 (got 1 columns instead of 3)
Line #29 (got 1 columns instead of 3)
Line #37 (got 1 columns instead of 3)
Line #43 (got 1 columns instead of 3)
Line #49 (got 1 columns instead of 3)
Line #54 (got 1 columns instead of 3)
Line #70 (got 1 columns instead of 3)
Line #79 (got 1 columns instead of 3)
Line #85 (got 1 columns instead of 3)
Line #101 (got 1 columns instead of 3)
Line #111 (got 1 columns instead of 3)
Line #123 (got 1 columns instead of 3)
Line #142 (got 1 columns instead of 3)
Line #158 (got 1 columns instead of 3)
Line #166 (got 1 columns instead of 3)
Line #181 (got 1 columns instead of 3)
Line #195 (got 1 columns instead of 3)
Line #206 (got 1 columns instead of 3)
Line #212 (got 1 columns instead of 3)
Line #222 (got 1 columns instead of 3)
Line #226 (got 1 columns instead of 3)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/hegro/enrico/bin/enrico_sed", line 55, in
sed(config,infile)
File "/home/hegro/enrico/bin/enrico_sed", line 13, in sed
run(infile)
File "/home/hegro/enrico/enrico/RunGTlike.py", line 213, in run
FitRunner,Fit,ListOfAnalysisObjects = GenAnalysisObjects(config)
File "/home/hegro/enrico/enrico/RunGTlike.py", line 183, in GenAnalysisObjects
Analyse = Analysis(folder, config, configgeneric=config,tag=typeirfs[evt], verbose = verbose)
File "/home/hegro/enrico/enrico/RunGTlike.py", line 30, in Analysis
FitRunner.GenerateFits() #Generates fits files for the rest of the products
File "/home/hegro/enrico/enrico/fitmaker.py", line 71, in GenerateFits
self.obs.DiffResps()#run gtdiffresp
File "/home/hegro/enrico/enrico/gtfunction.py", line 514, in DiffResps
self.run_retry_compress(diffResps)
File "/home/hegro/enrico/enrico/gtfunction.py", line 167, in run_retry_compress
run_retry(macro,tries,compress)
File "/home/hegro/enrico/enrico/gtfunction.py", line 32, in run_retry
orig_name = str(macro['outfile']).replace(".gz","")
File "/home/hegro/miniconda3/envs/fermi/lib/python3.9/site-packages/fermitools/GtApp.py", line 46, in getitem
return self.pars[key]
File "/home/hegro/miniconda3/envs/fermi/lib/python3.9/site-packages/fermitools/pil.py", line 58, in getitem
value = (self.params[name][2]).strip('"').strip("'")
KeyError: 'outfile'

Seems like it is enrico related error.
Can you please help?

[abhradeep-1996]

Seems like the problem is in the gtdiffrsp step, the "diffResps" function goes as the "macro" in the File "/home/hegro/enrico/enrico/gtfunction.py", line 167, in run_retry_compress
run_retry(macro,tries,compress) function.

But "diffResps" does not have any "outfile" keyword, which is giving the error.

So, in "gtfunction.py" line 514, I commented out "self.run_retry_compress(diffResps)".
And uncommented liine 513- "diffResps.run()".

Now the error disappeared.