-
Notifications
You must be signed in to change notification settings - Fork 7
/
datamodel.txt
1287 lines (983 loc) · 58.4 KB
/
datamodel.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
A Datamodel for PFS
===================
(The cognoscenti will recognise certain similarities to the SDSS model,
http://data.sdss3.org/datamodel/files/BOSS_SPECTRO_REDUX/RUN2D/PLATE4
)
There are implementations of python classes that represent many of the files in this document, supporting
reading and writing FITS files that conform to this model. To use them, add the python directory in this
package to your $PYTHONPATH and import e.g. pfs.datamodel.pfsConfig
I shall use these symbolic names in the discussion:
NFIBER The number of fibres in a spectrograph (some spectrographs have "missing" fibres)
NCOLUMN The number of columns in our detectors (different for CCDs and HgCgTe?)
NROW The number of rows that spectra extend over in the dispersion direction.
In practice, the number of rows in the usable part of the CCD.
NSIMROW Number of rows for simulated data. May be different in different files, depending
on the desired oversampling.
NCOARSE Number of wavelengths used in computing the coarse
(low-resolution) covariance matrix of the spectra
The values of NCOLUMN and NROW may be different in the optical and IR arms, and will differ for
raw and reduced data. Possible values are given in the section, "Parameters"
In various places I refer to a SHA-1, which is a strong 160-bit hash, as used by e.g. git
(https://en.wikipedia.org/wiki/SHA-1). We truncate these hashes to 63bits (so as to fit
in standard 64-bit signed integers). Sixty-three bits would produce up to 2^63 ~ 9e18 values.
These SHA-1 values are encoded into 16 hexadecimal digits in filenames.
In filenames, hexadecimal characters shall be
written in lower case (0x12345abcdef not 0X12345ABCDEF or 0x12345ABCDEF).
The following variables are used in filename formats (which are written in python notation):
site
J: JHU
L: LAM
X: Subaru offline
I: IPMU
A: ASIAA
S: Summit
P: Princeton
F: simulation (fake)
category (camera)
A: Science
B: UTR (Up The Ramp)
C: Metrology Camera
D: Auto-guider
visit
An incrementing exposure number, unique at any site
spectrograph
1-4
armNum
1: Blue
2: Red
3: IR
4: Medium resolution red
(used only in raw filenames)
arm
b: Blue
r: Red
n: IR
m: Medium resolution red
tract
An integer in the range (0, 99999) specifying an area of the sky
patch
A string of the form "m,n" specifying a region within a tract
objId
A unique 64-bit object ID for an object. For example, the HSC object ID from the database.
objIds are written out using %016x formats for compactness reasons.
catId
A small integer specifying the source of the objId. Currently the following catIds are defined.
catId | description
-------|--------------------------------------------------
0 | Simulated catalog
1 | Gaia Data Release 1
2 | Gaia Data Release 2
3 | Gaia Early Data Release 3
4 | Gaia Data Release 3
5 | HSC-SSP Public Data Release 1 (Wide)
6 | HSC-SSP Public Data Release 1 (Deep+UltraDeep)
7 | HSC-SSP Public Data Release 2 (Wide)
8 | HSC-SSP Public Data Release 2 (Deep+UltraDeep)
9 | HSC-SSP Public Data Release 3 (Wide)
10 | HSC-SSP Public Data Release 3 (Deep+UltraDeep)
11 | HSC-SSP Public Data Release 4 (Wide)
12 | HSC-SSP Public Data Release 4 (Deep+UltraDeep)
1001 | Sky positions from S21A HSC-SSP (Wide)
1002 | Sky positions from PS1
1003 | Sky positions from Gaia
1004 | Sky positions for regions without PS1 data
90001 | Messier 15 stars for the engineering observation
90002 | NGC1904 for the engineering run in November 2022
90003 | NGC2419 for the engineering run in November 2022
90004 | NGC5272 for the engineering run in February 2023
90005 | NGC5904 for the engineering run in February 2023
90006 | NGC7078 for the engineering run in July 2023
90007 | NGC7089 for the engineering run in July 2023
90008 | NGC7099 for the engineering run in July 2023
pfsDesignId
An integer uniquely specifying the configuration of the PFI; specifically a SHA-1 of the
(fiberId, ra, dec) tuples (with position rounded to the nearest arcsecond) truncated to 64 bits.
See calculate_pfsDesignId() in
python/pfs/datamodel/utils.py
We include the standard prefix "0x" in filenames to visually separate SHA-1s from 64-bit objId; this is
especially important in pfsObject filenames which have both an objId and a hash.
An alternative to a SHA-1 would be the visit number of the first data taken with this configuration but that
would have the disadvantage that it would not be available until the data was taken.
Lab data may choose to set the pfsDesignId to 0x0.
A pfsDesignId of 0x0 means that no fibres are illuminated.
pfsVisitHash
An integer uniquely defining the set of visits contributing to a reduced spectrum;
this will be calculated as a SHA-1 truncated to 64 bits
See calculate_pfsVisitHash() in
python/pfs/datamodel/utils.py
As a special case, if the only visit is 0, pfsVisitHash shall be taken to be 0x0 for the convenience
of simulation code.
MaskedImage
A data type which consists of three images:
A floating point image
An integer mask image
A floating point variance image
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Auto-Guider image data
The file name format is as follows:
"agcc_{pfs_visit:06d}_{agc_exposure_id:08d}.fits"
Where agc_exposure_id is a unique integer (monotonically increasing exposure by exposure).
This data contain the following mandatory tables:
HDU #0 PDU
HDU #1 CAM1 [16-bit INT] NPIXEL
HDU #2 CAM2 [16-bit INT] NPIXEL
HDU #3 CAM3 [16-bit INT] NPIXEL
HDU #4 CAM4 [16-bit INT] NPIXEL
HDU #5 CAM5 [16-bit INT] NPIXEL
HDU #6 CAM6 [16-bit INT] NPIXEL
And the following optional tables:
HDU #7 TABLE1 FITS binary table
HDU #8 TABLE2 FITS binary table
HDU #9 TABLE3 FITS binary table
HDU #10 TABLE4 FITS binary table
HDU #11 TABLE5 FITS binary table
HDU #12 TABLE6 FITS binary table
Each TABLE[1-6] HDU contains :
moment_00 32-bit int The (0,0) image moment
centroid_x 32-bit int The x-centroid
centroid_y 32-bit int The y-centroid
moment_20 32-bit int The (2,0) image moment
moment_11 32-bit int The (1,1) image moment
moment_02 32-bit int The (0,2) image moment
peak_x 16-bit int x-location of peak value
peak_y 16-bit int y-location of peak value
peak_intensity 32-bit int peak intensity value
background 32-bit int measured background level
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Raw Data
From the spectrograph cameras:
"PF%1sA%06d%1d%1d.fits" % (site, visit, spectrograph, armNum)
Format: as written by the DAQ system. The "A" (science) frames are a single image
extension of size > NCOLUMN*NROW due to extended registers and overclocks, even for the red and blue arms
whose "detectors" consist of two physical devices.
N.b. the restriction to filenames of the form 4 letters followed by 8 decimal digits comes from Subaru.
Note that the order of the last two numbers is "spectrograph, armNum" whereas in all other filenames
we use the order "arm, spectrograph" but now arm is a string (e.g. r2, n4).
From the MCS:
"PF%1sC%06d%02d.fits" % (site, visit, sequenceId)
Where a sequenceId is used to identify the individual frames.
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Calibration products (i.e. processed flats/biases/darks ready to be used in processing)
"pfsFlat-%s-%06d-%1s%1d.fits" % (calibDate, visit0, arm, spectrograph)
"pfsBias-%s-%06d-%1s%1d.fits" % (calibDate, visit0, arm, spectrograph)
"pfsDark-%s-%06d-%1s%1d.fits" % (calibDate, visit0, arm, spectrograph)
(While theoretically we don't need to distinguish between the r and m chips, it's simpler to be
consistent, so we will have 16, not 12, biases/darks for the complete spectrograph)
Visit0 is the first visit for which these flats/biases/darks are valid; in general
there will be many files for each (arm, spectrograph) but with different values of visit0. Note that we
don't know the upper range of validity when these files are created, so it cannot be encoded in the filename.
Visit0 is usually the last input visit used in preparing the calibration files. The calibration file that is
used by the pipeline is selected based on a list of valid dates provided to the calibration database; visit0
is only used to provide a unique filename and a sensible sort order.
Single extension fits files of size NCOLUMN*NROW
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Calibrated images
"postISRCCD-%1s%1s%06d-%1s%1d.fits" % (site, category, visit, arm, spectrograph)
"calexp-%1s%1s%06d-%1s%1d.fits" % (site, category, visit, arm, spectrograph)
These are persisted versions of LSST's lsst.afw.image.Exposure class. As such,
they may include a variety of HDUs for persisting PSFs, etc., but most notable
are the following:
HDU #0 PDU
HDU #1 IMAGE [32-bit FLOAT] NPIXEL
HDU #2 MASK [32-bit INT] NPIXEL
HDU #3 VARIANCE [32-bit FLOAT] NPIXEL
...
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
The profile of fibers in the 2-D flat-field image:
"pfsFiberProfiles-%s-%06d-%1s%1d.fits" % (calibDate, visit0, arm, spectrograph)
Note that visit0 is defined under "Calibration products"
For each fiber, an empirical, oversampled profile (in the spatial dimension, columns) is recorded for several
positions along the fiber trace in the spectral dimension (rows).
HDU #0 PDU
HDU #1 FIBERS Profile parameters for each fiber [BINARY FITS TABLE] NFIBER
HDU #2 PROFILES Profile measurements [BINARY FITS TABLE] NPROFILES
Profile parameters in the FIBERS HDU are:
fiberId 32-bit int Fiber identifier (primary key)
radius 32-bit int Half-size of profiles, in native pixels
oversample 32-bit float Oversampling factor (> 1)
norm 32-bit float vector Normalisation to apply to extracted spectra
Profile measurements in the PROFILES HDU are:
fiberId 32-bit int Fiber identifier (primary key)
rows 32-bit float Detector row at which the profile is measured (primary key)
profiles 32-bit float vector Measured profile
masks boolean vector Whether values in the profiles should be used (1 = don't use)
The length of a profile (and mask) vector is: int(2*(radius + 1)*oversample) + 1.
The PDU has keys to define which data were used to construct the FiberTrace:
SPECTROGRAPH Which spectrograph
ARM The arm of the spectrograph
CALIB_INPUT_%d Visits which were used
I.e. if only visit 5830 were used, the r1 file would have ARM='r', SPECTROGRAPH=1, CALIB_INPUT_0=5830
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
The mapping of fiberId,wavelength to x,y on the detector:
"pfsDetectorMap-%06d-%s%1d.fits" % (visit0, arm, spectrograph)
Note that visit0 is defined under "Calibration products"
We have multiple separate formats for representing the "detectorMap", which are distinguished by the value of
the "HIERARCH pfs_detectorMap_class" header card:
* SplinedDetectorMap: splines for each fiber provide the wavelength and column as a function of row.
* DoubleDetectorMap: mapping of x,y to dx,dy distortion for left and right CCDs to a SplineDetectorMap.
This is deprecated in favor of MultipleDistortionsDetectorMap.
* DifferentialDetectorMap: a mapping of fiberId,wavelength to x,y corrections to a SplinedDetectorMap.
This is deprecated and due to be removed.
* DistortedDetectorMap: a mapping of x,y to dx,dy corrections to a SplinedDetectorMap.
This is deprecated and due to be removed.
* MultipleDistortionsDetectorMap: multiple mappings of x,y to dx,dy corrections to a SplinedDetectorMap.
* LayeredDetectorMap: layered corrections to a SplinedDetectorMap.
SplinedDetectorMap format:
HDU #0 PDU
HDU #1 FIBERID Identifier of each fiber [32-bit INT] NFIBER
HDU #2 SLITOFF Slit offsets in x,y,z [32-bit FLOAT] 3*NFIBER
HDU #3 CENTER Spline for column with trace center [BINARY FITS TABLE] NFIBER
HDU #4 WAVELENGTH Spline for wavelength (nm) [BINARY FITS TABLE] NFIBER
The primary header records the dimensions of the detector in the MINX,MINY,MAXX,MAXY keywords,
and the polynomial order of the distortion field in the ORDER keyword.
The CENTER and WAVELENGTH tables contain:
index 64-bit int Index of fiber (primary key)
knot 32-bit float Row value of knot (primary key)
value 32-bit float Corresponding value at knot
MultipleDistortionsDetectorMap format:
HDU #0 PDU
HDU #1 FIBERID Identifier of each fiber [32-bit INT] NFIBER
HDU #2 SLITOFF Slit offsets in x,y,z [32-bit FLOAT] 3*NFIBER
HDU #3 CENTER Spline for column with trace center [BINARY FITS TABLE] NFIBER
HDU #4 WAVELENGTH Spline for wavelength (nm) [BINARY FITS TABLE] NFIBER
HDU #5 DISTORTIONS Distortion names [BINARY FITS TABLE] NDISTORTION
HDU #6...
The primary header records the dimensions of the detector in the MINX,MINY,MAXX,MAXY keywords.
The DISTORTIONS table contains a sole column, "name", which is a list of names of distortions to apply.
The distortions are in separate HDUs, with formats peculiar to their type (specified by the name):
* PolynomialDistortion: HDU PolynomialDistortion_<index> contains the polynomial order (ORDER) and
distortion range (MINX,MINY,MAXX,MAXY) in the header, and the x and y coefficients as columns in a table.
* DoubleDistortion: HDU DoubleDistortion_<index> contains the polynomial order (ORDER) and distortion
range (MINX,MINY,MAXX,MAXY) in the header and the x and y coefficients for the left and right CCDs
in a table.
* RotScaleDistortion: HDU RotScaleDistortion_<index> contains the distortion range in the header
(MINX,MINY,MAXX,MAXY) and the distortion parameters as pixels in a one-dimensional image.
* DoubleRotScaleDistortion: HDU DoubleRotScaleDistortion_<index> contains the distortion range in the header
(MINX,MINY,MAXX,MAXY) and the distortion parameters as pixels in a one-dimensional image.
DoubleDetectorMap format:
HDU #0 PDU
HDU #1 FIBERID Identifier of each fiber [32-bit INT] NFIBER
HDU #2 SLITOFF Slit offsets in x,y,z [32-bit FLOAT] 3*NFIBER
HDU #3 CENTER Spline for column with trace center [BINARY FITS TABLE] NFIBER
HDU #4 WAVELENGTH Spline for wavelength (nm) [BINARY FITS TABLE] NFIBER
HDU #5 COEFFICIENTS Distortion coefficients [BINARY FITS TABLE] NCOEFF
The primary header records the dimensions of the detector in the MINX,MINY,MAXX,MAXY keywords.
It also contains the polynomial order of the distortion in the ORDER keyword.
DifferentialDetectorMap format:
HDU #0 PDU
HDU #1 FIBERID Identifier of each fiber [32-bit INT] NFIBER
HDU #2 SLITOFF Slit offsets in x,y,z [32-bit FLOAT] 3*NFIBER
HDU #3 CENTER Spline for column with trace center [BINARY FITS TABLE] NFIBER
HDU #4 WAVELENGTH Spline for wavelength (nm) [BINARY FITS TABLE] NFIBER
HDU #5 COEFFICIENTS Distortion coefficients [BINARY FITS TABLE] NCOEFF
HDU #6 HIGHCCD Coefficients for high-fiberId CCD [BINARY FITS TABLE] 6
The primary header records the dimensions of the detector in the MINX,MINY,MAXX,MAXY keywords.
It also contains the following HIERARCH keywords which are used in scaling the fiberId,wavelength values:
scaling.fiberPitch
scaling.dispersion
scaling.wavelengthCenter
scaling.minFiberId
scaling.maxFiberId
scaling.height
scaling.buffer
The FIBERID, SLITOFF, CENTER and WAVELENGTH HDUs are as for the SplinedDetectorMap.
The COEFFICIENTS table contains:
x 32-bit float x coefficients
y 32-bit float y coefficients
The HIGHCCD table contains:
coefficients 32-bit float Affine transform coefficients
DistortedDetectorMap format:
HDU #0 PDU
HDU #1 FIBERID Identifier of each fiber [32-bit INT] NFIBER
HDU #2 SLITOFF Slit offsets in x,y,z [32-bit FLOAT] 3*NFIBER
HDU #3 CENTER Spline for column with trace center [BINARY FITS TABLE] NFIBER
HDU #4 WAVELENGTH Spline for wavelength (nm) [BINARY FITS TABLE] NFIBER
HDU #5 COEFFICIENTS Distortion coefficients [BINARY FITS TABLE] NCOEFF
HDU #6 RIGHTCCD Coefficients for right CCD [BINARY FITS TABLE] 6
The primary header records the dimensions of the detector in the MINX,MINY,MAXX,MAXY keywords.
The FIBERID, SLITOFF, CENTER and WAVELENGTH HDUs are as for the SplinedDetectorMap.
The COEFFICIENTS table contains:
x 32-bit float x coefficients
y 32-bit float y coefficients
The RIGHTCCD table contains:
coefficients 32-bit float Affine transform coefficients
LayeredDetectorMap format:
HDU #0 PDU
HDU #1 FIBERID Identifier of each fiber [32-bit INT] NFIBER
HDU #2 SLITOFF Slit offsets in x,y,z [32-bit FLOAT] 3*NFIBER
HDU #3 CENTER Spline for column with trace center [BINARY FITS TABLE] NFIBER
HDU #4 WAVELENGTH Spline for wavelength (nm) [BINARY FITS TABLE] NFIBER
HDU #5 LAYERS Slit offsets, CCD transform [BINARY FITS TABLE]
HDU #6 DISTORTIONS Distortion list [BINARY FITS TABLE]
HDUs ...
The LAYERS HDU contains arrays of parameters for the spatial and spectral offsets,
and the affine transformation for the right-hand CCD.
The DISTORTIONS HDU contains a list of distortions to be applied, each of which
is appended as a separate HDU.
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
The design of a PFI setup, i.e., the targetting of fibers, is a PfsDesign:
"pfsDesign-0x%016x.fits" % (pfsDesignId)
The choice of a hex format is because the the pfsDesignId is a SHA-1
of the intended fiber positions.
FITS file format:
HDU #0 PDU
HDU #1 FITS binary table named "DESIGN"
HDU #2 FITS binary table named "PHOTOMETRY"
HDU #3 FITS binary table named "GUIDESTARS"
The PDU contains the following keys:
DSGN_NAM Human-readable name for the design
RA Intended telescope boresight Right Ascension (degrees)
DEC Intended telescope boresight Declination (degrees)
POSANG Intended Position Angle of the PFI (degrees).
This corresponds to the angle
from the North Celestial Pole to the PFI_Y axis,
measured clockwise looking along the positive PFI_Z axis
(ie., the telescope pointing direction).
ARMS The arms that will be exposed.
This can be any combination
of the arms 'b', 'r', 'm', and 'n'
(where each arm can
appear only once
and 'r' and 'm' cannot be present
in the same combination)
examples: 'brn', 'bmn', 'rb'.
The DESIGN table lists for each object:
fiberId 32-bit int
catId 32-bit int
tract 32-bit int
patch string
objId 64-bit int
ra 64-bit float (degrees)
dec 64-bit float (degrees)
targetType 32-bit int (enumerated type: e.g. SCIENCE,SKY,FLUXSTD)
fiberStatus 32-bit int (enumerated type: e.g. GOOD,BROKENFIBER,BLOCKED,BLACKSPOT)
pfiNominal pair of 32-bit floats (millimeters on the PFI)
epoch 7-character string (e.g., J2000.0, see also the definition for the GUIDESTARS table)
pmRa 32-bit float (mas/yr)
pmDec 32-bit float (mas/yr)
parallax 32-bit float (mas)
proposalId string
obCode string
Note that:
* the fiberIds start at 1.
* the combination of (catId, objId) must be unique
[with the exception where (catId=-1 objId=-1) which corresponds to an untargeted fiber.]
* the proposalId is a string for each proposal assigned by Subaru Telescope (e.g., S24B-001QN)
* the obCode is a string for an Observing Block (OB) within a proposal. An OB consists of a combination of
a target, exposure setup, and QA constraints. The combination of (proposalId, obCode) must be unique.
The 'targetType' in the DESIGN table is an enumerated type, with values:
SCIENCE = 1: The fiber is intended to be on a science target.
SKY = 2: The fiber is intended to be on blank sky, and used for sky subtraction.
FLUXSTD = 3: The fiber is intended to be on a flux standard, and used for flux calibration.
UNASSIGNED = 4: The fiber is not targeted on anything in particular.
ENGINEERING = 5: The fiber is an engineering fiber.
SUNSS_IMAGING = 6: The fiber is fed by SuNSS imaging leg.
SUNSS_DIFFUSE = 7: The fiber is fed by SuNSS diffuse leg.
DCB = 8: Fiber is fed by DCB/DCB2.
HOME = 9: Cobra is going to its home position.
BLACKSPOT = 10: Cobra is going behind the black spot.
AFL = 11: The fiber is fed by all fiber lamp cable.
The 'fiberStatus' in the DESIGN table is an enumerated type, with values:
GOOD = 1: The fiber is working normally.
BROKENFIBER = 2: The fiber is broken, and any flux should be ignored.
BLOCKED = 3: The transmission through the fiber is temporarily blocked. Any flux should be ignored.
BLACKSPOT = 4: The fiber is hidden behind its spot, and any flux should be ignored.
UNILLUMINATED = 5: The fiber is not being illuminated.
BROKENCOBRA = 6: The cobra cannot move, but the fiber still carries flux.
NOTCONVERGED = 7: The Cobra did not converge to the target.
The PHOTOMETRY table lists:
fiberId 32-bit int
fiberFlux [nJy] 32-bit float
psfFlux [nJy] 32-bit float
totalFlux [nJy] 32-bit float
fiberFluxErr [nJy] 32-bit float
psfFluxErr [nJy] 32-bit float
totalFluxErr [nJy] 32-bit float
filterName string
Where the:
* fiberFlux corresponds to the integrated flux within an aperture of a fiber
(approx 1 arcsec in diameter). This is seeing-corrected.
* psfFlux is the flux computed by fitting the appropriate PSF to the image,
to an infinite radius.
* totalFlux is the complete flux for that object. This is the same as the psfFlux
for unresolved sources, but for extended sources the appropriate extended model
is applied to compute the flux.
A fiberId may be listed multiple times in the PHOTOMETRY table in order to
provide measurements in multiple filters for a single object.
The 'filterName' values in the PHOTOMETRY table will specify particular
transmission curves used by the pipeline, and therefore the range of
permitted values is limited to a set to be specified by the DRP team.
There will be a mechanism for adding to this set.
The GUIDESTARS table lists:
objId 64-bit int
epoch 7-character string
ra 64-bit float (degrees)
dec 64-bit float (degrees)
pmRa 32-bit float (mas/year)
pmDec 32-bit float (mas/year)
parallax 32-bit float (mas)
magnitude 32-bit float
passband string
color 32-bit float
agId 32-bit int
agX 32-bit float (pixels)
agY 32-bit float (pixels)
Where the:
* objId corresponds to the unique object identifer for the guide star
* epoch corresponds to the reference epoch for the stellar parameters,
in the form PYYYY.x, where P is the prefix (J for Julian or B for Besselian epochs),
YYYY is the whole year for the epoch,
and x is the fraction of Julian years from the whole year, or 0 for Besselian epochs.
* ra corresponds to the Right Ascension of the guide star
* dec corresponds to the Declination of the guide star
* pmRa is the proper motion of the guide star along the RA direction
* pmDec is the proper motion of the guide star along the DEC direction
* parallax is the observed parallax of the guide star
* magnitude is the magnitude for the guide star
* passband is the passband over which the magnitude is measured.
* color is the Gaia broadband BP-RP color for that guide star.
* agId is the identifier for the AG camera that will observe that guide star.
This can have a value from 0 to 5 inclusive.
* agX is the X-position on the corresponding AG camera where the guide star
is expected to be detected
* agY is the Y-position on the corresponding AG camera where the guide star
is expected to be detected
Additional keywords are required for the GUIDESTARS HDU:
* EPOCH the reference epoch for the guide star data.
All the guide star information share the same epoch.
This should be accurate to the order of weeks to ensure that the
parallax can be incorporated into the measurements.
The format for the time is the ISO-8601 yyyy-mm-ddThh:mm:ss format
and the time coordinate is TCB.
* TEL_ELEV the telescope elevation (degrees)
* GS_CATID the identifier of the catalogue containing the guide stars.
This implies that all the guide stars in this table originate from
a single catalogue.
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
The realisation of a PfsDesign for a particular exposure is a PfsConfig.
The most important differences from PfsDesign are that the actual fiber
positions and object photometry are recorded for use in the DRP.
"pfsConfig-0x%016x-%06d.fits" % (pfsDesignId, visit)
The choice of a hex format is because the the pfsDesignId is a SHA-1.
FITS file format:
HDU #0 PDU
HDU #1 FITS binary table named "CONFIG"
HDU #2 FITS binary table named "PHOTOMETRY"
The PDU has keys indicating the actual pointing of the telescope boresight.
RA Actual telescope boresight Right Ascension (degrees).
DEC Actual telescope boresight Declination (degrees).
The CONFIG table lists for each object:
fiberId 32-bit int
catId 32-bit int
tract 32-bit int
patch string
objId 64-bit int
ra 64-bit float (degrees)
dec 64-bit float (degrees)
targetType 32-bit int (enumerated type: SCIENCE,SKY,FLUXSTD,UNASSIGNED,ENGINEERING)
fiberStatus 32-bit int (enumerated type: GOOD,BROKENFIBER,BLOCKED,BLACKSPOT,UNILLUMINATED)
pfiCenter pair of 32-bit floats (millimeters on the PFI)
pfiNominal pair of 32-bit floats (millimeters on the PFI)
epoch 7-character string
pmRa 32-bit float (mas/yr)
pmDec 32-bit float (mas/yr)
parallax 32-bit float (mas)
proposalId string
obCode string
(I.e., the same as for PfsDesign, except for the addition of the
"pfiCenter" column.)
The 'targetType' and 'fiberStatus' in the CONFIG table are the same enumerated
types as in the PfsDesign.
The PHOTOMETRY table is the same as for the PfsDesign.
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Reduced but not combined single spectra from a single exposure (wavelength calibrated but not flux-calibrated)
"pfsArm-%06d-%1s%1d.fits" % (visit, arm, spectrograph)
N.b. visit numbers are only unique at a site, but we don't preserve this in derived product names. There will
keywords in the header to disambiguate this.
The file will have several HDUs:
HDU #0 PDU
HDU #1 FIBERID Fiber identifier [32-bit INT] NFIBER
HDU #2 WAVELENGTH Wavelength (nm, vacuum) [64-bit FLOAT] NROW*NFIBER
HDU #3 FLUX Flux (electrons) [FLOAT] NROW*NFIBER
HDU #4 MASK Pixel mask [32-bit INT] NROW*NFIBER
HDU #5 SKY Sky flux (electrons) [FLOAT] NROW*NFIBER
HDU #6 NORM Normalisation (electrons) [FLOAT] NROW*NFIBER
HDU #7 COVAR Near-diagonal part of flux covariance [FLOAT] NROW*3*NFIBER
HDU #8 CONFIG Identification of the exposure [BINARY TABLE]
HDU #9 NOTES Reduction notes [BINARY TABLE] NFIBER*NNOTES
Note that the data need not be resampled onto a uniform grid, as a wavelength is provided for each pixel.
The COVAR data contains the diagonal COVAR[fiberId][0][0:]
+-1 COVAR[fiberId][1][0:-1]
and
+-2 COVAR[fiberId][2][0:-2]
terms in the covariance matrix.
At a minimum the CONFIG table contains columns for pfsDesignId and visit (and only one row! But it's
a table not header keywords for consistency with the pfsObject file).
The reduction notes (NOTES) HDU is a FITS table HDU with one row for each fiber and a variety of columns.
The values record operations performed and measurements made during reduction for that fiber.
It would be possible to denormalise additional information about the observation into this HDU. I do not
know if it should also contain things we've learned about the observation from the analysis that created
this file; I'd be inclined to put it in a separate temporary file, and then build a single file describing
the entire PFS from the 12 temporaries. Alternatively, this could be done at the database level.
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Combined spectra from a single exposure (wavelength calibrated but not flux calibrated)
"pfsMerged-%06d.fits" % (visit,)
The format will be identical to that of the pfsArm files, except that the fluxes
(in the FLUX, SKY and NORM HDUs) will be in units of electrons per nm.
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Flux-calibrated arm-merged spectrum from a single exposure
"pfsSingle-%05d-%05d-%s-%016x-%06d.fits" % (catId, tract, patch, objId, visit)
The format will be identical to that of the pfsObject files.
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
2D spectral line measurements
The 2D pipeline measures the position and flux of sky and arc lines as part of the reduction process.
Note that science target lines are not included here.
As of 2022-07-22 (with the LSST Gen2-based pipeline), these are written in a single file:
"arcLines-%06d-%s%d.fits" % (visit, arm, spectrograph)
In the future (with the LSST Gen3-based pipeline), the centroid and photometry measurements will be
written to separate files:
"lineCentroids-%06d-%1s%1d.fits" % (visit, arm, spectrograph)
"linePhotometry-%06d-%1s%1d.fits" % (visit, arm, spectrograph)
These FITS files will include a single table HDU (in addition to the mandatory empty PHDU) with columns:
fiberId 32-bit int
wavelength 64-bit float (nm)
x 32-bit float (pixels)
y 32-bit float (pixels)
xErr 32-bit float (pixels)
yErr 32-bit float (pixels)
xx 32-bit float (pixels^2)
yy 32-bit float (pixels^2)
xy 32-bit float (pixels^2)
flux 32-bit float (arbitrary units)
fluxErr 32-bit float (arbitrary units)
fluxNorm 32-bit float (arbitrary units)
norm 32-bit float (arbitrary units)
flag boolean; indicates whether the measurement is bad
status 32-bit int; bitmask
description string
transition string; indicates line transition if appropriate
source 32-bit int; bitmask
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
2D spectral calibrations
The 2D pipeline measures and applies multiple calibrations to the spectra as part of the reduction process.
"apCorr-%06d-%s%d.fits" % (visit, arm, spectrograph)
"sky2d-%06d-%s%d.fits" % (visit, arm, spectrograph)
"sky1d-%06d-%s.fits" % (visit, arm)
"fluxCal-%06d.fits" % (visit,)
Each of these are functions of wavelength and position on the focal plane, but different representations.
The particular representation used is written in the PHDU as the value of the
"HIERARCH pfs_focalPlaneFunction_class" keyword.
* PfsConstantFocalPlaneFunction: a constant spectrum for the whole focal-plane.
The FITS file includes the following HDUs:
HDU #0 PDU
HDU #2 WAVELENGTH Wavelength (nm, vacuum) [64-bit FLOAT] LENGTH
HDU #1 VALUE Value of the function [32-bit FLOAT] LENGTH
HDU #3 MASK Is the value bad? [8-bit UINT] LENGTH
HDU #4 VARIANCE Error^2 of the value [32-bit FLOAT] LENGTH
* PfsOversampledSpline: a spline in wavelength for the whole focal-plane.
The FITS file includes a "SPLINE" table HDU with ORDER (spline order) and
DEFAULT (default value) header keywords and the following columns:
knots 64-bit float
coeffs 64-bit float
wavelength 64-bit float
variance 64-bit float
* PfsBlockedOversampledSpline: a spline for blocks of fiberId. The FITS file
includes a "BLOCKSPLINE" HDU with the following columns:
fiberId 32-bit float
splineOrder 32-bit integer
defaultValue 64-bit float
knots 64-bit float array
coeffs 64-bit float array
wavelength 64-bit float array
variance 64-bit float array
* PfsPolynomialPerFiber: a polynomial in wavelength for each fiber. The FITS file
includes a "POLYPERFIBER" table HDU with MIN_WL (minimum wavelength) and
MAX_WL (maximum wavelength) header keywords and the following columns:
fiberId 32-bit integer
coeffs 64-bit float array
rms 64-bit float
* PfsFluxCalib: Flux calibration vector: h(lam) exp g(x,y,lam), where h(lam)
is PfsConstantFocalPlaneFunction, and g(x,y,lam) is a polynomial.
The FITS file includes the same HDUs as PfsConstantFocalPlaneFunction does,
plus a binary table HDU named "POLYNOMIAL" defining g(x,y,lam). The last
table has only a single row with the following columns:
params 64-bit float array, length combination(order+3, 3)
min 64-bit float array, length 3.
max 64-bit float array, length 3.
params[i] is the coefficient by which to multiply the term
x^e[i][0] y^e[i][1] lam^e[i][2]. "e", array of 3-tuple of exponents, is
in descending order of e[.][0] + e[.][1] + e[.][2]. Ties are broken in
the reverse order of 3-tuple. For example, e = [(2,0,0),(1,1,0),(1,0,1),
(0,2,0),(0,1,1),(0,0,2),(1,0,0),(0,1,0),(0,0,1),(0,0,0)] if order=2.
min (max) is the vertex of the rectangular-parallelepipedal domain of the
polynomial at which (x, y, lam) are minimal (maximal). Arguments of the
polynomial are normalized so that min will be mapped to (0, 0, 0) and max
to (1, 1, 1). This nomalization happens before the arguments are
substituted for the variables of the polynomial.
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
The PSF for a single exposure
"pfsPSF-%06d-%1s%1d.fits" % (visit, arm, spectrograph)
Format: TBD
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Fiber normalizations
The fiber normalizations consist of polynomial coefficients used to correct the
normalization of fiber fluxes, so that the fluxes in different fibers may be
compared accurately.
"pfsFiberNorms-%s-%06d-%s.fits" % (calibDate, visit0, arm)
The file contains multiple image HDUs:
HDU #0 PDU
HDU #1 FIBERID Fiber identifiers [32-bit INT] NFIBER
HUD #2 WAVELENGTH Wavelength (nm) [64-bit FLOAT] NFIBER*NROW
HDU #3 VALUES Normalization values [64-bit FLOAT] NFIBER*NROW
HDU #4 MODEL Model details
The MODEL HDU contains the details of the model used to create the normalization
values. The format is not specified here, and varies according to the model.
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Reference model
The reference model set is a product from `calculateReferenceFlux` and is stored in the pfsFluxReference file.
pfsFluxReference is used in the flux calibration procedure together with pfsMerged to produce flux-calibrated
spectra. pfsFluxReference include the wavelength and flux of the reference spectra of standard stars, as well
as fitting results. We propose that pfsFluxReference includes multiple objects in a visit like pfsMerged.
The reference model set is saved to:
"pfsFluxReference-%06d.fits" % (visit,)
The file has several HDUs:
HDU #0 PDU
HDU #1 FIBERID Fiber identifier [32-bit INT] NFLUXSTD
HDU #2 FLUX Flux of reference models in units of nJy [FLOAT] (the number of pixels of a model)*NFLUXSTD
HDU #3 FITFLAG Flag if the fitting was successful or not [32-bit INT] NFLUXSTD
HDU #4 FITPARAMS Table of by-products [BINARY TABLE]
NFLUXSTD is the number of fibers for flux standards, FLUXSTD.
PDU includes the visit number. HDU #3 includes the flag of whether the fitting process was successful or not.
This flag also includes the causes of failure. HDU #4 includes a binary table of by-products from
the calculateReferenceFlux process. HDU #4 includes at least the stellar parameters of the reference models,
the estimated radial velocity, the fitting parameters to continuum, the scaling factor, and Galactic
extinction.
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Combined spectra.
In SDSS we used "spPlate" files, named by the plugplate and MJD of observation but this is not suitable for
PFS where we:
1. Will split observations of the same object over multiple nights
2. Will potentially reconfigure the PFI between observations.
I don't think it makes sense to put multiple spectra together based on sky coordinates as we may go back and
add more observations later, so I think we're forced to separate files for every object. That's a lot of
files, but maybe not too bad? We could use a directory structure based on HSC's (tract, patch) -- note that
these are well defined even if we are not using HSC data to target. An alternative would be to use a
healpix or HTM id.
Because we may later obtain more data on a given object, or decide that some data we have already taken is
bad, or process a number of subsets of the available data, there may be more than one set of visits used
to produce a pfsObject file for a given object. We therefore include both the number of visits (nVisit)
and a SHA-1 hash of the visits, pfsVisitHash. We use both as nVisits may be ambiguous, while pfsVisitHash
isn't human-friendly; in particular it doesn't sort in a helpful way. It seems improbable that we will
ever have more than 1000 visits, but as the pfsVisitHash is unambiguous it seemed safer to only allow for
larger values of nVisit, but record them only modulo 1000.
"pfsObject-%05d-%05d-%s-%016x-%03d-0x%016x.fits"
% (catId, tract, patch, objId, nVisit % 1000, pfsVisitHash)
The path would be
catId/tract/patch/pfsObject-*.fits
The file will have several HDUs:
HDU #0 PDU
HDU #1 FLUX Flux in units of nJy [FLOAT] NROW
HDU #2 MASK Pixel mask [32-bit INT] NROW
HDU #3 TARGET Binary table [FITS BINARY TABLE] NFILTER
Columns for:
filterName [STRING]
fiberFlux [FLOAT]
HDU #4 SKY Sky flux in same units as FLUX [FLOAT] NROW
HDU #5 COVAR Near-diagonal part of FLUX's covariance [FLOAT] NROW*3
HDU #6 COVAR2 Low-resolution non-sparse estimate covariance [FLOAT] NCOARSE*NCOARSE
HDU #7 OBSERVATIONS Binary table [FITS BINARY TABLE] NOBS
Columns for:
visit [32-bit INT]
arm [STRING]
spectrograph [32-bit INT]
pfsDesignId [64-bit INT]
fiberId [32-bit INT]
nominal PFI position (millimeters) [FLOAT]*2
actual PFI position (millimeters) [FLOAT]*2
HDU #8 FLUXTABLE Binary table [FITS BINARY TABLE] NOBS*NROW
Columns for:
wavelength in units of nm (vacuum) [64-bit FLOAT]
intensity in units of nJy [FLOAT]
intensity error in same units as intensity [FLOAT]
mask [32-bit INT]
HDU #9 NOTES Reduction notes [FITS BINARY TABLE] NNOTES
The wavelengths are specified via the WCS cards in the header (e.g. CRPIX1,
CRVAL1) for the FLUX, MASK, SKY, COVAR extensions and explicitly in the table
for the FLUXTABLE. We chose these two representations for the data due to the
difficulty in resampling marginally sampled data onto a regular grid, while
recognising the convenience of such a grid when rebinning, performing PCAs, or
stacking spectra. For highest precision the data in the FLUXTABLE is likely to
be used.
The TARGET HDU must contain at least the keywords
catId Catalog identifier INT
tract Tract identifier INT
patch Patch identifier STRING
objId Object identifier INT
ra Right Ascension (degrees) DOUBLE
dec Declination (degrees) DOUBLE
targetType Target type enum INT
(N.b. the keywords are case-insensitive). Other HDUs should specify INHERIT=T.
See pfsArm for definition of the COVAR data
What resolution should we use for HDU #1? The instrument has a dispersion per pixel which is roughly constant
(in the blue arm Jim-sensei calculates that it varies from 0.70 to 0.65 (going red) A/pix; in the red, 0.88 to
0.82, and in the IR, 0.84 to 0.77). We propose that we sample at 0.8 A/pixel.
The second covariance table (COVAR2) is the full covariance at low spectral resolution, maybe 10x10. It's
really only 0.5*NCOARSE*(NCOARSE + 1) numbers, but it doesn't seem worth the trouble to save a few bytes.
This covariance is needed to model the spectrophotometric errors.
The reduction notes (NOTES) HDU is a FITS table HDU with a single row and a variety of columns.
The values record operations performed and measurements made during reduction for the spectrum.
Note that we don't keep the SDSS "AND" and "OR" masks -- if needs be we could set two mask bits to capture
the same information, but in practice SDSS's OR masks were not very useful.
For data taken with the medium resolution spectrograph, HDU #1 is expected to be at the resolution of
the medium arm, and to omit the data from the blue and IR arms.
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Line-spread functions
The line-spread function (LSF) is the response of the spectrograph to a single
narrow line; it is the one-dimensional equivalent of the point-spread function
(PSF).
The datamodel representation for the LSF has not yet been settled. However, they
are temporarily represented as python pickles of instances of classes defined in
drp_stella (subclasses of pfs.drp.stella.Lsf; see the docstrings for API
details) so that they can be used in the mean time.
The LSFs will be saved to:
"pfsArmLsf-%06d-%1s%1d.pickle" % (visit, arm, spectrograph)
"pfsMergedLsf-%06d.pickle" % (visit,)
"pfsSingleLsf-%05d-%05d-%s-%016x-%06d.pickle" % (catId, tract, patch, objId, visit)
"pfsObjectLsf-%05d-%05d-%s-%016x-%03d-0x%016x.pickle" % (catId, tract, patch, objId, nVisit, pfsVisitHash)
These correspond to the corresponding FITS product without the "Lsf", e.g.
"pfsArm-%06d-%1s%1d.fits" % (visit, arm, spectrograph)
The pickle files can be read in python using the following function:
def readLsfPickle(filename):
"""Read the line-spread function from a pickle file"""
import pickle
import pfs.drp.stella # Load class definitions
with open(filename, "rb") as fd:
return pickle.load(fd)
This will return either a single Lsf object (for pfsSingleLsf and pfsObjectLsf
files) or a python dictionary of Lsf objects, indexed by fiberId (in the case of
pfsArmLsf and pfsMergedLsf files).
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Pixel mask values
We use bit masks to flag pixels in the images and spectra. The bit masks have
symbolic names that are recorded in the FITS headers with a prefix "MP_" (or