Processing RINEX provided by DORIS

[ChristopherRabotin]

Hi Guillaume,

I'm trying to build measurement models for orbit determination ground stations. The approach is to find the autocorrelation in the range and Doppler measurements from ground stations using the HY-2A spacecraft and use that for tuning the ground station models. That data is public and available on DORIS. I've attached the RINEX v3 I have to this discussion, but here is the header:

      3.00           O                   D                   RINEX VERSION / TYPE
Expert              CNES                20200914 083839 UTC PGM / RUN BY / DATE 
G = GPS R = GLONASS E = GALILEO S = GEO M = MIXED D = DORIS COMMENT             
HY-2A                                                       SATELLITE NAME      
2011-043A                                                   COSPAR NUMBER       
SPA_BN1_5.5.2       CNES                                    OBSERVER / AGENCY   
CHAIN1              DGXX                1.00                REC # / TYPE / VERS 
DORIS               STAREC                                  ANT # / TYPE        
        0.8500       -0.7500        1.3060                  APPROX POSITION XYZ 
        1.2464        0.0000        0.0008                  CENTER OF MASS: XYZ 
D   10  L1  L2  C1  C2  W1  W2   F   P   T   H              SYS / # / OBS TYPES 
  2020    09    11    00    00   28.8483914     DOR         TIME OF FIRST OBS   
D  100   2  C1  C2                                          SYS / SCALE FACTOR  
D          -1.900                                           L2 / L1 DATE OFFSET 
    52                                                      # OF STATIONS       
D01  SYQB SYOWA                         66006S005  3   0    STATION REFERENCE   
D02  CRQC CROZET                        91301S004  4   0    STATION REFERENCE   
D03  MAUB MARION ISLAND                 30313S005  3   0    STATION REFERENCE   
D04  ADHC TERRE ADELIE                  91501S005  3   0    STATION REFERENCE   

(...)

D46           2.061           6.786                         TIME REF STATION    
  2020    09    11    00    00    0.0000000                 TIME REF STAT DATE  
                                                            END OF HEADER       
> 2020 09 11 00 00 32.759949800  0  3       -3.911558370 0 
D01  -1405364.026     -276928.863   109246558.05211 109246808.40511      -122.200 7
         -111.350 7      3113.319         969.000 0       -19.700 0        83.000 0
D02  -3656537.266    -4088328.967   120137088.98012 120136837.15612      -128.150 7
         -116.600 7      3113.319        1010.000 0         1.000 0        66.000 0
D03  -2213995.947     -436273.113   130619626.99813 130618910.50213      -129.900 7
         -121.850 7      3113.319        1003.652 1         5.174 1        76.087 1
> 2020 09 11 00 00 35.759949800  0  3       -3.911558370 0 
D01  -1507546.423     -297064.066   109245053.65411 109245303.99211      -122.200 7
         -111.350 7      3113.319         969.000 0       -19.700 0        83.000 0
D02  -3599435.993    -4077076.981   120137929.68012 120137677.85612      -128.150 7
         -116.600 7      3113.319        1010.000 0         1.000 0        66.000 0
D03  -2204389.106     -434379.997   130619768.44913 130619051.94713      -129.900 7
         -121.850 7      3113.319        1003.652 1         5.174 1        76.087 1

I know nothing about RINEX files, so I was hoping to use your project to start learning how to parse this data. When I try to load the RINEX data, rinex-cli throws the following exception:

╰$ RUST_BACKTRACE=1 rinex-cli -f data/h2arx20255.001.rinex
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: TypeError(UnknownType("O"))', /home/chris/.cargo/registry/src/github.com-1ecc6299db9ec823/rinex-0.8.0/src/lib.rs:473:14
stack backtrace:
   0: rust_begin_unwind
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/std/src/panicking.rs:575:5
   1: core::panicking::panic_fmt
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/core/src/panicking.rs:64:14
   2: core::result::unwrap_failed
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/core/src/result.rs:1791:5
   3: rinex::Rinex::from_file
   4: rinex_cli::main
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

I'm guessing that's because DORIS data isn't supported, but I'm not sure. If it isn't supported, that's OK, I can maybe write an extremely basic parser that only extracts the range and doppler data.

h2arx20255.001.rinex.zip

Thanks!

[larsnaesbye]

Hi Christopher.

Yes, the D type for DORIS isn't supported. I had to look it up to be honest.

Is it something you think would have a somewhat wide appeal?

[ChristopherRabotin]

Hi Lars, I have no idea. I had never heard of it either until a few hours ago when "Claude" (by Anthropic, https://www.anthropic.com/product) recommended I use DORIS data for calibration of ground station measurement noise. It seems like I can "get away" with a very simple parser because I am only looking for the range and doppler data from some tracking passes. I'll likely implement this in Python to make it quick and dirty. If that works, I can try adding it to this crate in Rust, after I learn about all the features. Best regards, Chris

…

On Tue, Apr 11, 2023, 12:32 Lars Næsbye Christensen < ***@***.***> wrote: Hi Christopher. Yes, the D type for DORIS isn't supported. I had to look it up <https://ids-doris.org/analysis-coordination/about-doris-rinex-format.html> to be honest. Is it something you think would have a somewhat wide appeal? — Reply to this email directly, view it on GitHub <#77 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABEZV2H34AL6P3PCXWJNXXTXAWPSZANCNFSM6AAAAAAW2TA3WQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[larsnaesbye]

Hi @ChristopherRabotin ,

Since DORIS isn't supported in many other tools either as far as I know it might come in handy, so that would be appreciated, I guess. @gwbres, what do you think about it?

[ChristopherRabotin]

In case this is something that you want to implement, I've attached a ZIP with some documentation from the ftp server. It's just easier to have here instead of asking folks to connect to the FTP.

DORIS Docs.zip

For reproduction, here's the connection log:

ftp> open
(to) ftp.ids-doris.org
Connected to ftpsedr.cls.fr.
220 ::ffff:91.207.20.75 FTP server ready
Name (ftp.ids-doris.org:chris): anonymous
331 Anonymous login ok, send your complete email address as your password
Password: 
230 Anonymous access granted, restrictions apply
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> cd pub
250 CWD command successful
ftp> cd ids
250 CWD command successful
ftp> cd data
250 CWD command successful
ftp> ls
229 Entering Extended Passive Mode (|||51426|)
150 Opening ASCII mode data connection for file list
drwxr-xr-x   2 261      ids          4096 Nov  7  2017 archives
-rw-r--r--   1 261      ids        108579 May 13  2009 detailed_POD_configuration_200801.pdf
lrwxrwxrwx   1 261      ids            37 Oct 13  2014 detailed_POD_configuration.pdf -> detailed_POD_configuration_200801.pdf
-rw-r--r--   1 261      ids          4002 Dec 11  2002 doris10.fmt
-rw-r--r--   1 261      ids          6121 Dec 11  2002 doris21.fmt
-rw-r--r--   1 261      ids          7242 Jun 23  2008 doris22.fmt
-rw-r--r--   1 261      ids        119738 Mar 25  2015 DORIS_models&solutions_v1.0.pdf
-rw-r--r--   1 261      ids        340244 Apr  1  2015 DORIS_RINEX_implementation_in_GINS.v2.0.pdf
-rw-r--r--   1 261      ids         46592 Nov 19  2014 DORIS_RINEX_subroutines_from_GINS.tar
drwxr-xr-x   2 261      ids            54 Jun  3  2008 for_tests_of_RINEX_DORIS_format
drwxr-xr-x   2 261      ids           174 Nov  3  2009 for_tests_RINEX_PANDOR_time_tagging
-rw-r--r--   1 261      ids        235455 Jan  8  2015 IntroductionToDORISRINEX.pdf
-rw-r--r--   1 261      ids         43569 May 11  2012 POD_configuration_GDRD.pdf
-rw-r--r--   1 261      ids         49433 May 29  2015 POD_configuration_GDRE.pdf
-rw-r--r--   1 261      ids         10166 May 13  2009 POD_configuration_Jason_Envisat_GDRB_GDRC.pdf
-rw-r--r--   1 261      ids         59320 Sep 11  2018 POD_configuration_POEF.pdf
-rw-r--r--   1 261      ids          2079 Nov  7  2017 README.txt
-rw-r--r--   1 261      ids          9550 Apr  2  2015 RecommendationsAboutDORISRinexDataProcessing_20150315.pdf
-rw-r--r--   1 261      ids        175826 Oct 10  2015 RINEX_DORIS.pdf
-rw-r--r--   1 261      ids         88915 May  7  2013 SOD_processing.pdf
226 Transfer complete
ftp>

[ChristopherRabotin]

This document is the most relevant on how to parse the data: IntroductionToDORISRINEX.pdf

[gwbres]

Hello @ChristopherRabotin, @larsnaesbye

Sorry I missed out on this discussion.

The D record type will most likely cause a panic.

Fields like this from the Header section would get dropped since we don't know how to interpret them :

    52                                                      # OF STATIONS       
D01  SYQB SYOWA                         66006S005  3   0    STATION REFERENCE   
D02  CRQC CROZET                        91301S004  4   0    STATION REFERENCE   
D03  MAUB MARION ISLAND                 30313S005  3   0    STATION REFERENCE   
D04  ADHC TERRE ADELIE                  91501S005  3   0    STATION REFERENCE   

And the Epoch content will most likely panic

                                                            END OF HEADER       
2020 09 11 00 00 32.759949800  0  3       -3.911558370 0 
D01  -1405364.026     -276928.863   109246558.05211 109246808.40511      -122.200 7

because D01 is not a ""standard"" satellite vehicle ID.

I'll run through the documentation you posted and will get back.

[gwbres]

@ChristopherRabotin, @larsnaesbye,

I initiated a doris branch with the basis and will continue to work on this topic.
I downloaded a couple of DORIS RINEX from CDDIS to get started. Although I don't understand the other DORIS products (EOP, GEOC etc..), DORIS RINEX is pretty straightforward.

@ChristopherRabotin, I presume your work will have you interested in EOP Earth Orientation Parameters for accurate geodetic frame knwoledge. It seems that several formats answer that need (I am far from familiar with this topic). Note that RINEX Navigation V4 (latest revision), that we correctly support, is one of those (EOP broadcasted models). I have never used them nor do I truly understand what they represent. My applications only use the Ionosphere models (ION broadcasted models).

[ChristopherRabotin]

Great to see that you've started the work on the DORIS format. Back when I opened this discussion, I made a very basic DORIS parser in Python here: https://github.com/nyx-space/nyx/blob/ef28f76f5a1c41d5c730fe880438a456010e710c/python/nyx_space/utils/doris.py . Some of it good, some of it bad, and all of it ugly because it isn't tested.

You're right that I mostly use the Earth Orientation Parameters (EOP). I use them to rebuild the high precision Earth body fixed frame in ANISE, and the toolkit can automatically download the file from JPL (via this "MetaAlmanac" entry ) and rebuild a frame, which can then be used for translations and rotations.

[larsnaesbye]

Real nice. I just saw that the ESA Genesis mission might include DORIS :-)

[gwbres]

and the toolkit can automatically download the file from JPL

Is that the simplistic DORIS EOP text files that NASA/CDDIS proposes along DORIS RINEX and DORIS GEOC (other simplistic text file) we're talking about here ?

[ChristopherRabotin]

No these are SPICE BPC files, for Binary Planetary Constants. They store Chebyshev polynomials to interpolate the right ascension, declination, and "twist", and their respective rates, over time. It was a real pain to get these to work and be validated.

[gwbres]

@ChristopherRabotin,

I'm in the process of thinking how we should expose DORIS features in the CLI apps.

My first question is what do you think is best on how to deal with the (worldwide) stations described in a single file.
We will soon have the ability to retain only the stations we're interested in (preprocessing ops), to reduce data quantity.
But is - for example - your application requiring to perform the operation on all stations worldwide ? I presume that is rarely the case

My second question is, things get more complicated if we consider several DORIS files loaded at once.
Maybe - like in -p opmode and 1D positioning - we should have the first version panic if we detect more than one DORIS spacecraft and only allow analysis of a single spacecraft source at once. Otherwise, it requires the app to classify everything by spacecraft and not mix up analysis results. We will eventually get there, but it's easier to start simple

We could develop the very operation you are interested in, especially since we scale the pseudo range correctly (two extra digits in precision)

from ground stations using the HY-2A spacecraft

any reasons for this one in particular and not the others also part of the DORIS program ?

PS: next RINEX version will have the ability to classify data sources by Station ID (usually the DOMES ID number and the name of the agency) and GNSS receiver(s). This will enable the application to analyze complex datasets, perform differential positioning, etc.. The DORIS case is special in the sense it requires to classify by both the station and the spacecraft. For spacecraft classification, we have the COSPAR ID number.

[ChristopherRabotin]

I was only using DORIS to try to validate the computation of range and range rate in Nyx. But I've since validated it in ANISE indirectly, by checking the equations of range rate, and ensuring that the position vectors match SPICE to machine precision. As such, I don't have any direct need for Doris right now.

…

On Fri, May 10, 2024, 12:46 gwbres ***@***.***> wrote: @ChristopherRabotin <https://github.com/ChristopherRabotin>, I'm in the process of thinking how we should expose DORIS features in the CLI apps. My first question is what do you think is best on how to deal with the (worldwide) stations described in a single file. We will soon have the ability to retain only the stations we're interested in (preprocessing ops), to reduce data quantity. But is - for example - your application requiring to perform the operation on all stations worldwide ? I presume that is rarely the case We could develop the very operation you are particularly interested, we don't have issues with that, especially since we scale the pseudo range correctly (two extra digits in precision) from ground stations using the HY-2A spacecraft any reasons for this one in particular and not the others also part of the DORIS program ? — Reply to this email directly, view it on GitHub <#77 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABEZV2GAES7M52BNWU7T5ETZBT2XPAVCNFSM6AAAAAAW2TA3WSVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TGOBWHE2TQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>