gnatss.loaders module

gnatss.loaders module#

gnatss.loaders.get_atd_offsets(config: Configuration) → NDArray[Shape[3], Float] | None#

Retrieves the Antenna Transducer Offset values from configuration and turn them into a numpy array. Returns None if atd_offsets not present in configuration.

Parameters:

configConfiguration
The configuration object

Returns:

NDArray[Shape[“3”], Float] or None: Numpy array containing the forward, rightward and downward antenna transducer offsets. Return None if atd_offsets not present in configuration.

gnatss.loaders.load_configuration(config_yaml: str | None = None) → Configuration#: Loads configuration yaml file into a Config object to be used throughout the pre-processing

gnatss.loaders.load_csrs_positions(files: list[str], time_round: int = 6)#

Loads gps positions into a pandas dataframe from a list of files. This assumes input data in a .pos format from CSRS processing.

The .pos format features a header of variable length that must be skipped. Furthermore, positions are in lat/lon/ellp and time is in string format. Thus, these must be converted to XYZ and J2000 seconds.

Parameters:

fileslist of str: The list of path string to files to load
time_roundint: The precision value to round the time values

Returns:

pd.DataFrame: Pandas DataFrame containing all of the gps positions data. The numbers are column number. 1: Time unit seconds in J2000 epoch 2: Geocentric x in meters 3: Geocentric y in meters 4: Geocentric z in meters 5 - 7: Standard deviations std_x, std_y, std_z 8 - 16: Covariance matrix Vxyz

gnatss.loaders.load_deletions(config: Configuration, file_paths: list[str] | None = None, time_scale='tt', remove_outliers: bool = False) → DataFrame#

Loads the raw deletion text file into a pandas dataframe

Parameters:

file_pathslist[str]: Path to the deletion file to be loaded
configConfiguration: The configuration object
time_scalestr: The time scale of the datetime string input. Default is ‘tt’ for Terrestrial Time, Must be one of the following: (tai, tcb, tcg, tdb, tt, ut1, utc)

Returns:

pd.DataFrame: Deletion ranges data pandas dataframe

gnatss.loaders.load_dfo(config: Configuration, file_paths: list[str], time_round: int = 6)#

Loads Two-way travel time data from SV-3 DFOP00.raw data files. These files store data in a JSON format, which includes information on the interrogation and reply epochs of each ping.

Parameters:

configConfiguration: The configuration object
fileslist of str: The list of path string to files to load
time_roundint: The precision value to round the time values

Returns:

pd.DataFrame: Pandas DataFrame containing all of the raw data required for parsed pre-processing. The numbers are column number. 1: Time unit seconds in J2000 epoch 2: Transponder ID 3: Two-way Travel Time 4: Transmit Time

gnatss.loaders.load_gps_positions(files: list[str], time_round: int = 6)#

Loads gps positions into a pandas dataframe from a list of files. Usually named GPS_POS_FREED.

Parameters:

fileslist of str: The list of path string to files to load
time_roundint: The precision value to round the time values

Returns:

pd.DataFrame: Pandas DataFrame containing all of the gps positions data. The numbers are column number. 1: Time unit seconds in J2000 epoch 2: Geocentric x in meters 3: Geocentric y in meters 4: Geocentric z in meters 5 - 7: Standard deviations std_x, std_y, std_z

gnatss.loaders.load_gps_solutions(files: list[str], time_round: int = 6, from_legacy: bool = False) → DataFrame#

Loads gps solutions into a pandas dataframe from a list of files.

Parameters:

fileslist of str: The list of path string to files to load
time_roundint: The precision value to round the time values
from_legacy: bool: Defaults to False. If True, parses gps solutions legacy format files.

Returns:

pd.DataFrame: Pandas DataFrame containing all of the gps solutions data. The numbers are column number. 1: Time unit seconds in J2000 epoch 2: Geocentric x in meters 3: Geocentric y in meters 4: Geocentric z in meters 5 - 13: Covariance matrix (3x3) xx, xy, xz, yx, yy, yz, zx, zy, zz

gnatss.loaders.load_novatel(data_files: list[str]) → DataFrame#

Read from Novatel Level-1 data files and return its dataframe representation. Link to data format specifications: INSPVAA: https://docs.novatel.com/OEM7/Content/SPAN_Logs/INSPVA.htm?tocpath=Commands%20%2526%20Logs%7CLogs%7CSPAN%20Logs%7C_____22

Parameters:

data_files: list pf str: Paths to the Novatel Level-1 data files to be loaded

Returns:

pd.DataFrame: Parsed Novatel Level-1 data in a pandas dataframe

gnatss.loaders.load_novatel_std(data_files: list[str]) → DataFrame#

Read from Novatel Level-1 data files and return its dataframe representation. Link to data format specifications: INSSTDEVA: https://docs.novatel.com/OEM7/Content/SPAN_Logs/INSSTDEV.htm?tocpath=Commands%20%2526%20Logs%7CLogs%7CSPAN%20Logs%7C_____30

Parameters:

data_files: list pf str: Paths to the Novatel Level-1 data files to be loaded

Returns:

pd.DataFrame: Parsed Novatel Level-1 data in a pandas dataframe

gnatss.loaders.load_pride_positions(files: list[str], time_round: int = 6)#

Loads gps positions into a pandas dataframe from a list of files. This assumes input data in a kin file format generated by PRIDE PPP-AR processing.

The kin file format features a header of variable length that must be skipped. Positions are logged in XYZ so may be directly imported. However, time records are logged in two columns for modified julian day (MJD) and second of day (Sod), so must be converted to J2000 seconds.

In addition, PRIDE PPP-AR does not calculate position uncertainties for kinematic processing. As a temporary fix, we assume a constant position uncertainty of 0.01 m in every component.

Parameters:

fileslist of str: The list of path string to files to load
time_roundint: The precision value to round the time values

Returns:

pd.DataFrame: Pandas DataFrame containing all of the gps positions data. The numbers are column number. 1: Time unit seconds in J2000 epoch 2: Geocentric x in meters 3: Geocentric y in meters 4: Geocentric z in meters 5 - 7: Standard deviations std_x, std_y, std_z

gnatss.loaders.load_quality_control(qc_files: list[str], time_scale='tt') → DataFrame#

Loads the quality controls csv files into a pandas dataframe

Parameters:

qc_fileslist[str]: Path to the quality control files to be loaded
time_scalestr: The time scale of the datetime string input. Default is ‘tt’ for Terrestrial Time, Must be one of the following: (tai, tcb, tcg, tdb, tt, ut1, utc)

Returns:

pd.DataFrame: Deletion ranges data pandas dataframe

gnatss.loaders.load_roll_pitch_heading(files: list[str]) → DataFrame#

Loads roll pitch heading data into a pandas dataframe from a list of files.

Parameters:

fileslist[str]: The list of path string to files to load.

Returns:

pd.DataFrame: Pandas DataFrame containing all of the roll pitch heading data. Expected columns will have ‘time’ and the ‘roll’, ‘pitch’, ‘heading’ values. Union roll, pitch, heading covariance columns will be dropped from the dataframe. Return empty Dataframe if files is empty string.

gnatss.loaders.load_sound_speed(sv_files: list[str]) → DataFrame#

Loads sound speed file data into pandas dataframe

Parameters:

sv_fileslist[str]: The list of path string to the sound speed files to be loaded

Returns:

pd.DataFrame: Sound speed profile pandas dataframe

gnatss.loaders.load_sv3_targz(config: Configuration, file_paths: list[str], time_round: int = 6)#

Loads SV-3 parsed raw data into a pandas dataframe from a list of files. These files should be zipped binary files containing .pin files. Each .pin file contains a json structure with an interrogation event and events for each reply. These events should contain the timing and position data at ping transmit and reply, but the specific structure has been modified over time. This function also aims to accommodate these variations.

Parameters:

configConfiguration: The configuration object
fileslist of str: The list of path string to files to load
time_roundint: The precision value to round the time values

Returns:

pd.DataFrame: Pandas DataFrame containing all of the raw data required for parsed pre-processing. The numbers are column number. 1: Transponder ID 2: Time unit seconds in J2000 epoch 3: Transmit Time 4: Two-way Travel Time 5-7: Antenna positions (Geocentric x,y,z in meters) 8: Roll 9: Pitch 10: Heading 11-13: Antenna positions at Transmit (Geocentric x,y,z in meters)

gnatss.loaders.load_travel_times(files: list[str], transponder_ids: list[str], is_j2k: bool = False, time_scale: str = 'tt') → DataFrame#

Loads travel times data into a pandas dataframe from a list of files.

Time conversions from date and time string will occur under the hood using astropy package. The final time with epoch J2000 will be returned in the pandas dataframe.

Parameters:

fileslist: The list of path string to files to load
transponder_idslist: A list of transponder ids corresponding to the transponder data within the travel times. Note that ids order must match to travel times column order for transponders
is_j2kbool: A flag to signify to expect j2000 time column only rather than date and time. No conversion will be performed if this is the case.
time_scalestr: The time scale of the datetime string input. Default is ‘tt’ for Terrestrial Time, Must be one of the following: (tai, tcb, tcg, tdb, tt, ut1, utc)

Returns:

pd.DataFrame: Pandas DataFrame containing all of the travel times data. Expected columns: ‘time’, ‘transponder-x1’, …, ‘transponder-xid’

Notes

The input travel time data is assumed to follow the structure below.

Structure 1:

Date, Time, TT Transponder 1, TT Transponder 2, …, TT Transponder N

Date is a date string. e.g. ‘27-JUL-22’
Time is a time string. e.g. ‘17:37:45.00’
TT is the 2 way travel times in microseconds (µs). e.g. 2263261

Structure 2:

Time, TT Transponder 1, TT Transponder 2, …, TT Transponder N

Time is a J2000 epoch time float (seconds since Jan 1, 2000 12:00 UTC). e.g. ‘712215465.000’
TT is the 2 way travel times in microseconds (µs). e.g. 2263261

These files are often called pxp_tt or pxp_tt_j2k.

Additionally, there’s an assumption that wave glider delays have been removed from the data.

gnatss.loaders module

Contents

gnatss.loaders module#