Advanced Processing Flow

PDF Download

Summary

This document describes how to build an advanced processing flow, capable of cleaning out-of-spec data according to the requirements of typical project QAPPs. The following processing steps will be configured to run in the P Flow 1 processing flow.

Example Advanced Processing Flow

1. Interp: IMU: Interpolation of IMU channels outside of the range -5 to 5 radians over a maximum range of 3 meters.
2. Interp: Fix: Interpolation of GPS channels with fix quality outside of the range 4.0 to 4.1 over a maximum range of 3 meters.
3. Re-Calc GPS Heading: 2.0m: which fixes heading flips when the platform has little or no forward motion.
4. Raw to Merged (Required): Merge the raw data to the time base of the EM sensors.
5. Snip: Fix: Snip all data outside of the fix quality range of 4.0 to 4.1. This removes all data that could not be interpolated in step 1, due to the maximum range of 3 meters constraint.
6. Snip: Tx current: Snip all data below a transmitter current MQO threshold of 15.0 Amps.
7. Snip: Velocity: Snip all data with a velocity of 0.2m/s.
8. -a Detrend: 10m (Required): Background detrend the data using a 10m window and data cutoff values of 30% and 70%.

NOTE: the videos below show the configuration needed to setup each function. When adding a new configuration you will need to click the new button to set a new configuration and you will need to create a new mask.

Interp: IMU

Interp: Fix

Re-Calc GPS Heading: 2.0m

Raw to Merged

No configuration needed, this function merges the raw data to the time base of the EM sensor data.

Snip: Fix

Snip: Tx current

Snip: Velocity

This function will remove data where the sensor was not moving very fast, i.e. data at the start and end of lines.

-a Detrend: 10m

A standard de-median filter for removing background trends.

User Interface Descriptions and Additional Details

Interp

An interpolation cleaner. Capable of replacing out of range values under user specified conditions.

1. Cfg: Currently selected configuration for the Interp tool. It will always match the Name field.
2. Name: This field is editable only when in edit mode and will rename the Cfg.
3. Type: Specify the type of sensor to perform the Interp process on.
4. In: Specify a named data state to operate on. Typically we work with either the UnFilt or -a data states. The Interp cleaner is normally only applied to the UnFilt state.
5. Default Mask: Specify which channels within the Type sensor to operate on.
6. Max interp dist: Specify the maximum distance to interpolate out of range values. Values that are out of range beyond this distance are not interpolated and are left in the data.
7. Min/Max accept value: Together these values define the range of acceptable values in the data. Values outside of this range are interpolated up-to the Max interp dist.
8. Remove Spikes / Max jump: Enabling the Remove Spikes allows this function to detect jumps or changes in the sequential data of a channel beyond the Max jump threshold.
9. interp all members when bad data found: This setting will apply the interpolation to all channels within the Default Mask. For example, one could detect jumps in the Roll channel, but then apply interpolation to the Pitch, Roll and Yaw channels. The idea is that detection of an outlier in one channel may indicate suspect data quality in another, even though an outlier was not detect.

Re-Calc GPS Heading

This function will re-calculate the heading channel. The heading channel is generated from calculations on the POS data. During acquisition the heading is generated in real-time by looking behind in a time window. In post processing we can recompute the heading using a centered window, where we use data both behind and ahead in time.

1. Cfg: Currently selected configuration for the Interp tool. It will always match the Name field.
2. Name: This field is editable only when in edit mode and will rename the Cfg.
3. Wnd: Size of the window to perform the heading calculation from.

Raw to Merged

There are no settings to configure for this required step. This process will merge the time base of all individual sensors (POS, ATT and EM).

Snip

A snip cleaner. Capable of removing data outside of user defined thresholds. This cleaner has been purposed for advanced use to deal with data issues that can occur after prolong abuse of hardware in difficult field conditions. Under conditions of high wind and loose connectors, late time decay spikes or abnormal decay patterns can be identified and removed from the data.

1. Cfg: Currently selected configuration for the function. It will always match the Name field.
2. Name: This field is editable only when in edit mode and will rename the Cfg.
3. Type: Specify the type of sensor to perform the process on.
4. Default Mask: Specify which channels within the Type sensor to operate on.
5. Min accept value / Max accept value: These values define the range of acceptable data, with values outside to be snipped as outliers.
6. Snip all sesnors: When selected all sensors for the Tx sounding event will be snipped. For example, we may wish to snip data from a single Rx cube or alternatively snip data from all Rx’s when an outlier is detected.
   
7. Use slope: When selected all the Min / Max accept values will apply to the value of the slope of the data. Note: If the Type is Rx, the user can also check Apply to decays.
   
8. Slope wnd: When selected all the Min / Max accept values will apply to the value of the slope of the data. The length (in # of stations or time gates) of a moving window that will calculate the slope.
   
9. snip zero slope: This gives the user the option to ignore zero slopes when detecting bad decays.
   
10. snip zero slope: This gives the user the option to ignore zero slopes when detecting bad decays.
    
11. Apply to decays: When enabled and the Type is set to Rx, we can snip decays where the slope within a decay window is outside of the accept values.
    
12. Remove spikes: Remove spikes that exceed the Max jump value.
13. Max jump: Threshold for the Remove spikes.

Detrend

This function applies a standard de-median filter to the Rx type data. A background is estimated by computing the median over a time window, while rejecting data outside of the upper/lower cutoffs. The user has the option to process all lines or a subset of lines. It is recommended when filtering a subset of lines to create an additional detrend function in the flow to handle other lines.

1. Cfg: Currently selected configuration for the function. It will always match the Name field.
2. Name: This field is editable only when in edit mode and will rename the Cfg.
3. Type: Specify the type of sensor to perform the process on.
4. Default Mask: Specify which channels within the Type sensor to operate on.
5. In: Specify a named data state to operate on. Typically we work with either the UnFilt or -a data states. Typically this value is set to UnFilt for the detrend function.
6. Out: Specify a named data state to assign the result to. Typically we assign the result of the detrend filter to the -a data state.
7. Process all lines / Select lines: Configure this function to apply filtering to all lines or a subset of lines.
8. Filter Wnd: The size of window to compute the background from.
9. Upper CutOff: Upper percentile cutt-off to exclude data from being used to compute the median background for the window.
10. Lower CutOff: Lower percentile cutt-off to exclude data from being used to compute the median background for the window.
11. Recalculate cumulative distance: Option to re-calculate the cumulative distance. All Detrend functions have this defaulted to an ON (checked) state.