pylawr.transform.temporal#

pylawr.transform.temporal.Extrapolator([...])

This class can be used to extrapolate a field based on two previous fields using template matching.

pylawr.transform.temporal.NoiseRemover([...])

Noise filter that dynamically determines an appropriate spatially independent noise threshold to subtract from the reflectivity field.

pylawr.transform.temporal.Extrapolator#

class pylawr.transform.temporal.Extrapolator(cut_percentage=0.15, correlation_threshold=0.5, max_timediff=600.0)[source]#

Bases: Transformer, MemoryMixin

This class can be used to extrapolate a field based on two previous fields using template matching. The template matching finds similar areas between two fields. Based on the distance between similar pixels a vector of pixel movement between to time steps is calculated. With the vector the current field can be shifted to a field of a following time step.

vector#

fitted movement vector in meters per second

Type

tuple, float

time#

timestep of fitted vector

Type

numpy.datetime64

cut_percentage#

Offset defines cut of range for old data.

Type

float

correlation_threshold#

Minimal threshold of maximal correlation of match template between data arrays. Is the correlation lower than threshold, the vector of movement is set to zero.

Type

float

max_timediff#

Maximal difference between time steps until warning.

Type

int

_abc_impl = <_abc._abc_data object>#
static _check_grids(grid_now, grid_pre)[source]#

Checks if the grids of the arrays are equal and child of pylawr.grid.cartesian.CartesianGrid

Parameters

  • grid_now (child of BaseGrid) – grid of data of actual timestep

  • grid_pre (child of BaseGrid) – grid of data of previous timestep

_determine_timedelta(time, time_pre)[source]#

Get the timedelta between two data arrays.

Parameters

  • time (numpy.datetime64) – actual timestep.

  • time_pre (numpy.datetime64) – previous timestep.

Returns

timedelta – Timedelta in seconds.

Return type

numpy.timedelat64 (seconds)

calc_match_matrix(array, array_pre)[source]#

Calculates the correlation matrix based on match template between two arrays

Parameters
Returns

Correlation matrix

Return type

numpy.ndarray (float)

fit(array, array_pre, grid=None, *args, **kwargs)[source]#

Fits a vector of movement between to arrays, of e.g. reflectivity, based on match template.

Parameters
property fitted#
property time#
to_xarray()[source]#

Serialize this filter’s parameters to an xarray.Dataset

Returns

the filter’s parameters as dataset

Return type

xarray.Dataset

transform(array, grid=None, time=None, *args, **kwargs)[source]#

Extrapolate the given given array with a cartesian grid according to the fitted movement vector.

Parameters
Returns

The transformed array

Return type

xarray.DataArray

property vector#

pylawr.transform.temporal.NoiseRemover#

class pylawr.transform.temporal.NoiseRemover(noise_percentile=10, max_noise_percentile_tendency=0.006666666666666667, max_time_constant_noiselevel=10800, noiselevel_multiplier=1.03, remembrance_time=300, noiselevel=8e-07)[source]#

Bases: Transformer, MemoryMixin

Noise filter that dynamically determines an appropriate spatially independent noise threshold to subtract from the reflectivity field.

Assumptions:

  • reflectivity data is the raw measurement, i.e. no range dependency correction etc. has happened yet

noise_percentile#

percentile of all sorted values to use for determination of a noise level. Defaults to 10 which means \(10\%\).

Type

float

remembrance_time#

time in seconds to do a moving average of older values. Defaults to 300 seconds, 5 minutes.

Type

float

max_noise_percentile_tendency#

maximum relative tendency per time of the actual noise_percentile value. Defaults to \(\frac{20\%}{30s}\). If a higher relative tendency occurs, the previous noiselevel will be used for filtering.

Type

float

max_time_constant_noiselevel#

maximum time in seconds where the noiselevel can be constant before it is reset to the default. Defaults to three hours.

Type

float

noiselevel_constant_since#

the time where noiselevel last changed

Type

numpy.datetime64

noiselevel_multiplier#

the noiselevel is multiplied with this constant value. Defaults to 1.03.

Type

float

times#

recorded threshold times

Type

numpy.ndarray(numpy.datetime64)

thresholds#

recorded thresholds

Type

numpy.ndarray

noiselevel#

current noise level

Type

float

_abc_impl = <_abc._abc_data object>#
_determine_noiselevel(time_obj)[source]#
_determine_noiselevel_from_array(array)[source]#

Calculate the noise level from given array. At the moment, the noise level is the noise_percentile-th percentile of the given array.

Parameters

array (xarray.DataArray) – This array is used to calculate the noise level.

Returns

noiselevel – The calculated noise level, based on the given array.

Return type

float

_get_last_changed_time()[source]#

Get the index, where the noise levels changed the last time.

Returns

time_obj – The time of the last changed value within the thresholds. If the index is np.nan, then no noise level was saved or the noise level was saved without setting the appropriate time.

Return type

numpy.datetime64 or numpy.nan

static _get_mean_noise(noiselevels)[source]#

Calculate the mean noise level from given noise levels. At the moment this is only the nan median of the noise levels. In the future it will be possible to calculate the mean noise with a weighted mean.

Parameters

noiselevels – The mean from these noise levels is calculated.

Returns

noise_level – The calculated noise level.

Return type

float

_get_old_noise_level(time_obj)[source]#

Get the noise level from the old thresholds, which is the nearest to the given time object.

Parameters

time_obj (numpy.datetime64) – The nearest noise level to this time object will be returned.

Returns

old_noiselevel – The old_noiselevel is either determined from nearest old noise level or if not fitted it is the default value.

Return type

float

_get_remembrance_interval(time_obj)[source]#

Calculate the remembrance interval based on given time object and set remembrance time.

Parameters

time_obj (datetime-like) – The remembrance interval is calculated based on this time object.

Returns

  • start_date (pandas.datetime) – The start date of the remembrance interval. This is calculated based on \(time_obj-remembrance_time\)

  • end_date (pandas.datetime) – The end date of the remembrance interval. Basically, this is the given time object as pandas-datetime.

_is_rainy_field(array)[source]#

Check if RadarField has more rain than a threshold. The threshold is calculated as \(100 - noise_percentile\).

Parameters

array (xarray.DataArray) – This array is used to check if the field is rainy or not. The array is converted into reflectivity and all reflectivity above 0 dBZ will be set to rain.

Returns

is_rainy – If the rain fraction is larger than the threshold, this will be True, else False.

Return type

bool

_prune_thresholds(time_obj)[source]#

Prune the thresholds based on the given time object and set remembrance time.

Parameters

time_obj (numpy.datetime64) – This time object will be used as base time for the pruning process.

_save_noiselevel(noiselevel, time_obj)[source]#

Save the given noiselevel and time object pair to the attributes of this filter.

Parameters

time_obj (numpy.datetime64) – The valid time of the given noise level.

fit(array, time_obj=None)[source]#

Given a reflectivity array, determine the noiselevel for use in transform

Parameters
property fitted#
get_noiselevel(time_obj=None)[source]#

Determine an appropriate noiselevel from the available data. If time is given, interpolate an appropriate noiselevel from thresholds. If that is impossible, use the value of noiselevel.

Parameters

time_obj (datetime.datetime or numpy.datetime64, optional) – The time used to determine the noiselevel. If None the value of the last`noiselevel` is returned.

Returns

the noiselevel to use

Return type

float

property noiselevel#
property noiselevel_constant_since#

Get the index, where the noise levels changed the last time.

Returns

time_obj – The time of the last changed value within the thresholds. If the index is np.nan, then no noise level was saved or the noise level was saved without setting the appropriate time.

Return type

numpy.datetime64 or numpy.nan

property thresholds#
property times#
to_xarray()[source]#

Convert parameters to a dataset for easy serialisation

Returns

parameters as dataset

Return type

xarray.Dataset

transform(array, time_obj=None, replace_negative=0.0, *args, **kwargs)[source]#

Subtract an appropriate noiselevel (get_noiselevel) from the given linear reflectivity array, optionally replacing (unphysical) negative results.

Parameters

  • array (array-like) – The linear reflectivity array to remove the noise level from

  • time_obj (datetime.datetime or numpy.datetime64, optional) – The time used to determine the noiselevel to use. Defaults to array.time if available. If unavailable, the value of noiselevel is used for transforming.

  • replace_negative (float, optional) – The value to replace negative values in the resulting array with. Defaults to \(0.\,\mathrm{mm^6m^{-3}}\). Set to None to prevent replacing.

Returns

The array with linear reflectivity and the noise level is removed.

Return type

array-like

property xr_thresholds#

Get the thresholds as xarray.DataArray.

Returns

thres – The thresholds as xarray.DataArray with times as axis.

Return type

xarray.DataArray