Validation

This module contains the functions for calculating parameter based validation statistics for a set of matchup records.

Dependencies [External]:

os, scipy, numpy, copy

Dependencies [Internal]:

waveval.ModelData, waveval.ObsData, waveval.MatchUpDatabase, waveval.Geometry

Validation.bias(model, obs)

Validation Metric: Bias

Parameters:

• model (numpy.array, float) – model data time series

• obs (numpy.array, float) – observation time series mapped onto model time stamps

Returns:

bias (b)

Return type:

float

Validation.calculate_circular_metrics(model, obs, nsampl, degrees=True)

Calculate circlar validation metrics for a pair of modelled and observation time series

Validation.calculate_metrics(model, obs, nsampl)

Calculate standard validation metrics for a pair of modelled and observation time series

Validation.circ_corr(theta1, theta2, degrees=True)

Validation Metric (Circular): Correlation Coefficient

Parameters:

• theta1 (numpy.array,float) – vector of angular data (modelled)

• theta1 – vector of angualr data (observed)

• degrees (bool) – angular data units flag. The default is degrees = True

Returns:

circular correlation coefficient (c_corr)

Return type:

float

Validation.circ_mean(theta, degrees=True)

Calculate mean of angular data for range (-180 to 180)

Parameters:

• theta (numpy.array, float) – vector of angular values.

• degrees (bool, optional) – angular data units flag. The default is degrees = True.

Returns:

mean_theta – mean of the angular data in same units as passed in.

Return type:

float

Validation.circ_mean_abs_error(theta1, theta2, degrees=True)

Validation Metric (Circular): Mean absolute error

Parameters:

• theta1 (numpy.array,float) – vector of angular data (modelled)

• theta1 – vector of angualr data (observed)

• degrees (bool) – angular data units flag. The default is degrees = True

Returns:

circular mean absolute error (mae)

Return type:

float

Validation.circ_mean_bias(theta1, theta2, degrees=True)

Validation Metric (Circular): Mean Bias

Parameters:

• theta1 (numpy.array,float) – vector of angular data (modelled)

• theta1 – vector of angualr data (observed)

• degrees (bool) – angular data units flag. The default is degrees = True

Returns:

circular mean bias (mb)

Return type:

float

Validation.circ_mod_pi(theta)

Fix angular range in radians to -pi <= theta <= pi

Parameters:

theta (numpy.array, float) – vector of angular values in radians

Returns:

theta in range -pi to pi

Return type:

numpy.array, float

Validation.circ_norm_ma_error(theta1, theta2, degrees=True)

Validation Metric (Circular): Normalised mean absolute error

Parameters:

• theta1 (numpy.array,float) – vector of angular data (modelled)

• theta1 – vector of angualr data (observed)

• degrees (bool) – angular data units flag. The default is degrees = True

Returns:

circular normalised mean absolute error (mae) as a percentage.

Return type:

float

Validation.circ_norm_mean_bias(theta1, theta2, degrees=True)

Validation Metric (Circular): Normalised mean Bias

Parameters:

• theta1 (numpy.array,float) – vector of angular data (modelled)

• theta1 – vector of angualr data (observed)

• degrees (bool) – angular data units flag. The default is degrees = True

Returns:

circular normalised mean bias (nmb) as a percentage

Return type:

float

Validation.circ_norm_rmse(theta1, theta2, degrees=True)

Validation Metric (Circular): Normalised root-mean-square error

Parameters:

• theta1 (numpy.array,float) – vector of angular data (modelled)

• theta1 – vector of angualr data (observed)

• degrees (bool) – angular data units flag. The default is degrees = True

Returns:

circular normalised RMS erro (circ)nrmse as a percentage

Return type:

float

Validation.circ_rms_error(theta1, theta2, degrees=True)

Validation Metric (Circular): Root-mean-square error

Parameters:

• theta1 (numpy.array,float) – vector of angular data (modelled)

• theta1 – vector of angualr data (observed)

• degrees (bool) – angular data units flag. The default is degrees = True

Returns:

circular RMS error (circ_rmse)

Return type:

float

Validation.circ_scatter_index(theta1, theta2, degrees=True)

Validation Metric (Circular): Scatter index

Parameters:

• theta1 (numpy.array,float) – vector of angular data (modelled)

• theta1 – vector of angualr data (observed)

• degrees (bool) – angular data units flag. The default is degrees = True

Returns:

circular scatter index (circ_si)

Return type:

float

Validation.circ_std(theta, degrees=True)

Calculate standard deviation of angular data for range (-180 to 180)

Parameters:

• theta (numpy.array, float) – vector of angular values.

• degrees (bool, optional) – angular data units flag. The default is degrees = True.

Returns:

std_theta – stanadrd deviation of the angular data in same units as passed in.

Return type:

float

Validation.corr_coef(model, obs)

Validation Metric: Correlation Coefficient

Parameters:

• model (numpy.array, float) – model data time series

• floatobs (numpy.array,) – observation time series mapped onto model time stamps

Returns:

correlation coefficient (R)

Return type:

float

Validation.display_tabulated_results(store, mVarName)

Display validation results as a table on the standard output

Validation.extract_model(rscdfile, mVarName: str)

Extract model output for a specified variable

Validation.extract_obs(buoyfile, buoyDataFmt: str, varOptions: list)

Extract in situ buoy data for a specified data foramt and variable

Validation.getSurfaceIndx(record, dataFmt)

Get index of surface layer in depth data

Validation.getVarName(record, Options: list)

Get variable names based on processsing options list

Validation.initialize_results()

Initialise a dictionary for collecting validation results

Validation.load_binary_results(platform, mVarName, datapath)

load binary results from a matlab file

Validation.mean_absolute_error(model, obs)

Validation Metric: Mean absolute error

Parameters:

• model (numpy.array, float) – model data time series

• obs (numpy.array, float) – observation time series mapped onto model time stamps

Returns:

mae

Return type:

float

Validation.mean_bias(model, obs)

Validation Metric: Mean Bias

Parameters:

• model (numpy.array, float) – model data time series

• obs (numpy.array, float) – observation time series mapped onto model time stamps

Returns:

mb

Return type:

float

Validation.model_efficiency(model, obs)

Validation Metric: Model efficiency

Parameters:

• model (numpy.array, float) – model data time series

• obs (numpy.array, float) – observation time series mapped onto model time stamps

Returns:

mef

Return type:

float

Validation.norm_ma_error(model, obs)

Validation Metric: Normalised mean absolute error

Parameters:

• model (numpy.array, float) – model data time series

• obs (numpy.array, float) – observation time series mapped onto model time stamps

Returns:

nmae

Return type:

float

Validation.norm_mean_bias(model, obs)

Validation Metric: Normalised mean bias

Parameters:

• model (numpy.array, float) – model data time series

• obs (numpy.array, float) – observation time series mapped onto model time stamps

Returns:

nmb

Return type:

float

Validation.norm_rmse(model, obs)

Validation Metric: Normalised root-mean-square error

Parameters:

• model (numpy.array, float) – model data time series

• obs (numpy.array, float) – observation time series mapped onto model time stamps

Returns:

nrmse

Return type:

float

Validation.process_record(rec, buoyFmt: str, platform: str, mVarName: str, varOptions: list, pVarName: str, results_dir: str, angularData=False, plot_results=False)

Process a matched model/buoy data record

Validation.reliability_index(model, obs)

Validation Metric: Reliability index

Parameters:

• model (numpy.array, float) – model data time series

• obs (numpy.array, float) – observation time series mapped onto model time stamps

Returns:

ri

Return type:

float

Validation.results_record(modelFileName, obsFileName, obsVarName, year, month, metrics)

Create results recod from calculated validation metrics

Validation.rms_error(model, obs)

Validation Metric: Root-mean-square error

Parameters:

• model (numpy.array, float) – model data time series

• obs (numpy.array, float) – observation time series mapped onto model time stamps

Returns:

rmse

Return type:

float

Validation.save_cleaned_ts_data(t, rscd, buoy, platform, mVarName, outpath, year, month)

Save cleaned matched model and buoy timeseries as a matlab *.mat binary file

Validation.save_tabulated_results(store, platform, mVarName, outpath)

Save tabulated validation results as an ASCII file

Validation.scatter_index(model, obs)

Validation Metric: Scatter index

Parameters:

• model (numpy.array, float) – model data time series

• obs (numpy.array, float) – observation time series mapped onto model time stamps

Returns:

si

Return type:

float

Validation.skill_score(model, obs)

Validation Metric: Skill score

Parameters:

• model (numpy.array, float) – model data time series

• obs (numpy.array, float) – observation time series mapped onto model time stamps

Returns:

ss

Return type:

float

Validation.store_valid_results(store, results)

Store validation results record in the results dictionary

Validation.time_interpolate_obs(t_obs, t_var, t_model)

Interpolation of observation data onto the model output timestamps

Validation.validate_records(records, dataFmt: str, platform: str, mVarName: str, varOptions: list, pVarName, results_dir, plot_results=False)

Process a set of matched model/buoy data records