evaluate

Some tools to help evaluate and plot performance, used in optimization and in jupyter notebooks

pynumdiff.utils.evaluate.error_correlation(u, v, padding=0)

Calculate the error correlation (pearsons correlation coefficient) between the first vector and the difference between the two vectors.

Parameters:

  • u (np.array[float]) – e.g. true value of dxdt, if known, optional

  • v (np.array[float]) – e.g. estimated dxdt_hat

  • padding (int) – number of snapshots on either side of the array to ignore when calculating the metric. If 'auto', defaults to 2.5% of the size of inputs

Returns:

(float) – r-squared correlation coefficient

pynumdiff.utils.evaluate.plot(x, dt, x_hat, dxdt_hat, x_truth, dxdt_truth, xlim=None, show_error=True, markersize=5)

Make comparison plots of ‘x (blue) vs x_truth (black) vs x_hat (red)’ and ‘dxdt_truth (black) vs dxdt_hat (red)’

Parameters:

  • x (np.array[float]) – array of noisy data

  • dt (float) – a float number representing the step size

  • x_hat (np.array[float]) – array of smoothed estimation of x

  • dxdt_hat (np.array[float]) – array of estimated derivative

  • x_truth (np.array[float]) – array of noise-free time series

  • dxdt_truth (np.array[float]) – array of true derivative

  • xlim (list[int]) – a list specifying range of x

  • show_error (bool) – whether to show the rmse

  • markersize (int) – marker size of noisy observations

Returns:

(tuple) – figure and axes

pynumdiff.utils.evaluate.plot_comparison(dt, dxdt_truth, dxdt_hat1, title1, dxdt_hat2, title2, dxdt_hat3, title3)

This is intended to show method performances with different choices of parameter

pynumdiff.utils.evaluate.rmse(u, v, padding=0)

True RMSE between vectors

Parameters:

  • u (np.array[float]) – e.g. known true derivative

  • v (np.array[float]) – e.g. estimated derivative

  • padding (int) – number of snapshots on either side of the array to ignore when calculating the metric. If 'auto', defaults to 2.5% of the size of inputs

Returns:

true_rmse (float) – RMS error between dxdt_hat and dxdt_truth, returns None if dxdt_hat is None

pynumdiff.utils.evaluate.robust_rme(u, v, padding=0, M=6)

Robustified/Huberized Root Mean Error metric, used to determine fit between smooth estimate and data. Equals np.linalg.norm(u[s] - v[s]) / np.sqrt(N) if M=float(‘inf’), and dang close for M=6 or even 2.

Parameters:

  • u (np.array[float]) – e.g. noisy data

  • v (np.array[float]) – e.g. estimated smoothed signal, reconstructed from derivative

  • padding (int) – number of snapshots on either side of the array to ignore when calculating the metric. If 'auto', defaults to 2.5% of the size of inputs

  • M (float) – Huber loss parameter in units of ~1.4*mean absolute deviation of population of residual errors, intended to approximate standard deviation robustly.

Returns:

(float) – Robust root mean error between u and v

pynumdiff.utils.evaluate.total_variation(x, padding=0)

Calculate the total variation of an array. Used by optimizer.

Parameters:

  • x (np.array[float]) – data

  • padding (int) – number of snapshots on either side of the array to ignore when calculating the metric. If 'auto', defaults to 2.5% of the size of x

Returns:

(float) – total variation