Possible EMOS amendments #11
Conversation
… for use in testing.
…nd longitude, so that the truth cubelist is mergeable.
…l standardisation.
…se of this distribution including to support the use of additional predictors for this distribution.
| predictor, | ||
| ).x.astype(np.float32) | ||
| ) | ||
| for index, (truth_slice, fv_slice) in enumerate(zip(truth.slices_over(sindex), forecast_var.slices_over(sindex))): |
There was a problem hiding this comment.
Preprocessing of arrays for additional predictors before passing to minimising,
| forecast_predictor_data = flatten_ignoring_masked_data( | ||
| forecast_predictor.data | ||
| ) | ||
| fp_data_list = [] |
There was a problem hiding this comment.
More preprocessing for additional predictors
| initial_guess[-2], | ||
| initial_guess[-1], | ||
| ) | ||
| a_b = np.array([a] + b.tolist(), dtype=np.float64) | ||
|
|
||
| if predictor.lower() == "mean": |
There was a problem hiding this comment.
Again additional predictors and more initiall guesses
| forecast_predictor_flattened = flatten_ignoring_masked_data( | ||
| forecast_predictor | ||
| forecast_predictor, preserve_leading_dimension=True |
| for (truths_slice, fp_slice) in zip( | ||
| truths.slices_over(index), forecast_predictor.slices_over(index) | ||
| ): | ||
| for truths_slice in truths.slices_over(index): |
There was a problem hiding this comment.
Similar to above, pre processing of additional predictors
| "beta coefficients in order to create a calibrated forecast.") | ||
| raise ValueError(msg) | ||
|
|
||
| if self.standardise_cubelist: |
There was a problem hiding this comment.
All the equations in this section to do with applying coefficients if the standardisation was used when calculating the coefficients are now correct to the best of my understanding.
| return std_forecast, std_truth, forecast_mean, forecast_sd, truth_mean, truth_sd | ||
|
|
||
|
|
||
| def standardise_forecasts(historic_forecasts, hf_coords=["realization", "time"]): |
There was a problem hiding this comment.
This function and the one below aren't currently used.
improver/calibration/utilities.py
Outdated
|
|
||
| if truths.coords("time", dim_coords=True): | ||
| truth_mean = truths.collapsed(truth_coords, nanmean) | ||
| else: |
There was a problem hiding this comment.
Why are these else statements needed?
There was a problem hiding this comment.
This is dealing with a spin up problem. It will not impact the testing we are currently doing as we should always have more than one truth. For now it is probably best to fail if this else statement is reached.
| coord_values={y_name: lambda cell: any(np.isclose(cell.point, truth_slice.coord(axis="y").points)), | ||
| x_name: lambda cell: any(np.isclose(cell.point, truth_slice.coord(axis="x").points))}) | ||
| new_fp_data = [] | ||
| for fp_cube in forecast_predictors: |
There was a problem hiding this comment.
I think I need some help explaining this
There was a problem hiding this comment.
This is for static predictors where they don't change with time. Give them a time coordinate so they match the shape of the other predictor.
There was a problem hiding this comment.
I've added a comment to indicate that this is to account for static predictors.
improver/calibration/utilities.py
Outdated
| @@ -215,13 +218,38 @@ def filter_non_matching_cubes( | |||
| if truth_slice: | |||
There was a problem hiding this comment.
| if truth_slice: | |
| if truth_slice and af_slices: | |
| matching_historic_forecasts.append(hf_slice) | |
| matching_truths.append(truth_slice) | |
| matching_additional_fields.extend(af_slices) | |
There was a problem hiding this comment.
I've modified this to avoid the possible issue of truths and historic forecasts being added potentially without a required additional predictor.
gavinevans
force-pushed
the
impro_2102_additional_predictors
branch
from
b72eb70 to
4aebecf
Compare
Description
A branch containing some possible EMOS amendments. Tests have not been updated. or added to support the new features.
This PR is only to help facilitate an initial look and will not be merged.
Testing: