Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support initial values for constrained parameters #372

Merged
merged 10 commits into from
Jul 3, 2024

Conversation

gowerc
Copy link
Collaborator

@gowerc gowerc commented Jun 27, 2024

Closes #268

Generally tried to shield the user from this as much as possible and pushed all code to the behind the scenes. Each model (e.g. LongitudinalGSF) is responsible for setting the constraints on the priors which is done via set_limits().

initialValues() then attempts to draw 100 values and discards any that do not meet the constraint. If there is more than 1 value that meets the constraint then 1 of these is sampled at random.

For random effects parameters that were having their distribution set based on the initial value of the parent I've tried to reduce the variance of the values being produced by implementing a median function. In short this just generates 250 values following the above described method and then just takes the median across these. Originally we were just taking the centre point of the distribution but as users can now specify distributions whose centre point is potentially not a valid value this alternative was required.

Misc changes

  • Changed the prior@init slot to prior@centre which better represents what this parameter is doing (its meant to denote the centre of the distribution or more accurately the point at which we shrink our initial values towards).
  • Reduced the default prior of the longitudinal mu slope. As the new initial value selection has more noise / variance to it it started to generate values for the random effects that caused initial sampler errors so tightened the default prior to help reduce this
  • For reasons I don't understand the shared environment test_data_01 which is generated by a helper function started throwing errors when the unit tests were run in parallel. I've had to change the code slightly such that the environment is now individually created in each file that needs it. This does unfortunately add some run time but I couldn't find another solution :(

@gowerc gowerc requested a review from gravesti June 27, 2024 16:13
Copy link
Contributor

github-actions bot commented Jun 27, 2024

badge

Code Coverage Summary

Filename                           Stmts    Miss  Cover    Missing
-------------------------------  -------  ------  -------  --------------------------------
R/brier_score.R                      167       0  100.00%
R/DataJoint.R                         76       2  97.37%   264, 270
R/DataLongitudinal.R                 119       1  99.16%   244
R/DataSubject.R                       85       1  98.82%   142
R/DataSurvival.R                      98       0  100.00%
R/defaults.R                          10       6  40.00%   18-57, 84
R/generics.R                          34       4  88.24%   52, 396, 416, 471
R/Grid.R                              27       1  96.30%   173
R/GridEven.R                          32       0  100.00%
R/GridEvent.R                         22       0  100.00%
R/GridFixed.R                         30       0  100.00%
R/GridGrouped.R                       52       0  100.00%
R/GridManual.R                        23       3  86.96%   78-80
R/GridObserved.R                      20       0  100.00%
R/GridPopulation.R                    31       4  87.10%   61, 69-71
R/GridPrediction.R                    36       6  83.33%   83, 91-95
R/JointModel.R                       126      10  92.06%   146-151, 201, 205, 247, 293, 299
R/JointModelSamples.R                 66       0  100.00%
R/link_generics.R                     13       3  76.92%   59, 76, 91
R/Link.R                              62       4  93.55%   200-203
R/LinkComponent.R                     19       3  84.21%   88, 120-121
R/LongitudinalClaretBruno.R           88       6  93.18%   164-168, 183
R/LongitudinalGSF.R                   89       0  100.00%
R/LongitudinalModel.R                 19       0  100.00%
R/LongitudinalQuantities.R            71       0  100.00%
R/LongitudinalRandomSlope.R           44       5  88.64%   102-106
R/LongitudinalSteinFojo.R             80       5  93.75%   155-159
R/Parameter.R                         14       0  100.00%
R/ParameterList.R                     42       1  97.62%   184
R/Prior.R                            254       6  97.64%   507, 651-665
R/Promise.R                           25       0  100.00%
R/Quantities.R                        60       0  100.00%
R/QuantityGeneratorPopulation.R       22       0  100.00%
R/QuantityGeneratorPrediction.R       48       0  100.00%
R/QuantityGeneratorSubject.R          19       0  100.00%
R/settings.R                          12      12  0.00%    55-69
R/SimGroup.R                           5       0  100.00%
R/SimJointData.R                      72       1  98.61%   103
R/SimLongitudinal.R                    5       2  60.00%   22, 40
R/SimLongitudinalClaretBruno.R        52       0  100.00%
R/SimLongitudinalGSF.R                60       0  100.00%
R/SimLongitudinalRandomSlope.R        42       0  100.00%
R/SimLongitudinalSteinFojo.R          50       0  100.00%
R/SimSurvival.R                      104       0  100.00%
R/StanModel.R                         15       0  100.00%
R/StanModule.R                       176       6  96.59%   192-193, 235, 246, 383, 411
R/SurvivalExponential.R               10       0  100.00%
R/SurvivalLoglogistic.R               11       0  100.00%
R/SurvivalModel.R                     19       0  100.00%
R/SurvivalQuantities.R               170       1  99.41%   125
R/SurvivalWeibullPH.R                 11       0  100.00%
R/utilities.R                        144       2  98.61%   13, 335
R/zzz.R                               28      25  10.71%   3, 8-12, 14-35, 47
TOTAL                               3009     120  96.01%

Diff against main

Filename                       Stmts    Miss  Cover
---------------------------  -------  ------  --------
R/generics.R                      +1       0  +0.36%
R/LongitudinalClaretBruno.R       +5       0  +0.41%
R/LongitudinalGSF.R               +5       0  +100.00%
R/LongitudinalRandomSlope.R       +2       0  +0.54%
R/LongitudinalSteinFojo.R         +4       0  +0.33%
R/Prior.R                        +18      -2  +1.03%
TOTAL                            +35      -2  +0.11%

Results for commit: a0a8a6f

Minimum allowed coverage is 80%

♻️ This comment has been updated with latest results

Copy link
Contributor

github-actions bot commented Jun 27, 2024

Unit Tests Summary

    1 files    160 suites   6m 26s ⏱️
  139 tests   136 ✅ 3 💤 0 ❌
1 052 runs  1 048 ✅ 4 💤 0 ❌

Results for commit a9c9f1a.

♻️ This comment has been updated with latest results.

Copy link
Contributor

github-actions bot commented Jun 27, 2024

Unit Test Performance Difference

Test Suite $Status$ Time on main $±Time$ $±Tests$ $±Skipped$ $±Failures$ $±Errors$
JointModelSamples 💔 $30.05$ $+2.01$ $0$ $0$ $0$ $0$
extract_quantities 💚 $7.52$ $-2.56$ $0$ $0$ $0$ $0$
misc_models 💔 $0.84$ $+4.40$ $0$ $0$ $0$ $0$
model_multi_chain 💚 $4.54$ $-1.41$ $0$ $0$ $0$ $0$
Additional test case details
Test Suite $Status$ Time on main $±Time$ Test Case
Grid 💔 $2.51$ $+52.53$ Grid_objects_work_with_QuantityGenerator_and_QuantityCollapser
GridPrediction 💔 $117.08$ $+2.72$ GridPrediction_works_as_expected_for_Survival_models
JointModelSamples 💔 $30.05$ $+2.01$ print_works_as_expected_for_JointModelSamples
LongitudinalClaretBruno 👶 $+0.01$ Can_generate_valid_initial_values
LongitudinalClaretBruno 💔 $4.38$ $+1.50$ LongitudinalClaretBruno_works_as_expected_with_default_arguments
LongitudinalGSF 👶 $+0.01$ Can_generate_valid_initial_values
LongitudinalQuantiles 💚 $45.99$ $-42.58$ Test_that_LongitudinalQuantities_works_as_expected
LongitudinalRandomSlope 👶 $+0.00$ Can_generate_valid_initial_values
LongitudinalSteinFojo 👶 $+0.01$ Can_generate_valid_initial_values
LongitudinalSteinFojo 💔 $3.95$ $+1.15$ LongitudinalSteinFojo_works_as_expected_with_default_arguments
Prior 👶 $+0.00$ Limits_work_as_expected
Prior 👶 $+0.00$ median_Prior_works_as_expected
SurvivalExponential 💔 $22.29$ $+2.27$ SurvivalExponential_can_recover_true_parameter_including_covariates_
SurvivalLoglogistic 💚 $22.88$ $-8.48$ sim_os_loglogistic_is_consistant_with_flexsurv
SurvivalQuantities 💚 $3.49$ $-3.23$ SurvivalQuantities_and_autoplot.SurvivalQuantities_works_as_expected
brierScore 💔 $26.46$ $+10.16$ brierScore_SurvivalQuantities_returns_same_results_as_survreg
extract_quantities 💚 $7.52$ $-2.56$ extract_quantities_works_as_expected
misc_models 💔 $0.84$ $+4.40$ Longitudinal_Model_doesn_t_print_sampler_rejection_messages
model_multi_chain 💚 $4.54$ $-1.41$ Can_recover_known_distribution_parameters_from_random_slope_model_when_using_multiple_chains
stan_functions 💔 $26.18$ $+1.35$ GSF_SLD_function_works_as_expected

Results for commit 39f98f9

♻️ This comment has been updated with latest results.

Copy link
Collaborator

@gravesti gravesti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gowerc A few things to clarify :)

R/generics.R Show resolved Hide resolved
R/Prior.R Outdated Show resolved Hide resolved
R/Prior.R Show resolved Hide resolved
gravesti
gravesti previously approved these changes Jul 2, 2024
Copy link
Collaborator

@gravesti gravesti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very nice @gowerc !

@gowerc gowerc merged commit 2f57166 into main Jul 3, 2024
@gowerc gowerc deleted the 268-constrained-priors branch July 3, 2024 15:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Document that users need to set their own initial values for truncated distributions
2 participants