[21314] Improve resilience against clock adjustments #5018

ma30002000 · 2024-07-02T12:55:27Z

Description

When investigating a system undergoing larger system clock adjustments, I noticed that in some places, std::chrono::system_clock and std::chrono::high_resolution_clock are used for handling timeouts and status check intervals.
However, std::chrono::system_clock is definitely not steady, and std::chrono::high_resolution_clock is not steady quite often (see cppreference).

When undergoing clock adjustments (manually or due to clock server synchronization), timeouts and status checks might no longer be triggered when relying on timestamps based on std::chrono::system_clock.

@Mergifyio backport 2.14.x 2.10.x

Contributor Checklist

Commit messages follow the project guidelines.
The code follows the style guidelines of this project.
Tests that thoroughly check the new feature have been added/Regression tests checking the bug and its fix have been added; the added tests pass locally
N/A Any new/modified methods have been properly documented using Doxygen.
N/A Any new configuration API has an equivalent XML API (with the corresponding XSD extension)
Changes are backport compatible: they do NOT break ABI nor change library core behavior.
Changes are API compatible.
N/A New feature has been added to the versions.md file (if applicable).
N/A New feature has been documented/Current behavior is correctly described in the documentation.
Applicable backports have been included in the description.

Reviewer Checklist

The PR has a milestone assigned.
The title and description correctly express the PR's purpose.
Check contributor checklist is correct.
N/A If this is a critical bug fix, backports to the critical-only supported branches have been requested.
Check CI results: changes do not issue any warning.
Check CI results: failing tests are unrelated with the changes.

ma30002000 · 2024-07-09T14:21:47Z

Is there anything I need to to in order to get CI results?

JesusPoderoso · 2024-07-09T14:29:50Z

Hi @ma30002000, thanks for your contribution.
We will look at it in the following days and order CI.
Thanks for your patience!

…high_resolution_clock might not be steady depending on STL impl) Signed-off-by: Matthias Schneider <[email protected]>

Signed-off-by: Matthias Schneider <[email protected]>

Signed-off-by: Miguel Company <[email protected]>

MiguelCompany

@ma30002000

First of all, thank you for your patience waiting for this to be reviewed.

I took the liberty of rebasing this PR, and added a commit to fix one build issue.

Except for the changes in SharedMemGlobal.hpp, the changes look good to me.

src/cpp/rtps/transport/shared_mem/SharedMemGlobal.hpp

Signed-off-by: Miguel Company <[email protected]>

…stments. Signed-off-by: Miguel Company <[email protected]>

…checks (high_resolution_clock might not be steady depending on STL impl)" This reverts commit d69eb91.

MiguelCompany · 2024-08-30T06:29:55Z

Manual CIs:

src/cpp/utils/time_t_helpers.hpp

Mario-DL

LGTM with green CI

MiguelCompany · 2024-09-02T14:17:06Z

@Mergifyio backport 2.14.x 2.10.x

mergify · 2024-09-02T14:17:11Z

backport 2.14.x 2.10.x

✅ Backports have been created

#5189 [21314] Improve resilience against clock adjustments (backport #5018) has been created for branch 2.14.x but encountered conflicts
#5190 [21314] Improve resilience against clock adjustments (backport #5018) has been created for branch 2.10.x but encountered conflicts

* Use steady_clock instead of high_resolution_clock for status checks (high_resolution_clock might not be steady depending on STL impl) Signed-off-by: Matthias Schneider <[email protected]> * Use steady_clock instead for system_clock for calculating timeouts Signed-off-by: Matthias Schneider <[email protected]> * Use correct clock's duration for duration_cast Signed-off-by: Matthias Schneider <[email protected]> * Use Time_t::now() Signed-off-by: Matthias Schneider <[email protected]> * Fix build. Signed-off-by: Miguel Company <[email protected]> * Refs #21314. Refactor on DataWriterImpl. Signed-off-by: Miguel Company <[email protected]> * Refs #21314. Refactor on DataReaderImpl. Signed-off-by: Miguel Company <[email protected]> * Refs #21314. Refactor on StatefulWriter. Signed-off-by: Miguel Company <[email protected]> * Refs #21314. Protect current_time_since_unix_epoch against clock adjustments. Signed-off-by: Miguel Company <[email protected]> * Revert "Use steady_clock instead of high_resolution_clock for status checks (high_resolution_clock might not be steady depending on STL impl)" This reverts commit d69eb91. --------- Signed-off-by: Matthias Schneider <[email protected]> Signed-off-by: Miguel Company <[email protected]> Co-authored-by: Miguel Company <[email protected]> (cherry picked from commit ccc690c) # Conflicts: # include/fastdds/rtps/writer/StatefulWriter.h # src/cpp/fastdds/publisher/DataWriterImpl.cpp # src/cpp/fastdds/subscriber/DataReaderImpl.cpp # src/cpp/utils/time_t_helpers.hpp

* Use steady_clock instead of high_resolution_clock for status checks (high_resolution_clock might not be steady depending on STL impl) Signed-off-by: Matthias Schneider <[email protected]> * Use steady_clock instead for system_clock for calculating timeouts Signed-off-by: Matthias Schneider <[email protected]> * Use correct clock's duration for duration_cast Signed-off-by: Matthias Schneider <[email protected]> * Use Time_t::now() Signed-off-by: Matthias Schneider <[email protected]> * Fix build. Signed-off-by: Miguel Company <[email protected]> * Refs #21314. Refactor on DataWriterImpl. Signed-off-by: Miguel Company <[email protected]> * Refs #21314. Refactor on DataReaderImpl. Signed-off-by: Miguel Company <[email protected]> * Refs #21314. Refactor on StatefulWriter. Signed-off-by: Miguel Company <[email protected]> * Refs #21314. Protect current_time_since_unix_epoch against clock adjustments. Signed-off-by: Miguel Company <[email protected]> * Revert "Use steady_clock instead of high_resolution_clock for status checks (high_resolution_clock might not be steady depending on STL impl)" This reverts commit d69eb91. --------- Signed-off-by: Matthias Schneider <[email protected]> Signed-off-by: Miguel Company <[email protected]> Co-authored-by: Miguel Company <[email protected]> Signed-off-by: paxifaer <[email protected]>

* Use steady_clock instead of high_resolution_clock for status checks (high_resolution_clock might not be steady depending on STL impl) Signed-off-by: Matthias Schneider <[email protected]> * Use steady_clock instead for system_clock for calculating timeouts Signed-off-by: Matthias Schneider <[email protected]> * Use correct clock's duration for duration_cast Signed-off-by: Matthias Schneider <[email protected]> * Use Time_t::now() Signed-off-by: Matthias Schneider <[email protected]> * Fix build. Signed-off-by: Miguel Company <[email protected]> * Refs #21314. Refactor on DataWriterImpl. Signed-off-by: Miguel Company <[email protected]> * Refs #21314. Refactor on DataReaderImpl. Signed-off-by: Miguel Company <[email protected]> * Refs #21314. Refactor on StatefulWriter. Signed-off-by: Miguel Company <[email protected]> * Refs #21314. Protect current_time_since_unix_epoch against clock adjustments. Signed-off-by: Miguel Company <[email protected]> * Revert "Use steady_clock instead of high_resolution_clock for status checks (high_resolution_clock might not be steady depending on STL impl)" This reverts commit d69eb91. --------- Signed-off-by: Matthias Schneider <[email protected]> Signed-off-by: Miguel Company <[email protected]> Co-authored-by: Miguel Company <[email protected]> (cherry picked from commit ccc690c) # Conflicts: # include/fastdds/rtps/writer/StatefulWriter.h # src/cpp/fastdds/publisher/DataWriterImpl.cpp # src/cpp/fastdds/subscriber/DataReaderImpl.cpp # src/cpp/utils/time_t_helpers.hpp

* Use steady_clock instead of high_resolution_clock for status checks (high_resolution_clock might not be steady depending on STL impl) Signed-off-by: Matthias Schneider <[email protected]> * Use steady_clock instead for system_clock for calculating timeouts Signed-off-by: Matthias Schneider <[email protected]> * Use correct clock's duration for duration_cast Signed-off-by: Matthias Schneider <[email protected]> * Use Time_t::now() Signed-off-by: Matthias Schneider <[email protected]> * Fix build. Signed-off-by: Miguel Company <[email protected]> * Refs #21314. Refactor on DataWriterImpl. Signed-off-by: Miguel Company <[email protected]> * Refs #21314. Refactor on DataReaderImpl. Signed-off-by: Miguel Company <[email protected]> * Refs #21314. Refactor on StatefulWriter. Signed-off-by: Miguel Company <[email protected]> * Refs #21314. Protect current_time_since_unix_epoch against clock adjustments. Signed-off-by: Miguel Company <[email protected]> * Revert "Use steady_clock instead of high_resolution_clock for status checks (high_resolution_clock might not be steady depending on STL impl)" This reverts commit d69eb91. --------- Signed-off-by: Matthias Schneider <[email protected]> Signed-off-by: Miguel Company <[email protected]> Co-authored-by: Miguel Company <[email protected]> (cherry picked from commit ccc690c)

ma30002000 · 2024-10-15T07:12:59Z

Hi, sorry for the late response, could you please provide some more details on why you have not included the change of std::chrono::high_resolution_clock to std::chrono::steady_clock? Using a (on some systems) non-steady clock for timeout checks will have undesired effects. I also do not see a drawback of using steady_clock instead.

MiguelCompany · 2024-10-15T09:21:32Z

@ma30002000 Because the changes affected the PortNode structure, which is part of the Shared Memory port segment. This means the changes would break interoperability with previous versions, which we should not do on a patch release.

ma30002000 · 2024-10-15T15:22:42Z

@MiguelCompany I understand - how and when can we address the issue in the future then? Unfortunately this change did not make it into 3.x...

* Use steady_clock instead of high_resolution_clock for status checks (high_resolution_clock might not be steady depending on STL impl) Signed-off-by: Matthias Schneider <[email protected]> * Use steady_clock instead for system_clock for calculating timeouts Signed-off-by: Matthias Schneider <[email protected]> * Use correct clock's duration for duration_cast Signed-off-by: Matthias Schneider <[email protected]> * Use Time_t::now() Signed-off-by: Matthias Schneider <[email protected]> * Fix build. Signed-off-by: Miguel Company <[email protected]> * Refs #21314. Refactor on DataWriterImpl. Signed-off-by: Miguel Company <[email protected]> * Refs #21314. Refactor on DataReaderImpl. Signed-off-by: Miguel Company <[email protected]> * Refs #21314. Refactor on StatefulWriter. Signed-off-by: Miguel Company <[email protected]> * Refs #21314. Protect current_time_since_unix_epoch against clock adjustments. Signed-off-by: Miguel Company <[email protected]> * Revert "Use steady_clock instead of high_resolution_clock for status checks (high_resolution_clock might not be steady depending on STL impl)" This reverts commit d69eb91. --------- Signed-off-by: Matthias Schneider <[email protected]> Signed-off-by: Miguel Company <[email protected]> Co-authored-by: Miguel Company <[email protected]> (cherry picked from commit ccc690c)

ma30002000 force-pushed the bugfix/steady-clock branch 3 times, most recently from b5fad03 to 519524c Compare July 2, 2024 13:19

ma30002000 mentioned this pull request Jul 2, 2024

Dropped samples after system clock adjustments #5019

Open

1 task

EduPonz added this to the v3.0.0 milestone Jul 3, 2024

elianalf added the needs-review PR that is ready to be reviewed label Jul 11, 2024

EduPonz modified the milestones: v3.0.0, v3.0.1 Jul 19, 2024

ma30002000 and others added 5 commits August 28, 2024 10:50

Use steady_clock instead of high_resolution_clock for status checks (…

d69eb91

…high_resolution_clock might not be steady depending on STL impl) Signed-off-by: Matthias Schneider <[email protected]>

Use steady_clock instead for system_clock for calculating timeouts

03532f3

Signed-off-by: Matthias Schneider <[email protected]>

Use correct clock's duration for duration_cast

8c993da

Signed-off-by: Matthias Schneider <[email protected]>

Use Time_t::now()

d27f4ad

Signed-off-by: Matthias Schneider <[email protected]>

Fix build.

d4d2ba4

Signed-off-by: Miguel Company <[email protected]>

MiguelCompany force-pushed the bugfix/steady-clock branch from 519524c to d4d2ba4 Compare August 28, 2024 10:48

MiguelCompany requested changes Aug 28, 2024

View reviewed changes

src/cpp/rtps/transport/shared_mem/SharedMemGlobal.hpp Outdated Show resolved Hide resolved

Refs #21314. Refactor on DataWriterImpl.

6078402

Signed-off-by: Miguel Company <[email protected]>

MiguelCompany changed the title ~~More use of std::chrono::steady_clock for resilience against clock adjustments~~ [21314] More use of std::chrono::steady_clock for resilience against clock adjustments Aug 29, 2024

MiguelCompany added 4 commits August 29, 2024 10:45

Refs #21314. Refactor on DataReaderImpl.

c3867e7

Signed-off-by: Miguel Company <[email protected]>

Refs #21314. Refactor on StatefulWriter.

d5e2ecc

Signed-off-by: Miguel Company <[email protected]>

Refs #21314. Protect current_time_since_unix_epoch against clock adju…

3d20ab3

…stments. Signed-off-by: Miguel Company <[email protected]>

Revert "Use steady_clock instead of high_resolution_clock for status …

f94d111

…checks (high_resolution_clock might not be steady depending on STL impl)" This reverts commit d69eb91.

MiguelCompany changed the title ~~[21314] More use of std::chrono::steady_clock for resilience against clock adjustments~~ [21314] Improve resilience against clock adjustments Aug 29, 2024

MiguelCompany requested a review from richiprosima August 29, 2024 11:31

Mario-DL reviewed Aug 30, 2024

View reviewed changes

src/cpp/utils/time_t_helpers.hpp Show resolved Hide resolved

Mario-DL approved these changes Aug 30, 2024

View reviewed changes

JesusPoderoso added ready-to-merge Ready to be merged. CI and changes have been reviewed and approved. and removed needs-review PR that is ready to be reviewed labels Sep 2, 2024

MiguelCompany merged commit ccc690c into eProsima:master Sep 2, 2024
25 of 31 checks passed

mergify bot mentioned this pull request Sep 2, 2024

[21314] Improve resilience against clock adjustments (backport #5018) #5189

Open

11 tasks

mergify bot mentioned this pull request Sep 2, 2024

[21314] Improve resilience against clock adjustments (backport #5018) #5190

Closed

11 tasks

MiguelCompany mentioned this pull request Sep 9, 2024

[21609] Revert behavior change on current_time_since_unix_epoch #5213

Merged

10 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[21314] Improve resilience against clock adjustments #5018

[21314] Improve resilience against clock adjustments #5018

ma30002000 commented Jul 2, 2024 •

edited by JesusPoderoso

Loading

ma30002000 commented Jul 9, 2024

JesusPoderoso commented Jul 9, 2024

MiguelCompany left a comment

MiguelCompany commented Aug 30, 2024

Mario-DL left a comment •

edited

Loading

MiguelCompany commented Sep 2, 2024

mergify bot commented Sep 2, 2024 •

edited

Loading

ma30002000 commented Oct 15, 2024

MiguelCompany commented Oct 15, 2024

ma30002000 commented Oct 15, 2024

[21314] Improve resilience against clock adjustments #5018

[21314] Improve resilience against clock adjustments #5018

Conversation

ma30002000 commented Jul 2, 2024 • edited by JesusPoderoso Loading

Description

Contributor Checklist

Reviewer Checklist

ma30002000 commented Jul 9, 2024

JesusPoderoso commented Jul 9, 2024

MiguelCompany left a comment

Choose a reason for hiding this comment

MiguelCompany commented Aug 30, 2024

Mario-DL left a comment • edited Loading

Choose a reason for hiding this comment

MiguelCompany commented Sep 2, 2024

mergify bot commented Sep 2, 2024 • edited Loading

✅ Backports have been created

ma30002000 commented Oct 15, 2024

MiguelCompany commented Oct 15, 2024

ma30002000 commented Oct 15, 2024

ma30002000 commented Jul 2, 2024 •

edited by JesusPoderoso

Loading

Mario-DL left a comment •

edited

Loading

mergify bot commented Sep 2, 2024 •

edited

Loading