Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[🐛 Bug]: HeapProfiler.addHeapSnapshotChunk not fired in order #13845

Closed
zixinyin opened this issue Apr 19, 2024 · 8 comments
Closed

[🐛 Bug]: HeapProfiler.addHeapSnapshotChunk not fired in order #13845

zixinyin opened this issue Apr 19, 2024 · 8 comments
Labels
C-devtools BiDi or Chrome DevTools related issues C-java I-defect needs-triaging

Comments

@zixinyin
Copy link

What happened?

For CDP session HeapProfiler.addHeapSnapshotChunk() event, Selenium is not returning the large chunk data in sequence and there is not way to place the returned data in order without low level access.

How can we reproduce the issue?

DevTools devTools = ((HasDevTools) driver).getDevTools();
        devTools.createSession();
        devTools.send(HeapProfiler.enable());
        Queue<String> queue = new LinkedList<>();
        devTools.addListener(HeapProfiler.addHeapSnapshotChunk(), queue::add);
        devTools.send(HeapProfiler.takeHeapSnapshot(Optional.of(false), Optional.of(false), Optional.of(false), Optional.of(false)));
        try {
            FileOutputStream fileOutputStream = new FileOutputStream("/tmp/output.heapsnapshot");
            while (!queue.isEmpty()) {
                fileOutputStream.write(queue.poll().getBytes());
            }
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
        devTools.clearListeners();
        devTools.close();



Another way to reproduce is to print the string size every time the event is fired.
final ReentrantLock lock = new ReentrantLock(true);
devTools.addListener(HeapProfiler.addHeapSnapshotChunk(), target -> {
            lock.lock();
            System.out.println(target.length());
            lock.unlock();
        });
And you would still see the output end up like:

102400
102400
102400
102400
102400
32441
102400
102400
102400
102400

Relevant log output

Usually the log output will start with {"snapshot":{"meta":{"node_fields": but not always. 
The log output should be a complete json file but you would often see the file ends with broken output.
Each large chunk data's length is 102400.

Operating System

maci and windows

Selenium version

open jdk 21.0.2

What are the browser(s) and version(s) where you see this issue?

Chrome 118

What are the browser driver(s) and version(s) where you see this issue?

ChromeDriver 118

Are you using Selenium Grid?

No response

Copy link

@zixinyin, thank you for creating this issue. We will troubleshoot it as soon as we can.


Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

@pujagani pujagani added C-java C-devtools BiDi or Chrome DevTools related issues labels May 6, 2024
@pujagani
Copy link
Contributor

pujagani commented May 6, 2024

Thank you for the details. Have you checked the logs to ensure that the events are returned in order from CDP itself? Can you please share the debug logs here. Thank you!

@joerg1985
Copy link
Member

The CDP events are processed concurrently by different threads, i think this will destory the order.

Other areas rely on this, e.g. the NetworkInterceptor:
One thread will wait here for another thread to complete the future:
https://github.com/SeleniumHQ/selenium/blob/trunk/java/src/org/openqa/selenium/devtools/idealized/Network.java#L242
And this is the line where the other thread does complete the future:
https://github.com/SeleniumHQ/selenium/blob/trunk/java/src/org/openqa/selenium/devtools/idealized/Network.java#L226

@pujagani
Copy link
Contributor

pujagani commented May 8, 2024

Makes sense. Thank you @joerg1985! So what should be done regarding this issue?

@joerg1985
Copy link
Member

@pujagani this is a very special case, so i would empower the listener to handle this, e.g. like in the PR i will raise in the next minutes.

@pujagani
Copy link
Contributor

pujagani commented May 9, 2024

Thank you for the PR! It looks good to me.

@joerg1985
Copy link
Member

joerg1985 commented May 10, 2024

@zixinyin Starting with the next release it is possible to register a two argument listener, to sort the data:

TreeMap<Long, String> chunks = new TreeMap<>();
devTools.addListener(HeapProfiler.addHeapSnapshotChunk(), chunks::put);
devTools.send(HeapProfiler.takeHeapSnapshot(Optional.of(false), Optional.of(false), Optional.of(false), Optional.of(false)));
try (FileWriter fileWriter = new FileWriter("dump.json")) {
  for (String chunk : chunks.values()) {
     fileWriter.write(chunk);
  }
}

Copy link

github-actions bot commented Jun 9, 2024

This issue has been automatically locked since there has not been any recent activity since it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked and limited conversation to collaborators Jun 9, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
C-devtools BiDi or Chrome DevTools related issues C-java I-defect needs-triaging
Projects
None yet
Development

No branches or pull requests

3 participants