Disable Image Automatic Resizing in Positron’s Python Kernel to Prevent IDE Freezing When Meeting Complex Pic

[ZhimingYe]

System details:

Positron and OS details:

Positron Version: 2024.08.0 (Universal) build 83
Code - OSS Version: 1.92.0
Commit: f2238e0
Date: 2024-08-29T02:37:56.178Z
Electron: 30.1.2
Chromium: 124.0.6367.243
Node.js: 20.14.0
V8: 12.4.254.20-electron.0
OS: Darwin arm64 23.5.0

Interpreter details:

Python 3.9.18

Describe the issue:

The Python section of Positron is based on Ipykernel, and changing the image size in the preview pane within Ipykernel seems to occupy the ipykernel, rendering the console unusable. When printing a large image (e.g., a faceted scatter plot with hundreds of thousands of points), accidentally resizing the image pane can trigger the console to wait for the image to be re-rendered, which may take a significant amount of time. Since the image pane in Positron is placed together with the variable pane, sometimes I just want to enlarge the variable pane, but unintentionally, I trigger the image resizing event as well. This issue is particularly evident with images that take several seconds to render, although it doesn't occur with smaller images.

Moreover, when there are several complex plots like this in the history, repeatedly clicking on them can potentially cause Positron to freeze.

This need is quite common in this field; it’s not that I’m creating such an image to deliberately challenge the Positron team.

Expected or desired behavior:

I suggest that Positron's Python kernel should implement a timer for each image rendering to record the time it takes. If this time becomes unacceptable (e.g., greater than 3 seconds), automatic resizing of the image along with the pane should be disabled. Additionally, switching between image history should use a more lazy approach (e.g., using previously saved thumbnails).

graph TD
    A[Start Image Rendering] --> B{Is Rendering Time Acceptable?}
    B -- Yes, <= 3 seconds --> C[Allow Automatic Resizing]
    B -- No, > 3 seconds --> D[Disable Automatic Resizing]
    D --> E[Switching Image History]
    E --> F{Use Lazy Loading?}
    F -- Yes --> G[Use Previously Saved Thumbnails]
    F -- No --> H[Render Full Image]
    H --> I[Complete]
    G --> I[Complete]
    C --> I[Complete]

Loading

Steps to reproduce the issue:

# wget "https://datasets.cellxgene.cziscience.com/981bcf57-30cb-4a85-b905-e04373432fef.h5ad"
import scanpy as sc
test=sc.read_h5ad("/home/yezhiming/redoSC/981bcf57-30cb-4a85-b905-e04373432fef.h5ad")
test
sc.set_figure_params(dpi=80,dpi_save=300)
sc.pl.umap(test,color=['ENSG00000081237','ENSG00000119888','ENSG00000261371','ENSG00000164692','ENSG00000107796'],use_raw=True,legend_loc="on data")
sc.pl.umap(test,color=['ENSG00000081237','ENSG00000119888','ENSG00000261371','ENSG00000164692','ENSG00000107796'],use_raw=True,legend_loc="on data")
sc.pl.umap(test,color=['ENSG00000081237','ENSG00000119888','ENSG00000261371','ENSG00000164692','ENSG00000107796'],use_raw=True,legend_loc="on data")
sc.pl.umap(test,color=['ENSG00000081237','ENSG00000119888','ENSG00000261371','ENSG00000164692','ENSG00000107796'],use_raw=True,legend_loc="on data")
testdf=test.obs # Generate a table

This generate scatter plot with 483152 points.

Then, resizing the figure or viewing history

Case 1: Viewing history makes console and other parts of IDE freeze

like this:

quick.switch.pic.using.console.mp4

Sometimes an error was throw:

Case 2:

In this case, you can see resizeing the right side planel make console freeze. at ~15s, you can see multi-command "freezed" before send immediately.

resize.burdens.console.mp4

Case 3:

Similar to Case 2, simply resize the right planel, the data viewer is also freeze, after a long time wait, many data viewer is opened.

Stops.data.viewer.mp4

For my field, creating such images is almost commonplace. So I hope this situation can improve in future Positron. Thank you for your work.

[jmcphers]

Some other notes:

• this isn't specific to Python; some R plots are also very expensive to draw, and any solution should be language agnostic
• a lot of the really annoying behavior here comes from the console not dealing with a busy state very well, which is also a problem for other long-running operations
• it might be better for us to have a much longer timeout on the first render of each plot (up to, say, a couple of minutes) so that truly expensive plots can still be displayed
• if a plot in R or Python is expensive to redraw, we could disable the auto resize but would also want to display an affordance for manually redrawing if the user wants to (maybe along with an estimate of the time needed)

[ZhimingYe]

Some other notes:

• this isn't specific to Python; some R plots are also very expensive to draw, and any solution should be language agnostic
• a lot of the really annoying behavior here comes from the console not dealing with a busy state very well, which is also a problem for other long-running operations
• it might be better for us to have a much longer timeout on the first render of each plot (up to, say, a couple of minutes) so that truly expensive plots can still be displayed
• if a plot in R or Python is expensive to redraw, we could disable the auto resize but would also want to display an affordance for manually redrawing if the user wants to (maybe along with an estimate of the time needed)

@jmcphers
Thank you for your prompt response. Before raising this issue, I was concerned that the request might be too niche, but this need is actually quite common in bioinformatics field.

I worry that some of the requests might seem too demanding for Positron Team—like why not rasterize these images in advance so that they don’t need to be output as vector graphics, or why not use some better optimization methods. In short, I feel that some of the issues might not lie on Positron’s side, but the package itself. However, unfortunately, the default behavior of the Scanpy package—the most popular Python package for single-cell sequencing data analysis at the moment—outputs a PDF file as large as 100MB without specifying the DPI （in the example I mentioned above, I set the dpi to 80, but default is without this limitaion), which can lead beginner users to mistakenly blame the Positron IDE for this problem if they aren’t aware of the options.

What I want to convey is that if rendering the output is difficult, Positron could entirely avoid rendering and leave this choice to the user, while focusing on ensuring that the console’s processes are not interrupted by it. It’s true that many packages in this field are not written by professional software engineers, so some strange outputs may occur. For those of us working in interdisciplinary fields, what we write usually just needs to function. As long as Positron ensures that no matter what, the IDE doesn’t become sluggish or cause console issues, that would be sufficient.

Additionally, in ARK, I don’t recall encountering similar issues, perhaps because #3628 prevented me from trying similar cases. I will do further test after #3628 is fixed. Also, when generating similarly scaled images in RStudio using R (though Python and R are not directly comparable), such issues don’t occur. I think some of RSession’s strategies in RStudio are worth referencing. I’ve been doing similar analyses for over five years now, and in RStudio, I’ve never had to worry about such things—it’s incredibly stable, and I can confidently make each click and preview without worrying about data size causing any problems.

Again, thank you very much for your work!