With Plotly version >= 2.0.0-beta9, FSharp.Formatting Evaluator does not return a plot

[nhirschey]

Description

Do you know if it is possible to generate fsdocs content with plots using nuget versions of Plotly.NET >= beta9?

Related to fsprojects/FSharp.Formatting#635, I was trying to upgrade some fsdocs content to the new Plotly.NET version and the plots were not evaluated. The fsdocs evaluator returned "No value returned by any evaluator".

It looks like the change happens between Plotly.NET versions beta8 (works) and beta9 (doesn't work).

Repro steps

I can reproduce it with a simple script and calling the FSharp.Formatting evaluator directly:

#r "nuget: FSharp.Formatting, 14.0.1"

open FSharp.Formatting.Literate
open FSharp.Formatting.Literate.Evaluation


let source version = $"""
(***hide***)
#r "nuget: Newtonsoft.JSON, 12.0.3"
#r "nuget: DynamicObj"
#r "nuget: Plotly.NET, 2.0.0-{version}"

open Plotly.NET

let x  = [1.; 2.; 3.; 4.; 5.; 6.; 7.; 8.; 9.; 10.; ]
let y = [2.; 1.5; 5.; 1.5; 3.; 2.5; 2.5; 1.5; 3.5; 1.]

Chart.Line(x,y)
|> GenericChart.toChartHTML
(***include-it-raw***)
"""

let parsed version =
    let source = source version
    let fsi = FsiEvaluator()
    let doc = Literate.ParseScriptString(source, fsiEvaluator = fsi)
    Literate.ToHtml(doc)

The plot gets made with beta8:

> parsed "beta8";;
val it: string =
  "<div id="8f65bf44-f610-4cc0-ba06-0794a0a1dc3c" style="width: 600px; height: 600px;"><!-- Plotly chart will be drawn inside this DIV --></div>
<script type="text/javascript">

            var renderPlotly_8f65bf44f6104cc0ba060794a0a1dc3c = function() {
            var fsharpPlotlyRequire = requirejs.config({context:'fsharp-plotly',paths:{plotly:'https://cdn.plot.ly/plotly-latest.min'}}) || require;
            fsharpPlotlyRequire(['plotly'], function(Plotly) {

            var data = [{"type":"scatter","x":[1.0,2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0,10.0],"y":[2.0,1.5,5.0,1.5,3.0,2.5,2.5,1.5,3.5,1.0],"mode":"lines","marker":{}}];
            var layout = {};
            var config = {};
            Plotly.newPlot('8f65bf44-f610-4cc0-ba06-0794a0a1dc3c', data, layout, config);
});
            };
            if ((typeof(requirejs) !==  typeof(Function)) || (typeof(requirejs.config) !== typeof(Function))) {
                var script = document.createElement("script");
                script.setAttribute("src", "https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.6/require.min.js");
                script.onload = function(){
                    renderPlotly_8f65bf44f6104cc0ba060794a0a1dc3c();
                };
                document.getElementsByTagName("head")[0].appendChild(script);
            }
            else {
                renderPlotly_8f65bf44f6104cc0ba060794a0a1dc3c();
            }
</script>
..."

The plot is not made with either beta9 or preview.16:

> parsed "beta9";;
val it: string =
  "<table class="pre"><tr><td><pre><code>No value returned by any evaluator</code></pre></td></tr></table>
<div class="fsdocs-tip" id="fs1">namespace Plotly</div>
<div class="fsdocs-tip" id="fs2">namespace Plotly.NET</div>
<div class="fsdocs-tip" id="fs3">val x: float list</div>
<div class="fsdocs-tip" id="fs4">val y: float list</div>
<div class="fsdocs-tip" id="fs5">type Chart =
..."

> parsed "preview.16";;
val it: string =
  "<table class="pre"><tr><td><pre><code>No value returned by any evaluator</code></pre></td></tr></table>
<div class="fsdocs-tip" id="fs1">namespace Plotly</div>
<div class="fsdocs-tip" id="fs2">namespace Plotly.NET</div>
<div class="fsdocs-tip" id="fs3">val x: float list</div>
<div class="fsdocs-tip" id="fs4">val y: float list</div>
<div class="fsdocs-tip" id="fs5">type Chart =
...

Or using the 14.0.1 version of the fsdocs tool itself:

#r "nuget: Plotly.NET, 2.0.0-preview.16"

open Plotly.NET

let x  = [1.; 2.; 3.; 4.; 5.; 6.; 7.; 8.; 9.; 10.; ]
let y = [2.; 1.5; 5.; 1.5; 3.; 2.5; 2.5; 1.5; 3.5; 1.]
let line1 = Chart.Line(x,y)

(** try 1*)
line1 |> GenericChart.toChartHTML
(***include-it-raw***)


(** try 2 *)
line1 |> GenericChart.toChartHTML
(***include-fsi-output***)

Expected behavior

Working version with beta 8 using the 14.0.1 version of the fsdocs tool itself:

Known workarounds

Use old versions of Plotly.NET

Related information

• Windows 11
• .net 6.0.101

[kMutagene]

i think this might have to do with fsharp.formatting not working well with .net 6. i have had many problems with it recently as well. It is very unlikely an error on our end though, as the docs of Plotly.NET are built with fsdocs as well, just with .net 5.

[kMutagene]

I did some digging across some blogs and docs i maintain, here are some versions that work together. It seems like overall the more complex the repo, the harder it is to make fsdocs working well for all kinds of evaluations, but here are some combinations that should work:

Plotly.NET repo docs:

• fsharp.formatting.commandtool: 11.5.1
• Plotly.NET: latest
• .NET 5
• OS: windows and ubuntu (locally)

Other examples:

https://github.com/fslaborg/datasets

• fsharp.formatting.commandtool: 11.4.0
• Plotly.NET: 2.0.0-preview.6
• .NET 5.0.302
• OS: ubuntu (GH actions CI)

https://github.com/CSBiology/CSBlog

• fsdocs-tool (new nuget package): 14.0.1
• Plotly.NET: 2.0.0-preview.16
• .NET 6.0.101
• OS: windows (GH actions CI)

[nhirschey]

Thank you for kindly documenting these configurations where it works. The CSBlog is my exact local setup, which was helpful to see.

Overall, I can sometimes get preview.16 to work, but it's not reliable and fails more often than not. It seems to work more often (but not always) with a clean folder (rm .fsdocs and fsdocs watch --eval --clean). I haven't been able to isolate the problem, but I agree it looks more like either an issue with fsdocs itelf or my local computer.

I don't have a solution but I'll add a comment with the solution if I find one.

An oddity: I have seen a page using preview.16 graphs work with fsdocs watch locally, then I add an innocuous markdown paragraph like (** this is a test *) and then when the page is rebuilt in watch mode the evaluator fails on preview.16 graphs.

[nhirschey]

For reference,

1. I cloned the CSBlog. When I build it with dotnet fsdocs watch --eval everything works on the first build with plots showing. This is what would happen in a github action.
2. But then if I add (** this is a test *) to line 72 of the 4_qvalues.fsx file, it rebuilds that page (because it is in watch mode) and the plot evaluator isn't working.

Anyway, I'll let you know if I figure it out.

[nhirschey]

@bvenn I was looking at the great fslab website just now and noticed that your qvalues page has this problem with plotly charts not being evaluated. See the distributionChartAB at https://fslab.org/content/tutorials/010_q_values.html.

The workflows appear to be using dotnet 5.0.102 and Ubuntu, which is a different toolchain than the one I've used when seeing it locally.

[bvenn]

Thanks @nhirschey, we're in the process of fixing it right now 👍

[bvenn]

We also noticed that the watch mode seems broken after the first rendering. The evaluator fails to report any cell on the second try.

It looks like the change happens between Plotly.NET versions beta8 (works) and beta9 (doesn't work). @nhirschey

Have you tried a fresh render for beta9 or was it a sequential test from 8 to 9?

[nhirschey]

Going from 8 -> 9 I was iterating through versions using Literate.ParseScriptString directly with the parsed function in my first post in this issue.

Just now I was trying to reproduce beta9 failures in a fresh repo and beta9 worked. BUT, then while watch is running I switched to beta8 and the evaluator no longer worked. I switched back to beta9 and it worked. Then I tried beta5 and that didn't work either.

So in fact it seems like maybe the issue is that it fails whenever you switch plotly versions within a docs build? Does that make sense?

Update:
FYI this was the sample code where I saw changing beta9 to beta5 in watch mode caused the evaluator to fail. It was the only file in the docs folder, but I also tried adding another file that was exactly the same except it used beta 5. Both would build the first time in watch mode, but if I changed one to a version other than 8 or 9 while watch was running then it would fail.

#r "nuget: Plotly.NET, 2.0.0-beta9"

open Plotly.NET

let x  = [1.; 2.; 3.; 4.; 5.; 6.; 7.; 8.; 9.; 10.; ]
let y = [2.; 1.5; 5.; 1.5; 3.; 2.5; 2.5; 1.5; 3.5; 1.]
let line1 = Chart.Line(x,y)

(** try 1*)
line1 |> GenericChart.toChartHTML
(***include-it-raw***)


(** try 2 *)
line1 |> GenericChart.toChartHTML
(***include-fsi-output***)

[kMutagene]

Fits in the issues we see when referencing different versions of a lib across different scripts. Looks like there is not an isolated execution context per script in fsdocs, or maybe some things get cached wrong

[kMutagene]

I'll close this because there is nothing we can do from the plotly.net side of things - the problem is on the fsharp.formatting side.