FSharp.Formatting for F# 3.1 implicitly reading FSharp.Core 4.4.0.0

[dsyme]

I'm using the latest FSharp.Formatting on a Windows machine with both F# 4.0 and F# 3.1 installed. I'm running from a F# 3.1 command prompt.

I'm getting an error where the FSharp.Core 4.4.0.0 (from F# 4.0) is now being used by the FSharp.Formatting toolchain (for example the error below). This is a bug - the FSharp.Formatting toolchain is currently built on FSharp.Compiler.Service for F# 3.1 and should never be reading FSharp.Core 4.4.0.0 unless explicitly directed to do so.

Has the change to Yaaf.FSharp.Scripting caused this? Or is it an implicit reference to FSharp.Core 4.4.0.0 flowing from elsewhere, e.g. FCS itself? Either way the FSharp.Formatting tooling feels somewhat broken right now, or at least not working in an isolated way,

I feel we need to move to a model where the reference set being used by interactive evaluation with FSharp.Formatting is more locked down.

    unknown(1,1): error FS0229: Error opening binary file 'C:\Program Files (x86)\Reference Assemblies\Microsoft\FSharp\.NETFramework\v4.0\4.4.0.0\FSharp.Co re.dll': Error reading/writing metadata for the F# compiled DLL 'C:\Program Files (x86)\Reference Assemblies\Microsoft\FSharp\.NETFramework\v4.0\4.4.0.0\FSharp.Core.dll'. ...).

cheers
don

[tpetricek]

My impression was that (when invoked via F# Interactive and not as a stand-alone application) F# Formatting will always load the FSharp.Core.dll that is used by the current process. Or is that not the case?

I think this would be sensible behavior - otherwise, the values you get from FsiEvaluator might involve different FSharp.Core.dll types (say, list) than the ones your current process is using... (but I'm not sure if this is actually possible?)

[matthid]

How can I reproduce this? I have installed both F# 4.0 and F# 3.1 as well, but everything seems to be working fine for me.

I agree with @tpetricek and I tested with Yaaf.FSharp.Scripting with the F# 3.1 fsi.exe and the F# 4.0 fsi.exe and it was always using the current FSharp.Core. But it is quite possible that I got something wrong in the FSharp.Core resolution of Yaaf.FSharp.Scripting.

[7sharp9]

I cant get fsfromatting to work either I get FSharp.Core version mismatches and also Microsoft.Build.Utilities masmatches when runing the documentation generator for FCS:

FSharp.Formatting Information: 0 : FSharp.Formatting Logging setup!
Yaaf.FSharp.Scriping Information: 0 : Yaaf.FSharp.Scripting Logging setup!
Copying file: /Users/dave/code/FCS/docs/output/images/en.png
Copying file: /Users/dave/code/FCS/docs/output/images/ja.png
Copying file: /Users/dave/code/FCS/docs/output/images/logo.png
Copying file: /Users/dave/code/FCS/docs/output/content/fcs.css
Copying file: /Users/dave/code/FCS/docs/output/content/style.ja.css
Copying styles and scripts: /Users/dave/code/FCS/docs/output/content/style.css
Copying styles and scripts: /Users/dave/code/FCS/docs/output/content/style.css.bak
Copying styles and scripts: /Users/dave/code/FCS/docs/output/content/style_light.css
Copying styles and scripts: /Users/dave/code/FCS/docs/output/content/tips.js
Copying styles and scripts: /Users/dave/code/FCS/docs/output/content/img/github-blue.png
Copying styles and scripts: /Users/dave/code/FCS/docs/output/content/img/github.png
Generating '/Users/dave/code/FCS/docs/tools/../content/caches.html'
FSharp.Formatting Critical: 0 :
Processing the file 'docpage' failed
Source written to: '/var/folders/5v/p_5h35nd70s8x7fh8fqn7hyh0000gn/T/tmp47c63951.tmp.cs'
Compilation errors:
 - warning: (0, 0) Assuming assembly reference `FSharp.Core, Version=4.3.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' matches assembly `FSharp.Core, Version=4.4.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. You may need to supply runtime policy
 - error: (0, 0) Assembly `Microsoft.Build.Utilities.v12.0, Version=12.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' references `Microsoft.Build.Framework, Version=12.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' which has a higher version number than imported assembly `Microsoft.Build.Framework, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'

System.Exception: Generating HTML failed.
   at FSharp.Literate.RazorRender.handleCompile[a] (System.String source, Microsoft.FSharp.Core.FSharpFunc`2 f) in <filename unknown>:line 0
   at FSharp.Literate.RazorRender.ProcessFileModel[b] (System.Type modelType, System.Object model, Microsoft.FSharp.Core.FSharpOption`1 properties) in <filename unknown>:line 0
   at FSharp.Literate.Templating.generateFile (Microsoft.FSharp.Core.FSharpOption`1 references, System.String contentTag, IEnumerable`1 parameters, Microsoft.FSharp.Core.FSharpOption`1 templateOpt, System.String output, IEnumerable`1 layoutRoots) in <filename unknown>:line 0
   at FSharp.Literate.Templating.processFile (Microsoft.FSharp.Core.FSharpOption`1 references, FSharp.Literate.LiterateDocument doc, System.String output, FSharp.Literate.ProcessingContext ctx) in <filename unknown>:line 0
   at FSharp.Literate.Literate.ProcessScriptFile (System.String input, Microsoft.FSharp.Core.FSharpOption`1 templateFile, Microsoft.FSharp.Core.FSharpOption`1 output, Microsoft.FSharp.Core.FSharpOption`1 format, Microsoft.FSharp.Core.FSharpOption`1 formatAgent, Microsoft.FSharp.Core.FSharpOption`1 prefix, Microsoft.FSharp.Core.FSharpOption`1 compilerOptions, Microsoft.FSharp.Core.FSharpOption`1 lineNumbers, Microsoft.FSharp.Core.FSharpOption`1 references, Microsoft.FSharp.Core.FSharpOption`1 fsiEvaluator, Microsoft.FSharp.Core.FSharpOption`1 replacements, Microsoft.FSharp.Core.FSharpOption`1 includeSource, Microsoft.FSharp.Core.FSharpOption`1 layoutRoots, Microsoft.FSharp.Core.FSharpOption`1 generateAnchors, Microsoft.FSharp.Core.FSharpOption`1 assemblyReferences, Microsoft.FSharp.Core.FSharpOption`1 customizeDocument) in <filename unknown>:line 0
   at <StartupCode$FSharp-Literate>[email protected] (System.String file, System.String output) in <filename unknown>:line 0
   at Microsoft.FSharp.Core.FSharpFunc`2[T,TResult].InvokeFast[V] (Microsoft.FSharp.Core.FSharpFunc`2 func, Microsoft.FSharp.Core.T arg1, Microsoft.FSharp.Core.TResult arg2) in <filename unknown>:line 0
   at <StartupCode$FSharp-Literate>.$Main.processDirectory@199 (Microsoft.FSharp.Core.FSharpOption`1 format, Boolean processRecursive, Microsoft.FSharp.Core.FSharpFunc`2 processScriptFile, Microsoft.FSharp.Core.FSharpFunc`2 processMarkdown, System.String indir, System.String outdir) in <filename unknown>:line 0
   at FSharp.Literate.Literate.ProcessDirectory (System.String inputDirectory, Microsoft.FSharp.Core.FSharpOption`1 templateFile, Microsoft.FSharp.Core.FSharpOption`1 outputDirectory, Microsoft.FSharp.Core.FSharpOption`1 format, Microsoft.FSharp.Core.FSharpOption`1 formatAgent, Microsoft.FSharp.Core.FSharpOption`1 prefix, Microsoft.FSharp.Core.FSharpOption`1 compilerOptions, Microsoft.FSharp.Core.FSharpOption`1 lineNumbers, Microsoft.FSharp.Core.FSharpOption`1 references, Microsoft.FSharp.Core.FSharpOption`1 fsiEvaluator, Microsoft.FSharp.Core.FSharpOption`1 replacements, Microsoft.FSharp.Core.FSharpOption`1 includeSource, Microsoft.FSharp.Core.FSharpOption`1 layoutRoots, Microsoft.FSharp.Core.FSharpOption`1 generateAnchors, Microsoft.FSharp.Core.FSharpOption`1 assemblyReferences, Microsoft.FSharp.Core.FSharpOption`1 processRecursive, Microsoft.FSharp.Core.FSharpOption`1 customizeDocument) in <filename unknown>:line 0
   at FSI_0003.buildDocumentation () in <filename unknown>:line 0
   at <StartupCode$FSI_0003>.$FSI_0003.main@ () in <filename unknown>:line 0
   at (wrapper managed-to-native) System.Reflection.MonoMethod:InternalInvoke (System.Reflection.MonoMethod,object,object[],System.Exception&)
   at System.Reflection.MonoMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) in <filename unknown>:line 0
Stopped due to error

[7sharp9]

I have tried the standalone command line tool with the metadataformat option with pretty much the same errors.

[halcwb]

Same here, building on mac os x (v 10.11.1) I am getting:

Compilation errors:
 - warning: (0, 0) Assuming assembly reference `FSharp.Core, Version=4.3.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' matches assembly `FSharp.Core, Version=4.3.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. You may need to supply runtime policy
 - error: (0, 0) Assembly `FakeLib, Version=4.7.1.0, Culture=neutral, PublicKeyToken=null' references `FSharp.Core, Version=4.4.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' which has a higher version number than imported assembly `FSharp.Core, Version=4.3.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'

System.Exception: Generating HTML failed.
  at FSharp.Literate.RazorRender.handleCompile[String] (System.String source, Microsoft.FSharp.Core.FSharpFunc`2 f) [0x00000] in <filename unknown>:0 
  at FSharp.Literate.RazorRender.ProcessFileModel[IDictionary`2] (System.Type modelType, System.Object model, Microsoft.FSharp.Core.FSharpOption`1 properties) [0x00000] in <filename unknown>:0
...

But when building on Windows or on Linux, everything runs fine. So, it seems to be a mono version dependency issue? On mac os x I am running mono version 4.0.5. The build that passes on Travis uses mono 4.0.4.

[7sharp9]

Its not a particular mono version I get the same with F# master at https://github.com/fsharp/fsharp

[tpetricek]

I'm still not sure what's happening here.

One thing I figured out recently is that when running things via FAKE, we always get the version of FSharp.Compiler.Service that comes with FAKE (and is used to invoke the script that then calls F# Formatting which then uses F.C.S.).

This is a bit problematic - in practice, it means we need to keep in sync with FAKE (in some way) - though this is not really a package dependency in one or the other direction. The most recent releases are synced though, so should work, I think...

But that does not answer the issues with FSharp.Core versions.

[matthid]

There are really 2 use cases for FSharp.Formatting

1. Library: This means downstream is responsible for either providing a compatible FSharp.Core or otherwise making sure one is available (we always prefer a bundled FSharp.Core).
2. In build scripts: Since we upgraded to F#4 I think the initial issue from @dsyme is no longer valid (please correct me if I'm wrong). It might come back with the next F# update. I hope I can reproduce it then...
   However another issue appeared since upgrading:
   Because we call "fsharpi generate.fsx" we will run with a older version of FSharp.Core on systems where F#3 or F#3.1 is installed. This might very well be the reason for the various issues popping up (but I could not reproduce the reported issues instead I got another one).

@7sharp9 @halcwb Can you try to use the following patch (fsprojects/ProjectScaffold#217) in your build scripts and report if that fixes the issues you are seeing?

Let's continue discussing on #356 as I think the initially reported bug is no longer valid (until the next F# upgrade) and sadly I could never reproduce it...

[7sharp9]

Ill try out the projectscaffold fix although one of my problems was I could not build Fsharp.Formatting, has that now being addressed?

[matthid]

Yes FSF has the same problem in its build script, will update as soon as it is sure that this in fact fixes the problem. Could you test?

[7sharp9]

See fsprojects/ProjectScaffold#217 (comment)