FAKE Dotnetcore

[matthid]

TLDR: Lets combine FAKE and Paket to make the head-start awesome in any environment and remove all dependencies.

TODO , see #1523

---

Old Post (the thing is now real, see #1281)

Why?

This are just some thoughts on how we could widen the F# usage, even in projects outside of the regular .NET scope and across platforms. This issue is to discuss various ideas, get feedback and maybe even start a little POC if the thing does make any kind of sense to somebody :)

The basic idea is to be able to use FAKE everywhere, while this is possible just right now there is a lot of overhead:

• paket.dependencies
• paket.lock
• build.fsx
• paket bootstrapper to restore paket to restore FAKE
• paket knowledge
• you need mono (on unix)

You might use nuget.exe but with nuget.exe it's a lot more frustrating in the long run.
Its especially painful because the project we are speaking about might already have some package resolution in place so paket.exe might be quite some overhead on top of that.

Proposal

I propose to make the following possible:

• build.fsx
• fake-bootstrapper (whatever that is)

and the following "extensions" to .fsx (maybe that would make that a .fake):

#paket "Fake.IO"
#paket "MyBuildLibrary"
open Fake.IO
open MyBuildLibrary

// ... continue regular build script as we know today

• The fake-bootstrapper would then download a special version of fake build with coreclr and ilmerged with paket.core. Ideally a single binary running on the current platform
• We would then resolve/install the packages given in the script (via #paket or some other syntax - potentially one to still be a valid .fsx file)
• We would then replace the #paket lines with a #load of the packages include scripts (see Implement a loading script generator. Paket#1619)
• Finally FAKE would run the targets as today and potentially cache the script.

Thoughts

This would give us the possibility to completely clean up FAKE and reduce it to its core (everything else can be added easily via nuget packages). I'm not sure if there is any migration path.

In essence this idea would combine FAKE and Paket. We could get back to our well known infrastructure by adding additional commands like #enablePaketDependencies which would make sure that an existing paket.dependencies is restored before running the script.

It would be a good way to use F# in some alien environments - if we can make it "just work".

Dependencies

Finally we'd need the following:

• FSharp.Compiler.Service -> CoreCLR - for FAKE (done)
• Argu -> CoreCLR - for Paket and FAKE (fsc overload resolution failure in .NET Core (coreclr) dotnet/fsharp#983)
• Mono.Cecil -> CoreCLR - for Paket and FAKE
• FAKE -> CoreCLR (probably easy after above is done)
• Chessie -> CoreCLR - for Paket (done)
• Newtonsoft.Json -> CoreCLR - for Paket (done)
• Paket -> CoreCLR (?)
• figure out what "fake-bootstrapper" can and should be,
  • currently I'm thinking about a bash and powershell (i really hate to suggest this) script downloading and running the "correct" version. Problem here is that I don't want to depend on mono, and ideally not even on a installed dotnet cli.
  • @enricosada suggested using a project.json to bootstrap (but then dotnet needs to be installed).

Roadmap

Moved to #1281

What do you think, would that actually improve anything? Is it feasible?

/cc @forki @tpetricek @enricosada

[enricosada]

Chessie -> CoreCLR - for Paket (?)

Is already, fsprojects/Chessie#43 is merged. dunno about the package

[enricosada]

figure out what "fake-bootstrapper" can and should be, currently I'm thinking about a bash and powershell (i really hate to suggest this) script downloading and running the "correct" version. Problem here is that I don't want to depend on mono, and ideally not even on a installed dotnet cli.

Two possibilities (i prefer the cli command)

1) make a dotnet cli command

use dotnet cli command, so you can write dotnet fake.
dotnet cli will download, restore all deps and run fake. And that work xplat. ref docs

It's just a console app. and inside a project.json there is a "tool" { "dotnet-fake": "1.*" }

also if paket is a dotnet too, you can dotnet paket, and in tools you have both paket and fake. And all the needed command by fake too. lot of command are rewritten like that (all asp.net / ef helpers, etc)

2) publish a different package for each platform, and download+ run it

or use dotnet cli to publish a self contained app, so one each platform. so it's easy as curl + unzip and exec. ref doc
no need for mono or other deps

[smoothdeveloper]

I feel #paket "PackageName" is as creepy as #nuget "PackageName" which was discussed on the compiler.

Otherwise there is food for thought on how to consolidate the very useful tools that Paket and FAKE are toward better user experience / easy cross platform support.

For all the paket stuff, it feels like the FAKE script should just reference the tools exposed in Paket.Core (albeit the API is not to be considered stable) rather than introducing some special dialect of what is a F# script.

[matthid]

Thanks for the feedback, keep it coming!

or use dotnet cli to publish a self contained app, so one each platform. so it's easy as curl + unzip and exec

Yes I imagined using something like this. Like I said, ideally I want to avoid a dependency on dotnet cli on the target platform running this special version of FAKE. So I would go with the self contained app.

I feel #paket "PackageName" is as creepy as #nuget "PackageName" which was discussed on the compiler.

Note that I'm not proposing to add nuget support or anything like that to the F# compiler / fsi. We can change how it would look, but I would force all packages to be listed on top of the file to clearly separate them from the rest of the script.

Something like

(* package "Fake.IO "*)
(* package "MyBuildLibrary" *)

would work as well. I just want it to be part of the script file to simplify the entry level (like I said on non .net projects). A nice side effect would be that we would no longer require a build group in paket.dependencies.

[smoothdeveloper]

Maybe something which still does the same if called from FSI?

#r "fake.exe"
Paket.Install("""
Fake.IO
MyBuildLibrary
""");;

[matthid]

Problem with that is that the paket command should run before executing the script apparently

[smoothdeveloper]

I mean if FAKE contains Paket, I'd expect the above to work as expected. Am I misunderstanding something?

[matthid]

Yes it probably would but

#r "fake.exe"
Paket.Install("""
Fake.IO
MyBuildLibrary
""");;
#r "packages/Fake.IO/lib/net40/Fake.IO.dll"
open Fake.IO

will probably not work as expected. Especially considering FAKE caching and things like that.

[matthid]

@enricosada I think we can actually do both of your proposals. Provide standalone packages (to be downloaded and executed via scripts) or let the user use a project.json to handle the bootstrapping.

It could continue the same way from there...

[matthid]

Okay I'm making some progress with this. However I'm now stuck with:

$ dotnet run -f net46 -- --help
Project FakeLib.netcore (.NETFramework,Version=v4.6) was previously compiled. Skipping compilation.
Project Fake.netcore (.NETFramework,Version=v4.6) was previously compiled. Skipping compilation.
 1.0.0-alpha1
ERROR(S):
  Verb
'C:\PROJ\FAKE\src\app\Fake.netcore\bin\Debug\net46\win7-x64\Fake.netcore.exe'
is not recognized.
  --help       Display this help screen.
  --version    Display version information.

dotnet run without parameter fails as well:

$ dotnet run
Couldn't find target to run. Possible causes:
1. No project.lock.json file or restore failed - run `dotnet restore`
2. project.lock.json has multiple targets none of which is in default list (DNXCore, .NETStandardApp)

@enricosada Any Idea whats going on?

To reproduce checkout my branch:

cd src/app/FakeLib.netcore
dotnet pack
cd ../Fake.netcore
dotnet run

[enricosada]

@matthid i'll check tonight the branch

To get more info, you can run all dotnet commands with --verbose before the command like

dotnet --verbose run

to see more info

[smoothdeveloper]

@matthid, I haven't yet tried out your branch (nor dotnet core), but I believe ultimately you'd expect to have a fake.netcore.dll and rename conhost.exe (which is dotnet core console application "host") into fake.netcore.exe.

In visualfsharp repository, a build step performs similar renaming for fsc.exe and fsi.exe.

conhost.exe is supposed to find a .dll whose name is same as the exe (assuming you renamed conhost.exe into fake.netcore.exe, it will look for fake.netcore.dll and run this).

Hope this helps

[enricosada]

@smoothdeveloper doing the rename is wrong, the visualfsharp repo do that beacuse doesnt use dotnet sdk (the new name of dotnet cli) and the project.json. Doing that doesnt copy references etc.

if you want a clean xcopy executable (with all the references) you should use dotnet publish, it already do that

also an exe should depends on netcore.app package, not directly netstandard.library

[smoothdeveloper]

@enricosada thanks for the precisions!

[isaacabraham]

Note that if dotnetcli is extensible we should be able to hook into dot net build and restore etc for these features.

[matthid]

@isaacabraham Yeah definitely looking forward to the docs of those extension points. Is there a list somewhere?

I definitely had this in mind from the beginning. But first step is to make the code run :)

[matthid]

@enricosada thanks for the tip. Will try it as soon as possible and report back if it gets me somewhere...

[isaacabraham]

@matthid Nope, I think with everything happening now it's all in a state of flux, but in my personal opinion this will probably happen.

[matthid]

I managed to get it working with the preview 1 it seems like the old version was giving some additional parameters to the started application.

I'm now trying to run with netstandard1.5 (net46 is working as expected now) and get:

$ dotnet --verbose run -f netstandard1.5
Telemetry is: Enabled
Project FakeLib.netcore (.NETStandard,Version=v1.5) was previously compiled. Skipping compilation.
Project Fake.netcore (.NETStandard,Version=v1.5) was previously compiled. Skipping compilation.
Running C:\Program Files\dotnet\dotnet.exe exec --additionalprobingpath C:\Users\dragon\.nuget\packages C:\PROJ\FAKE\src\app\Fake.netcore\bin\Debug\netstandard1.5\Fake.netcore.dll
Process ID: 22840
Unhandled Exception: System.IO.FileLoadException: Could not load file or assembly 'FSharp.Core, Version=4.4.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
   at Program.main(String[] args)

@enricosada FYI (maybe for some future docs) I got a weird nullreferenceexception which disappeared after converting project.json of the dependency (fakelib) to the new layout.
What warning DOTNET1015: The 'compileFiles' option is deprecated. Use 'compile' in 'buildOptions' instead. doesn't say is that you need an additional includeFiles element :)

$ dotnet --verbose run -f net46
Telemetry is: Enabled
Project FakeLib.netcore (.NETFramework,Version=v4.6) will be compiled because Input items removed from last build
        C:/PROJ/FAKE/src/app/FakeLib/AssemblyInfo.fs
        C:/PROJ/FAKE/paket-files/matthid/Yaaf.FSharp.Scripting/src/source/Yaaf.FSharp.Scripting/YaafFSharpScripting.fs
        C:/PROJ/FAKE/src/app/FakeLib/EnvironmentHelper.fs
        C:/PROJ/FAKE/src/app/FakeLib/StringHelper.fs
        C:/PROJ/FAKE/src/app/FakeLib/BuildServerHelper.fs
        C:/PROJ/FAKE/src/app/FakeLib/TraceListener.fs
        C:/PROJ/FAKE/src/app/FakeLib/TraceHelper.fs
        C:/PROJ/FAKE/src/app/FakeLib/FSIHelper.fs
Compiling FakeLib.netcore for .NETFramework,Version=v4.6
Running C:\Program Files\dotnet\dotnet.exe compile-fsc @C:\PROJ\FAKE\src\app\FakeLib.netcore\obj\Debug\net46\dotnet-compile.rsp
Process ID: 15804
Unhandled Exception: System.NullReferenceException: Object reference not set to an instance of an object.
   at Microsoft.DotNet.Cli.Utils.ProjectToolsCommandResolver.ResolveCommandSpecFromToolLibrary(LibraryRange toolLibraryRange, String commandName, IEnumerable`1 args, ProjectContext projectContext)
   at Microsoft.DotNet.Cli.Utils.ProjectToolsCommandResolver.ResolveCommandSpecFromAllToolLibraries(IEnumerable`1 toolsLibraries, String commandName, IEnumerable`1 args, ProjectContext projectContext)
   at Microsoft.DotNet.Cli.Utils.ProjectToolsCommandResolver.ResolveFromProjectTools(String commandName, IEnumerable`1 args, String projectDirectory)
   at Microsoft.DotNet.Cli.Utils.ProjectToolsCommandResolver.Resolve(CommandResolverArguments commandResolverArguments)
   at Microsoft.DotNet.Cli.Utils.CompositeCommandResolver.Resolve(CommandResolverArguments commandResolverArguments)
   at Microsoft.DotNet.Cli.Utils.CommandResolver.TryResolveCommandSpec(String commandName, IEnumerable`1 args, NuGetFramework framework, String configuration, String outputPath)
   at Microsoft.DotNet.Cli.Utils.Command.Create(String commandName, IEnumerable`1 args, NuGetFramework framework, String configuration, String outputPath)
   at Microsoft.DotNet.Cli.Program.ProcessArgs(String[] args, ITelemetry telemetryClient)
   at Microsoft.DotNet.Cli.Program.Main(String[] args)
C:\Program Files\dotnet\dotnet.exe compile-fsc @C:\PROJ\FAKE\src\app\FakeLib.netcore\obj\Debug\net46\dotnet-compile.rsp returned Exit Code -1073741819

[enricosada]

See https://github.com/dotnet/netcorecli-fsc/ sample and wiki.
For issues join #dotnetcore channel in fsharp slack.
I'll try to checkout this or this weekend

[matthid]

Moving implementation and technical problems to #1281 . Such that people can leave in-line comments and focus here on the discussion of usefulness/design.

@enricosada Thanks for the link, I tried some things but couldn't figure out the problem. I did send a request for the fsharp slack channel :).

[alexpantyukhin]

Hi. How is progress of this task?

[matthid]

I hope I can push this further after chrismas... currently I'm thinking about moving some of this functionality to paket and generalizing this concept even further...

[ErikSchierboom]

Any updates on this issue?

[matthid]

sadly not :( The good thing is that netcore has some time to mature ...

[ErikSchierboom]

True. Let's hope everything will be mature enough soon :)

[matthid]

real alpha is coming soon, for now we have https://chocolatey.org/packages/fake and the https://ci.appveyor.com/nuget/fake nuget feed. So I'm closing this.