Add optional precompiling of build script

[leonidborisenko]

I've tried to precompile build script into executable and found that running executable saves me about 5-7 seconds. Build script isn't being changed often, so i'd like to see option --precompile-build-script=<path_to_executable> with approximately following behavior (in some pseudocode):

if (not dirname(path_to_executable).exists()) {
  mkdir_p(path_to_executable)
}
if (not path_to_executable.exists()) {
  exec("fsharpc", [buildScriptPath, "--out:"+path_to_executable])
}
if (modification_time(path_to_executable) < modification_time(buildScriptPath)) {
  exec("fsharpc", [buildScriptPath, "--out:"+path_to_executable])
}

setPassedFakeOptionsInExecEnvironment()

try {
  exec(path_to_executable)
} catch (CanNotExecute) {
  exec("mono", [path])
}

---

Right now I'm using external script for precompiling.

It's necessary to use #I in build script to let fsharpc find FAKE libs without passing any option to compiler:

#I @"packages/FAKE/tools"
#r @"FakeLib.dll"

Then this script should work (create it in the same dir as build script named build.fsx, compiled executable is placed in build/ subdirectory):

#!/bin/sh

THIS_SCRIPT_DIR="$(realpath "$(dirname "$0")")"

# Define invocation of command binaries.
FSHARPC="${FAKE_FSHARPC:-fsharpc}"
MONO="${FAKE_MONO:-mono}"
STAT_MOD_TIME="stat --format=%Y"

# Define path to build script and resulting executable.
BUILD_SCRIPT="${THIS_SCRIPT_DIR}/build.fsx"
COMPILED_BUILD_SCRIPT_DIR="${THIS_SCRIPT_DIR}/build"
COMPILED_BUILD_SCRIPT="${COMPILED_BUILD_SCRIPT_DIR}/build.exe"

# Invoke mono with including FAKE libs in MONO_PATH.
mono_with_fake_libs () {
  MONO_PATH="${THIS_SCRIPT_DIR}/packages/FAKE/tools:${MONO_PATH}" \
  "${MONO}" "$@"
}

# Get modification time (in seconds) of passed path.
modtime () {
  local STAT_OUTPUT="$(${STAT_MOD_TIME} "${1}")"
  echo "${STAT_OUTPUT#=}"
}

recompile () {
  ${FSHARPC} "${BUILD_SCRIPT}" --out:"${COMPILED_BUILD_SCRIPT}"
  if [ $? -gt 0 ]; then exit 1; fi
  mono_with_fake_libs --aot -O=all,-shared "${COMPILED_BUILD_SCRIPT}"
}

# Create output directory.
[ -d "${COMPILED_BUILD_SCRIPT_DIR}" ] || mkdir "${COMPILED_BUILD_SCRIPT_DIR}"

if [ ! -f "${COMPILED_BUILD_SCRIPT}" ]; then
  recompile
elif [ "$(modtime "${BUILD_SCRIPT}")" \
       -gt "$(modtime "${COMPILED_BUILD_SCRIPT}")" ]; then
  recompile
fi

mono_with_fake_libs "${COMPILED_BUILD_SCRIPT}"

When build script includes RunTargetOrDefault "TargetName", running executable will invoke default target. FAKE options are passed to executable as environment variables:

$ target=Build build/build.exe

# env is required
# because `single-target` is not directly accepted by shell as a variable name
$ env target=RunTests single-target=true build

[forki]

Yes we thought about a precompile mode. PullRequests are very welcome.
This should be built directly into FAKE, since we also have the F# compiler available (via FCS)

[leonidborisenko]

I'm trying to determine how to make precompiled build script executable being able to load assemblies referenced in original build script when executable is executed on Microsoft .NET runtime (i.e. without possibility to use MONO_PATH). Unfortunately, I don't have access to Windows machine for experiments and don't have any prior .NET knowledge, so I need help.

After scanning web for relevant information, I found that Microsoft runtime hasn't equivalent of MONO_PATH (there is DEVPATH, but it requires editing of Machine.config in system-wide directory, so it's not user-friendly). Setting <codeBase> in application configuration file requires setting of concrete assembly version and also forbids setting path to weak-named assembly to location outside of application directory.

It seems like Microsoft runtime just doesn't provide official way for aiding it in searching of arbitrary assembly on disk without prior knowledge of assembly version and assembly type (whether it's strong-named or weak-named).

I think, it's only possible to put executable (build.exe) in the same directory as build.fsx and additionally create application configuration file (build.exe.config) with <probing> elements pointing to subdirectories found by scanning build.fsx for #r and #I directives.

Is there any other way?

[leonidborisenko]

So far, I'm continuing to collect information necessary for creating cross-platform PR (as I don't have any prior .NET ecosystem/infrastructure knowledge).

My mono (version 3.2.8) supports <probing> in application configuration file, for example:

<configuration>
   <runtime>
      <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
         <probing privatePath="packages/FAKE/tools;build/CustomFakeTask"/>
      </assemblyBinding>
   </runtime>
</configuration>

So, build.exe.config for precompiled build script could be used also on Mono instead of setting MONO_PATH.

There is Microsoft knowledge base article 837908: "How to load an assembly at runtime that is located in a folder that is not the bin folder of the application". It describes three methods: using GAC, using <codebase> in application configuration file and using the AssemblyResolve event.

However using AssemblyResolve event requires mutating of original build script code, so it brings "magic" in process and IMHO should not be considered as solution. AssemblyResolve handler also should be set as early as possible (in module initializer, with the help of something like https://github.com/einaregilsson/InjectModuleInitializer or https://github.com/fody/moduleinit, or at least in static constructor), and has some other quirks, making its' implementation convoluted.

So creating build.exe.config with <probing> is a way to go, I suppose. It means that FAKE command-line option for precompilation should not support parameter (path where resulting executable is placed/found). This option should be a flag (named -exe and --precompile-build-script).

Contents of <probing> element could be determined by looking for lines with #I and #r in build.fsx. When both several #I and several #rare defined and some #r are containing leading directories, all combinations of directories should be scanned before precompilation in search of real placement of such referenced assemblies,

Also, usage of FSharp.Compiler.Service bundled with FAKE (useful for compiling FAKE script without invoking external compiler, as forki said) is described in tutorial and reference.

[xavierzwirtz]

I have been doing experiments on saving the AssemblyBuilder that FSI generates to disk. It requires a small change to FSharp.Compiler.Service, but the implementation on the FAKE side looks to be quite clean. After running session.EvalScript we would add

    let assemBuilder = session.DynamicAssembly :?> System.Reflection.Emit.AssemblyBuilder
    assemBuilder.Save("FSI-ASSEMBLY.dll")

Then, save that to a cache location(.fake/build.dll perhaps), along with a crc32 hash of the script files used to compile. Then on next run, check if that cache assembly exists, do a fresh crc32 hash of build.fsx and any #load-ed files, and if it matches use the cache assembly.

That would eliminate any need to muck around with the compiler.

The is dependent on fsharp/fsharp-compiler-docs#365 getting accepted.