Embedding F# Compiler

UPDATE: If you are looking for a ready to use solution please check out my fsc-host NuGet package. Also I wrote a blog post about it.

Embedded scripting is one of the ways of providing flexibility to compiled applications. It is not free from problems (performance penalty, security issues, additional tooling is required). But if your goal is a ceremony-free flexibility for the end-user then scripting is a very robust way1 . F# comes with great capabilities in this area. I want to show you how you can extend your otherwise statically compiled program with scripting.

We’ll try to load, parse, and dynamically compile an fsx file and then consume the compiled code in the hosting application. Let’s start with creating a new F# console app and add the FSharp.Compiler.Service package which contains all the necessary compiler APIs.

The core type for compilation is FSharpChecker and we can obtain an instance with FSharpChecker.Create(). We want to compile a script and load it as an assembly so our app can make use of it. The suitable method is CompileToDynamicAssembly. It takes a list of compiler options that get passed to fsc 2. The output is a tuple of compilation messages (diagnostics), and an assembly if compilation is successful.

let scriptPath = "./path/to/your/script.fsx"

let checker = FSharpChecker.Create()
let compilerArgs = [|
  sprintf "--reference:%s" (Assembly.GetEntryAssembly().GetName().Name)

let errors, _, maybeAssembly =
  checker.CompileToDynamicAssembly(compilerArgs, None) |> Async.RunSynchronously

Fig. 1. Compile a script using a preview F# version into a module with the netcore profile and include the host application assembly as a reference.

Given a successfully compiled assembly let’s extract and execute a part of our script. The script looks like the below:

namespace This.Is.A.Namespace

module HelloHost =
  let original (x:string) = async {
    printfn "Invoked by: %s" x
  let myFunc = original

Fig. 2. An example script to be compiled

I am aiming to extract and invoke myFunc. The easiest way to point to it is to use its fully-qualified name: This.Is.A.Namespace.HelloHost.myFunc. We want to invoke the function statically so ideally it should be accessible as string -> Async<unit>. To achieve that we can retrieve myFunc’s PropertyInfo (with Public and Static bindings flags), invoke GetValue(null) method and dynamically down-cast :?> the value to string -> Async<unit>. Voila!

let extract name assembly : Result<'a,Error> =

  let (fqTypeName, memberName) =
    let splitIndex = name.LastIndexOf(".")
    name.[0..splitIndex - 1], name.[splitIndex + 1..]

  let candidates =
    assembly.GetTypes() |> Seq.where (fun t -> t.FullName = fqTypeName) |> Seq.toList

  match candidates with
  | [t] ->
    match t.GetProperty(memberName, BindingFlags.Static ||| BindingFlags.Public) with
    | null -> Error ScriptsPropertyNotFound
    | p ->
        Ok (p.GetValue(null) :?> 'a)
      | :? System.InvalidCastException -> Error ScriptsPropertyHasInvalidType

  | [] -> Error ExpectedMemberParentTypeNotFound
  | _ -> Error MultipleMemberParentTypeCandidatesFound

let result = assembly |> extract<string -> Async<unit>> "This.Is.A.Namespace.HelloHost.myFunc"

Fig. 3. Retrieving property value.

Inferring what F# code compiles into may by a non-trivial task at times. You may be wondering why I am not trying to access the original function. It’s because F# functions get compiled into MethodInfo which do not easily downcast to F# functions. By creating another let binding it gets transformed into a function value that gets compiled into PropertyInfo. We can retrieve its value and cast it to the desired function type.

If your script contains a module and a type with the same name on the same nesting level then the compiled module gets the Module suffix. Example:

type HelloHost = {

module HelloHost =
  let myFunc = ... // The path would be 'This.Is.A.Namespace.HelloHostModule.myFunc'

All the types used in the extracted member signature, (being de facto a public interface the script must implement), must be statically-known to the host application. It means the types should be either declared in the host application or in a referenced NuGet package. They cannot be declared in the script itself.

As you can see extending F# apps with scripting is not too difficult. However, I didn’t cover one important aspect. Referencing NuGet packages with the #r directive requires a bit more work which I’ll try to cover in the next post. The full source code for this post is available here. If you want to learn more about F# Compiler Services read the docs.

  1. There are many ways of providing flexible behaviour to compiled tools. Mature tools usually come with great configuration capabilities (usually using YAML or TOML). Other come with a plugin architecture. Or both. It is a great deal of work to create a decent configurability. Not only from the technology point of view. It’s rather the inability to predict what the users may want to configure. Plugins leave the implementation to the user. However, they’re are on the heavy side of things. They usually need to be compiled with the matching SDK and packaged in a digestible form. Every change comes with that cost. It makes the iterations slow. ↩︎

  2. The host application must have access to the F# compiler fsc binary. The easiest way is to package the host app as a Docker image basing on one of the .NET SDK Docker images. Alternatively one may try using the FSharp.Compiler.Tools but it doesn’t seem to be actively maintained any more. ↩︎