Configurator Reborn¶
With dune 1.0 around the corner, there’s a pressing need to create some hype raise awareness of the upcoming features. In this post, I’d like to talk about one such transition: the move to dune’s own configuration kit called configurator. Since the library itself is still fairly new, I’ll first introduce configurator itself and show what problems it attempts to solve.
What is Configurator?¶
Before I describe what configurator is, I’ll briefly motivate the need for configuring builds in OCaml projects. In this context, configuration often means conditionally selecting flags, or generating sources based on the environment where the project is build. Here are some examples where this is useful:
Bindings to C libraries should use compilation flags that reflect where the C library is installed. Often this means obtaining such flags using the pkg-config utility.
Detecting system capabilities by compiling C code and checking the variables
ocamlc -config.Extracting C constants and generating header files and other source files.
Configurator is a library that provides simple, portable, and cross compilation aware solution to these problems. Using configurator is fairly simple: use it as a library to create robust scripts that will generate build flags and other sources.
Configurator Today (jbuilder)¶
Configurator was initially created by janestreet to help package some of their
own libraries. It enjoyed quick adoption by the community because of its good
support for pkg-config, support for accessing ocamlc -config variables, and
an API for obtaining #define constants, and generating header files.
The interaction between configurator and dune is generally as follows:
Write a configuration script discover.ml. I’ll use an example taken from
async_ssl. This is a configuration script that adds support for building the
bindings against a pkg-config installed openssl. It also handles cases where
pkg-config is absent or openssl isn’t in its search path by defaulting to some
common sense flags.
open Base
open Stdio
module C = Configurator
let write_sexp fn sexp =
Out_channel.write_all fn ~data:(Sexp.to_string sexp)
let () =
C.main ~name:"async_ssl" (fun c ->
let default : C.Pkg_config.package_conf =
{ libs = ["-lssl"; "-lcrypto"]
; cflags = []
}
in
let conf =
match C.Pkg_config.get c with
| None -> default
| Some pc ->
Option.value (C.Pkg_config.query pc ~package:"openssl") ~default
in
write_sexp "openssl-cclib.sexp" [%sexp (conf.libs : string list)];
write_sexp "openssl-ccopt.sexp" [%sexp (conf.cflags : string list)];
Out_channel.write_all "openssl-cclib" ~data:(String.concat conf.libs ~sep:" ");
Out_channel.write_all "openssl-ccopt" ~data:(String.concat conf.cflags ~sep:" "))
The above script is the source of a binary that will produce
openssl-cclib.sexp, openssl-ccopt.sexp, openssl-cclib,
openssl-ccopt when executed. We just need to make dune aware of the targets
that this script will generate via a custom rule:
(rule
((targets (openssl-ccopt.sexp openssl-cclib.sexp openssl-ccopt openssl-cclib))
(deps (config/discover.exe))
(action (run ${<} -ocamlc ${OCAMLC}))))
Finally, we may now tell dune it must read a file to obtain the falgs for the
c_library_flags field:
;; I've removed some fields here to keep things brief
(library
((name async_ssl_bindings)
(public_name async_ssl.bindings)
(c_names (openssl_helpers))
(c_flags ((:standard \ -Werror -pedantic -Wall -Wunused)))
(c_library_flags (:standard (:include openssl-cclib.sexp)))
(libraries (ctypes.stubs ctypes ctypes.foreign.threaded))))
dune will now handle building the configuration script and running it to produce
the flag files for building the async_ssl.bindings library.
Configurator Tomorrow (dune 1.0)¶
While configurator works quite well in its current form, for dune 1.0 we’ve
decided to make some important changes to configurator and how it interacts with
dune. The biggest change is that we’ve decided to bundle configurator with dune
itself (available as the dune.configurator package). This means that the
independent configurator is now deprecated, and users are encouraged to
transition their projects to use the new one once dune 1.0 is released.
Transitioning to the new configurator is documented in the manual, so I will not reproduce that information here. Instead, I will list the main reasons that should persuade users to make the switch. Even though bundling alone has numerous benefits, there’s a few other important benefits that should compel users to upgrade. However, we will not force anyone’s hand here. The old configurator still works just the same and will continue to work. Users are encouraged to upgrade only at their own convenience.
Some users have rightfully raised some concerns using an external library for configuration - especially when this library had its own set of dependencies. Truthfully, even though Configurator’s dependency profile was fairly slim, it was hurting its adoption. Since the new configurator is now bundled with dune, it doesn’t have any external dependencies (aside from dune itself). We hope that that this will lower the barrier for using configurator even for simple configuration scripts.
Another neat little improvement in usability is that the path to the ocamlc
of the current context is now passed automatically. Passing it through
-ocamlc is no longer necessary. We felt that it was too error prone to have
users to remember to pass this.
dune is committed to providing a cross compilation story for OCaml projects. Therefore, we wanted to release a configuration tool that doesn’t break in cross compilation contexts. The old configurator relied on building and running C code to query things like C constants which doesn’t work when cross compiling. The new configurator is smarter and relies on tricks picked up from ctypes to avoid running binaries for these queries (credit goes to whitequark)
The configurator API is now versioned. The dune team feels that the current
configurator API can be improved, but given that many people already rely on the
current API, we will not make any breaking changes to it. We think that the
backwards compatibility afforded to our users here will have many benefits down
the line. For example, the ability to create dune workspaces by just symlinking
directories together can only work if dune supports building the widest set of
packages simultaneously. This versioning technique can be quite a useful tool
that I think is underused today. I hope to blog about its virtues another time.
In practice, this means that users will need to select a particular version of
the API in their scripts. At the time of writing, Configurator.V1 is the
only available version.
What’s Next?¶
Truthfully, we don’t have any concrete plans for Configurator. We’re hoping that making this library as part of dune will make it much more widely used and encourage users to report their experiences with it. If you have a library with configuration needs that aren’t covered by configurator, get in touch on discuss or dune’s bugtracker.