Publishing an OPAM Package - a Checklist

The process of publishing an opam package has come a long way from its modest beginnings. Nevertheless the opam team deserves praise for choosing an extremely simple and flexible model for contribution - the git commit. To me that explains how it aged gracefully with improvements such as:

  • Travis CI for every commit. With the number of platforms tested is slowly growing (OSX has been added recently).

  • A lint tool that points out common errors or omissions in your opam metadata. This lint tools run against every commit in ocaml/opam-repository as part of the CI so you don’t even have to run it yourself.

  • A utility that automates the boring publishing steps for you. Steps such as verifying the checksum for your distribution and writing the actual commit message. opam-publish.

Nevertheless, there’s still a few blind spots that the current tooling easily overlooks. The purpose of this post is to list some of the common problems with the hopes that more automated ways of dealing with them will come up, and also to raise awareness among current and potential opam package authors.

Debugging information -g

Let’s quote the man page for ocamlc for this:

Add debugging information while compiling and linking. This option is required
in order to be able to debug the program with ocamldebug(1) and to produce
stack  back- traces when the program terminates on an uncaught exception.

Which means that if you omit this, users of your library will suffer from bad stacktraces. Don’t be that guy! Just add -g to your compiler invocation or debug: true to your _tags file if you’re using ocamlbuild.

Omitting -bin-annot

Citing TFM again:

Dump detailed information about the compilation (types, bindings, tail-calls,
etc) in binary format. The information for file src.ml is put into file
src.cmt.  In case of  a type error, dump all the information inferred by the
type-checker before the error.  The annotation files produced by -bin-annot
contain more information and are much more compact than the files produced by
-annot.

The presence of .cmt and .cmti files is essential for the functionality of common OCaml tooling. For example, the absence of these files will cause .merlin to fail looking up definitions in external packages. To generate these just add true: bin_annot to your _tags for ocamlbuild or -bin-annot to your compiler flags.

Not installing .mli, .cmt, cmti files

Generating .cmt and .cmti is fine and all but please make sure that these are being copied appropriately as part of your installation step. This is only a separate point because I’d also like to remind you to install .mli files as well. These are very useful as this is where merlin will land you after you go to definition or navigate to a module. Finally, some packages also install the sources directly. I think the jury is still out on whether this a good practice or not so for now I’ll abstain from recommending it. I am leaning in favour of it however.

Work for byte code only environments

Some systems do not have ocamlopt available, but luckily, opam makes it possible for us to be considerate towards such systems. You can easily test for the availability of ocamlopt with the {ocaml-native} variable.

Here’s an example where this is put to good use: https://github.com/ocaml/opam-repository/pull/5231

Generating (and installing) .cmxs files

.cmxs objects are necessary for your package to be used through Dynlink in native code. Make sure these are generated using the -shared option:

-shared

Build a plugin (usually .cmxs) that can be dynamically loaded with the  Dyn-
link module. The name of the plugin must be set with the -o option. A plugin
can include a number of  OCaml  modules  and  libraries,  and  extra  native
objects  (.o, .a files).  Building native plugins is only supported for some
operating system. Under some systems (currently, only Linux AMD 64), all the
OCaml code linked in a plugin must have been compiled without the -nodynlink
flag. Some constraints might also apply to the way the extra native  objects
have  been  compiled  (under  Linux AMD 64, they must contain only position-
independent code).

If you’re using ocamlbuild then building an appropriate .mldylib will generate the necessary artifacts for you.

Riffing off my last tip, you should also make sure that native dynlink is in fact available. If you’re using makefiles a useful trick I’ve picked up from ppx_tools for doing that is:

include $(shell ocamlc -where)/Makefile.config

ifeq ($(NATDYNLINK),true)
all: $(YOUR_PACKAGE).cmxs
endif

This Makefile.config has other goodies as well to make your packages portable. Such as the ARCH variable which is set to none for bytecode only systems. Platform specific object extensions such EXT_DLL and EXE.

That finishes my 4 common opam publishing mistakes, and yet I’m sure I’ve forgotten to mention some important points myself. To me that hints at the necessity of more automation around this process. Hopefully in the future, less diligence will be required of maintainers.

Happy OCaml programming.

Comments

comments powered by Disqus