Benchmarking OCaml Json Libraries

According to opam OCaml has 2 popular libraries for parsing json:

At the time of writing, Yojson is actually the 6th most popular OCaml library overall!

All things being equal we’d like to use the fastest json library. Unfortunately for us, things aren’t equal. In fact, Jsonm strives to be extremely minimalist and low level and the API that it provides non-blocking encoding and decoding.

Therefore, we will instead be benchmarking Yojson against Ezjsonm. Ezjsonm is a wrapper library designed provide a user friendly API on top of Jsonm. With this wrapper, serialization/deserialization code looks almost identical with both libraries. In my benchmarks I test the speed in which I read/write the following datatype:

module Event_type = struct
  type t =
    | Login
    | Purchase
    | Logout
    | Cancel
end
type event = {
  username: string;
  date: int;
  event_type: Event_type.t;
  payload: string;
}

This is a simple data type, however it’s representative of at least some of the applications json is used for.

Results

In my benchmarks I’ve opted for 2 simple tests:

  • How fast can a list of a 1000 event types be transformed to a big json list written to an array

  • How fast can a json string of a 1000 events be transformed into event list

The tests are done using the excellent Core_bench library and here is the “pretty” output for the first test:

 ─────────────────────────┬──────────┬────────────┬──────────┬──────────┬────────────┐
│ Name                    │ Time/Run │    mWd/Run │ mjWd/Run │ Prom/Run │ Percentage │
├─────────────────────────┼──────────┼────────────┼──────────┼──────────┼────────────┤
│ ezjsonm write           │   2.67ms │ 1_668.16kw │ 124.34kw │  41.75kw │    100.00% │
│ yojson write            │   1.24ms │    61.27kw │  86.60kw │   4.27kw │     46.35% │
└─────────────────────────┴──────────┴────────────┴──────────┴──────────┴────────────┘

For the second test:

┌──────────────┬──────────┬────────────┬──────────┬──────────┬────────────┐
│ Name         │ Time/Run │    mWd/Run │ mjWd/Run │ Prom/Run │ Percentage │
├──────────────┼──────────┼────────────┼──────────┼──────────┼────────────┤
│ ezjsonm read │   5.47ms │ 1_857.55kw │  67.00kw │  67.00kw │    100.00% │
│ yojson read  │   2.04ms │   155.36kw │  39.14kw │  22.10kw │     37.30% │
└──────────────┴──────────┴────────────┴──────────┴──────────┴────────────┘

The verdict is: Yojson is at least twice as fast as Ezjsonm in both benchmarks.

Of course, recall that benchmarks often lie (on behalf of their author) and it’s entirely possible that this difference can be attributed to my code. Therefore, I welcome you to go over my code here and draw your own conclusions.

Even in the case where I did not make a mistake in benchmarking, don’t let me turn you off from using Jsonm, which allows you to write json parsers at a very low level. I’m certain that a specialized reader/writer in Jsonm will easily beat Yojson.

Future work

This set of tests is not enough to draw any conclusions just yet. Here’s what I’d like to do in the future:

  • OCaml has more implementations of the json serialization format. They are uncommonly used, however it would still be nice to include them: yajl, tiny_json.

  • Test serializers/deserializer that are automatically generated from type definitions: atdgen, cow, deriving-yojson

  • See how well a hand rolled Jsonm reader/writer fairs.

  • Add more tests for different use cases, data types, write to channels, not just strings.

Document Search Using Cosine Similarity In Haskell Using Linear   Omegle clone in Flask + Gevent + WebSockets

Comments

comments powered by Disqus