lfe-rebar3/compile updates

Eric Bailey

Written on 11 February, 2016
Updated on 18 December, 2023
Tags: lfe, lisp, erlang, beam, rebar3, tooling, open-source

Since much of my time these days is spent (or 'writing 'developing 'evangelizing) LFE, it should come as no surprise that I use the rebar3 compiler plugin extensively. In doing so, I noticed it always recompiles every .lfe file it finds, irrespective of the staleness of the corresponding .beam file. Having compiled a fair amount of vanilla Erlang via rebar3, I knew it didn't have to be that way.

Replacing run/4 with run/7

To further my quest, Tristan pointed me to run/7, specifically its use of [check_last_mod]. Before I forked lfe-rebar3/compile, it was using run/4, which does NOT make use of [check_last_mod], hence the unnecessary recompilation.

lfe-rebar3/compile/src/lr3_comp.erl:74 

rebar_base_compiler:run(Opts, [], Files, DoCompile).

After some experimentation, and _checkouts symlinking tricks, I came up with an appropriate run/7 incantation and made a pull request.

quasiquoting/lfe-compile/src/lr3_comp.erl:37-39 

rebar_base_compiler:run(Config, FirstFiles,
                        SourceDir, ".lfe", TargetDir, ".beam",
                        fun compile/3).

With those changes, rebar3 lfe compile will now skip unmodified .lfe files. Not only is this The Right Thing™ to do, but it should be time saver in the long run.

Examples of time saving

Here's an example of the difference when compiling Lodox…

… using run/4 via 0.2.1: 

$ time rebar3 lfe compile

 ~~>    Compiling ./src/lodox-html-writer.lfe ...
 ~~>    Compiling ./src/lodox-p.lfe ...
 ~~>    Compiling ./src/lodox-parse.lfe ...
 ~~>    Compiling ./src/lodox-util.lfe ...
 ~~>    Compiling ./src/lodox.lfe ...
 ~~>    Compiling ./src/pandoc.lfe ...
 ~~>    Compiling ./test/unit-lodox-tests.lfe ...
        1.39 real         1.14 user         0.24 sys

… using run/7 via 0.3.0: 

$ time rebar3 lfe compile

        0.94 real         0.73 user         0.20 sys

A little more 🐄💩 microbenchmarking on a proprietary work project: 

# run/4
        3.58 real         3.02 user         0.58 sys

# run/7
        1.83 real         1.46 user         0.35 sys