Building

Building

DAISY Pipeline's build system makes use of a variety of build tools: Maven, Gradle, Makefiles, Autotools, etc. This is a concequence of using different programming languages and third-party software. The build system tries to hide as much as possible the complexity this brings.

The system of connecting the individual components is (almost) entirely Maven based. This means that upon building a certain component, the artifact is installed in a Maven repository (which can be online or offline) to other components that want to make use of it. Dependencies are resolved and downloaded automatically and do not have to be copied explicitely.

Gradle is Maven compatible. Some components such as Liblouis are built with Autotools which is not inherently Maven compatible, but the binary artifacts are made part of the Maven system by packaging them into a special kind of JAR file (see liblouis-nar.

The "super-build" project attempts to further simplify things by wrapping the entire code base in a single Make based build script that allows you to build everything with one simple command. The script also supports building incrementally which is an indispensable feature for a project this size.

Important: the super-build script is intended to be run on MacOS or Linux. In theory it could also be run with Cygwin although some issues are definitely expected there. Contact us if you have ideas for making the script cross-platform.

For typical modifications coming from external contributors, it is enough to recompile only a small part of Pipeline, i.e. one or two components, so the super-build script is not really required in these cases.

Prerequisites

The following tools are required to build Pipeline completely from scratch. Some tools are only needed for some components or third party libraries. It is propably a good idea to start with the two essential ones, namely Java and Maven, and install more as you go.

• Java >= 8
• Maven >= 3.0.0
• Go (for CLI and updater)
• GNU Make (for super-project, assembly, Liblouis, CLI, ...)
• xmllint (for super-project)
• curl (for assembly)
• NSIS (for Windows installer)
• Ruby and Bundler (for website)
• Autotools and a C compiler (for Liblouis)

Very common tools that most of you already have were not listed. Please let us know if something is missing from this list.

Building individual components

Individual components may be built by checking out the git repository in question and following the build instructions specific to that repository, which can normally be found in a README file.

In case a component has dependencies from other repositories, those dependencies must have been built beforehand. The Maven dependency resolution system takes care of the rest. There are two kinds of Maven dependencies: released artifacts and snapshots. Released artifacts are built only once after which they are made available online. Snapshots on the other hand are development versions which can be built more than once with the same version number. This makes snapshot dependencies unstable. Snapshots are built automatically for all Pipeline components and are available online. These "official" snapshot always correspond to the "master" branch of their respective git repositories.

You can use "official" snapshots by adding the following repositories in the "profiles" element of ~/.m2/settings.xml or /etc/maven/settings.xml:

<profile>
    <id>sonatype-snapshots</id>
    <activation>
        <activeByDefault>true</activeByDefault>
    </activation>
    <repositories>
        <repository>
            <id>sonatype-snapshots</id>
            <name>OSS Sonatype repo (snapshots)</name>
            <url>https://oss.sonatype.org/content/repositories/snapshots/</url>
            <releases>
                <enabled>false</enabled>
            </releases>
            <snapshots>
                <enabled>true</enabled>
                <updatePolicy>always</updatePolicy>
                <checksumPolicy>warn</checksumPolicy>
            </snapshots>
        </repository>
    </repositories>
    <pluginRepositories>
        <pluginRepository>
            <id>sonatype-snapshots</id>
            <name>OSS Sonatype repo (snapshots)</name>
            <url>https://oss.sonatype.org/content/repositories/snapshots/</url>
            <releases>
                <enabled>false</enabled>
            </releases>
            <snapshots>
                <enabled>true</enabled>
                <updatePolicy>always</updatePolicy>
                <checksumPolicy>warn</checksumPolicy>
            </snapshots>
        </pluginRepository>
    </pluginRepositories>
</profile>

As a result, a dependency only needs to be built locally when local changes have been made to the source code, or if the required version does not match what is currently on the master branch.

To get an overview of all the different Pipeline repositories and what they contain, refer to the page about the source code.

The order in which individual components need to be built might not always be immediately clear. This is one of the reasons for having the super-build.

Super-build

The super-build makes abstraction of dependencies and allows you to build the whole Pipeline project as if it were in one piece. The whole Pipeline code base is aggregated in the so-called super-project. It also includes some (forks of) third-party libraries that require changes frequently. A Makefile contains logic to compute interdependencies and build and connect the different components.

With a simple command you can build and package the main application into a DMG, a EXE, a ZIP (for Linux), a DEB and a RPM:

make dist

There are also commands to run all tests, to build and run the application, to build the website, to create an Eclipse project, etc. To get a list of all available commands run make help.

Note that all individual snapshot artifacts are internal to the super-build, meaning you can not expect to be able to use a snapshot dependency that was built as part of the super-build when building an individual component, or the other way around. However there is a trick to enable this possibility anyway: after running eval $(make dump-maven-cmd) in the super-project, mvn will be a shell alias that behaves like you expect.

Docker

To get people started quickly, we also provide a Dockerfile. With this Dockerfile you create an image that has all the build prerequisites and runtime dependencies installed.

FROM debian:stretch
RUN apt-get update && apt-get install -y openjdk-8-jdk \
                                         maven \
                                         golang \
                                         ruby \
                                         build-essential \
                                         zip \
                                         libxml2-utils \
                                         git \
                                         curl \
                                         lame \
                                         espeak
RUN gem install bundle
# the following lines are only for 64-bit systems, change to i368 if needed
RUN curl -L -o /tmp/#1 http://ftp.br.debian.org/debian/pool/main/n/nsis/{nsis-common_3.04-1_all.deb,nsis_3.04-1_amd64.deb}
RUN dpkg -i /tmp/nsis*.deb

Please let us know if something is missing from this Dockerfile.