Pipeline as Service

Pipeline as a Service

Most people will use DAISY Pipeline 2 as a standalone desktop application. It is easy to install and ready to go. But the Pipeline can also be used as a service to which client programs can connect, through a web API.

There are several use cases:

• You want to run batch jobs through the command line interface.
• You want to use our web application.
• You want to include the Pipeline in a larger setup where it needs to talk to other components.

Installation

Choose a package from the downloads page that includes the server, then follow the installation instructions for that package.

Starting and stopping the service

Windows

TBD

Debian/Ubuntu

If the Pipeline was installed via the Debian package, the service is launched automatically on system startup. To start or stop the service manually, use service daisy-pipeline2 start|stop|restart|status.

Other distros

In order to start the service, execute the file pipeline2. Where exactly this file is located on the file system depends on the installation. To shut down the service press Ctrl-C.

Configuration

Configuration files

When using DAISY Pipeline 2 as a service, especially when integrating it in a custom setup, you probably want to configure a thing or two. Two files are intended to be edited by you for this purpose: pipeline.properties and config-logback.xml. See Configuration files for details. Where on the file system these files are located depends on the installation.

Windows

TBD

Debian/Ubuntu

If the Pipeline was installed via the Debian package, pipeline.properties and config-logback.xml are located in /etc/opt/daisy-pipeline2. In addition there is a third configuration file available: /etc/default/daisy-pipeline2. All available environment variables listed below, except for PIPELINE2_DATA, can be specified in this file in the format export VAR=value.

Other distros

Environment variables

A number of environment variables will influence the program:

JAVA : Location of the "java" executable. Must be Java 11 or higher.

JAVA_HOME : Location where the Java JRE or JDK is installed. Must be Java 11 or higher.

JAVA_MIN_MEM : Sets the initial size of the heap. See Java's -Xms option.

JAVA_MAX_MEM : Sets the maximum size of the memory allocation pool. See Java's -Xmx option.

JAVA_PERM_MEM : Sets the permanent generation space size. See Java's -XX:PermSize option.

JAVA_MAX_PERM_MEM : Sets the maximum permanent generation space size. See Java's -XX:MaxPermSize option.

JAVA_OPTS : Any other Java options.

JAVA_DEBUG_OPTS : Additional Java options to be passed on when PIPELINE2_DEBUG is set.

PIPELINE2_DEBUG : When "true", passes debug options to Java ("true" or "false", default is "false").

PIPELINE2_DATA : Directory for storing program data. Defaults to the "data" folder inside the Pipeline installation folder.

In addition to the environment variable listed above, all available user properties can be set through environment variables as well. For example, the org.daisy.pipeline.ws.host property can be set with the environment variable PIPELINE2_WS_HOST. The environment variable settings will have precedence over settings in the pipeline.properties file.

Arguments for pipeline2 executable

The following command line arguments are available. Combinations are possible.

remote : Run Pipeline server in "remote" mode. Has the same effect as setting PIPELINE2_WS_LOCALFS=false and PIPELINE2_WS_AUTHENTICATION=true.

local : Run Pipeline server in "local" mode. Has the same effect as setting PIPELINE2_WS_LOCALFS=true and PIPELINE2_WS_AUTHENTICATION=false.

gui : Launch the graphical user interface instead of the web service. See also Desktop Application.

clean : Clean the program data.

debug : Enable debugging. Has the same effect as setting PIPELINE2_DEBUG=true.

osgi : Run Pipeline inside an OSGi framework (Apache Felix).

shell : Enable an interactive shell for low-level system manipulation. Run help for a list of commands, help COMMAND for more info about a specific command.