Mule Flow Diagrams [mulefd]

Table of Contents

• Introduction
• Features
• Requirements
  • Graphviz Engine
• Installation
  • SDKMan [linux] [apple]
  • Homebrew [apple]
  • Docker [docker]
  • Scoop [windows]
  • Manual install [apple] [windows] [linux]
  • Running with JBang
  • Build and Run
• Configuration
  • Supported Properties
• Usage
  • Source directory
  • Generate Individual flow diagrams
  • Flowname
  • Diagram Types
  • Output arguments
• Known modules/connectors
  • Registering custom modules
• Troubleshooting
  • Getting None of the provided engines could be initialized
• Copyright & License

Introduction

Mule application can contain multiple flows and sub-flows. All these can reference each other for processing messages. Data can also be exchanged synchronously or asynchronously using connectors.

A project can easily have multiple configuration files with flows/sub-flows that are spread across those files. When analyzing mule application, it could be difficult to see the complete data processing flow and their dependencies.

Have you ever wished to visualize your flow dependencies? How does message flows through your application?

If so, then try this mulefd application. It can read your configuration files and generate a diagrams that can easily tell you how your flows are connected.

mulefd CLI commands

$ mulefd
mulefd is a tool to generate beautiful diagrams for your mule application flows.
Usage: mulefd [-hV] [COMMAND]

Some command examples:
- mulefd graph <MULE_APP_PATH>
	Generate a Graph diagram for mule configuration in mule app
- mulefd help graph
	See help for Graph generation

-h, --help      display this help message
-V, --version   display version info
Commands:
help     Displays help information about the specified command
version  Display version info
graph    Create beautiful graph diagrams for mule flows
config   Manage MuleFD Global configuration

Copyright: 2020-2022 Manik Magar, License: MIT
Website: https://github.com/manikmagar/mulefd

For an example mule configuration ./itests/test-hello-app.xml, run following command to generate a flow diagram -

$ mulefd graph -t ./itests/output ./itests/test-hello-app.xml

Features

• Visualize mule flow dependencies and data flow.

• Visualize mule flow dependencies for a single flow.

• Identify unused mule flows and sub-flows.

• Identify flow spaghetti situations.

• Visualize synchronous and asynchronous data processing points.

• Identify connector dependencies and usage.

• Detect a possible recursive flow executions.

Requirements

At the minimum, Java 8 is required. The build pipeline also tests it against JDK 11+.

Tested and verified to use on OSX, Linux and Windows.

Graphviz Engine

This tool uses Graphviz Engine to generate Graph diagrams. It will use the first available engine, in following order -

• Native DOT engine if installed. This is preferred and much more performant engine compared to other engines (see issue#286).

• EclipseSource J2V8 Engine, bundled with this utility.

  • J2V8 engine is NOT available for Apple Silicon (M1). Consider install DOT engine or use JDK 8 or 11 for Nashorn.

• JDK JavaScript Nashorn engine. Slowest compared to the other two.

  • Nashorn engine has been deprecated in JDK 11 and remove in JDK 15.

Installation

SDKMan [linux] [apple]

Recommended way to install both java and Mule flow diagrams is sdkman for Linux and OSX.

To install SDKMAN and Java -

curl -s "https://get.sdkman.io" | bash
source "$HOME/.sdkman/bin/sdkman-init.sh"

sdk install java

Once Java is installed and ready, you install Mule Flow Diagrams -

sdk install mulefd

To test your installation run:

mulefd help

This should print out usage information.

For upgrading existing installations via SDKMAN,run:

sdk upgrade mulefd

Homebrew [apple]

On OSX you can install 'java' and mulefd with Homebrew using custom taps.

To install Java 8:

brew tap AdoptOpenJDK/openjdk
brew cask install adoptopenjdk8

Once Java is installed you can use brew with manikmagar/tap to get mulefd:

brew install manikmagar/tap/mulefd

To upgrade to the latest version:

brew upgrade manikmagar/tap/mulefd

Test running mulefd help in CLI.

Docker [docker]

If you don’t want to install mulefd, you can run it via docker.

docker run -v `pwd`:/app manikmagar/mulefd /app

This will generate diagrams in pwd or mounted directory.

Docker container resources are limited. Based on the size of your application, you may see outofmemory errors when executing with docker. You can allocate more memory with -m option to docker run command, Eg. -m 512m.

Scoop [windows]

On Windows, you can install mulefd using Scoop - A command-line installer for Windows.

Once you have Scoop installed and JDK configured, you can run following commands to get mulefd -

scoop bucket add manikmagar https://github.com/manikmagar/scoop-bucket
scoop install mulefd

To upgrade, you can run -

scoop update mulefd

Manual install [apple] [windows] [linux]

1. Unzip the latest binary release.

2. Add mulefd-<version>/bin folder in to your $PATH

3. Test running mulefd help in CLI.

Running with JBang

You can run MuleFD using JBang. If you have jbang installed, you can run jbang mulefd@manikmagar/mulefd to run latest release of MuleFD.

Build and Run

This project uses Gradle to compile and build. To create a zip distribution, run below command in project directory -

./gradlew distZip

To install the distribution -

1. Clone the project

2. Run ./gradlew installDist. This will explode the generated zip file to ./build/install directory.

3. You can verify binaries by executing -

   1. [apple] [linux] : sh build/install/mulefd/bin/mulefd help

   2. [windows] : ./build/install/mulefd/bin/mulefd.bat help

Configuration

mulefd supports external properties configuration to change the behavior of the tool. You can create a file ${user.home}/.mulefd/mulefd.properties and provide configuration properties.

To see the current properties, including the defaults of the tool, run following command -

List config properties

$ mulefd config list

Supported Properties

Following properties are supported in the config properties:

• diagram.font.name: To change the font used for graph text, set this to the name of the font available on target system. Eg. Arial or Courier.

• mule.components.csv: Path of the file containing list of Known modules/connectors.

📎	If generated diagram has node label’s going out of the shape, try changing the font to the one available on the systm.

Usage

mulefd support various arguments for generating diagrams.

Example: mulefd graph ~/AnypointStudio/studio-workspace/mulefd-demo

Generating Graph diagram

$ mulefd graph
Missing required parameter: '<sourcePath>'
Create beautiful graph diagrams for mule flows.
Usage: mulefd graph [-hV] [-gs] [-fl=<flowName>] [-o=<outputFilename>]
					[-t=<targetPath>] <sourcePath>
	<sourcePath>        Source directory path containing mule configuration
							files
	-fl, --flowname=<flowName>
						Target flow name to generate diagram for. All
							flows/subflows not related to this flow will be
							excluded from the diagram.
	-gs, --genSingles   Generate individual diagrams for each flow.
-h, --help              display this help message
-o, --out=<outputFilename>
						Name of the output file
-t, --target=<targetPath>
						Output directory path to generate diagram
-V, --version           display version info

Out of memory errors?

If your application is large and contains too many flows, process could fail with Exception in thread "main" java.lang.OutOfMemoryError: Java heap space error.

Try increasing the JVM allocated memory using -Xmx flag.

$ JAVA_OPTS='-Xmx1024m' mulefd graph ~/AnypointStudio/studio-workspace/mulefd-demo

For windows, you may need to set it at environment level -

set JAVA_OPTS=%JAVA_OPTS% -Xmx1024m
mulefd.bat graph ~/AnypointStudio/studio-workspace/mulefd-demo

Source directory

Source directory is a required argument. This argument specifies where mule xml configuration files be searched.

This argument value can be one of the following:

• Path of the individual mule xml file. Eg. ~/Downloads/test-app-config.xml. In this case, diagram for just this file is generated.

• Path to a standard mule 4 or mule 3 project root. Eg. ~/AnypointStudio/studio-workspace/mulefd-demo.

  • Mule 3: All configurations from src/main/app/ are scanned to generate a diagram.

  • Mule 4: All configurations from src/main/mule/ are scanned to generate a diagram.

• Path to any non-mule project directory.

Generate Individual flow diagrams

When running against a large mule application, the generated mule-diagram.png can contain too many flows. To simplify understanding each flow, it can be helpful to generate diagrams per flow (not sub-flows).

You can specify -gs or --genSingles option to generate individual flow diagrams, in addition to the consolidated one.

These diagrams are generated at {targetPath}/single-flow-diagrams/{currentDateTime} directory. Each generated diagram has the same name as flow it represents.

📎	Some flow names, especially the APIKit generated flows can contain characters not valid for some OS. Names are thus sanitized. Any character not in a-zA-Z0-9.- is replaced with _.

Flowname

If you just want to generate diagram for a single flow then specify it with -fl or --flowname option. This will exclude all flows and subflows that are not related to this target flow.

Diagram Types

Current release supports generating Graph diagram only.

Output arguments

Target directory to output generated diagram can be specified with -t {directoryPath} option. This is an optional argument and defaults to the source directory (or parent directory if source is a file).

The file name for diagram defaults to mule-diagram.png. This can be changed by specifying -o {filename} argument.

Known modules/connectors

Mule has many connectors/modules that can be used for building flows and sub-flows, and list keeps growing. This tool maintains a list of known components with their supported operations for including in the generated diagram.

You can find this list in source code src/main/resources/default-mule-components.csv.

Each record in this file has following columns -

prefix, operation,  sourceFlag, path,   configName, async

• prefix: Namespace of module/connector. Eg. vm, http etc.

• operation: Name of an operation or input source in that namespace. Eg. listener, consume in vm etc. This supports wildcard entries (values defined as *) for non-source (sourceFlag=false) entries.

• sourceFlag: true if operation is an input source withing that namespace. Eg. listener in vm.

• path: Name of the attribute on operation which can help identify resource path. Eg. queueName for vm:listener or path for http:request. This value will be visible on source nodes in the diagram.

  • For complex paths, it is possible to specify xpath expression to extract value. xpath: prefix must be added to the expression when providing relative xpath.

Scheduler Element for Complex Path selector

<scheduler doc:name="Scheduler">
	<scheduling-strategy >
		<cron expression="${cron.expreesion}" /> // (1)
	</scheduling-strategy>
</scheduler>

1. For example, the scheduler component’s path can be xpath:scheduling-strategy/cron/@expression. Notice that it is relative to <scheduler> element.

• configName: Name of the module configuration. Eg. config-ref attribute name for vm:listener. For sources, this is visible on source nodes in diagram. This can help identify source uniquely when multiple configuration exists.

• async: true if this is an asynchronous operation. Defaults to false.

The generated diagram will represent these known components using the Component share.

Figure 1. Known Component rendering for s3:create-object to test2 bucket

Registering custom modules

If any module is missing in the default list - either being a new module or a custom module, then it is possible for users to register their modules.

If mulefd-components.csv named CSV file exists in sourcePath, then all modules/connectors within that file are registered as known components. Structure of this file must be same as default components file explained above.

See example file at src/test/resources/mulefd-components.csv.

MuleFD will load components file from predefined locations in the given order as below -

1. Default Components file shipped with the tool

2. Components file from the path defined in Configuration file with key mule.components.csv

3. Components file located in the source directory

Troubleshooting

Getting None of the provided engines could be initialized

This tool uses Graphviz-Java library to generate diagrams. Graphviz-Java describes how the engines are leveraged. To execute the graphviz layout engine, one of these options is used:

1. If the machine has graphviz installed and a dot command is available, spawn a new process running dot.

2. Use this javascript version of graphviz and execute it on the V8 javascript engine. This is done with the bundled J2V8 library.

3. Alternatively, the javascript can be executed on Java’s own Nashorn or GraalVM engine (preferring Graal if both are available).

This tool bundles the J2V8 library but it does not support Apple M1. The Java’s Nashorn engine was deprecated starting JDK 9 and removed in JDK 15.

So, if your diagram generation is failing with that error then check one of the following -

1. Which JDK is used? Using JDK’s prior to 15 will at-least make the Nashorn engine available. It may be slower to run when compared to others.

2. If JDK 15+ is being used on Apple M1, make sure the dot is installed. It is a part of Graphviz, so check here for installing graphviz with brew install graphviz.

Copyright & License

Licensed under the MIT License, see the LICENSE file for details.