User-defined VRL modules

[uralmetal]

This is an idea for improving reuse and code style VRL. I suggest to realize mechanism like python import or Jinja include. tthis is necessary to use user vrl code in different pipelines without using the Pipeline to Pipeline communication
For example:

user library

function set_http_status_code() {
    http_status_code = parse_int!(.http_status)
    del(.http_status)

    if http_status_code >= 200 && http_status_code <= 299 {
        .status = "success"
    } else {
        .status = "error"
    }
}

pipeline code

from library import set_http_status_code

del(.foo)

.timestamp = now()

set_http_status_code()

[uralmetal]

example from fluentd example from fluentd https://docs.fluentd.org/configuration/config-file-yaml#id-7.-reuse-your-config-the-include-yaml-tag

[jszwedko]

Hi @uralmetal !

Thanks for opening this. I can see how it would be useful. I think there are few separate requests here:

1. User-defined functions, which we have a request for: User defined VRL functions #47
2. Modules (related: add (syntax-only) modules for functions #145)
3. Including files in VRL configuration. The fluentd example you referenced is a feature of some YAML engines to include files. Unfortunately I don't think the serde-yaml, which we use, supports that feature.

It seems like you don't really want 3, but just 1 and 2. Since we have an issue for 1 already, I'll repurpose this issue to be about 2.

[uralmetal]

Yeah, i agree with you. Then i wait news about User-defined VRL functions and modules

[lddlww]

that's awesome feature