Vector v0.40.0 release notes

VRL was updated to v0.17.0. This includes the following changes:

Breaking Changes & Upgrade Guide

• parse_logfmt now processes 3 escape sequences when parsing: , " and \. This means that for example, in the input will be replaced with an actual newline character in parsed keys or values. (https://github.com/vectordotdev/vrl/pull/777)

Now the GELF codec with stream-based sources uses null byte (\0) by default as messages delimiter instead of newline (\n) character. This better matches GELF server behavior.

Configuration changes

In order to maintain the previous behavior, you must set the framing.method option to the character_delimited method and the framing.character_delimited.delimiter option to \n when using GELF codec with stream-based sources.

Example configuration change for socket source

Previous

sources:
  my_source_id:
    type: "socket"
    address: "0.0.0.0:9000"
    mode: "tcp"
    decoding:
      codec: "gelf"

Current

sources:
  my_source_id:
    type: "socket"
    address: "0.0.0.0:9000"
    mode: "tcp"
    decoding:
      codec: "gelf"
    framing:
      method: "character_delimited"
    character_delimited:
      delimiter: "
"

Reduce transforms can now properly aggregate nested fields.

This is a breaking change because previously, merging object elements used the “discard” strategy. The new behavior is to use the default strategy based on the element type.

Example

Config

group_by = [ "id" ]
merge_strategies.id = "discard"
merge_strategies."a.b[0]" = "array"

Event 1

{
  "id": 777,
  "an_array": [
    {
      "inner": 1
    }
  ],
  "message": {
    "a": {
      "b": [1, 2],
      "num": 1
    }
  }
}

Event 2

{
  "id": 777,
  "an_array": [
    {
      "inner": 2
    }
  ],
  "message": {
    "a": {
      "b": [3, 4],
      "num": 2
    }
  }
}

Reduced Event

Old behavior:

{
  "id": 777,
  "an_array": [
    {
      "inner": 2
    }
  ],
  "message": {
    "a": {
      "b": [1, 2],
      "num": 1
    }
  }
}

New behavior:

{
  "id": 777,
  "an_array": [
    {
      "inner": 1
    }
  ],
  "message": {
    "a": {
      "b": [
        [1, 2],
        [3,4]
      ],
      "num": 3
    }
  }
}