TSV

Parse Elastic Compatible

Synopsis

Parses TSV (Tab-Separated Values) data from a field and assigns the extracted values to specified target fields.

Schema

- tsv:
    field: <ident>
    target_fields: <ident[]>
    description: <text>
    empty_value: <string>
    if: <script>
    ignore_failure: <boolean>
    ignore_missing: <boolean>
    on_failure: <processor[]>
    on_success: <processor[]>
    tag: <string>
    trim: <boolean>

Configuration

The following fields are used to define the processor:

Field	Required	Default	Description
field	Y		Field containing the TSV string to parse
target_fields	Y		Array of field names to assign the parsed values to
description	N		Explanatory note
empty_value	N		Value to use for empty fields. If not specified, empty fields remain empty
if	N		Condition to run
ignore_failure	N	false	Continue processing if parsing fails
ignore_missing	N	false	Skip processing if source field doesn't exist
on_failure	N		See Handling Failures
on_success	N		See Handling Success
tag	N		Identifier
trim	N	false	Trim whitespace from parsed fields

Details

The processor reads a single line of TSV data and maps each tab-delimited column to the corresponding target field. The tab separator is hardcoded and cannot be configured — use the csv processor with separator: "\t" equivalent or the csv processor's separator field for other delimiters.

note

The processor expects the TSV data to be a single line. For multi-line TSV processing, pre-process the data to extract individual lines first.

warning

If the number of columns in the TSV data is less than the number of target fields, the processor will fail unless ignore_missing is set to true.

Examples

Basic

Parsing a log entry with timestamp, level, and message columns...	{ "raw": "2024-01-15T10:23:45Z\tERROR\tconnection timeout" } - tsv: field: raw target_fields: ["timestamp", "level", "message"]
creates the specified fields:	{ "raw": "2024-01-15T10:23:45Z\tERROR\tconnection timeout", "timestamp": "2024-01-15T10:23:45Z", "level": "ERROR", "message": "connection timeout" }

Trim Whitespace

Trimming surrounding whitespace from each parsed column...	{ "raw": " 192.168.1.10 \t 10.0.0.5 \t 4096 " } - tsv: field: raw target_fields: ["src_ip", "dst_ip", "bytes_sent"] trim: true
whitespace is stripped from each value:	{ "raw": " 192.168.1.10 \t 10.0.0.5 \t 4096 ", "src_ip": "192.168.1.10", "dst_ip": "10.0.0.5", "bytes_sent": "4096" }

Empty Values

Filling empty columns with a default value...	{ "raw": "192.168.1.10\t\t443" } - tsv: field: raw target_fields: ["src_ip", "dst_ip", "port"] empty_value: "N/A"
empty column receives the configured default:	{ "raw": "192.168.1.10\t\t443", "src_ip": "192.168.1.10", "dst_ip": "N/A", "port": "443" }

Column Mismatch

Handling fewer TSV columns than target fields using ignore_missing...	{ "raw": "val1\tval2\tval3" } - tsv: field: raw target_fields: ["field1", "field2", "field3", "field4"] ignore_missing: true
only columns present in the data are written; the fourth target field is skipped:	{ "raw": "val1\tval2\tval3", "field1": "val1", "field2": "val2", "field3": "val3" }