Miscellaneous operators

Use these functions in the Advanced Search interface to perform relatively complex data transformations. The Logs Query Builder does not natively support these functions.

backshift

Shifts a column in a table down by the specified number.

For the views table.

The default shift is 1.

The default alias, when used on the command line, is _backshift.

Syntax

| backshift <field>
| backshift <field> as <alias>
| backshift <field>, <number>
| backshift <field>, <number> as <alias>

base64Decode

Converts a base64 string into to an ASCII/UTF-8 string.

Syntax

| base64Decode(<base64String>) as <alias>

Example

The following example returns K with a value of Hello World.

| base64Decode("SGVsbG8gV29ybGQ=") as K

base64Encode

Converts an ASCII/UTF-8 string to a base64 string.

Syntax

| base64Encode(<string>) as <alias>

Example

The following example returns K with a value of SGVsbG8gV29ybGQ=.

| base64Encode("Hello World") as K

compareCIDRPrefix

Checks if the network prefixes of two IPv4 addresses match.

Syntax

| compareCIDRPrefix(<ipv4String>, <ipv4String>) as <alias>

Examples

The following example returns K with a value of true.

| compareCIDRPrefix("10.10.1.35", "10.10.1.35", 24) as K

The following example returns K with a value of false.

| compareCIDRPrefix("10.10.1.35", "11.11.2.35", 24) as K

concat

Concatenates multiple strings and numbers into a new string.

Syntax

| concat(<field1>, <field2>, ...) as <alias>

Example

The following example returns K with a value of Hello World 1.

| concat("Hello", " ", "World", " ", 1) as K

decToHex

Converts long value to hexadecimal value.

Syntax

| decToHex(<longField>) as <alias>

Example

The following example returns K with a value of 461.

| decToHex("70") as K

dedup

Removes duplicate results in a table by a set of columns.

For the views table.

The default number is 1.

Syntax

| dedup by <field1>, <field2>, ...
| dedup by <number> <field1>, <field2>, ...

format

Returns a formatted string given a format specifier and arguments.

Syntax

| format(<formatSpecifierString>, <field1>, ...) as <alias>

Example

The following example returns K with a value of The count is: 1.

| format("%s : %s", "The count is", 1) as K

formatDate

Returns a date string given a date, format, and timezone (default UTC).

Syntax

| formatDate(<dateMilliseconds>) as <alias>

| formatDate(<dateMilliseconds>, <formatString>) as <alias>

| formatDate(<dateMilliseconds>, <formatString>, <timezoneString>) as <alias>

Examples

The following example returns K with a value of 2024-11-06 20:33:37.

| formatDate(1730925217838) as K

The following example returns K with a value of 2024-11-06.

| formatDate(1730925217838, "YYYY-MM-dd") as K

getCIDRPrefix

Retrieves the network prefix from an IPv4 address.

Syntax

| getCIDRPrefix(<ipv4String>) as <alias>

Example

The following example returns K with a value of 24.

| getCIDRPrefix("10.10.1.35") as K

hash

Hashes data into a string value using the specified hash algorithm.

Supported hash algorithms are MD5, SHA1, SHA2, and MurmurHash3.

Syntax

| hash(<field>) as <alias>

| hash(<field>, <hashAlgorithm>) as <alias>

where <hashAlgorithm> is either md5(default), sha1, sha2_256, or murmur3_128

Examples

The following example returns K with a value of 5eb63bbbe01eeed093cb22bb8f5acdc3.

| hash("hello world") as K

The following example returns K with a value of 2aae6c35c94fcfb415dbe95f408b9ce91ee846ed.

| hash("hello world", "sha1") as K

hexToAscii

Converts hexadecimal string to ASCII value.

Syntax

| hexToAscii(<hexString>) as <alias>

Example

The following example returns K with a value of Hello Gopher!.

| hexToAscii("48656c6c6f20476f7068657221") as K

hexToDec

Converts hexadecimal string to a long value.

Syntax

| hexToDec(<hexString>) as <alias>

Example

The following example returns K with a value of 4919.

| hexToDec("0000000000001337") as K

in

Checks if a field is in a set of string values or number values.

Syntax

| where <field> in (<value_1>[, <value_2>, <value_3>, ...])

Examples

| source="pinot-server" | parse "[*]" as class | where class in ("ServerQueryLogger", "AppInfoParser")

ipv4ToNumber

Converts an Internet Protocol version 4 (IPv4) IP address from the octet dot-decimal format to a decimal format.

Syntax

| ipv4ToNumber(<ipv4String>) as <alias>

Example

The following example returns K with a value of 178455296.

| ipv4ToNumber("10.163.3.0") as K

isBlank

Checks if a string value is null, empty, or contains only empty characters.

Syntax

| isBlank(<string>) as <alias>

Example

The following example returns K with a value of true.

| isBlank("  ") as K

isEmpty

Checks if a string value is an empty string that contains no characters or whitespace.

Syntax

| isEmpty(<string>) as <alias>

Example

The following example returns K with a value of false.

| isEmpty(" a ") as K

isNumeric

Checks if a string value can be parsed as a number.

Syntax

| isNumeric(<string>) as <alias>

Example

The following example returns K with a value of true.

| isNumeric("1.234") as K

isPrivateIP

Checks if an IPv4 address is private.

Syntax

| isPrivateIP(<ipv4String>) as <alias>

Example

The following example returns K with a value of true.

| isPrivateIP("192.168.0.1") as K

isPublicIP

Checks if an IPv4 address is public.

Syntax

| isPublicIP(<ipv4String>) as <alias>

Example

The following example returns K with a value of false.

| isPublicIP("192.168.0.1") as K

isValidIP

Checks if an IPv4 or IPv6 address is valid.

Syntax

| isValidIP(<ipString>) as <alias>

Examples

The following example returns K with a value of true.

| isValidIP("192.168.0.1") as K

The following example returns K with a value of false.

| isValidIP("192.168.400.1") as K

len

Returns the length of a string.

Syntax

| len(<string>) as <alias>

Example

The following example returns K with a value of 4.

| len("1234") as K

luhn

Validates credit card numbers in a string value using Luhn’s algorithm.

Syntax

| luhn(<string>) as <alias>

Examples

The following example returns K with a value of true.

| luhn("6666-7777-6666-8888") as K

The following example returns K with a value of false.

| luhn("00000000000000031") as K

maskFromCIDR

Returns the subnet mask given a prefix length for IPv4 addresses.

Syntax

| maskFromCIDR(<prefixLength>) as <alias> where <prefixLength> is 0-32

Examples

The following example returns K with a value of 255.255.255.255.

| maskFromCIDR(32) as K

The following example returns K with a value of 255.255.255.255.

| maskFromCIDR(32) as K

matches

Matches strings using the RE2-compliant regex format.

Syntax

| where <string expression> matches "<regex>"

Examples

| source="pinot-server" | parse "[*]" as class | where class matches "Server.*"

now

Returns the current epoch time in milliseconds.

Syntax

| now() as <alias>

Examples

| now() as K
| formatDate(now(), "YYYY-MM-dd") as today

queryEndTime

Returns the end time of the search in milliseconds.

Syntax

| queryEndTime() as <alias>

Examples

| queryEndTime() as K

queryStartTime

Returns the start time of the search in milliseconds.

Syntax

| queryStartTime() as <alias>

Examples

| queryStartTime() as K

queryTimeRange

Returns the time range for the query being executed, in milliseconds.

Syntax

| queryTimeRange() as <alias>

Examples

| queryTimeRange() as K

replace

Replaces all occurrences of a specified string with another string.

The specified string can be a literal or regex.

Syntax

| replace(<sourceString>, <searchString>, <replaceString>) as <alias>

| replace(<sourceString>, <regexString>, <replaceString>) as <alias>

Examples

The following example returns K with a value of hello gopher.

| replace("hello world", "world", "gopher") as K

The following example returns K with a value of http://kloudfuse.com/products//logs.

| replace("http://kloudfuse.com/products/12345678/logs", "[0-9]{5,}", "") as K

substring

Extracts a part of a given string and start/end offsets.

Syntax

| substring(<sourceString>, <startOffsert>) as <alias>

| substring(<sourceString>, <startOffsert>, <endOffset>) as <alias>

Example

The following example returns K with a value of hello.

| substring("hello world", 0, 5) as K

toBytes

Parses a string representation of bytes (KB, MB, GB, and so on) into the number of bytes it represents.

Syntax

| toBytes(<storageSize>) as <alias>

Examples

The following example returns K with a value of 1500.

| toBytes("1.5KB") as K

toDuration

Parses a string representation of time (ns, µs, ms, s, m, h) to milliseconds.

Syntax

| toDuration(<timeString>) as <alias>

Example

The following example returns K with a value of 2000000.

| toDuration("2ms") as K

toFloat

Parses a string representation of a number or a number to a float.

Syntax

| toFloat(<number>) as <alias>

| toFloat(<numberString>) as <alias>

Examples

The following example returns K with a value of 1.25.

| toFloat("1.25") as K

toInt

Parses a string representation of a number, or a number to an int.

Syntax

| toInt(<number>) as <alias>

| toInt(<numberString>) as <alias>

Example

The following example returns K with a value of 1.

| toInt("1.5") as K

toLowerCase

Converts all letters of a string to lowercase.

Syntax

| toLowerCase(<string>) as <alias>

Example

The following example returns K with a value of hello world.

| toLowerCase("HELLO WORLD") as K

toUpperCase

Converts all letters of a string to uppercase.

Syntax

| toUpperCase(<string>) as <alias>

Example

The following example returns K with a value of HELLO WORLD.

| toUpperCase("hello world") as K

transpose

Converts aggregate query results from a long format into a wide, tabular format by pivoting row values into column headers.

Syntax

| transpose row <row_field1>[, <row_field2>, ...] column <column_field1>[, <column_field2>, ...]

Examples

Without transpose:

source="pinot-server" | timeslice 5m | count by _timeslice, status_code

With transpose:

source="pinot-server" | timeslice 5m | count by _timeslice, status_code | transpose row _timeslice column status_code

trim

Removes starting and trailing empty spaces in a string.

Syntax

| trim(<string>) as <alias>

Example

The following example returns K with a value of hello world.

| trim("  hello world   ") as K

urlDecode

Returns an unescaped URL string.

Syntax

| urlDecode(<urlString>) as <alias>

Example

The following example returns K with a value of http://example-server123.org/api/v1/data.php?auth=AbCdEfGhIjKlMnOpQrStUvWxYz123456&;.

| urlDecode("http%3A%2F%2Fexample-server123.org%2Fapi%2Fv1%2Fdata.php%3Fauth%3DAbCdEfGhIjKlMnOpQrStUvWxYz123456%26") as K

urlEncode

Encodes a URL into the ASCII character set.

Syntax

| urlEncode(<urlString>) as <alias>

Example

The following example returns K with a value of http%3A%2F%2Fexample-server123.org%2Fapi%2Fv1%2Fdata.php%3Fauth%3DAbCdEfGhIjKlMnOpQrStUvWxYz123456%26.

| urlEncode("http://example-server123.org/api/v1/data.php?auth=AbCdEfGhIjKlMnOpQrStUvWxYz123456&") as K