Skip to content

Iteration

Iterables

Prior to the introduction of iterators in Integration Toolkit, any function or operator in an Ignition expression binding or expression tag would be executed, at most, once per overall expression execution.

Now, with these tools, expression bindings (and expression tags) can execute a nested expression many times over a collection of values, building lists or other output data structures from the separate results.

Iteration is accomplished by setting up one or more thread-local variables to carry the necessary loop values, then executing a nested expression once for each value in the loop. The loop values are accessed inside the nested expression with special expression functions, it() and idx().

Each looping function that repeatedly executes one or more nested expressions defines the types of source data they accept. The most flexible handlers of source data are:

  • forEach()

  • groupBy()

  • orderBy()

  • where()

These four expression function accept datasets, lists, arrays, mapping objects, and simple integers. The loop value retrieved by it() within the looping expressions of these functions, for the various source types, are as follows:

dataset
When looping through a dataset, the loop value it() is assigned a wrapping (indirect) dataset of the same column names and column types, but having just the one (current) row. You can therefore refer to values in the current row using Ignition’s normal dataset subscript operator (square brackets) with a column index or column name as usual. That is, it()[0] or it()['colName'] are the expected forms when looping through a dataset.
list
When looping through a list, the loop value it() is assigned each element of the list in turn. Since a list can hold either simple types or nest other complex types, the operators that apply will vary.
array
Looping through arrays is indistinguishable from looping through lists–arrays hold the same kinds of data.
map
When looping through mapping objects (like dictionaries), the loop value it() is assigned a two-element array (pair) of [key,value] for each key/value pair in the mapping object. Since Ignition v8.1.8, the subscript operator (square brackets) supports arrays and lists, so it()[0] yields the current pair’s key, and it()[1] yields the current pair’s value.
integer
When given an integer as the loop source, the loop value it() is assigned the zero-based loop index, so it behaves just like idx(). In this case, the source is treated as if it were jython’s range() function with that integer.

it()

This expression function retrieves the loop value for an iterator from inside the nested expression, using a thread-local variable. It optionally accepts a depth argument to look up the loop value for an outer iterator. See the source datatypes descriptions for Iteration above to determine what return type to expect.

it([depth]) returns Object

depth Optional number of enclosing iterators to skip to reach the iterator from which to retrieve the loop value. Zero when omitted.

This expression function throws an exception when used outside of a looping function or when the depth is too large.

lag()

This expression function retrieves the previous loop value for an iterator from inside the nested expression, using a thread-local variable. It optionally accepts a depth argument to look up the previous loop value for an outer iterator. See the source datatypes descriptions for Iteration above to determine what return type to expect.

The return value is always null on the first pass through any loop.

lag([depth]) returns Object

depth Optional number of enclosing iterators to skip to reach the iterator from which to retrieve the previous loop value. Zero when omitted.

This expression function throws an exception when used outside of a looping function or when the depth is too large.

idx()

This expression function retrieves the loop index for an iterator from inside the nested expression, using a thread-local variable. It optionally accepts a depth argument to look up the loop index for an outer iterator. Behaves as if a jython for loop included jython’s enumerate() built-in function.

idx([depth]) returns Integer

depth Optional number of enclosing iterators to skip to reach the iterator from which to retrieve the loop index. Zero when omitted.

This expression function throws an exception when used outside of a looping function or when the depth is too large.

forEach()

This function is the most generic of all of the looping functions, as it assembles a list for its output from its nested expression (or expressions) on a one-for-one basis. It is most similar to jython’s simplest list comprehensions.

forEach(source, expr...) returns List

source Any iterable.
expr A nested expression yielding any object, typically using it() and/or idx() to operate upon the source data element-by-element. Expression functions that need to be able to retrigger (like polling, or tag or property references) are not allowed.

When only one expr is given, the output list directly contains the return values from that expression, in this shape:

[result0, result1, ... resultN]

When multiple expression are given, possibly like exprA, exprB, … exprZ , the output list contains a nested list of each nested expressions’ return value, in this shape:

[
    [result0a, result0b, ... result0z],
    [result1a, result1b, ... result1z],
    ...
    [resultNa, resultNb, ... resultNz]
]

groupBy()

This function is the most complex of all of the looping functions, as it assembles a list of lists of lists to group the source data by arbitrary key expressions.

groupBy(source, keyExpr...) returns List

source Any iterable.
keyExpr A nested expression yielding a java Comparable, typically using it() and/or idx() with suitable operators to extract the information needed to segregate the source data into separate lists. Expression functions that need to be able to retrigger (like polling, or tag or property references) are not allowed.

When a single keyExpr is supplied, and source is a dataset, the output list has the following shape, with source rows recombined into grouped datasets:

[
    [key0, dataset0],
    [key1, dataset1],
    ...
    [keyN, datasetN]
]

When a single keyExpr is supplied, and source is not a dataset, the output list has the following shape:

[
    [key0, [value0, valueJ...]],
    [key1, [valueK, valueT...]],
    ...
    [keyN, [valueM, valueZ...]]
]

Multiple keyExpr are permitted, and change the output shape. If the source is a dataset, the output list has the following shape, with source rows recombined into grouped datasets:

[
    [[key0a, key0b...], dataset0],
    [[key1a, key1b...], dataset1],
    ...
    [[keyNa, keyNb...], datasetN]
]

When multiple keyExpr are supplied, and the source is not a dataset, the output list has the following shape:

[
    [[key0a, key0b...], [value0, valueJ...]],
    [[key1a, key1b...], [valueK, valueT...]],
    ...
    [[keyNa, keyNb...], [valueM, valueZ...]]
]

In all cases, keys or sets of keys are added to the output list in the order that key or combination is first encountered in the source data. The output datasets or value lists for a given key or combination are also ordered as they were encountered in the source data.

orderBy()

This function reorders the given source data per the given key expressions. Output data is the sequence of loop values assembled into a list, or assembled into a new dataset.

orderBy(source, keyExpr...) returns Dataset or List

source Any iterable.
keyExpr A nested expression yielding a java Comparable, typically using it() and/or idx() with suitable operators to extract the information needed to provide the desired ordering. See also the descending(), naturalOrder(), and naturalCasedOrder() functions below for complex ordering cases. Expression functions that need to be able to retrigger (like polling, or tag or property references) are not allowed.

Multiple keyExpr may be supplied.

Note that while the return value is a dataset when source is a dataset, any other source datatype produces a list.

Also see the companion scripting function (datasets only) system.dataset.orderBy().

where()

This function prunes the source data according to supplied conditional expressions. The output length is no greater than the source length, and can be zero.

where(source, condExpr...) returns Dataset or List

source Any iterable.
condExpr A nested expression yielding a java Boolean, typically using it() and/or idx() with suitable comparison operators and/or logical operators. Expression functions that need to be able to retrigger (like polling, or tag or property references) are not allowed.

Multiple condExpr may be supplied. Additional conditionals are skipped when a false result is obtained. Each loop value that yields a true for all conditionals will be included in the output.

Note that while the return value is a dataset when source is a dataset, any other source datatype produces a list.

asList()

This expression function unconditionally assembles all of its arguments into a List. The return is an ArrayList, so is compatible with Identity Provider attribute mapping.

asList([arg...]) returns List

arg Any expression.

When no arguments are given, the return is an empty list.

asMap()

Given a source dataset, or a list, or an arbitrary series of paired arguments, produce a Java Map with string keys. If a dataset, the column names will be the keys and the first row will provide the output values. If a list, the elements are expected to be lists of length two, supplying key and value. Otherwise, the arguments themselves provide key/value pairs for output.

asMap(dataset OR list OR array) returns Map

dataset Any single Dataset.
list Any single List, containing nested two-element lists or arrays.
array Any single Array, containing nested two-element lists or arrays.

asMap([key, value [, key, value...]]) returns Map

key String to use as the key for the following value.
value Any expression.

Multiple key/value pairs may be supplied.

When no arguments are given, the return is an empty mapping object.

The returned mapping object is an instance of LinkedHashMap, which maintains the order that keys are inserted. (Once assigned to a Perspective property, that ordering is lost.)

asPairs()

This expression function produces a list of key/value pairs (as nested lists) from any number and combination of dataset or map or list arguments. The first element of the pair is the dataset column name or the map’s key. The second element is the corresponding value.

asPairs(dataset OR map OR list [, dataset OR map OR list...]) returns List

dataset Any dataset, from which only the first row is taken. Column names and first-row values become the values of a two-element list per column.
map Any mapping object, which yield a two-element list per key/value pair.
list Any list or array object. It is assumed to contain two-element nested lists, but is not required to, unless the result is supplied to an outer asMap() function.

The output is suitable to use as input to asMap(), providing a convenient means to combine or update mappings with new values. When the inputs are from the columnsOf() function, or similar information from asMap(), the output is suitable as the columnsInfo argument to unionAll().

Because of the unchecked handling of arguments that are lists, this function can perform the equivalent of python’s list concatenation on any provided lists, regardless of content. This is the same functionality as python’s sequence.extend() method.

descending()

Reverses the comparison order of any comparable handed to it. Only useful in a key expression of the orderBy() function, as no other methods of the original comparable are proxied.

descending(comparable) returns Comparable

comparable Any java Comparable, typically from an ordering key expression.

Also see the companion scripting function (datasets only) system.dataset.descending().

naturalOrder()

Stringifies any value supplied to it, and wraps it so that it automatically orders using a non-case-sensitive alphanumeric “human-friendly” algorithm.

naturalOrder(value) returns Comparable

value Any java Object, typically from an ordering key expression.

Also see the companion scripting function (datasets only) system.dataset.naturalOrder().

naturalCasedOrder()

Stringifies any value supplied to it, and wraps it so that it automatically orders using a case-sensitive alphanumeric “human-friendly” algorithm.

naturalCasedOrder(value) returns Comparable

value Any java Object, typically from an ordering key expression.

Also see the companion scripting function (datasets only) system.dataset.naturalCasedOrder().

flatten()

This expression function produces a list from the inner values of nested lists within its arguments.

flatten(list [, list...]) returns List

list Any list or array that contains nested lists or arrays.

Multiple lists may be supplied. Only the outermost two levels of each list are flattened into the result list.

unMap()

Given an object, and if not a mapping, return it unchanged. Otherwise try given keys in order to extract a desired value. Stops when a non-mapping object is obtained or all keys are used.

unMap(object, key [, key...]) returns Object

object Any object expression.
key Any value of the type required by the object's keys, if it is a mapping.

Use when processing a list of values where some values are mappings instead of primitive types, and the mapping is expected to contain the actual primitive type at some inner path.

Many Perspective components accept data or items properties that follow this rule. If you wish to iterate over the “plain” values of such component properties, you have to conditionally unMap().

It is conceptually similar to an expression like the following, but doesn’t try to apply a key to a null:

coalesce(object[key0][key1][key2], object[key0][key1], object[key0], object)