CREATE FLEX Statement

A flex operator takes input from one or more streams/windows and produces a derived stream or window as its output. It allows the use of SPLASH code to specify customizable processing logic.

Note: The name of the Flex operator exists only for labeling in Studio and cannot be referred to in queries. Instead, refer to the output element.

Syntax

CREATE FLEX procedureName
IN input1 [KEEP keep_spec] [,...]
OUT [OUTPUT|LOCAL] [STREAM|DELTA STREAM|WINDOW] name schema_clause
[PRIMARY KEY (column1 [,...])] [store_clause] [keep_clause] [aging_clause]
[GUARANTEE DELIVERY]
BEGIN
[local-declare-block]
ON input1 { [statement1 [,...]] };
[EVERY interval { [statement1 [,...]] };]
[ON START TRANSACTION { [statement1 [,...]] };]
[ON END TRANSACTION { [statement1 [,...]] };]
END;

Components

`procedureName`	The name of the Flex operator being created.
`IN input1`	Inputs to the Flex operator are declared in the IN clause. The inputs can be streams, delta streams, windows, or outputs of another flex operator.
`KEEP keep_spec`	The KEEP clause modifies the retention policy of existing input elements that are either delta streams or windows.
`OUT output_element`	The output of the Flex operator is defined in this clause. A Flex stream can have only one output. It can be a stream, a delta stream or a window, either local or output.
`name`	The name of the output element must be included in the OUT clause. This name is used when running queries against the flex operator.
`schema_clause`	This clause is mandatory for all output elements: stream, delta stream, and window. See SCHEMA Clause for more details.
`PRIMARY KEY (column1 [,...])`	(Optional) This clause may be used when specifying either a delta stream or a window as the output element. See PRIMARY KEY Clause for more details.
`store_clause`	(Optional) This clause may only be used when specifying a window as the output element. See STORE Clause for more details.
`keep_clause`	(Optional) This clause may only be used when specifying a window as the output element. See KEEP Clause for more details.
`aging_clause`	(Optional) This clause may only be used when specifying a window as the output element. See AGING Clause for more details.
`GUARANTEE DELIVERY`	(Optional) Enables guaranteed delivery for this window. Any window with guaranteed delivery must also be assigned to a log store. Streams and delta streams do not support guaranteed delivery. They do support persistent subscribe pattern, which provides similar benefits.
`local-declare-block`	(Optional) The DECLARE block can define variables and functions of all types, including complex data types such as records, vectors, dictionaries and event caches.
`ON input1`	The ON input clause must be declared for every input of the Flex operator. The SPLASH code specified in this block is executed each time an input record is received. If an input element does not require processing, use an empty ON input clause.
`EVERY interval`	(Optional) The SPLASH statements specified in this block are executed every time the interval expires. The interval can be specified explicitly, or specified through an interval type parameter.
`ON START TRANSACTION`	(Optional) The SPLASH statements specified in this block are executed at the start of each transaction.
`ON END TRANSACTION`	(Optional) The SPLASH statements specified in this block are executed at the end of each transaction.
`statement1`	(Optional) SPLASH statement to specify the processing logic.

Note: The EVERY interval must be:

At least sixteen milliseconds on a Windows system
At least one millisecond on any other system

If you enter a shorter interval, the compiler changes it to the system’s minimum at run-time.

Usage

The CREATE FLEX statement creates a Flex operator that accepts any number of input elements and produces one output element. The input elements are previously existing streams, delta streams, and windows defined in the project. If the input element is a delta stream or window, you can modify its retention policy by specifying a KEEP clause. The output element is a stream, delta stream, or window with an unique name generated by the Flex operator. Specification of the SCHEMA clause is mandatory for all output element types. Specification of the PRIMARY KEY is mandatory for output elements that are delta streams or windows.

When you enable guaranteed delivery on a window that has registered guaranteed delivery subscribers, the window stores a copy of every event it produces in its log store until all the registered guaranteed delivery subscribers acknowledge receiving the events.

Note: The window stores copies of events only if there are registered guaranteed delivery subscribers.

Because copies of events are kept in the same log store the window is assigned to, the log store for a guaranteed delivery window must be significantly larger than the log store for a similar window without guaranteed delivery. Ensure that the log store for every guaranteed delivery window is large enough to accommodate the required events. If the log store runs out of room, the project server shuts down.

The ON input clause contains the processing logic for inputs arriving on a particular input element. Specification of the ON input clause is mandatory for each input of the Flex operator. The ON START TRANSACTION and ON END TRANSACTION clauses are optional and contain processing logic that should be executed at the start/end of each transaction respectively. The optional EVERY interval clause contains logic that is executed periodically based on a fixed time interval independent of any incoming events.

Restrictions

You can use a KEEP clause for the input of a Flex operator if the input element is a window or a delta stream.
You cannot associate a KEEP clause with a reference table query. The compiler will reject it, and the Studio will not let you add one.
Because a reference table query does not stream data to the Flex operator, it cannot have an ON method. The compiler will reject it, and the Studio will not let you add one.
Use GUARANTEE DELIVERY to enable guaranteed delivery only for a window stored in a log store; guaranteed delivery is not available to windows stored in memory stores.
Do not use GUARANTEE DELIVERY inside a module because you cannot directly attach adapters to elements in modules.
You cannot declare functions in the ON input and EVERY clauses.
You can define event cache types only in the local DECLARE block associated with the statement.
A Flex delta stream (a Flex stream for which the output is a delta stream) cannot generate records with update or upsert opcodes. To generate records with these opcodes, use a Flex window instead of a Flex delta stream.
You can use a SPLASH output statement inside the body of a function defined only in the local declare block of a Flex operator and not in a global declare block or a local declare block of any other element.

Example

This example computes the average trade price every five seconds. Guaranteed delivery is enabled on the output window so that subscribers to the window are assured of receiving all the output.

CREATE FLEX ComputeAveragePrice
 IN NASDAQ_Trades
 OUT OUTPUT WINDOW AverageTradePrice SCHEMA (Symbol string, 
 AveragePrice money(4) ) PRIMARY KEY(Symbol) STORE store1 GUARANTEE DELIVERY
 BEGIN
  DECLARE
   typedef [|money(4) TotalPrice; integer NumOfTrades] totalRec_t;
   dictionary(string,totalRec_t) averageDictionary;
 END;
 ON NASDAQ_Trades {
  totalRec_t rec := averageDictionary[NASDAQ_Trades.Symbol];
  if( isnull(rec) ) {
   averageDictionary[NASDAQ_Trades.Symbol] := 
   [|TotalPrice = NASDAQ_Trades.Price; NumOfTrades = 1];
  } else {
   // accumulate the total price and number of trades per input record
   averageDictionary[NASDAQ_Trades.Symbol] := 
   [|TotalPrice=rec.TotalPrice + NASDAQ_Trades.Price;
   NumOfTrades=rec.NumOfTrades + 1];
  }
 };
 EVERY 5 SECONDS {
  totalRec_t rec;
  for (sym in averageDictionary ) {
   rec := averageDictionary[sym];
   output setOpcode([Symbol=sym;|AveragePrice=(rec.TotalPrice/rec.NumOfTrades);], upsert);
  }
 };
 END;