Splitting Up Incoming Data

Use the SPLITTER construct to separate incoming data according to filtering rules and write it out to different target streams.

When you want to separate incoming data into several subsets and process those subsets differently, use the CREATE SPLITTER construct, which operates like the ANSI case statement. It reads the incoming data, applies the specified filtering conditions and writes out each subset of the data to one or more target streams.

The target stream or delta streams are implicitly defined by the compiler. The schema for the target streams are derived based on the column_list specification. All the targets are defined as either local or output depending on the visibility clause defined for the splitter. The default is local. Note that when the splitter has an output visibility, output adapters can be directly attached to the splitter targets, even though those targets are implicitly defined.

The first condition that evaluates to true (non-zero value) causes the record as projected in the column_list to be inserted into the corresponding target streams. Subsequent conditions are neither considered nor evaluated. If the source is a:

Stream, the targets are also streams.
Delta stream or window, the targets are delta streams.

If the source is a window or delta stream, the primary keys need to be copied as-is. The other columns can be changed.

Note: When the source is a window or a delta stream, the warning about unpredictable results being produced if one of the projections contains a non-deterministic expressions that applies for delta streams also applies for splitters.

Example

The example creates a schema named TradeSchema and applies that schema to the input window Trades. IBM_MSFT_Splitter evaluates and routes data to one of three output windows. Event records with the symbol IBM or MSFT are sent to the IBM_MSFT_Tradeswin window. Event records where the product of trw.Price * trw.Volume is greater than 25,000 are sent to the Large_TradesWin window. All event records that do not meet the conditions placed on the two previous output windows are sent to the Other_Trades window.

CREATE SCHEMA TradeSchema (
Id long,
Symbol STRING,
Price MONEY(4),
Volume INTEGER,
TradeTime DATE
) ;
CREATE INPUT WINDOW Trades
SCHEMA TradeSchema
PRIMARY KEY (Id) ;
CREATE SPLITTER IBM_MSFT_Splitter
AS
WHEN trw.Symbol IN ('IBM', 'MSFT') THEN IBM_MSFT_Trades
WHEN trw.Price * trw.Volume > 25000 THEN Large_Trades
ELSE Other_Trades
SELECT trw. * FROM Trades trw ;
CREATE OUTPUT WINDOW IBM_MSFT_TradesWin
PRIMARY KEY DEDUCED
AS SELECT * FROM IBM_MSFT_Trades ;
CREATE OUTPUT WINDOW Large_TradesWin
PRIMARY KEY DEDUCED
AS SELECT * FROM Large_Trades ;
CREATE OUTPUT WINDOW Other_TradesWin
PRIMARY KEY DEDUCED
AS SELECT * FROM Other_Trades ;