Notes on Transactions

A Flex operator method processes one event at a time. The Event Stream Processor can, however, be fed data in transaction blocks (groups of insert, update, and delete events).

In such cases, the method is run on each event in the transaction block. The Event Stream Processor maintains an invariant: a stream takes in a transaction block, and produces a transaction block. It's always one block in, one block out. The Flex operator pulls apart the transaction block, and runs the method on each event within the block. All of the events that output are collected together. The Flex operator then atomically applies this block to its records, and sends the block to downstream streams.

If you happen to create a bad event in processing an event, the whole block is rejected. For example, if you try to output a record with any null key columns.

output [ | Shares = InStream.Shares; Price = InStream.Price; ];

This whole transaction block would be rejected. Likewise, if you try the following implicit insert:

output [Id = 4; | 
        Shares = InStream.Shares; 
        Price = InStream.Price; ];

If there is already a record in the Flex operator with Id set to 4, the block is rejected. You can get a report of bad transaction blocks by starting the Event Stream Processor with the -B option. Often it's better to ensure that key columns are not null, and use setOpcode to create upsert or safedelete events so that the transaction block is accepted.

Transaction blocks are made as small as possible before they are sent to other streams. For instance, if your code outputs two updates with the same keys, only the second update is sent downstream. If your code outputs an insert followed by a delete, both events are removed from the transaction block. Thus, you might output many events, but the transaction block might contain only some of them.