Retention

In CCL, you can specify a retention policy when defining a Window. You can also create an Unnamed Window by specifying a retention policy on a Window or Delta Stream when it is used as source to another element.

Retention is specified through the KEEP clause. You can limit the number of records in a window based on either the number, or age, of records in the window. These methods are referred to as count-based retention and time-based retention, respectively. If you do not specify a retention policy, the window retains all records. Or, you can use the ALL modifier to explicitly specify that the window should retain all records.

Specifying the KEEP clause with no modifier produces a Sliding Window, which deletes individual rows once a maximum age is reached or the maximum number of rows are retained.

Count-based Retention

In a count-based policy, a constant integer specifies the maximum number of rows retained in the window. You can use parameters in the count expression.

A count-based policy also defines an optional SLACK value, which can enhance performance by requiring less frequent cleaning of memory stores. A SLACK value accomplishes this by ensuring that there are no more than N + S rows in the window, where N is the retention size and S is the SLACK value. When the window reaches N + S rows, the system purges S rows. The larger the SLACK value, the better the performance, since there is less cleaning required.

The default value for SLACK is 1, which means that after the window reaches the maximum number of records, every new record inserted deletes the oldest record. This causes a significant impact on performance. Larger slack values improve performance by reducing the need to constantly delete rows.

The following example creates a Sliding Window that retains the most recent 100 records that match the filter condition. Once there are 100 records in the window, the arrival of a new record causes the deletion of the oldest record in the window.

CREATE WINDOW Last100Trades PRIMARY KEY DEDUCED 
KEEP 100 ROWS
AS SELECT * FROM Trades
WHERE Trades.Volume > 1000;

Adding the SLACK value of 10 means the window may contain as many as 110 records before any records are deleted.

CREATE WINDOW Last100Trades PRIMARY KEY DEDUCED 
KEEP 100 ROWS SLACK 10
AS SELECT * FROM Trades
WHERE Trades.Volume > 1000;

Time-based Retention

In a time-based policy, a constant interval expression specifies the maximum age of the rows retained in the window.

The following example creates a Sliding Window that retains each record received for ten minutes. As each individual row exceeds the ten minute retention time limit, it is deleted.

CREATE WINDOW RecentPositions PRIMARY KEY DEDUCED
KEEP 10 MINS
AS SELECT * FROM Positions;