Internal Pulsing

A stock market feed is a good example of several updates flowing into a stream.

Suppose the stock market feed keeps the last tick for each symbol. Some of the downstream calculations might be computationally expensive, and you might not need to recalculate on every change. You might want to recalculate only every second or every ten seconds. How can you collect and pulse the updates so that the expensive recalculations are done periodically instead of continuously?

The dictionary data structure and the timer facility allow you to code internal pulsing. Let's suppose that the stream to control is called InStream. First, define two local variables in the Flex operator:

integer version := 0; 
dictionary(typeof(InStream), integer) versionMap;

These two variables keep a current version and a version number for each record. The SPLASH code handling events from the input stream is:

{
  versionMap[InStream] := version; 
}

The special Timer block within the Flex operator sends the inserts and updates:

{ 
  for (k in versionMap) { 
    if (version = versionMap[k]) 
      output setOpcode(k, upsert); 
  } 
  version++; 
}

You can configure the interval between runs of the Timer block in numbers of seconds. Only those events with the current version get sent downstream, and the version number is incremented for the next set of updates.

This code works when InStream has only inserts and updates. It's a good exercise to extend this code to work with deletes.