Example: Using the SpPersistentSubscribeSource Component

The SpPersistentSubscribeSource component subscribes to the Server using persistent subscribe (stores subscribed records until it processes them, and then deletes them).

To implement this, a log stream (Stream1_log) and truncate stream (TruncateStream1) are created for stream "Stream1". Stream1_log stores the data and TruncateStream1 has two columns: primary key and sequence number. See the model.ccl in the bin folder for more details.

Incoming records are transferred to Stream1_log with an additional sequencenumber column. Once records are processed from Stream1_log, the last sequence number is published to TruncateStream1. All records with sequence numbers smaller than or equal to the published sequencenumber are then deleted from the Stream1_log.

  1. Start Event Stream Processor.
    Operating System Step
    UNIX Open a terminal window:
    1. Start the example cluster:

      start_server_cluster.sh

    2. Start the project on the cluster:

      start_project.sh
    Windows Open a command window:
    1. Start the example cluster:

      start_server_cluster.bat

    2. Add project to the cluster, and start it on the cluster:

      start_project.bat
  2. Start esp_subscriber to subscribe to Stream1 of the project running on the cluster above.
    Operating System Step
    UNIX Open a terminal window and enter:

    ./esp-subscribe-Stream1.sh
    Windows Open a command window and enter:

    esp-subscribe-Stream1.bat
  3. Subscribe to the log stream, Stream1_Log.
    Operating System Step
    UNIX Open a terminal window and enter:

    ./esp-subscribe-Stream1_log.sh
    Windows Open a command window and enter:

    esp-subscribe-Stream1_log.bat
  4. Subscribe to the log stream, Truncate_stream1.
    Operating System Step
    UNIX Open a terminal window and enter:

    ./esp-subscribe-TruncateStream1.sh
    Windows Open a command window and enter:

    esp-subscribe-TruncateStream1.bat
  5. Start the SpPersistentSubscribeSource and FileSink components.
    Operating System Step
    UNIX Open a terminal window and enter:

    ./asapToFile.sh
    Windows Open a command window and enter:

    asapToFile.bat
  6. Upload data from the esp_insert.txt file to the Server.
    Operating System Step
    UNIX Open a terminal window and enter:

    ./esp_upload.sh
    Windows Open a command window and enter:

    esp_upload.bat
    SpPersistentSubscribeSource subscribes to the Server and Stream1_log, and passes these records to FileSink, which writes these records to the out.txt file. All the subscription script files show the respective subscriptions.
Parent topic: Examples
Related reference
SpPersistentSubscribeSource Properties