You can use three new features—guaranteed delivery, consistent recovery, and auto checkpoint—to protect a project against data loss in the event of a server failure or loss of connection. This set of features supports the goal of zero data loss.
For detailed discussions of guaranteed delivery, consistent recovery, and auto checkpoint, see the Programmers Guide. See the Studio Users Guide for instructions on enabling the new options.
With guaranteed delivery (GD), a GD-enabled subscriber to a GD-enabled window receives all the data processed by that window even if the client is not connected when the data is produced. GD is supported only on windows assigned to log stores. When you enable guaranteed delivery on a window that has registered guaranteed delivery subscribers, the window stores a copy of every event it produces in its log store until all the registered guaranteed delivery subscribers acknowledge receiving the events. You can subscribe to windows configured for GD using adapters, bindings, the SDKs, or the esp_subscribe tool. You can get information about windows configured for GD by subscribing to the _ESP_GD_Sessions and _ESP_Streams metadata streams—see the Configuration and Administration Guide for details.
The consistent recovery feature can restore all the windows in a project to a consistent state after a server or connection failure. (Recovery consistency depends on following guidelines for log stores.) When consistent recovery is enabled, the server uses coordinated checkpoints to save data in log stores. When any log store fails to complete a checkpoint, all the log stores for that project roll back to their state as of the previous successful checkpoint. This rule ensures that if a server stops unexpectedly, all log stores in a project remain consistent with one another. However, any input data that has not been checkpointed is not recovered upon restart.
Consistent recovery relies on data being checkpointed (registered and saved in the project’s log stores). Auto checkpoint, which is optional, lets you configure the checkpoint interval—the number of input transactions that triggers a checkpoint.