Achieving Zero Data Loss

A lost connection or a server crash can cause data produced by a project to be lost before it is delivered to a listening client. If you cannot afford to lose data, complete these tasks to configure zero data loss.

Task For Instructions, see...
Create a project with a guaranteed delivery window or add a GD window to an existing project. You can do this in Studio or by adding CREATE WINDOW or CREATE FLEX statements to the project’s CCL file. Studio Users Guide

Programmers Reference

Set up log stores for any windows in the project that cannot be recovered by an upstream provider. Review the guidelines, restrictions, and sizing instructions for log stores to ensure that your project can accurately and completely recreate its data after a restart.

Creating a Log Store and all subsections

Variables and SPLASH data structures (dictionaries, vectors, and event caches) do not persist in log stores and thus cannot be recovered after a failure. Use these structures with log stores only when:
  • You can provide logic to reconstruct the structures on restart, or
  • Processing will not be affected if the structures are missing after a restart.
– 
Enable guaranteed delivery on any bindings associated with GD-enabled windows. Studio Users Guide
Before you deploy the project, enable these options in the project configuration (CCR) file:
  • Failover
  • (Optional) Auto Checkpoint
    Note: SAP recommends that you enable Auto Checkpoint only if you do not configure publishers of inbound data to trigger checkpoints by issuing commits.
  • Consistent Recovery
    Note: SAP does not recommend enabling active-active HA mode for projects where consistent recovery is enabled. Active-active mode does not safeguard against data loss or data inconsistency when the project switches from the primary instance to the secondary.
Studio Users Guide

Configuration and Administration Guide

Enable guaranteed delivery on the project’s adapters.
Note: Some adapters do not support guaranteed delivery. See the Adapter Summary in the Adapters Guide for information on GD support.
Adapters Guide
If auto checkpoint is not enabled, configure publishers sending data to your project to issue commits to trigger server checkpoints. When the commit call returns, the publisher knows that the server has checkpointed the data.
Note: When a project receives data from more than one publisher, a commit issued by one publisher triggers checkpointing of data sent by all publishers.

Configure publishers to retransmit any data for which they do not receive a commit confirmation.

C SDK Guide

Java SDK Guide

.NET SDK Guide

Set up guaranteed delivery subscriptions for client applications that consume data from your project.To confirm that subscribers have received and processed data, configure them to send GD commit calls in response to checkpoint notifications. C SDK Guide

Java SDK Guide

.NET SDK Guide

Related concepts
Consistent Recovery
Auto Checkpoint
Guaranteed Delivery