Publishing

The SDK provides various options for publishing data to a project.

The steps involved in publishing data are:

  1. Create a NetEspPublisher from a previously connected NetEspProject instance.
  2. Create a NetEspMessageWriter for the stream to publish to. You can create multiple NetEspMessageWriters from a single NetEspPublisher.
  3. Create a NetEspRelativeRowWriter.
  4. Format the data buffer to publish using NetEspRelativeRowWriter methods.
  5. Publish the data.

While NetEspPublisher is thread-safe, NetEspMessageWriter and NetEspRelativeRowWriter are not. Therefore, ensure that you synchronize access to the latter two.

The SDK provides a number of options to tune the behavior of a NetEspPublisher. Specify these options using NetEspPublisherOptions when creating the NetEspPublisher. Once created, options cannot be changed. Like all other entities in the SDK, publishing also supports the direct, callback, and select access modes.

In addition to access modes, the SDK supports internal buffering. When publishing is buffered, the data is first written to an internal queue. This is picked up by a publishing thread and then written to the ESP project. Buffering is possible only in direct access mode. Direct and buffered publishing potentially provides the best throughput.

Two other settings influence publishing: batching mode and sync mode. Batching controls how data rows are written to the socket. They can be written individually or grouped together in either envelope or transaction batches. Envelopes group individual rows together to send to the ESP project and are read together from the socket by the project. This improves network throughput. Transaction batches, like envelope batches, are also written and read in groups. However, with transaction batches, the ESP project only processes the group if all the rows in the batch are processed successfully. If one fails, the whole batch is rolled back.

Sync mode settings control the publishing handshake between the SDK and the ESP project. By default, the SDK sends data to the ESP project without waiting for acknowledgement. But if sync mode is set to true, the SDK waits for acknowledgement from the ESP project before sending the next batch of data. This provides an application level delivery guarantee, but it reduces throughput.

Publishing in async mode improves throughput, but does not provide an application level delivery guarantee. Since TCP does not provide an application level delivery guarantee either, data in the TCP buffer could be lost when a client exits. Therefore, a commit must be executed before a client exit when publishing in async mode.

There are certain considerations to keep in mind when using callback or select mode publishing. These modes are driven by the NET_ESP_PUBLISHER_EVENT_READY event, which indicates that the publisher is ready to accept more data. In response, you can publish data or issue a commit, but only one such action is permitted in response to a single NET_ESP_PUBLISHER_EVENT_READY event.

Like all entities, if you intend to work in callback mode with a Publisher and want to get notified, register the callback handler before the event is triggered. For example:
net_esp_publisher_options_set_access_mode(options, CALLBACK_ACCESS, error);
net_esp_publisher_set_callback(publisher, events, callback, NULL, error)
net_esp_publisher_connect(publisher, error);