Incremental Parsing

Incremental parsing improves replication performance especially for replication environments with multiple Replication Servers in the replication route, and reduces memory consumption for HVAR, RTL, and DSI bulk-copy-in.

The single DSI Scheduler (DSI-S) thread parses data and then profiles transactions to determine whether to use the strategy of DSI bulk-copy-in, high volume adaptive replication (HVAR), or real-time loading (RTL) to apply transactions to the replicate database. DSI-S then conveys the parsed data and profiled transactions to one or more DSI Executor (DSI-E) threads that wait, in turn, for the single DSI-S thread to process data. To avoid a second parsing by the DSI-E thread to profile the data for the replicate database, Replication Server stores the entire parsed data in the SQT cache. However, this requires a large amount of memory, since Replication Server does not delete the data until more transactions are available to optimally compile transactions for HVAR or RTL, or to group transactions for DSI bulk-copy-in.

Incremental parsing allows the DSI-S thread to parse only table-level information, which is the first part of any DML command that the DSI-S thread processes. The DSI-S thread transfers the parsing tasks for the second part of the command to multiple DSI-E threads. With incremental parsing:

Performance improves because throughput increases and parsing is more efficient because the data is available to the DSI-E threads earlier and the parsing load is distributed over multiple DSI-E threads.
Because the SQT cache stores only the first part of the command, there is less memory consumption, leaving more memory available for effective use of the SQM cache and SQM command cache, inbound and outbound direct command replication, and metadata reduction.

Set dsi_incremental_parsing to on to enable incremental parsing by the DSI-S thread when you enable high volume adaptive replication (HVAR), real-time loading (RTL), or DSI bulk-copy-in.

Note: dsi_incremental_parsing takes effect only if either dsi_compile_enable or dsi_bulk_copy is also set to on. Otherwise, Replication Server ignores dsi_incremental_parsing.

Default: on

The replicate Replication Servers must be version 15.7.1 SP100 or later to support incremental parsing.

Use dsi_incremental_parsing with:

alter connection and create connection – to enable incremental parsing at the connection level for the specified database.
Any change to the parameter setting takes effect immediately.
configure replication server – to enable incremental parsing at the server level for all connections.
You must suspend and resume connections for any change to the parameter to take effect.

Connection-level settings override server-level settings.

See Real-Time Loading Solution in the Replication Server Heterogeneous Guide.