Knowledge of the replication environment and Replication Agent statistics is required to tune Replication Agent for optimal performance. A trial-and-error approach may be necessary to achieve optimal performance for each environment.
Memory Use
The maximum memory available to Replication Agent is the primary consideration: If too little memory is allocated for the configuration, schema, and transaction profile, Java Virtual Machine (JVM) garbage collection can degrade performance significantly. There are five Replication Agent statistics that provide information about memory use. Specifically, by monitoring the percentage of maximum memory used over time, you can determine whether or not the amount of memory is sufficient for a configuration.
The queue (or buffer) and cache size parameters are the primary configuration parameters that determine memory use. Using the ra_statistics or the ra_statrack command to monitor the queue and cache sizes, along with the percentage of memory used, can help you determine whether to change the configuration to accommodate the amount of memory use allowed or, conversely, whether to change the amount of memory use allowed to accommodate the configuration. By decreasing the queue and cache sizes, the percentage of maximum memory use should decrease, resulting in decreased garbage collection and improved performance. Conversely, increasing the maximum memory available should result in improved performance. In general, if the amount of memory used never drops below 50% of the maximum, garbage collection may be hampering Replication Agent performance.
Threads
Replication Agent is a multithreaded application, and several threads are active during processing, even though the log records are usually read and processed serially. This means that, in most cases, each thread is dependent on another thread, so a bottleneck in one thread can result in poor replication performance in the entire system.
Parallel LTL Formatting
Replication Agent can format LTL commands in parallel, improving performance on machines with multiple processors. The LTL formatter is an internal component of Replication Agent that converts change sets into LTL commands. In versions of Replication Agent earlier than 15.6, the LTL formatter has been a performance bottleneck because items in the log transfer interface (LTI) queue could be processed only serially. Replication Agent now provides a multithreaded LTL formatter that processes items in the LTI queue in parallel. This may improve performance on machines with multiple processors.
Replication Agent uses two configuration parameters for configuring the size of the LTI queue and the number of LTL formatter threads: lti_max_buffer_size and lti_formatter_count. Replication Agent also provides LTI statistics to assist you in performance and tuning.
Statistics and Queue Bottlenecks
Replication Agent statistics, analyzed over time, identify bottlenecks in Replication Agent. The queue (or buffer) size statistics provide a likely indication of where a Replication Agent bottleneck is occurring. For example, if the Log Transfer Interface (LTI) inbound queue is at or near capacity, but the LTI outbound queue is at or near zero, the LTI inbound queue—where LTL is formatted—is probably the bottleneck decreasing replication performance. In this way, queue sizes can identify specific areas of trouble:
- An LTI outbound queue size at or near capacity indicates that the Replication Agent is waiting on Replication Server. In this case, all queues in the Replication Agent are also at or near capacity.
- An LTI inbound queue size at or near capacity but with an LTI outbound queue size at or near zero indicates a bottleneck in Replication Agent LTL formatting. This may result from the primary database character set being different than the Replication Server character set, resulting in Replication Agent having to convert character data between character sets. Alternately, the bottleneck may result from the Replication Server RSSD not being used effectively, either because the RSSD is not being used at all—the use_rssd parameter is set to false—or because minimal columns functionality—as determined by the column_compression and lti_send_only_primary keys parameters—is not being used.
- A Log Reader operation queue size at or near capacity but with LTI queue sizes at or near zero indicates a bottleneck in Replication Agent operation processing. This may result from a problem with RASD access or with a cache configuration.
- A queue size at or near zero may indicate a bottleneck on the log scan, since the Log Reader scan queue is the first queue used in Replication Agent processing. This bottleneck may result from input-output inefficiency, possibly due to the read block count being too low or to Replication Agent having to read a log over the network rather than locally.
Character Set Conversion
Character set differences in the replication environment can also affect replication performance. For example, if the primary database character set is different from the Replication Server character set, Replication Agent performs character set conversion for all character data that is replicated. If Replication Agent has to perform character set conversion, a warning message is logged in the Replication Agent system log on replication start-up.