Ideally, an applications request rate relates linearly to response rate as shown in Figure 2-1.
However, performance of any application depends on availability of resources like CPU, memory, network connections, and swap space. These resources are limited, and when they are exhausted, the response rate degrades. Due to resource limits, the response rate is expected to level off when the number of incoming requests reaches the point where resources are exhausted, as shown in Figure 2-2. However, in practice, an unlimited increase in incoming requests can cause performance to degrade; the response rate can drop in this case. In extreme cases, the application may run out of memory and abend or hang.
Figure 2-1: Ideal response rate curve
Figure 2-2: Expected response rate curve
Threshold monitoring allows you to configure the system to operate at a constant response rate and avoid out-of-memory conditions under high load conditions. EAServer uses these algorithms to heuristically govern the request rate when high load conditions are detected:
Memory monitoring You can configure thresholds for memory usage. EAServer monitors the memory used and throttles external requests when the critical threshold is reached.
Response time monitoring You can configure expected average response times for network requests and component method invocations. EAServer keeps a running average of the actual response time, and throttles handling of new requests when the average response time rises above the configured threshold. When the average response time drops below the threshold, the throttle is removed.
Threshold monitoring requires associating a thread monitor with the performance constraint to be monitored. Under stress conditions, the thread monitors Maximum Active Threads property throttles the load. When the performance threshold is crossed, new client requests must acquire the thread monitor before proceeding. If the thread monitor has no available threads, the request blocks temporarily until a thread is available. You can modify the thread monitor’s Maximum Active Threads property to tune the response rate curve.
