As we think about it this way, we realize that the closer ⍴ gets to 1, the more likely it is that an incoming item of work will find a busy server, and so will be queued. So increasing ⍴ increases queue depth, which increases latency. By a lot. In fact, it increases latency by an alarming amount as ⍴ goes it 1. One way to think about this is in terms of the number of items of work in the system, including being serviced by the server, and in the queue. For tradition's sake, we'll call this N and its mean (expectation) E[N].

E[N] = ⍴/(1-p) Maybe we need to draw that to show how alarming it is. Exponential curve up

Latency Sneaks Up On You - Marc's Blog by brooker.co.za favicon