As we think about it this way, we realize that the closer ⍴ gets to 1, the more likely it is that an incoming item of work will find a busy server, and so will be queued. So increasing ⍴ increases queue depth, which increases latency. By a lot. In fact, it increases latency by an alarming amount as ⍴ goes it 1. One way to think about this is in terms of the number of items of work in the system, including being serviced by the server, and in the queue. For tradition's sake, we'll call this N and its mean (expectation) E[N].
E[N] = ⍴/(1-p) Maybe we need to draw that to show how alarming it is. Exponential curve up
brooker.co.za | Latency Sneaks Up On You - Marc's Blog
Josh BeckmanFiled under:
Same Source
Related Notes
- it's neat how *adding* affine measures is mathematically invali...from buttondown.email
- The student who overcomes this problem might learn the following us...from plover.com
- The `io_uring` interface works through two main data structures: th...from Glauber Costa
- as devices get extremely fast, interrupt-driven work is no longer a...from Glauber Costa
- In general, the chief compliance officer at any company has a dial ...from Matt Levine
- The discovery of scaling laws has typically preceded a boomtime for...from Josh Beckman
- Use a small model to generate a 'draft' output, then use a ...from Josh Beckman
- Let’s demonstrate the method with an example. Consider computing th...from Gregory Gundersen