Circuit Breaker Overhead
Table of Contents
Benchmarks
This file was last exported: 2014-10-21 Tue 16:33
Benchmarks run with 2,000,000 documents indexed with a numeric, incrementing field (long type). 100 queries are run per-thread with 100 warmup queries run. Each index has 2 primary shards and no replicas.
Times based on the took field in the response.
Each "query" is a match_all query with "search_type=count" with a single
aggregation, a terms aggregation on the numeric field for the top 25 numeric
terms.
Using a debugger I found that the size of the Request breaker (meaning the size of the LongHash using BigArrays) was 29.213mb for each request sent.
Laptop
Run on my MacBook Air
| Test | average time (ms) | queries per thread | threads |
|---|---|---|---|
| single-threaded, with breaker | 436 | 100 | 1 |
| single-threaded, noop breaker | 440 | 100 | 1 |
| single-threaded, with breaker (2nd run) | 476 | 100 | 1 |
| single-threaded, noop breaker (2nd run) | 465 | 100 | 1 |
| multi-threaded, with breaker | 1770 | 100 | 5 |
| multi-threaded, noop breaker | 1757 | 100 | 5 |
| multi-threaded, with breaker | 3432 | 100 | 10 |
| multi-threaded, noop breaker | 3450 | 100 | 10 |
Desktop
Run on my desktop
| Test | average time (ms) | queries per thread | threads |
|---|---|---|---|
| single-threaded, with breaker | 192 | 100 | 1 |
| single-threaded, noop breaker | 192 | 100 | 1 |
| single-threaded, with breaker (2nd run) | 205 | 100 | 1 |
| single-threaded, noop breaker (2nd run) | 207 | 100 | 1 |
| single-threaded, with breaker (3rd run) | 199 | 100 | 1 |
| single-threaded, noop breaker (3rd run) | 201 | 100 | 1 |
| single-threaded, with breaker (4th run) | 187 | 100 | 1 |
| single-threaded, noop breaker (4th run) | 189 | 100 | 1 |
| multi-threaded, with breaker | 398 | 100 | 5 |
| multi-threaded, noop breaker | 396 | 100 | 5 |
| multi-threaded, with breaker | 664 | 100 | 10 |
| multi-threaded, noop breaker | 663 | 100 | 10 |
| multi-threaded, with breaker | 1356 | 100 | 20 |
| multi-threaded, noop breaker | 1368 | 100 | 20 |
| multi-threaded, with breaker | 1939 | 100 | 30 |
| multi-threaded, noop breaker | 1972 | 100 | 30 |
Old benchmarks
These were before it was really enough data to count, and before I changed it to be a benchmark instead of just a test.
Info
This is with 20,000 documents indexed with a numeric incrementing field (long
type). 100 queries are run per-thread with 10 warmup queries run before taking
any kind of measurements. The times are from the took field in the response.
Each request is a terms aggregation on the "long" type field, so that it uses
LongHash, which uses BigArrays, which uses the circuit breaker.
Single-threaded
This test I ran 10 times for each test, to get a variety of results.
Without Circuit Breaker:
--> min: 43ms, max: 84ms, total: 5837ms, average: 58ms --> min: 42ms, max: 129ms, total: 5762ms, average: 57ms --> min: 42ms, max: 70ms, total: 5456ms, average: 54ms --> min: 46ms, max: 85ms, total: 5777ms, average: 57ms --> min: 49ms, max: 128ms, total: 6663ms, average: 66ms --> min: 47ms, max: 156ms, total: 6390ms, average: 63ms --> min: 45ms, max: 68ms, total: 5350ms, average: 53ms --> min: 45ms, max: 96ms, total: 5626ms, average: 56ms --> min: 45ms, max: 116ms, total: 5657ms, average: 56ms --> min: 46ms, max: 94ms, total: 5701ms, average: 57ms
With Circuit Breaker:
--> min: 47ms, max: 118ms, total: 6293ms, average: 62ms --> min: 45ms, max: 89ms, total: 5866ms, average: 58ms --> min: 43ms, max: 141ms, total: 5999ms, average: 59ms --> min: 44ms, max: 76ms, total: 5478ms, average: 54ms --> min: 44ms, max: 90ms, total: 5509ms, average: 55ms --> min: 45ms, max: 88ms, total: 5506ms, average: 55ms --> min: 44ms, max: 88ms, total: 5635ms, average: 56ms --> min: 45ms, max: 86ms, total: 5572ms, average: 55ms --> min: 45ms, max: 80ms, total: 5496ms, average: 54ms --> min: 45ms, max: 87ms, total: 5485ms, average: 54ms
Doesn't look like there is a big difference for single-threaded operations.
Multi-threaded (10 threads)
Without Circuit Breaker:
--> [55] min: 64ms, max: 494ms, total: 32744ms, average: 327ms --> [56] min: 100ms, max: 679ms, total: 32944ms, average: 329ms --> [57] min: 69ms, max: 494ms, total: 32364ms, average: 323ms --> [58] min: 106ms, max: 515ms, total: 32737ms, average: 327ms --> [59] min: 69ms, max: 531ms, total: 32370ms, average: 323ms --> [60] min: 78ms, max: 438ms, total: 31277ms, average: 312ms --> [61] min: 75ms, max: 468ms, total: 32975ms, average: 329ms --> [62] min: 77ms, max: 494ms, total: 32543ms, average: 325ms --> [63] min: 76ms, max: 484ms, total: 32945ms, average: 329ms --> [64] min: 117ms, max: 535ms, total: 33351ms, average: 333ms
With Circuit Breaker:
--> [62] min: 96ms, max: 758ms, total: 37192ms, average: 371ms --> [63] min: 148ms, max: 558ms, total: 37568ms, average: 375ms --> [64] min: 91ms, max: 584ms, total: 37921ms, average: 379ms --> [65] min: 67ms, max: 532ms, total: 38106ms, average: 381ms --> [66] min: 115ms, max: 511ms, total: 37846ms, average: 378ms --> [67] min: 152ms, max: 552ms, total: 37474ms, average: 374ms --> [68] min: 65ms, max: 706ms, total: 37146ms, average: 371ms --> [69] min: 109ms, max: 530ms, total: 36056ms, average: 360ms --> [70] min: 140ms, max: 611ms, total: 37739ms, average: 377ms --> [71] min: 89ms, max: 559ms, total: 36279ms, average: 362ms
Second run
Without Circuit Breaker:
--> [62] min: 127ms, max: 603ms, total: 33183ms, average: 331ms --> [63] min: 81ms, max: 633ms, total: 33674ms, average: 336ms --> [64] min: 68ms, max: 711ms, total: 33605ms, average: 336ms --> [65] min: 132ms, max: 701ms, total: 33279ms, average: 332ms --> [66] min: 52ms, max: 551ms, total: 32814ms, average: 328ms --> [67] min: 90ms, max: 721ms, total: 32409ms, average: 324ms --> [68] min: 124ms, max: 740ms, total: 33094ms, average: 330ms --> [69] min: 94ms, max: 654ms, total: 33062ms, average: 330ms --> [70] min: 77ms, max: 677ms, total: 33357ms, average: 333ms --> [71] min: 77ms, max: 626ms, total: 33348ms, average: 333ms
With Circuit Breaker:
--> [62] min: 72ms, max: 558ms, total: 32335ms, average: 323ms --> [63] min: 82ms, max: 639ms, total: 32296ms, average: 322ms --> [64] min: 76ms, max: 587ms, total: 31387ms, average: 313ms --> [65] min: 79ms, max: 659ms, total: 32142ms, average: 321ms --> [66] min: 78ms, max: 484ms, total: 31106ms, average: 311ms --> [67] min: 92ms, max: 665ms, total: 32882ms, average: 328ms --> [68] min: 76ms, max: 560ms, total: 30733ms, average: 307ms --> [69] min: 92ms, max: 583ms, total: 31302ms, average: 313ms --> [70] min: 93ms, max: 671ms, total: 32339ms, average: 323ms --> [71] min: 70ms, max: 661ms, total: 32587ms, average: 325ms
Multi-threaded (5 threads)
Without Circuit Breaker:
--> [63] min: 64ms, max: 301ms, total: 12021ms, average: 120ms --> [64] min: 67ms, max: 261ms, total: 12180ms, average: 121ms --> [65] min: 64ms, max: 292ms, total: 12300ms, average: 123ms --> [66] min: 43ms, max: 387ms, total: 12751ms, average: 127ms --> [67] min: 55ms, max: 287ms, total: 12726ms, average: 127ms
With Circuit Breaker:
--> [63] min: 60ms, max: 193ms, total: 10521ms, average: 105ms --> [64] min: 60ms, max: 184ms, total: 10843ms, average: 108ms --> [65] min: 60ms, max: 211ms, total: 10393ms, average: 103ms --> [66] min: 60ms, max: 250ms, total: 11279ms, average: 112ms --> [67] min: 56ms, max: 195ms, total: 10910ms, average: 109ms
Run 2 (because the first run contained results I did not expect)
Without Circuit Breaker:
--> [106] min: 57ms, max: 530ms, total: 13827ms, average: 138ms --> [107] min: 71ms, max: 485ms, total: 13849ms, average: 138ms --> [108] min: 70ms, max: 508ms, total: 14214ms, average: 142ms --> [109] min: 68ms, max: 593ms, total: 13263ms, average: 132ms --> [110] min: 73ms, max: 447ms, total: 13255ms, average: 132ms
With Circuit Breaker:
--> [62] min: 61ms, max: 227ms, total: 11946ms, average: 119ms --> [63] min: 63ms, max: 231ms, total: 11275ms, average: 112ms --> [64] min: 63ms, max: 241ms, total: 11795ms, average: 117ms --> [65] min: 63ms, max: 253ms, total: 12203ms, average: 122ms --> [66] min: 64ms, max: 241ms, total: 12050ms, average: 120ms