Browse Source
Introduced comprehensive benchmarks for `next-orly-badger`, `next-orly-neo4j`, and `nostr-rs-relay` backends, covering peak throughput, burst patterns, mixed read/write, query, and concurrent query/store tests. Reports include detailed performance metrics (e.g., events/sec, latency, success rates) and are saved as text and AsciiDoc formats. Aggregate summary also generated for testing consistency across relay implementations.main
mleku
1 month ago
13 changed files with 1800 additions and 95 deletions
@ -0,0 +1,9 @@ |
|||||||
|
{ |
||||||
|
"permissions": { |
||||||
|
"allow": [ |
||||||
|
"Bash(ls:*)" |
||||||
|
], |
||||||
|
"deny": [], |
||||||
|
"ask": [] |
||||||
|
} |
||||||
|
} |
||||||
@ -0,0 +1,176 @@ |
|||||||
|
================================================================ |
||||||
|
NOSTR RELAY BENCHMARK AGGREGATE REPORT |
||||||
|
================================================================ |
||||||
|
Generated: 2025-12-03T22:47:18+00:00 |
||||||
|
Benchmark Configuration: |
||||||
|
Events per test: 50000 |
||||||
|
Concurrent workers: 24 |
||||||
|
Test duration: 60s |
||||||
|
|
||||||
|
Relays tested: 8 |
||||||
|
|
||||||
|
================================================================ |
||||||
|
SUMMARY BY RELAY |
||||||
|
================================================================ |
||||||
|
|
||||||
|
Relay: rely-sqlite |
||||||
|
---------------------------------------- |
||||||
|
Status: COMPLETED |
||||||
|
Events/sec: 16694.18 |
||||||
|
Events/sec: 6270.43 |
||||||
|
Events/sec: 16694.18 |
||||||
|
Success Rate: 100.0% |
||||||
|
Success Rate: 100.0% |
||||||
|
Success Rate: 100.0% |
||||||
|
Avg Latency: 1.339216ms |
||||||
|
Bottom 10% Avg Latency: 779.552µs |
||||||
|
Avg Latency: 1.281976ms |
||||||
|
P95 Latency: 1.894111ms |
||||||
|
P95 Latency: 2.087148ms |
||||||
|
P95 Latency: 910.529µs |
||||||
|
|
||||||
|
Relay: next-orly-badger |
||||||
|
---------------------------------------- |
||||||
|
Status: COMPLETED |
||||||
|
Events/sec: 17987.95 |
||||||
|
Events/sec: 6246.39 |
||||||
|
Events/sec: 17987.95 |
||||||
|
Success Rate: 100.0% |
||||||
|
Success Rate: 100.0% |
||||||
|
Success Rate: 100.0% |
||||||
|
Avg Latency: 1.16914ms |
||||||
|
Bottom 10% Avg Latency: 675.419µs |
||||||
|
Avg Latency: 1.301155ms |
||||||
|
P95 Latency: 1.605171ms |
||||||
|
P95 Latency: 2.260728ms |
||||||
|
P95 Latency: 911.513µs |
||||||
|
|
||||||
|
Relay: next-orly-neo4j |
||||||
|
---------------------------------------- |
||||||
|
Status: COMPLETED |
||||||
|
Events/sec: 17437.04 |
||||||
|
Events/sec: 6060.82 |
||||||
|
Events/sec: 17437.04 |
||||||
|
Success Rate: 100.0% |
||||||
|
Success Rate: 100.0% |
||||||
|
Success Rate: 100.0% |
||||||
|
Avg Latency: 1.250926ms |
||||||
|
Bottom 10% Avg Latency: 735.971µs |
||||||
|
Avg Latency: 1.493295ms |
||||||
|
P95 Latency: 1.757814ms |
||||||
|
P95 Latency: 2.404304ms |
||||||
|
P95 Latency: 896.796µs |
||||||
|
|
||||||
|
Relay: khatru-sqlite |
||||||
|
---------------------------------------- |
||||||
|
Status: COMPLETED |
||||||
|
Events/sec: 17531.15 |
||||||
|
Events/sec: 6335.87 |
||||||
|
Events/sec: 17531.15 |
||||||
|
Success Rate: 100.0% |
||||||
|
Success Rate: 100.0% |
||||||
|
Success Rate: 100.0% |
||||||
|
Avg Latency: 1.233875ms |
||||||
|
Bottom 10% Avg Latency: 707.713µs |
||||||
|
Avg Latency: 1.239192ms |
||||||
|
P95 Latency: 1.713051ms |
||||||
|
P95 Latency: 1.880869ms |
||||||
|
P95 Latency: 918.848µs |
||||||
|
|
||||||
|
Relay: khatru-badger |
||||||
|
---------------------------------------- |
||||||
|
Status: COMPLETED |
||||||
|
Events/sec: 15958.89 |
||||||
|
Events/sec: 6352.78 |
||||||
|
Events/sec: 15958.89 |
||||||
|
Success Rate: 100.0% |
||||||
|
Success Rate: 100.0% |
||||||
|
Success Rate: 100.0% |
||||||
|
Avg Latency: 1.414398ms |
||||||
|
Bottom 10% Avg Latency: 791.675µs |
||||||
|
Avg Latency: 1.183812ms |
||||||
|
P95 Latency: 2.191322ms |
||||||
|
P95 Latency: 1.80172ms |
||||||
|
P95 Latency: 903.25µs |
||||||
|
|
||||||
|
Relay: relayer-basic |
||||||
|
---------------------------------------- |
||||||
|
Status: COMPLETED |
||||||
|
Events/sec: 17757.23 |
||||||
|
Events/sec: 6227.33 |
||||||
|
Events/sec: 17757.23 |
||||||
|
Success Rate: 100.0% |
||||||
|
Success Rate: 100.0% |
||||||
|
Success Rate: 100.0% |
||||||
|
Avg Latency: 1.193531ms |
||||||
|
Bottom 10% Avg Latency: 679.232µs |
||||||
|
Avg Latency: 1.304ms |
||||||
|
P95 Latency: 1.679267ms |
||||||
|
P95 Latency: 2.155365ms |
||||||
|
P95 Latency: 930.632µs |
||||||
|
|
||||||
|
Relay: strfry |
||||||
|
---------------------------------------- |
||||||
|
Status: COMPLETED |
||||||
|
Events/sec: 17794.50 |
||||||
|
Events/sec: 6252.01 |
||||||
|
Events/sec: 17794.50 |
||||||
|
Success Rate: 100.0% |
||||||
|
Success Rate: 100.0% |
||||||
|
Success Rate: 100.0% |
||||||
|
Avg Latency: 1.192717ms |
||||||
|
Bottom 10% Avg Latency: 676.594µs |
||||||
|
Avg Latency: 1.270724ms |
||||||
|
P95 Latency: 1.645564ms |
||||||
|
P95 Latency: 2.251457ms |
||||||
|
P95 Latency: 915.623µs |
||||||
|
|
||||||
|
Relay: nostr-rs-relay |
||||||
|
---------------------------------------- |
||||||
|
Status: COMPLETED |
||||||
|
Events/sec: 17174.61 |
||||||
|
Events/sec: 6311.06 |
||||||
|
Events/sec: 17174.61 |
||||||
|
Success Rate: 100.0% |
||||||
|
Success Rate: 100.0% |
||||||
|
Success Rate: 100.0% |
||||||
|
Avg Latency: 1.281647ms |
||||||
|
Bottom 10% Avg Latency: 742.249µs |
||||||
|
Avg Latency: 1.260479ms |
||||||
|
P95 Latency: 1.836808ms |
||||||
|
P95 Latency: 1.893887ms |
||||||
|
P95 Latency: 922.647µs |
||||||
|
|
||||||
|
|
||||||
|
================================================================ |
||||||
|
DETAILED RESULTS |
||||||
|
================================================================ |
||||||
|
|
||||||
|
Individual relay reports are available in: |
||||||
|
- /reports/run_20251203_222024/khatru-badger_results.txt |
||||||
|
- /reports/run_20251203_222024/khatru-sqlite_results.txt |
||||||
|
- /reports/run_20251203_222024/next-orly-badger_results.txt |
||||||
|
- /reports/run_20251203_222024/next-orly-neo4j_results.txt |
||||||
|
- /reports/run_20251203_222024/nostr-rs-relay_results.txt |
||||||
|
- /reports/run_20251203_222024/relayer-basic_results.txt |
||||||
|
- /reports/run_20251203_222024/rely-sqlite_results.txt |
||||||
|
- /reports/run_20251203_222024/strfry_results.txt |
||||||
|
|
||||||
|
================================================================ |
||||||
|
BENCHMARK COMPARISON TABLE |
||||||
|
================================================================ |
||||||
|
|
||||||
|
Relay Status Peak Tput/s Avg Latency Success Rate |
||||||
|
---- ------ ----------- ----------- ------------ |
||||||
|
rely-sqlite OK 16694.18 1.339216ms 100.0% |
||||||
|
next-orly-badger OK 17987.95 1.16914ms 100.0% |
||||||
|
next-orly-neo4j OK 17437.04 1.250926ms 100.0% |
||||||
|
khatru-sqlite OK 17531.15 1.233875ms 100.0% |
||||||
|
khatru-badger OK 15958.89 1.414398ms 100.0% |
||||||
|
relayer-basic OK 17757.23 1.193531ms 100.0% |
||||||
|
strfry OK 17794.50 1.192717ms 100.0% |
||||||
|
nostr-rs-relay OK 17174.61 1.281647ms 100.0% |
||||||
|
|
||||||
|
================================================================ |
||||||
|
End of Report |
||||||
|
================================================================ |
||||||
Loading…
Reference in new issue