- Add query semaphore limiting concurrent database queries to 3
- Reuse single iterator in FetchEventsBySerials instead of one per serial
- Disable expensive e-tag fallback lookup in deletion processing
- Add empty ID validation in access tracker loop
- Reduce log spam from GetSerialById empty ID errors
These changes reduce memory usage from ~5GB to ~150MB under load by
limiting concurrent Badger iterators which consume significant memory.
Files modified:
- pkg/database/database.go: Add query semaphore with acquire/release
- pkg/database/fetch-events-by-serials.go: Reuse iterator for sev lookups
- pkg/database/query-events.go: Add semaphore, disable e-tag fallback
- pkg/database/get-serial-by-id.go: Don't log empty ID errors
- app/handle-req.go: Validate event ID before GetSerialById
- pkg/version/version: Bump to v0.52.17
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Introduced benchmark tests for various database operations, including event saving, querying, and fetching by serials, to assess performance.
- Implemented optimizations to reduce memory allocations and improve efficiency by pre-allocating slices and maps in critical functions.
- Enhanced the `FetchEventsBySerials`, `GetFullIdPubkeyBySerials`, and `QueryForIds` methods with pre-allocation strategies to minimize reallocations.
- Documented performance improvements in the new PERFORMANCE_REPORT.md file, highlighting significant reductions in execution time and memory usage.
- Bumped version to v0.23.1 to reflect these changes.
Removed old benchmark reports and detailed logs from the repository to clean up unnecessary files. These reports appear to be auto-generated and no longer relevant for ongoing development.
Replaced legacy `*.orly` module imports with `next.orly.dev/pkg` paths across the codebase for consistency. Removed legacy `go.mod` files from sub-packages, consolidating dependency management. Added Dockerfiles and configurations for benchmarking environments.
woikos
mleku