Legacy events stored before compact format (v6 migration) may have IDs
that start with byte 0x01. The v6 migration incorrectly skipped these
events because it checked `eventData[0] == CompactFormatVersion` before
attempting to decode, causing them to never receive SerialEventId (sei)
mappings.
This fix:
1. Changes compact format detection to try sei lookup first - if the
mapping doesn't exist, fall back to legacy binary format instead
of returning an error
2. Adds v9 migration (BackfillMissingSerialEventIdMappings) that finds
legacy events with IDs starting with 0x01 that were skipped by the
v6 migration and creates their sei mappings
3. Quiets the "Key not found" logging for sei lookups since missing
mappings are now expected and handled gracefully
Files modified:
- pkg/database/fetch-events-by-serials.go: Fallback to legacy format
- pkg/database/fetch-event-by-serial.go: Fallback to legacy format
- pkg/database/export.go: Fallback to legacy format
- pkg/database/serial_cache.go: Quiet sei lookup logging
- pkg/database/migrations.go: Add v9 sei backfill migration
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add unicode_normalize.go with mappings for small caps and fraktur
- Map 77 decorative unicode characters to ASCII equivalents:
- Small caps (25 chars): ᴅᴇᴀᴛʜ → death
- Fraktur lowercase (26 chars): 𝔡𝔢𝔞𝔱𝔥 → death
- Fraktur uppercase (26 chars): 𝔇𝔈𝔄𝔗ℌ → death
- Fix broken utf8DecodeRuneInString() that failed on multi-byte UTF-8
- Add migration v7 to rebuild word indexes with normalization
- Add comprehensive unit tests for all character mappings
Files modified:
- pkg/database/unicode_normalize.go: New - character mapping tables
- pkg/database/unicode_normalize_test.go: New - unit tests
- pkg/database/tokenize.go: Integrate normalizeRune(), fix UTF-8 decoder
- pkg/database/migrations.go: Add version 7 migration
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Simplified event import to run synchronously, ensuring proper resource handling and accurate feedback. Enhanced frontend with real-time import status messages and error handling. Adjusted migrations to handle events individually, improving reliability and granular progress tracking.
- Introduced test files for the blossom and database packages to improve test coverage and ensure functionality.
- Updated logging practices by suppressing unnecessary log outputs during tests to enhance clarity and focus on relevant information.
- Refactored error handling in the `handle-message` and `handle-req` functions to avoid logging expected context cancellation errors during shutdown.
- Bumped version to v0.25.2 to reflect these updates.
Replaced legacy `*.orly` module imports with `next.orly.dev/pkg` paths across the codebase for consistency. Removed legacy `go.mod` files from sub-packages, consolidating dependency management. Added Dockerfiles and configurations for benchmarking environments.
woikos
mleku