Add WasmDB support and enhance query/cache/policy systems
Introduced WasmDB as a new IndexedDB backend for WebAssembly environments, replicating Badger's schema for compatibility. Enhanced the query caching system with optional configuration to improve memory usage efficiency. Improved the policy system with new permissive overrides and clarified read vs write applicability for better flexibility.
@ -8,11 +8,11 @@ ORLY is a high-performance Nostr relay written in Go, designed for personal rela
@@ -8,11 +8,11 @@ ORLY is a high-performance Nostr relay written in Go, designed for personal rela
**Key Technologies:**
- **Language**: Go 1.25.3+
- **Database**: Badger v4 (embedded) or Neo4j (social graph)
- **Database**: Badger v4 (embedded), Neo4j (social graph), or WasmDB (IndexedDB for WebAssembly)
- **Cryptography**: Custom p8k library using purego for secp256k1 operations (no CGO)
- **Web UI**: Svelte frontend embedded in the binary
- **WebSocket**: gorilla/websocket for Nostr protocol
- **Performance**: SIMD-accelerated SHA256 and hex encoding, query result caching with zstd compression
- **Performance**: SIMD-accelerated SHA256 and hex encoding, optional query result caching with zstd compression
- **Social Graph**: Neo4j backend with Web of Trust (WoT) extensions for trust metrics
- `github.com/gorilla/websocket` - WebSocket server
- `github.com/minio/sha256-simd` - SIMD SHA256
- `github.com/templexxx/xhex` - SIMD hex encoding
@ -686,8 +713,8 @@ Each level has these printer types:
@@ -686,8 +713,8 @@ Each level has these printer types:
## Performance Considerations
- **Query Cache**: 512MB query result cache (configurable via `ORLY_QUERY_CACHE_SIZE_MB`) with zstd level 9 compression reduces database load for repeated queries
- **Filter Normalization**: Filters are normalized before cache lookup, so identical queries with different field ordering produce cache hits
- **Query Cache**: Optional 512MB query result cache (disabled by default via `ORLY_QUERY_CACHE_DISABLED=true`) with zstd level 9 compression reduces database load for repeated queries; enable with `ORLY_QUERY_CACHE_DISABLED=false`
- **Filter Normalization**: When query cache is enabled, filters are normalized before cache lookup, so identical queries with different field ordering produce cache hits
- **Database Caching**: Tune `ORLY_DB_BLOCK_CACHE_MB` and `ORLY_DB_INDEX_CACHE_MB` for workload (Badger backend only)
- **Query Optimization**: Add indexes for common filter patterns; multiple specialized query builders optimize different filter combinations
- **Batch Operations**: ID lookups and event fetching use batch operations via `GetSerialsByIds` and `FetchEventsBySerials`
@ -699,8 +726,9 @@ Each level has these printer types:
@@ -699,8 +726,9 @@ Each level has these printer types:
ORLY has received several significant performance improvements in recent updates:
### Query Cache System (Latest)
- 512MB query result cache with zstd level 9 compression
### Query Cache System
- Optional 512MB query result cache with zstd level 9 compression (disabled by default to reduce memory usage)
- Enable with `ORLY_QUERY_CACHE_DISABLED=false`
- Filter normalization ensures cache hits regardless of filter field ordering
- Configurable size (`ORLY_QUERY_CACHE_SIZE_MB`) and TTL (`ORLY_QUERY_CACHE_MAX_AGE`)
- Dramatically reduces database load for repeated queries (common in Nostr clients)
3. GitHub Actions workflow builds binaries for multiple platforms
4. Release created automatically with binaries and checksums
## Recent Features (v0.31.x)
## Recent Features (v0.34.x)
### Directory Spider
The directory spider (`pkg/spider/directory.go`) automatically discovers and syncs metadata from other relays:
@ -789,11 +817,21 @@ The Neo4j backend (`pkg/neo4j/`) includes Web of Trust (WoT) extensions:
@@ -789,11 +817,21 @@ The Neo4j backend (`pkg/neo4j/`) includes Web of Trust (WoT) extensions:
- **WoT Schema**: See `pkg/neo4j/WOT_SPEC.md` for full specification
- **Schema Modifications**: See `pkg/neo4j/MODIFYING_SCHEMA.md` for how to update
- Uses IndexedDB as storage via `aperturerobotics/go-indexeddb`
- Full query compatibility with Badger's index schema
- Object stores map to index prefixes (evt, eid, kc-, pc-, etc.)
- Range queries use IndexedDB cursors with KeyRange bounds
- Build with `GOOS=js GOARCH=wasm go build`
### Policy System Enhancements
- **Default-Permissive Model**: Read and write are allowed by default unless restrictions are configured
- **Write-Only Validation**: Size, age, tag validations apply ONLY to writes
- **Read-Only Filtering**: `read_allow`, `read_follows_whitelist`, `privileged` apply ONLY to reads
- **Separate Follows Whitelists**: `read_follows_whitelist` and `write_follows_whitelist` for fine-grained control
- **Permissive Mode Overrides**: `read_allow_permissive` and `write_allow_permissive` (global rule only) override kind whitelist for independent read/write control
- **Scripts**: Policy scripts execute ONLY for write operations
- **Reference Documentation**: `docs/POLICY_CONFIGURATION_REFERENCE.md` provides authoritative read vs write applicability
- See also: `pkg/policy/README.md` for quick reference
@ -825,6 +863,8 @@ The Neo4j backend (`pkg/neo4j/`) includes Web of Trust (WoT) extensions:
@@ -825,6 +863,8 @@ The Neo4j backend (`pkg/neo4j/`) includes Web of Trust (WoT) extensions:
"read_deny": ["pubkey_hex"], // Pubkeys denied from reading
"read_follows_whitelist": ["pubkey_hex"], // Pubkeys whose follows can read
"write_follows_whitelist": ["pubkey_hex"], // Pubkeys whose follows can write
"read_allow_permissive": false, // Override kind whitelist for reads
"write_allow_permissive": false, // Override kind whitelist for writes
@ -843,9 +883,11 @@ The Neo4j backend (`pkg/neo4j/`) includes Web of Trust (WoT) extensions:
@@ -843,9 +883,11 @@ The Neo4j backend (`pkg/neo4j/`) includes Web of Trust (WoT) extensions:
| `read_allow` | READ | Only listed pubkeys can read |