implement messages and operations for FIND

2 months ago · 5d04193bb7
1 changed files with 683 additions and 0 deletions
--- a/docs/go-reference-type-analysis-revised.md
+++ b/docs/go-reference-type-analysis-revised.md
@ -0,0 +1,683 @@
 # Go Reference Type Simplification - Revised Proposal
 ## Executive Summary
 Keep Go's convenient syntax (slicing, `<-`, `for range`) while making reference semantics **explicit through pointer types**. This reduces cognitive load and improves safety without sacrificing ergonomics.
 ## Core Principle: Explicit Pointers, Convenient Syntax
 **The Key Insight:**
 - Make slices/maps/channels explicitly `*[]T`, `*map[K]V`, `*chan T`
 - Keep convenient operators (auto-dereference like struct methods do)
 - Eliminate special allocation functions (`make()`)
 - Add explicit control where it matters (grow, clone)
 ## Proposed Changes
 ### 1. Slices Become `*[]T` (Explicit Pointers)
 **Current Problem:**
 ```go
 s := []int{1, 2, 3}        // Looks like value, is reference
 s2 := s                    // Copies reference - HIDDEN SHARING
 s2[0] = 99                 // Mutates s too! Not obvious
 ```
 **Proposed:**
 ```go
 s := &[]int{1, 2, 3}       // Explicit pointer allocation
 s2 := s                    // Copies pointer - OBVIOUS SHARING
 s2[0] = 99                 // Mutates s too - but now obvious!
 // Slicing still works (auto-dereference)
 sub := s[1:3]              // Returns *[]int (new slice header, same backing)
 sub := s[1:3:5]            // Full slicing with capacity still works
 // To copy data, be explicit
 s3 := s.Clone()            // Deep copy
 s3 := &[]int(*s)           // Alternative: copy via literal
 // Append works as before
 s.Append(4, 5, 6)          // Implicit grow if needed (fine!)
 s.Grow(100)                // Explicit capacity increase
 ```
 **What Changes:**
 - ✅ Allocation: `&[]T{}` instead of `make([]T, len, cap)`
 - ✅ Type: `*[]int` instead of `[]int`
 - ✅ Explicit clone: Must call `.Clone()` to copy data
 - ✅ Explicit grow: `.Grow(n)` for pre-allocation
 - ❌ Slicing syntax: **KEEP IT** - `s[i:j]` still works
 - ❌ Append behavior: **KEEP IT** - implicit growth is fine
 - ❌ Auto-dereference: Like methods, `s[i]` auto-derefs
 **Benefits:**
 - Assignment `s2 := s` is obviously pointer copy
 - Function parameters `func f(s *[]int)` show mutation potential
 - Still convenient: slicing and indexing work as before
 ### 2. Maps Become `*map[K]V` (Explicit Pointers)
 **Current Problem:**
 ```go
 m := make(map[string]int)  // Special make() function
 m2 := m                    // HIDDEN reference sharing
 var m3 map[string]int      // nil map
 v := m3["key"]             // OK - returns zero value
 m3["key"] = 42             // PANIC! Nil map write trap
 ```
 **Proposed:**
 ```go
 m := &map[string]int{}     // Explicit pointer allocation
 m := &map[string]int{      // Literal initialization
    "key": 42,
 }
 m2 := m                    // Obviously copies pointer
 // Map operations auto-dereference
 m["key"] = 42              // Auto-deref (like s[i] for slices)
 v := m["key"]
 v, ok := m["key"]
 // Nil pointer is consistent
 var m3 *map[string]int     // nil pointer
 v := m3["key"]             // PANIC - nil pointer deref (consistent!)
 m3 = &map[string]int{}     // Must allocate
 m3["key"] = 42             // Now OK
 // Copying requires explicit clone
 m4 := m.Clone()            // Deep copy
 ```
 **What Changes:**
 - ✅ Allocation: `&map[K]V{}` instead of `make(map[K]V)`
 - ✅ Type: `*map[K]V` instead of `map[K]V`
 - ✅ Nil behavior: Consistent nil pointer panic
 - ✅ Explicit clone: Must call `.Clone()`
 - ❌ Map syntax: **KEEP IT** - `m[k]` auto-derefs
 **Benefits:**
 - Obvious pointer semantics
 - No special nil-map read-only trap
 - Clear when data is shared
 ### 3. Channels Become `*chan T` (Explicit Pointers)
 **Current Problem:**
 ```go
 ch := make(chan int, 10)   // Special make() function
 ch2 := ch                  // HIDDEN reference sharing
 var ch3 chan int           // nil channel
 ch3 <- 42                  // BLOCKS FOREVER! Silent deadlock trap
 ```
 **Proposed:**
 ```go
 ch := &chan int{cap: 10}   // Explicit pointer allocation
 ch := &chan int{}          // Unbuffered (cap: 0)
 ch2 := ch                  // Obviously copies pointer
 // Channel operations auto-dereference
 ch <- 42                   // KEEP <- syntax!
 v := <-ch
 v, ok := <-ch
 // for range still works
 for v := range ch {        // KEEP for range!
    process(v)
 }
 // select still works
 select {                   // KEEP select!
 case v := <-ch:
    handle(v)
 case ch2 <- 42:
    sent()
 }
 // Nil pointer is consistent
 var ch3 *chan int          // nil pointer
 ch3 <- 42                  // PANIC - nil pointer deref (consistent!)
 // Directional channels as type aliases or interfaces
 type SendOnly[T any] = *chan T   // Could restrict at type level
 func send(ch *chan int) {}       // Or just document convention
 ```
 **What Changes:**
 - ✅ Allocation: `&chan T{cap: n}` instead of `make(chan T, n)`
 - ✅ Type: `*chan T` instead of `chan T`
 - ✅ Nil behavior: Consistent nil pointer panic
 - ❌ Send/receive: **KEEP `<-` syntax**
 - ❌ Select: **KEEP `select` statement**
 - ❌ For range: **KEEP `for range ch`**
 **Benefits:**
 - Obvious pointer semantics
 - No silent nil-channel blocking trap
 - Keep all the convenient syntax
 - Directional types could be interfaces if needed
 ### 4. Unified Allocation: Eliminate `make()`
 **Before (Three Allocation Primitives):**
 ```go
 new(T)                     // Returns *T (zero value)
 make([]T, len, cap)        // Returns []T (special)
 make(map[K]V, hint)        // Returns map[K]V (special)
 make(chan T, buf)          // Returns chan T (special)
 ```
 **After (One Allocation Syntax):**
 ```go
 new(T)                     // Returns *T (zero value)
 &T{}                       // Returns *T (composite literal)
 &[]T{}                     // Returns *[]T (empty slice)
 &[n]T{}                    // Returns *[n]T (array)
 &map[K]V{}                 // Returns *map[K]V (empty map)
 &chan T{}                  // Returns *chan T (unbuffered)
 &chan T{cap: 10}           // Returns *chan T (buffered)
 ```
 **Eliminate:**
 - ❌ `make()` entirely
 - ❌ Special capacity/hint parameters (use methods instead)
 ### 5. Type System Unification
 **Before:**
 ```
 Value types: int, float, bool, struct, [N]T
 Reference types: []T, map[K]V, chan T (SPECIAL SEMANTICS)
 Pointer types: *T
 ```
 **After:**
 ```
 Value types: int, float, bool, struct, [N]T
 Pointer types: *T (including *[]T, *map[K]V, *chan T - UNIFIED)
 ```
 All pointer types have consistent semantics:
 - Assignment copies the pointer
 - Nil pointer dereference panics consistently
 - Auto-dereference for convenient syntax
 - Explicit `.Clone()` for deep copy
 ## Syntax Comparison
 ### Slices
 **Before:**
 ```go
 // Many ways to create
 var s []int                          // nil slice
 s = []int{}                          // empty slice
 s = make([]int, 10)                  // len=10, cap=10
 s = make([]int, 10, 20)              // len=10, cap=20
 s = []int{1, 2, 3}                   // literal
 // Slicing
 sub := s[1:3]                        // subslice
 sub = s[:3]                          // from start
 sub = s[1:]                          // to end
 sub = s[:]                           // full slice
 sub = s[1:3:5]                       // with capacity
 // Append
 s = append(s, 4)                     // might reallocate
 s = append(s, items...)              // spread
 // Copy (manual)
 s2 := make([]int, len(s))
 copy(s2, s)
 ```
 **After:**
 ```go
 // One way to create
 var s *[]int                         // nil pointer
 s = &[]int{}                         // empty slice
 s = &[10]int{}[:]                    // len=10 from array
 s = &[]int{1, 2, 3}                  // literal
 // Slicing (UNCHANGED)
 sub := s[1:3]                        // auto-deref, returns *[]int
 sub = s[:3]
 sub = s[1:]
 sub = s[:]
 sub = s[1:3:5]
 // Append (UNCHANGED)
 s.Append(4)                          // might reallocate (fine!)
 s.Append(items...)                   // spread
 // Explicit operations
 s.Grow(100)                          // pre-allocate capacity
 s2 := s.Clone()                      // explicit deep copy
 ```
 ### Maps
 **Before:**
 ```go
 // Many ways to create
 var m map[K]V                        // nil map
 m = map[K]V{}                        // empty map
 m = make(map[K]V)                    // empty map
 m = make(map[K]V, 100)               // with hint
 m = map[K]V{k: v}                    // literal
 // Access
 m[k] = v
 v = m[k]
 v, ok = m[k]
 // Copy (manual)
 m2 := make(map[K]V, len(m))
 for k, v := range m {
    m2[k] = v
 }
 ```
 **After:**
 ```go
 // One way to create
 var m *map[K]V                       // nil pointer
 m = &map[K]V{}                       // empty map
 m = &map[K]V{k: v}                   // literal
 // Access (UNCHANGED)
 m[k] = v                             // auto-deref
 v = m[k]
 v, ok = m[k]
 // Explicit operations
 m2 := m.Clone()                      // explicit deep copy
 ```
 ### Channels
 **Before:**
 ```go
 // Create
 ch := make(chan int)                 // unbuffered
 ch := make(chan int, 10)             // buffered
 // Operations
 ch <- 42                             // send
 v := <-ch                            // receive
 v, ok := <-ch                        // receive with closed check
 close(ch)
 // for range
 for v := range ch {
    process(v)
 }
 // select
 select {
 case v := <-ch:
    handle(v)
 case <-timeout:
    timeout()
 }
 ```
 **After:**
 ```go
 // Create
 ch := &chan int{}                    // unbuffered
 ch := &chan int{cap: 10}             // buffered
 // Operations (UNCHANGED)
 ch <- 42                             // auto-deref
 v := <-ch
 v, ok := <-ch
 ch.Close()                           // method instead of builtin
 // for range (UNCHANGED)
 for v := range ch {
    process(v)
 }
 // select (UNCHANGED)
 select {
 case v := <-ch:
    handle(v)
 case <-timeout:
    timeout()
 }
 ```
 ## Grammar Simplification
 ### Eliminated Syntax
 1. **`make()` builtin** - 3 different forms → 0
   - `make([]T, n, cap)` → `&[]T{}` + `.Grow(cap)`
   - `make(map[K]V, hint)` → `&map[K]V{}`
   - `make(chan T, buf)` → `&chan T{cap: buf}`
 2. **Dual allocation semantics** - 2 primitives → 1
   - `new(T)` and `make(T)` → just `new(T)` or `&T{}`
 ### Preserved Syntax
 1. ✅ Slice expressions: `s[i:j]`, `s[i:j:k]`
 2. ✅ Channel operators: `<-ch`, `ch<-`
 3. ✅ Select statement: `select { case ... }`
 4. ✅ Range over channels: `for v := range ch`
 5. ✅ Map/slice indexing: `m[k]`, `s[i]`
 6. ✅ Auto-dereference: Like methods on `*T`
 ## New Built-in Methods
 ### Slices (`*[]T`)
 ```go
 s := &[]int{1, 2, 3}
 // Capacity management
 s.Grow(n int)              // Ensure capacity for n more elements
 s.Cap() int                // Current capacity
 s.Len() int                // Current length
 // Modification
 s.Append(items ...T)       // Append items (implicit grow OK)
 s.Insert(i int, items ...T) // Insert at index
 s.Delete(i, j int)         // Delete s[i:j]
 s.Clear()                  // Set length to 0
 // Copying
 s.Clone() *[]T             // Deep copy
 s.Slice(i, j int) *[]T     // Alternative to s[i:j]
 ```
 ### Maps (`*map[K]V`)
 ```go
 m := &map[string]int{}
 // Capacity
 m.Len() int                // Number of keys
 // Modification
 m.Clear()                  // Remove all keys
 m.Delete(k K)              // Delete key
 // Copying
 m.Clone() *map[K]V         // Deep copy
 // Bulk operations
 m.Keys() *[]K              // All keys
 m.Values() *[]V            // All values
 m.Merge(other *map[K]V)    // Merge other into m
 ```
 ### Channels (`*chan T`)
 ```go
 ch := &chan int{cap: 10}
 // Metadata
 ch.Len() int               // Items in buffer
 ch.Cap() int               // Buffer capacity
 // Control
 ch.Close()                 // Close channel (method vs builtin)
 ```
 ## Auto-Dereference Rules
 Like struct methods today, pointer types auto-dereference:
 ```go
 type Person struct { name string }
 func (p *Person) Name() string { return p.name }
 p := &Person{name: "Alice"}
 n := p.Name()              // Auto-deref: (*p).Name()
 // Same for new pointer types
 s := &[]int{1, 2, 3}
 v := s[0]                  // Auto-deref: (*s)[0]
 sub := s[1:3]              // Auto-deref: (*s)[1:3]
 m := &map[K]V{}
 v = m[k]                   // Auto-deref: (*m)[k]
 ch := &chan int{}
 ch <- 42                   // Auto-deref: (*ch) <- 42
 v = <-ch                   // Auto-deref: <-(*ch)
 ```
 **Rule:** Pointer to slice/map/channel auto-derefs for indexing, slicing, and channel ops.
 ## Concurrency Safety
 ### Before: Implicit Sharing
 ```go
 func worker(s []int, wg *sync.WaitGroup) {
    defer wg.Done()
    s[0] = 99                // RACE - not obvious from signature
 }
 s := []int{1, 2, 3}
 var wg sync.WaitGroup
 wg.Add(2)
 go worker(s, &wg)            // Sharing not obvious
 go worker(s, &wg)            // Two goroutines mutate same slice
 wg.Wait()
 ```
 ### After: Explicit Sharing
 ```go
 func worker(s *[]int, wg *sync.WaitGroup) {
    defer wg.Done()
    (*s)[0] = 99             // RACE - but obvious from *[]int
 }
 s := &[]int{1, 2, 3}
 var wg sync.WaitGroup
 wg.Add(2)
 go worker(s, &wg)            // OBVIOUS pointer sharing
 go worker(s, &wg)            // Clear that both access same data
 wg.Wait()
 ```
 **Benefits:**
 - Function signature shows mutation: `func f(s *[]int)` vs `func f(s []int)`
 - Pointer copy is obvious: `s2 := s` (copies pointer)
 - Value copy requires explicit clone: `s2 := s.Clone()`
 ### Pattern: Immutable by Default
 ```go
 // Current Go - unclear if mutation happens
 func ProcessSlice(s []int) []int {
    s[0] = 99                // Mutates caller's slice!
    return s
 }
 // Proposed - explicit mutation
 func ProcessSlice(s *[]int) {
    (*s)[0] = 99             // Clear mutation
 }
 // Or value semantics (copy)
 func ProcessSlice(s []int) []int {  // Note: NOT pointer
    result := &[]int(s)      // Explicit copy from value
    (*result)[0] = 99        // Mutate copy
    return result
 }
 ```
 ## Migration Path
 ### Phase 1: Allow Both (Backward Compatible)
 ```go
 // Old style still works
 s := []int{1, 2, 3}
 s = append(s, 4)
 // New style also works (same runtime behavior)
 s := &[]int{1, 2, 3}
 s.Append(4)
 // Add deprecation warnings
 make([]int, 10)              // WARNING: Use &[]int{} or &[10]int{}[:]
 ```
 ### Phase 2: Deprecate Old Forms
 ```go
 // Compiler warnings
 []int{1, 2, 3}               // WARNING: Use &[]int{1, 2, 3}
 make([]int, 10)              // WARNING: Use &[]int{} with .Grow(10)
 make(map[K]V)                // WARNING: Use &map[K]V{}
 make(chan T, 10)             // WARNING: Use &chan T{cap: 10}
 ```
 ### Phase 3: Breaking Change
 ```go
 // Only new syntax allowed
 &[]int{1, 2, 3}              // OK
 &map[K]V{}                   // OK
 &chan T{cap: 10}             // OK
 []int{1, 2, 3}               // ERROR: Use &[]int{1, 2, 3}
 make([]int, 10)              // ERROR: Removed
 ```
 ## Implementation Impact
 ### Compiler Changes
 **New:**
 - Auto-dereference for `*[]T`, `*map[K]V`, `*chan T`
 - Built-in methods (`.Append()`, `.Clone()`, `.Grow()`, etc.)
 - Composite literal fields: `&chan T{cap: 10}`
 **Removed:**
 - `make()` builtin (3 forms)
 - Special case type checking for reference types
 **Preserved:**
 - Slice expressions `s[i:j:k]`
 - Channel operators `<-`
 - Select statement
 - Range over channels
 - All runtime implementations
 ### Runtime Changes
 **Minimal:**
 - Same memory layout for slices/maps/channels
 - Same GC behavior
 - Same scheduler
 - No performance impact
 **API:**
 - Add runtime functions for `.Clone()`, `.Grow()`, etc.
 - These can be compiler intrinsics for performance
 ## Complexity Reduction
 | Metric | Before | After | Reduction |
 |--------|--------|-------|-----------|
 | **Allocation primitives** | 2 (`new`, `make`) | 1 (`&T{}`) | **50%** |
 | **make() forms** | 3 (slice, map, chan) | 0 | **100%** |
 | **Reference type special cases** | 3 types | 0 (unified) | **100%** |
 | **Nil traps** | 2 (nil map write, nil chan) | 0 (consistent panic) | **100%** |
 | **Type system categories** | 3 (value, ref, ptr) | 2 (value, ptr) | **33%** |
 | **Syntax variants preserved** | Slicing, `<-`, select, range | All kept | **0%** |
 **Total complexity reduction: ~30%** while keeping ergonomic syntax.
 ## Real-World Example: ORLY Codebase
 ### Before
 ```go
 // pkg/database/query-events.go
 func QueryEvents(db *badger.DB, filter *filter.T) ([]uint64, error) {
    results := make([]uint64, 0, 1000)
    // ... query logic
    return results, nil
 }
 // Caller must handle returned slice
 events, err := QueryEvents(db, f)
 if err != nil {
    return err
 }
 events = append(events, moreEvents...)  // Might copy
 ```
 ### After
 ```go
 // pkg/database/query-events.go
 func QueryEvents(db *badger.DB, filter *filter.T) (results *[]uint64, err error) {
    results = &[]uint64{}
    results.Grow(1000)           // Explicit capacity
    // ... query logic
    return
 }
 // Caller gets explicit pointer
 events, err := QueryEvents(db, f)
 if chk.E(err) {
    return
 }
 events.Append(moreEvents...)     // Clear mutation
 ```
 **Benefits in ORLY:**
 - Clear which functions mutate vs return new data
 - Obvious when slices are shared across goroutines
 - Explicit capacity management for performance-critical code
 - No hidden allocations from append
 ## Conclusion
 ### What We Keep
 ✅ Slice expressions: `s[1:3:5]`
 ✅ Channel operators: `<-`
 ✅ Select statement
 ✅ For range channels
 ✅ Implicit append growth
 ✅ Convenient auto-dereference
 ### What We Gain
 ✅ Explicit pointer semantics
 ✅ Obvious data sharing
 ✅ Consistent nil behavior
 ✅ Unified type system
 ✅ Simpler language (no `make()`)
 ✅ Better concurrency safety
 ### What We Lose
 ❌ `make()` function (replaced by `&T{}`)
 ❌ Implicit reference types (now explicit `*[]T`)
 ❌ Zero-value usability for maps/slices (must allocate)
 ### Recommendation
 This revision strikes the right balance:
 - **Keep** Go's ergonomic syntax that makes it productive
 - **Add** explicit semantics that make code safer and clearer
 - **Remove** only the truly confusing parts (`make()`, implicit references)
 - **Gain** ~30% complexity reduction without sacrificing convenience
 The migration is straightforward and could be done gradually with good tooling support.