next.orly.dev/docs/IPC_SYNC_SERVICES.md

IPC Sync Services Architecture

ORLY supports splitting sync functionality into independent gRPC IPC services for improved modularity, resource isolation, and horizontal scaling.

Overview

The sync subsystem can run in two modes:

1. Local mode (default): All sync functionality runs in-process with the main relay
2. gRPC mode: Sync services run as separate processes communicating via gRPC

Architecture

┌─────────────────────────────────────────────────────────────────────────────┐
│                              orly-launcher                                   │
│  (Process supervisor - manages lifecycle of all services)                   │
└─────────────────────────────────────────────────────────────────────────────┘
                                      │
           ┌──────────────────────────┼──────────────────────────┐
           │                          │                          │
           ▼                          ▼                          ▼
┌─────────────────────┐   ┌─────────────────────┐   ┌─────────────────────┐
│     orly-db         │   │     orly-acl        │   │   Sync Services     │
│   (gRPC :50051)     │   │   (gRPC :50052)     │   │   (gRPC :50061-64)  │
│                     │   │                     │   │                     │
│ - Event storage     │   │ - Access control    │   │ - Distributed sync  │
│ - Query execution   │   │ - Follow lists      │   │ - Cluster sync      │
│ - Index management  │   │ - Permissions       │   │ - Relay groups      │
└─────────────────────┘   └─────────────────────┘   │ - Negentropy/NIP-77 │
           │                          │             └─────────────────────┘
           │                          │                          │
           └──────────────────────────┼──────────────────────────┘
                                      │
                                      ▼
                          ┌─────────────────────┐
                          │       orly          │
                          │   (Main relay)      │
                          │                     │
                          │ - WebSocket server  │
                          │ - HTTP endpoints    │
                          │ - NIP handling      │
                          └─────────────────────┘

Sync Services

1. orly-sync-distributed (Port 50061)

Serial-based peer-to-peer synchronization between relay instances.

Purpose: Keeps multiple relay instances in sync by exchanging events based on serial numbers (Lamport clocks).

Key Features:

• Periodic polling of peer relays
• NIP-11 relay info caching for peer identification
• NIP-98 authenticated sync requests
• Incremental sync using serial numbers

gRPC API:

service DistributedSyncService {
  rpc Ready(Empty) returns (ReadyResponse);
  rpc GetInfo(Empty) returns (SyncInfo);
  rpc GetCurrentSerial(CurrentRequest) returns (CurrentResponse);
  rpc GetEventIDs(EventIDsRequest) returns (EventIDsResponse);
  rpc HandleCurrentRequest(HTTPRequest) returns (HTTPResponse);
  rpc HandleEventIDsRequest(HTTPRequest) returns (HTTPResponse);
  rpc GetPeers(Empty) returns (PeersResponse);
  rpc UpdatePeers(UpdatePeersRequest) returns (Empty);
  rpc GetSyncStatus(Empty) returns (SyncStatusResponse);
  rpc GetPeerPubkey(PeerPubkeyRequest) returns (PeerPubkeyResponse);
  rpc IsAuthorizedPeer(AuthorizedPeerRequest) returns (AuthorizedPeerResponse);
  rpc NotifyNewEvent(NewEventNotification) returns (Empty);
}

Environment Variables:

Variable	Default	Description
ORLY_SYNC_DISTRIBUTED_LISTEN	127.0.0.1:50061	gRPC listen address
ORLY_SYNC_DISTRIBUTED_DB_TYPE	grpc	Database backend
ORLY_SYNC_DISTRIBUTED_DB_SERVER	127.0.0.1:50051	Database server address
ORLY_SYNC_DISTRIBUTED_PEERS		Comma-separated peer URLs
ORLY_SYNC_DISTRIBUTED_INTERVAL	30s	Sync polling interval

---

2. orly-sync-cluster (Port 50062)

Cluster replication with persistent state for high-availability deployments.

Purpose: Enables multiple relay instances to form a cluster with consistent event replication.

Key Features:

• Real-time event propagation to cluster members
• Cluster membership management via Kind 39108 events
• Persistent tracking of cluster member state
• Configurable privileged event propagation (DMs, gift wraps)

gRPC API:

service ClusterSyncService {
  rpc Ready(Empty) returns (ReadyResponse);
  rpc Start(Empty) returns (Empty);
  rpc Stop(Empty) returns (Empty);
  rpc HandleLatestSerial(HTTPRequest) returns (HTTPResponse);
  rpc HandleEventsRange(HTTPRequest) returns (HTTPResponse);
  rpc GetMembers(Empty) returns (MembersResponse);
  rpc UpdateMembership(UpdateMembershipRequest) returns (Empty);
  rpc HandleMembershipEvent(MembershipEventRequest) returns (Empty);
  rpc GetClusterStatus(Empty) returns (ClusterStatusResponse);
  rpc GetMemberStatus(MemberStatusRequest) returns (MemberStatusResponse);
  rpc GetLatestSerial(Empty) returns (LatestSerialResponse);
  rpc GetEventsInRange(EventsRangeRequest) returns (EventsRangeResponse);
}

Environment Variables:

Variable	Default	Description
ORLY_SYNC_CLUSTER_LISTEN	127.0.0.1:50062	gRPC listen address
ORLY_SYNC_CLUSTER_DB_TYPE	grpc	Database backend
ORLY_SYNC_CLUSTER_DB_SERVER	127.0.0.1:50051	Database server address
ORLY_CLUSTER_ADMINS		Authorized cluster admin pubkeys
ORLY_CLUSTER_PROPAGATE_PRIVILEGED_EVENTS	true	Replicate DMs/gift wraps

---

3. orly-sync-relaygroup (Port 50063)

Relay group configuration discovery using Kind 39105 events.

Purpose: Manages relay group configurations that define which relays should sync together.

Key Features:

• Discovery of authoritative relay group configs
• Validation of relay group events
• Authorized publisher management
• Dynamic peer list updates

gRPC API:

service RelayGroupService {
  rpc Ready(Empty) returns (ReadyResponse);
  rpc FindAuthoritativeConfig(Empty) returns (RelayGroupConfigResponse);
  rpc GetRelays(Empty) returns (RelaysResponse);
  rpc IsAuthorizedPublisher(AuthorizedPublisherRequest) returns (AuthorizedPublisherResponse);
  rpc GetAuthorizedPubkeys(Empty) returns (AuthorizedPubkeysResponse);
  rpc ValidateRelayGroupEvent(ValidateEventRequest) returns (ValidateEventResponse);
  rpc HandleRelayGroupEvent(HandleEventRequest) returns (Empty);
}

Environment Variables:

Variable	Default	Description
ORLY_SYNC_RELAYGROUP_LISTEN	127.0.0.1:50063	gRPC listen address
ORLY_SYNC_RELAYGROUP_DB_TYPE	grpc	Database backend
ORLY_SYNC_RELAYGROUP_DB_SERVER	127.0.0.1:50051	Database server address
ORLY_RELAY_GROUP_ADMINS		Authorized relay group config publishers

---

4. orly-sync-negentropy (Port 50064)

NIP-77 negentropy-based efficient set reconciliation.

Purpose: Provides efficient set reconciliation for both relay-to-relay sync and client-facing NIP-77 WebSocket protocol.

Key Features:

• NIP-77 client WebSocket support (NEG-OPEN, NEG-MSG, NEG-CLOSE)
• Relay-to-relay negentropy sync
• Session management for concurrent clients
• Configurable frame size and ID truncation

gRPC API:

service NegentropyService {
  rpc Ready(Empty) returns (ReadyResponse);
  rpc Start(Empty) returns (Empty);
  rpc Stop(Empty) returns (Empty);

  // Client-facing NIP-77
  rpc HandleNegOpen(NegOpenRequest) returns (NegOpenResponse);
  rpc HandleNegMsg(NegMsgRequest) returns (NegMsgResponse);
  rpc HandleNegClose(NegCloseRequest) returns (Empty);

  // Relay-to-relay sync
  rpc SyncWithPeer(SyncPeerRequest) returns (stream SyncProgress);
  rpc GetSyncStatus(Empty) returns (SyncStatusResponse);
  rpc GetPeers(Empty) returns (PeersResponse);
  rpc AddPeer(AddPeerRequest) returns (Empty);
  rpc RemovePeer(RemovePeerRequest) returns (Empty);
  rpc TriggerSync(TriggerSyncRequest) returns (Empty);
  rpc GetPeerSyncState(PeerSyncStateRequest) returns (PeerSyncStateResponse);

  // Session management
  rpc ListSessions(Empty) returns (ListSessionsResponse);
  rpc CloseSession(CloseSessionRequest) returns (Empty);
}

Environment Variables:

Variable	Default	Description
ORLY_SYNC_NEGENTROPY_LISTEN	127.0.0.1:50064	gRPC listen address
ORLY_SYNC_NEGENTROPY_DB_TYPE	grpc	Database backend
ORLY_SYNC_NEGENTROPY_DB_SERVER	127.0.0.1:50051	Database server address
ORLY_SYNC_NEGENTROPY_PEERS		Comma-separated peer WebSocket URLs
ORLY_SYNC_NEGENTROPY_INTERVAL	60s	Background sync interval
ORLY_SYNC_NEGENTROPY_FRAME_SIZE	4096	Negentropy frame size
ORLY_SYNC_NEGENTROPY_ID_SIZE	16	ID truncation size
ORLY_SYNC_NEGENTROPY_SESSION_TIMEOUT	5m	Client session timeout

---

Launcher Configuration

The orly-launcher manages all services. Enable sync services with these environment variables:

Variable	Default	Description
ORLY_LAUNCHER_SYNC_DISTRIBUTED_ENABLED	false	Enable distributed sync
ORLY_LAUNCHER_SYNC_DISTRIBUTED_BINARY	orly-sync-distributed	Binary path
ORLY_LAUNCHER_SYNC_DISTRIBUTED_LISTEN	127.0.0.1:50061	Listen address
ORLY_LAUNCHER_SYNC_CLUSTER_ENABLED	false	Enable cluster sync
ORLY_LAUNCHER_SYNC_CLUSTER_BINARY	orly-sync-cluster	Binary path
ORLY_LAUNCHER_SYNC_CLUSTER_LISTEN	127.0.0.1:50062	Listen address
ORLY_LAUNCHER_SYNC_RELAYGROUP_ENABLED	false	Enable relay group
ORLY_LAUNCHER_SYNC_RELAYGROUP_BINARY	orly-sync-relaygroup	Binary path
ORLY_LAUNCHER_SYNC_RELAYGROUP_LISTEN	127.0.0.1:50063	Listen address
ORLY_LAUNCHER_SYNC_NEGENTROPY_ENABLED	false	Enable negentropy
ORLY_LAUNCHER_SYNC_NEGENTROPY_BINARY	orly-sync-negentropy	Binary path
ORLY_LAUNCHER_SYNC_NEGENTROPY_LISTEN	127.0.0.1:50064	Listen address
ORLY_LAUNCHER_SYNC_READY_TIMEOUT	30s	Sync service startup timeout

---

Relay Configuration

When sync services are enabled, configure the relay to connect:

Variable	Default	Description
ORLY_SYNC_TYPE	local	Sync backend: local or grpc
ORLY_GRPC_SYNC_DISTRIBUTED		Distributed sync server address
ORLY_GRPC_SYNC_CLUSTER		Cluster sync server address
ORLY_GRPC_SYNC_RELAYGROUP		Relay group server address
ORLY_GRPC_SYNC_NEGENTROPY		Negentropy server address
ORLY_GRPC_SYNC_TIMEOUT	10s	gRPC connection timeout
ORLY_NEGENTROPY_ENABLED	false	Enable NIP-77 WebSocket support

---

Startup Order

The launcher starts services in dependency order:

1. Database (orly-db) - Must be ready first
2. ACL (orly-acl) - Depends on database
3. Sync Services (parallel) - All depend on database only
4. Relay (orly) - Depends on all above

Shutdown happens in reverse order.

---

Example: Enabling Negentropy

To enable NIP-77 negentropy support on an existing deployment:

# In systemd service or environment
Environment=ORLY_LAUNCHER_SYNC_NEGENTROPY_ENABLED=true
Environment=ORLY_LAUNCHER_SYNC_NEGENTROPY_BINARY=/path/to/orly-sync-negentropy
Environment=ORLY_NEGENTROPY_ENABLED=true

Build the binary:

CGO_ENABLED=0 go build -o orly-sync-negentropy ./cmd/orly-sync-negentropy

---

Proto Definitions

Proto files are located in:

• proto/orlysync/common/v1/types.proto - Shared types
• proto/orlysync/distributed/v1/service.proto - Distributed sync
• proto/orlysync/cluster/v1/service.proto - Cluster sync
• proto/orlysync/relaygroup/v1/service.proto - Relay group
• proto/orlysync/negentropy/v1/service.proto - Negentropy

Generate Go code with:

buf generate

---

NIP-77 Client Protocol

The negentropy service implements NIP-77 for efficient client synchronization:

Client Messages

• ["NEG-OPEN", subscription_id, filter, initial_message?] - Start reconciliation
• ["NEG-MSG", subscription_id, message] - Continue reconciliation
• ["NEG-CLOSE", subscription_id] - End session

Server Responses

• ["NEG-MSG", subscription_id, message] - Reconciliation response
• ["NEG-ERR", subscription_id, reason] - Error response

The relay automatically routes these messages to the negentropy service when ORLY_NEGENTROPY_ENABLED=true.