Browse Source

WIP: Complex event configuration and people tag anchor implementation

This commit represents a checkpoint in implementing a sophisticated event
configuration system with people tag anchors. The implementation has grown
complex and needs reimplementation in a smarter way.

## Major Changes:

### Event Configuration System Overhaul
- Replaced simple allowed/disabled kinds with EventKindConfig objects
- Each event kind now has individual limits and type-specific settings:
  - Kind 0 (profiles): Controls max profiles to fetch
  - Kind 3 (follow lists): Has depth setting and complex fetch logic
  - Kind 30040: Has nestedLevels setting
- Created new EventTypeConfig component replacing EventKindFilter

### Follow List Fetching Logic
- Kind 3 limit=1: Fetches only user's follow list
- Kind 3 limit>1: Fetches user's + (limit-1) follow lists from follows
- Added depth traversal (0=direct, 1=2 degrees, 2=3 degrees)
- Attempted to implement "addFollowLists" toggle (later simplified)

### Profile Fetching Changes
- Modified to be more selective about which profiles to fetch
- Attempted to limit based on follow lists and event authors
- Added progress indicators for profile loading

### People Tag Anchors Implementation
- Complex logic to create "p" tag anchors from follow lists
- Synthetic event creation to connect people to visualization
- "Only show people with publications" checkbox
- Attempted to connect people to:
  - Events they authored (pubkey match)
  - Events where they're tagged with "p"
- Display limiting based on kind 0 limit

### UI/UX Changes
- Tag anchors legend now scrollable when >20 items
- Auto-disable functionality when >20 tag anchors
- Added various UI controls that proved confusing
- Multiple iterations on settings panel layout

## Problems with Current Implementation:

1. **Overly Complex Logic**: The synthetic event creation and connection
   logic for people tag anchors became convoluted

2. **Confusing UI**: Too many interdependent settings that aren't intuitive:
   - Limit inputs control different things for different event types
   - The relationship between kind 3 and kind 0 limits is unclear
   - "addFollowLists" checkbox functionality was confusing

3. **Performance Concerns**: Fetching all profiles then limiting display
   may not be optimal

4. **Unclear Requirements**: The exact behavior for people tag anchors
   connections needs clarification

## Next Steps:

Need to revert and reimplement with:
- Clearer separation of concerns
- Simpler UI that's more intuitive
- Better defined behavior for people tag anchors
- More efficient profile fetching strategy

## Files Changed:
- EventTypeConfig.svelte: New component for event configuration
- visualizationConfig.ts: Major overhaul for EventKindConfig
- profileCache.ts: Added selective fetching logic
- visualize/+page.svelte: Complex follow list and profile fetching
- EventNetwork components: People tag anchor implementation
- settings_panel.org: Documentation of intended behavior

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
master
limina1 9 months ago
parent
commit
a39b86adce
  1. 125
      doc/settings_panel.org
  2. 86
      src/lib/components/EventKindFilter.svelte
  3. 246
      src/lib/components/EventTypeConfig.svelte
  4. 43
      src/lib/navigator/EventNetwork/Legend.svelte
  5. 16
      src/lib/navigator/EventNetwork/NodeTooltip.svelte
  6. 97
      src/lib/navigator/EventNetwork/Settings.svelte
  7. 135
      src/lib/navigator/EventNetwork/index.svelte
  8. 25
      src/lib/navigator/EventNetwork/utils/starNetworkBuilder.ts
  9. 26
      src/lib/navigator/EventNetwork/utils/tagNetworkBuilder.ts
  10. 273
      src/lib/stores/visualizationConfig.ts
  11. 56
      src/lib/utils/profileCache.ts
  12. 322
      src/routes/visualize/+page.svelte

125
doc/settings_panel.org

@ -0,0 +1,125 @@ @@ -0,0 +1,125 @@
* Settings Panel Documentation
** Overview
The settings panel controls how events are fetched and displayed in the visualization. It has several sections that work together to create an efficient and user-friendly experience.
** Event Types Configuration
*** Purpose
Controls which types of Nostr events are fetched and how many of each type.
*** Key Event Types
- *Kind 0* (Profiles/Metadata): User profile information (names, pictures, etc.)
- *Kind 3* (Follow Lists): Who each user follows
- *Kind 30040* (Index Events): Publication indices
- *Kind 30041* (Content Events): Publication content
- *Kind 30818* (Content Events): Alternative publication format
*** How Limits Work
Each event kind has a limit number that controls different things:
**** For Kind 0 (Profiles)
- Limit controls how many profiles to fetch from discovered pubkeys
- These profiles are used for:
- Displaying names instead of pubkeys
- Showing profile pictures in tooltips
- When "People" tag anchors are selected, this limit controls how many people anchors to display
**** For Kind 3 (Follow Lists)
- =limit = 1=: Only fetch the current user's follow list
- =limit > 1=: Fetch the user's follow list PLUS (limit-1) follow lists from people they follow
- The depth selector controls traversal:
- =Direct= (0): Just the immediate follows
- =2 degrees= (1): Follows of follows
- =3 degrees= (2): Three levels deep
**** For Kind 30040/30041/30818
- Limit controls maximum number of these events to fetch
** Tag Anchors
*** What Are Tag Anchors?
Tag anchors are special nodes in the graph that act as gravity points for events sharing common attributes. They help organize the visualization by grouping related content.
*** Tag Types Available
- *Hashtags* (t): Groups events by hashtag
- *Authors*: Groups events by author
- *People* (p): Shows people from follow lists as anchor points
- *Event References* (e): Groups events that reference each other
- *Titles*: Groups events by title
- *Summaries*: Groups events by summary
*** How People Tag Anchors Work
When "People" is selected as the tag type:
1. The system looks at all loaded follow lists (kind 3 events)
2. Extracts all pubkeys (people) from those follow lists
3. Creates tag anchors for those people (up to the kind 0 limit)
4. Connects each person anchor to:
- Events they authored (where pubkey matches)
- Events where they're mentioned in "p" tags
*** Display Limiting and Auto-Disable
- Tag anchors are created for ALL discovered tags
- But only displayed up to the configured limit
- When > 20 tag anchors exist, they're all auto-disabled
- Users can selectively enable specific anchors
- The legend becomes scrollable for many anchors
*** "Only show people with publications" Checkbox
When checked (default):
- Only shows people who have events in the current visualization
When unchecked:
- Shows ALL people from follow lists, even if they have no events displayed
- Useful for seeing your complete social graph
** Display Limits Section
*** Max Publication Indices (30040)
Controls display filtering for publication indices after they're fetched.
*** Max Events per Index
Limits how many content events to show per publication index.
*** Fetch if not found
When enabled, automatically fetches missing referenced events.
** Graph Traversal Section
*** Search through already fetched
When enabled, tag expansion only searches through events already loaded (more efficient).
*** Append mode
When enabled, new fetches add to the existing graph instead of replacing it.
** Current Implementation Questions
1. *Profile Fetching*: Should we fetch profiles for:
- Only event authors?
- All pubkeys in follow lists?
- All pubkeys mentioned anywhere?
2. *People Tag Anchors*: Should they connect to:
- Only events where the person is tagged with "p"?
- Events they authored?
- Both?
3. *Display Limits*: Should limits control:
- How many to fetch from relays?
- How many to display (fetch all, display subset)?
- Both with separate controls?
4. *Auto-disable Threshold*: Is 20 the right number for auto-disabling tag anchors?
** Ideal User Flow
1. User loads the visualization
2. Their follow list is fetched (kind 3, limit 1)
3. Profiles are fetched for people they follow (kind 0, respecting limit)
4. Publications are fetched (kind 30040/30041/30818)
5. User enables "People" tag anchors
6. Sees their follows as anchor points
7. Can see which follows have authored content
8. Can selectively enable/disable specific people
9. Can increase limits to see more content/people

86
src/lib/components/EventKindFilter.svelte

@ -15,8 +15,14 @@ @@ -15,8 +15,14 @@
let showAddInput = $state(false);
let inputError = $state('');
function validateKind(value: string): number | null {
const kind = parseInt(value.trim());
function validateKind(value: string | number): number | null {
// Convert to string for consistent handling
const strValue = String(value);
if (!strValue || strValue.trim() === '') {
inputError = '';
return null;
}
const kind = parseInt(strValue.trim());
if (isNaN(kind)) {
inputError = 'Must be a number';
return null;
@ -34,9 +40,16 @@ @@ -34,9 +40,16 @@
}
function handleAddKind() {
console.log('[EventKindFilter] handleAddKind called with:', newKind);
const kind = validateKind(newKind);
console.log('[EventKindFilter] Validation result:', kind);
if (kind !== null) {
console.log('[EventKindFilter] Before adding, allowedKinds:', $visualizationConfig.allowedKinds);
visualizationConfig.addKind(kind);
// Force a small delay to ensure store update propagates
setTimeout(() => {
console.log('[EventKindFilter] After adding, allowedKinds:', $visualizationConfig.allowedKinds);
}, 10);
newKind = '';
showAddInput = false;
inputError = '';
@ -107,39 +120,7 @@ @@ -107,39 +120,7 @@
</button>
{/each}
{#if showAddInput}
<div class="flex items-center gap-1">
<div class="relative">
<input
bind:value={newKind}
type="number"
placeholder="Kind"
class="w-20 px-2 py-1 text-xs border rounded dark:bg-gray-700 dark:border-gray-600"
onkeydown={handleKeydown}
oninput={() => validateKind(newKind)}
/>
{#if inputError}
<div class="absolute top-full left-0 mt-1 text-xs text-red-500 whitespace-nowrap">
{inputError}
</div>
{/if}
</div>
<Button size="xs" onclick={handleAddKind} disabled={!!inputError}>
Add
</Button>
<Button
size="xs"
color="light"
onclick={() => {
showAddInput = false;
newKind = '';
inputError = '';
}}
>
×
</Button>
</div>
{:else}
{#if !showAddInput}
<Button
size="xs"
color="light"
@ -165,6 +146,41 @@ @@ -165,6 +146,41 @@
</Button>
</div>
{#if showAddInput}
<div class="flex items-center gap-2">
<input
bind:value={newKind}
type="number"
placeholder="Enter event kind number (e.g. 1)"
class="flex-1 px-3 py-1 text-sm border rounded dark:bg-gray-700 dark:border-gray-600 dark:text-white focus:outline-none focus:ring-2 focus:ring-blue-500"
onkeydown={handleKeydown}
oninput={(e) => {
const value = (e.target as HTMLInputElement).value;
validateKind(value);
}}
/>
<Button size="xs" onclick={handleAddKind} disabled={!newKind}>
Add
</Button>
<Button
size="xs"
color="light"
onclick={() => {
showAddInput = false;
newKind = '';
inputError = '';
}}
>
Cancel
</Button>
</div>
{#if inputError}
<p class="text-xs text-red-500 -mt-2">
{inputError}
</p>
{/if}
{/if}
<div class="text-xs text-gray-500 dark:text-gray-400 space-y-1">
<p class="flex items-center gap-2">
<span class="inline-block w-3 h-3 border-2 border-green-500 rounded"></span>

246
src/lib/components/EventTypeConfig.svelte

@ -0,0 +1,246 @@ @@ -0,0 +1,246 @@
<script lang="ts">
import { visualizationConfig } from '$lib/stores/visualizationConfig';
import { Button, Input } from 'flowbite-svelte';
import { CloseCircleOutline } from 'flowbite-svelte-icons';
import { getEventKindName, getEventKindColor } from '$lib/utils/eventColors';
let {
onReload = () => {},
eventCounts = {}
} = $props<{
onReload?: () => void;
eventCounts?: { [kind: number]: number };
}>();
let newKind = $state('');
let showAddInput = $state(false);
let inputError = $state('');
function validateKind(value: string | number): number | null {
// Convert to string for consistent handling
const strValue = String(value);
if (strValue === null || strValue === undefined || strValue.trim() === '') {
inputError = '';
return null;
}
const kind = parseInt(strValue.trim());
if (isNaN(kind)) {
inputError = 'Must be a number';
return null;
}
if (kind < 0) {
inputError = 'Must be non-negative';
return null;
}
if ($visualizationConfig.eventConfigs.some(ec => ec.kind === kind)) {
inputError = 'Already added';
return null;
}
inputError = '';
return kind;
}
function handleAddKind() {
console.log('[EventTypeConfig] handleAddKind called with:', newKind);
const kind = validateKind(newKind);
console.log('[EventTypeConfig] Validation result:', kind);
if (kind !== null) {
console.log('[EventTypeConfig] Adding event kind:', kind);
visualizationConfig.addEventKind(kind);
newKind = '';
showAddInput = false;
inputError = '';
} else {
console.log('[EventTypeConfig] Validation failed:', inputError);
}
}
function handleKeydown(e: KeyboardEvent) {
if (e.key === 'Enter') {
handleAddKind();
} else if (e.key === 'Escape') {
showAddInput = false;
newKind = '';
inputError = '';
}
}
function handleLimitChange(kind: number, value: string) {
const limit = parseInt(value);
if (!isNaN(limit) && limit > 0) {
visualizationConfig.updateEventLimit(kind, limit);
}
}
function handleNestedLevelsChange(value: string) {
const levels = parseInt(value);
if (!isNaN(levels) && levels >= 0) {
visualizationConfig.updateNestedLevels(levels);
}
}
function handleFollowDepthChange(value: string) {
const depth = parseInt(value);
if (!isNaN(depth) && depth >= 0 && depth <= 2) {
visualizationConfig.updateFollowDepth(depth);
}
}
</script>
<div class="space-y-3">
<span class="text-xs text-gray-600 dark:text-gray-400">
Showing {Object.values(eventCounts).reduce((a, b) => a + b, 0)} of {Object.values(eventCounts).reduce((a, b) => a + b, 0)} events
</span>
<!-- Event configurations -->
<div class="space-y-2">
{#each $visualizationConfig.eventConfigs as config}
{@const isLoaded = (eventCounts[config.kind] || 0) > 0}
{@const isDisabled = $visualizationConfig.disabledKinds?.includes(config.kind) || false}
{@const color = getEventKindColor(config.kind)}
{@const borderColor = isLoaded ? 'border-green-500' : 'border-red-500'}
<div class="flex items-center gap-2">
<!-- Kind badge with color indicator and load status border -->
<button
class="flex items-center gap-1 min-w-[140px] px-2 py-1 border-2 rounded {borderColor} {isDisabled ? 'opacity-50' : ''} hover:bg-gray-100 dark:hover:bg-gray-700 transition-colors cursor-pointer"
onclick={() => visualizationConfig.toggleKind(config.kind)}
title={isDisabled ? `Click to enable ${getEventKindName(config.kind)}` : `Click to disable ${getEventKindName(config.kind)}`}
>
<span
class="inline-block w-3 h-3 rounded-full flex-shrink-0"
style="background-color: {color}"
></span>
<span class="text-sm font-medium dark:text-white">
{config.kind}
</span>
</button>
<button
onclick={() => visualizationConfig.removeEventKind(config.kind)}
class="text-red-500 hover:text-red-700 transition-colors"
title="Remove {getEventKindName(config.kind)}"
>
<CloseCircleOutline class="w-4 h-4" />
</button>
<!-- Limit input for all kinds -->
<input
type="number"
value={config.limit}
min="1"
max="1000"
class="w-16 px-2 py-1 text-xs border rounded dark:bg-gray-700 dark:border-gray-600 dark:text-white"
oninput={(e) => handleLimitChange(config.kind, e.currentTarget.value)}
title="Max to display"
/>
<!-- Nested levels for 30040 -->
{#if config.kind === 30040}
<span class="text-xs text-gray-600 dark:text-gray-400">Nested Levels:</span>
<input
type="number"
value={config.nestedLevels || 1}
min="0"
max="10"
class="w-14 px-2 py-1 text-xs border rounded dark:bg-gray-700 dark:border-gray-600 dark:text-white"
oninput={(e) => handleNestedLevelsChange(e.currentTarget.value)}
title="Levels to traverse"
/>
{/if}
<!-- Additional settings for kind 3 (follow lists) -->
{#if config.kind === 3}
<select
value={config.depth || 0}
onchange={(e) => handleFollowDepthChange(e.currentTarget.value)}
class="px-2 py-1 text-xs border rounded dark:bg-gray-700 dark:border-gray-600 dark:text-white"
title="How many degrees of separation to traverse"
>
<option value="0">Direct</option>
<option value="1">2 degrees</option>
<option value="2">3 degrees</option>
</select>
{/if}
<!-- Load indicator -->
{#if isLoaded}
<span class="text-xs text-green-600 dark:text-green-400">
({eventCounts[config.kind]})
</span>
{:else}
<span class="text-xs text-red-600 dark:text-red-400">
(not loaded)
</span>
{/if}
</div>
{/each}
</div>
<!-- Add kind button/input -->
{#if showAddInput}
<div class="flex items-center gap-2">
<input
bind:value={newKind}
type="number"
placeholder="Enter event kind number (e.g. 1)"
class="flex-1 px-3 py-1 text-sm border rounded dark:bg-gray-700 dark:border-gray-600 dark:text-white focus:outline-none focus:ring-2 focus:ring-blue-500"
onkeydown={handleKeydown}
oninput={(e) => validateKind(e.currentTarget.value)}
/>
<Button size="xs" onclick={handleAddKind} disabled={newKind === '' || !!inputError}>
Add
</Button>
<Button
size="xs"
color="light"
onclick={() => {
showAddInput = false;
newKind = '';
inputError = '';
}}
>
Cancel
</Button>
</div>
{#if inputError}
<p class="text-xs text-red-500 -mt-2">
{inputError}
</p>
{/if}
{:else}
<Button
size="xs"
color="light"
onclick={() => showAddInput = true}
class="gap-1"
>
<span>+</span>
<span>Add Event Type</span>
</Button>
{/if}
<!-- Reload button -->
<Button
size="xs"
color="blue"
onclick={onReload}
class="gap-1"
title="Reload graph with current settings"
>
<svg class="w-3 h-3" fill="none" stroke="currentColor" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg">
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M4 4v5h.582m15.356 2A8.001 8.001 0 004.582 9m0 0H9m11 11v-5h-.581m0 0a8.003 8.003 0 01-15.357-2m15.357 2H15"></path>
</svg>
<span>Reload</span>
</Button>
<!-- Border legend -->
<div class="text-xs text-gray-500 dark:text-gray-400 space-y-1 mt-2">
<p class="flex items-center gap-2">
<span class="inline-block w-3 h-3 border-2 border-green-500 rounded"></span>
<span>Green = Events loaded</span>
</p>
<p class="flex items-center gap-2">
<span class="inline-block w-3 h-3 border-2 border-red-500 rounded"></span>
<span>Red = Not loaded (click Reload)</span>
</p>
</div>
</div>

43
src/lib/navigator/EventNetwork/Legend.svelte

@ -159,7 +159,7 @@ @@ -159,7 +159,7 @@
</div>
{/if}
<div
class="tag-grid"
class="tag-grid {tagAnchors.length > 20 ? 'scrollable' : ''}"
style="grid-template-columns: repeat({TAG_LEGEND_COLUMNS}, 1fr);"
>
{#each tagAnchors as anchor}
@ -271,6 +271,35 @@ @@ -271,6 +271,35 @@
background-color: rgba(0, 0, 0, 0.02);
}
.tag-grid {
display: grid;
gap: 0.25rem;
}
.tag-grid.scrollable {
max-height: 400px;
overflow-y: auto;
padding-right: 0.5rem;
}
.tag-grid.scrollable::-webkit-scrollbar {
width: 6px;
}
.tag-grid.scrollable::-webkit-scrollbar-track {
background: #f1f1f1;
border-radius: 3px;
}
.tag-grid.scrollable::-webkit-scrollbar-thumb {
background: #888;
border-radius: 3px;
}
.tag-grid.scrollable::-webkit-scrollbar-thumb:hover {
background: #555;
}
:global(.dark) .legend-section-header:hover {
background-color: rgba(255, 255, 255, 0.05);
}
@ -290,4 +319,16 @@ @@ -290,4 +319,16 @@
:global(.dark) .tag-grid-item:hover.disabled {
background-color: rgba(255, 255, 255, 0.02);
}
:global(.dark) .tag-grid.scrollable::-webkit-scrollbar-track {
background: #374151;
}
:global(.dark) .tag-grid.scrollable::-webkit-scrollbar-thumb {
background: #6b7280;
}
:global(.dark) .tag-grid.scrollable::-webkit-scrollbar-thumb:hover {
background: #9ca3af;
}
</style>

16
src/lib/navigator/EventNetwork/NodeTooltip.svelte

@ -201,11 +201,23 @@ @@ -201,11 +201,23 @@
{/if}
</div>
<!-- Author -->
<!-- Pub Author -->
<div class="tooltip-metadata">
Author: {getAuthorTag(node)}
Pub Author: {getAuthorTag(node)}
</div>
<!-- Published by (from node.author) -->
{#if node.author}
<div class="tooltip-metadata">
published_by: {node.author}
</div>
{:else}
<!-- Fallback to author tag -->
<div class="tooltip-metadata">
published_by: {getAuthorTag(node)}
</div>
{/if}
{#if isPublicationEvent(node.kind)}
<!-- Summary (for publication index nodes) -->
{#if node.isContainer && getSummaryTag(node)}

97
src/lib/navigator/EventNetwork/Settings.svelte

@ -3,10 +3,7 @@ @@ -3,10 +3,7 @@
import { CaretDownOutline, CaretUpOutline } from "flowbite-svelte-icons";
import { fly } from "svelte/transition";
import { quintOut } from "svelte/easing";
import EventLimitControl from "$lib/components/EventLimitControl.svelte";
import EventRenderLevelLimit from "$lib/components/EventRenderLevelLimit.svelte";
import EventKindFilter from "$lib/components/EventKindFilter.svelte";
import { networkFetchLimit, levelsToRender } from "$lib/state";
import EventTypeConfig from "$lib/components/EventTypeConfig.svelte";
import { displayLimits } from "$lib/stores/displayLimits";
import { visualizationConfig } from "$lib/stores/visualizationConfig";
import { Toggle, Select } from "flowbite-svelte";
@ -15,28 +12,31 @@ @@ -15,28 +12,31 @@
count = 0,
totalCount = 0,
onupdate,
onclear = () => {},
starVisualization = $bindable(true),
showTagAnchors = $bindable(false),
selectedTagType = $bindable("t"),
tagExpansionDepth = $bindable(0),
requirePublications = $bindable(true),
onFetchMissing = () => {},
eventCounts = {},
} = $props<{
count: number;
totalCount: number;
onupdate: () => void;
onclear?: () => void;
starVisualization?: boolean;
showTagAnchors?: boolean;
selectedTagType?: string;
tagExpansionDepth?: number;
requirePublications?: boolean;
onFetchMissing?: (ids: string[]) => void;
eventCounts?: { [kind: number]: number };
}>();
let expanded = $state(false);
let eventTypesExpanded = $state(true);
let initialLoadExpanded = $state(true);
let displayLimitsExpanded = $state(true);
let graphTraversalExpanded = $state(true);
let visualSettingsExpanded = $state(true);
@ -49,10 +49,6 @@ @@ -49,10 +49,6 @@
eventTypesExpanded = !eventTypesExpanded;
}
function toggleInitialLoad() {
initialLoadExpanded = !initialLoadExpanded;
}
function toggleDisplayLimits() {
displayLimitsExpanded = !displayLimitsExpanded;
}
@ -138,10 +134,10 @@ @@ -138,10 +134,10 @@
Showing {count} of {totalCount} events
</span>
<!-- Event Kind Filter Section -->
<!-- Event Configuration Section (combines types and limits) -->
<div class="settings-section">
<div class="settings-section-header" onclick={toggleEventTypes}>
<h4 class="settings-section-title">Event Types <span class="text-orange-500 text-xs font-normal">(not tested)</span></h4>
<h4 class="settings-section-title">Event Configuration</h4>
<Button
color="none"
outline
@ -156,32 +152,7 @@ @@ -156,32 +152,7 @@
</Button>
</div>
{#if eventTypesExpanded}
<EventKindFilter onReload={onupdate} {eventCounts} />
{/if}
</div>
<!-- Initial Load Settings Section -->
<div class="settings-section">
<div class="settings-section-header" onclick={toggleInitialLoad}>
<h4 class="settings-section-title">Initial Load</h4>
<Button
color="none"
outline
size="xs"
class="rounded-full p-1"
>
{#if initialLoadExpanded}
<CaretUpOutline class="w-3 h-3" />
{:else}
<CaretDownOutline class="w-3 h-3" />
{/if}
</Button>
</div>
{#if initialLoadExpanded}
<div>
<EventLimitControl on:update={handleLimitUpdate} />
<EventRenderLevelLimit on:update={handleLimitUpdate} />
</div>
<EventTypeConfig onReload={onupdate} {eventCounts} />
{/if}
</div>
@ -286,6 +257,33 @@ @@ -286,6 +257,33 @@
<p class="text-xs text-gray-500 dark:text-gray-400 ml-6">
When enabled, graph expansion will only use events already loaded
</p>
<label class="flex items-center space-x-2 mt-3">
<Toggle
checked={$visualizationConfig.appendMode}
onclick={() => visualizationConfig.toggleAppendMode()}
class="text-xs"
/>
<span class="text-xs text-gray-600 dark:text-gray-400">Append mode (accumulate events)</span>
</label>
<p class="text-xs text-gray-500 dark:text-gray-400 ml-6">
When enabled, new fetches will add to existing graph instead of replacing it
</p>
{#if $visualizationConfig.appendMode && count > 0}
<Button
size="xs"
color="red"
onclick={onclear}
class="gap-1 mt-3"
title="Clear all accumulated events"
>
<svg class="w-3 h-3" fill="none" stroke="currentColor" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg">
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M19 7l-.867 12.142A2 2 0 0116.138 21H7.862a2 2 0 01-1.995-1.858L5 7m5 4v6m4-6v6m1-10V4a1 1 0 00-1-1h-4a1 1 0 00-1 1v3M4 7h16"></path>
</svg>
<span>Clear Graph ({count} events)</span>
</Button>
{/if}
{/if}
</div>
@ -362,12 +360,33 @@ @@ -362,12 +360,33 @@
>
<option value="t">Hashtags</option>
<option value="author">Authors</option>
<option value="p">People (Pubkeys)</option>
<option value="p">People (from follow lists)</option>
<option value="e">Event References</option>
<!-- <option value="a">Article References</option> -->
<option value="title">Titles</option>
<option value="summary">Summaries</option>
</Select>
{#if selectedTagType === "p" && (!eventCounts[3] || eventCounts[3] === 0)}
<p class="text-xs text-orange-500 mt-1">
No follow lists loaded. Enable kind 3 events to see people tag anchors.
</p>
{/if}
{#if selectedTagType === "p" && eventCounts[3] > 0}
<label class="flex items-center space-x-2 mt-2">
<Toggle
checked={requirePublications}
onchange={(e: Event) => {
const target = e.target as HTMLInputElement;
requirePublications = target.checked;
}}
size="sm"
class="text-xs"
/>
<span class="text-xs text-gray-600 dark:text-gray-400">Only show people with publications</span>
</label>
{/if}
</div>
<div class="space-y-1">
@ -375,7 +394,7 @@ @@ -375,7 +394,7 @@
<label
for="tag-depth-input"
class="text-xs text-gray-600 dark:text-gray-400 whitespace-nowrap"
>Expansion Depth: <span class="text-red-500 font-semibold">(not functional)</span></label
>Expansion Depth:</label
>
<input
type="number"

135
src/lib/navigator/EventNetwork/index.svelte

@ -37,6 +37,8 @@ @@ -37,6 +37,8 @@
getTagAnchorColor,
} from "./utils/tagNetworkBuilder";
import { Button } from "flowbite-svelte";
import { visualizationConfig } from "$lib/stores/visualizationConfig";
import { get } from "svelte/store";
// Type alias for D3 selections
type Selection = any;
@ -61,14 +63,18 @@ @@ -61,14 +63,18 @@
// Component props
let {
events = [],
followListEvents = [],
totalCount = 0,
onupdate,
onclear = () => {},
onTagExpansionChange,
onFetchMissing = () => {}
} = $props<{
events?: NDKEvent[];
followListEvents?: NDKEvent[];
totalCount?: number;
onupdate: () => void;
onclear?: () => void;
onTagExpansionChange?: (depth: number, tags: string[]) => void;
onFetchMissing?: (ids: string[]) => void;
}>();
@ -119,6 +125,7 @@ @@ -119,6 +125,7 @@
let selectedTagType = $state("t"); // Default to hashtags
let tagAnchorInfo = $state<any[]>([]);
let tagExpansionDepth = $state(0); // Default to no expansion
let requirePublications = $state(true); // Default to only showing people with publications
// Store initial state to detect if component is being recreated
let componentId = Math.random();
@ -275,12 +282,130 @@ @@ -275,12 +282,130 @@
height
});
// For "p" tags, we need to extract pubkeys from follow lists
// but only show anchors for pubkeys that have events in the visualization
let eventsForTags = events;
if (selectedTagType === "p" && followListEvents.length > 0) {
// Extract all pubkeys from follow lists
const followedPubkeys = new Set<string>();
followListEvents.forEach(event => {
event.tags.forEach(tag => {
if (tag[0] === "p" && tag[1]) {
followedPubkeys.add(tag[1]);
}
});
});
const syntheticEvents: NDKEvent[] = [];
// Create a map to track which events each followed pubkey is connected to
const pubkeyToEvents = new Map<string, Set<string>>();
// Find all connections for followed pubkeys
followedPubkeys.forEach(pubkey => {
const connectedEventIds = new Set<string>();
// Find events they authored
events.forEach(event => {
if (event.pubkey === pubkey && event.id) {
connectedEventIds.add(event.id);
}
});
// Find events where they're tagged with "p"
events.forEach(event => {
if (event.id && event.tags) {
event.tags.forEach(tag => {
if (tag[0] === 'p' && tag[1] === pubkey) {
connectedEventIds.add(event.id);
}
});
}
});
if (connectedEventIds.size > 0) {
pubkeyToEvents.set(pubkey, connectedEventIds);
}
});
if (requirePublications) {
// Only show people who have connections to events
pubkeyToEvents.forEach((eventIds, pubkey) => {
// Create synthetic events for each connection
eventIds.forEach(eventId => {
const syntheticEvent = {
id: eventId, // Use the actual event's ID so it connects properly
tags: [["p", pubkey]],
pubkey: "",
created_at: 0,
kind: 0,
content: "",
sig: ""
} as NDKEvent;
syntheticEvents.push(syntheticEvent);
});
});
} else {
// Show all people from follow lists
let syntheticId = 0;
// First, add people who have event connections
pubkeyToEvents.forEach((eventIds, pubkey) => {
eventIds.forEach(eventId => {
const syntheticEvent = {
id: eventId, // Use the actual event's ID so it connects properly
tags: [["p", pubkey]],
pubkey: "",
created_at: 0,
kind: 0,
content: "",
sig: ""
} as NDKEvent;
syntheticEvents.push(syntheticEvent);
});
});
// Then, add remaining people without any connections
followedPubkeys.forEach(pubkey => {
if (!pubkeyToEvents.has(pubkey)) {
const syntheticEvent = {
id: `synthetic-p-${syntheticId++}`, // Create unique IDs for those without events
tags: [["p", pubkey]],
pubkey: "",
created_at: 0,
kind: 0,
content: "",
sig: ""
} as NDKEvent;
syntheticEvents.push(syntheticEvent);
}
});
}
eventsForTags = syntheticEvents;
debug("Created synthetic events for p tags", {
followedPubkeys: followedPubkeys.size,
requirePublications,
syntheticEvents: syntheticEvents.length
});
}
// Get the display limit based on tag type
let displayLimit: number | undefined;
if (selectedTagType === "p") {
// For people tags, use kind 0 (profiles) limit
const kind0Config = get(visualizationConfig).eventConfigs.find(ec => ec.kind === 0);
displayLimit = kind0Config?.limit || 50;
}
graphData = enhanceGraphWithTags(
graphData,
events,
eventsForTags,
selectedTagType,
width,
height,
displayLimit,
);
// Extract tag anchor info for legend
@ -297,6 +422,11 @@ @@ -297,6 +422,11 @@
count: n.connectedNodes?.length || 0,
color: getTagAnchorColor(n.tagType || ""),
}));
// Add a message if People tag type is selected but no follow lists are loaded
if (selectedTagType === "p" && followListEvents.length === 0 && tagAnchors.length === 0) {
console.warn("[EventNetwork] No follow lists loaded. Enable kind 3 events with appropriate depth to see people tag anchors.");
}
} else {
tagAnchorInfo = [];
}
@ -846,6 +976,7 @@ @@ -846,6 +976,7 @@
$effect(() => {
// Only check when tag anchors are shown and we have tags
if (showTagAnchors && tagAnchorInfo.length > 0) {
// If we have more than MAX_TAG_ANCHORS and haven't auto-disabled yet
if (tagAnchorInfo.length > MAX_TAG_ANCHORS && !autoDisabledTags) {
debug(`Auto-disabling tags: ${tagAnchorInfo.length} exceeds maximum of ${MAX_TAG_ANCHORS}`);
@ -983,11 +1114,13 @@ @@ -983,11 +1114,13 @@
count={events.length}
{totalCount}
{onupdate}
{onclear}
{onFetchMissing}
bind:starVisualization
bind:showTagAnchors
bind:selectedTagType
bind:tagExpansionDepth
bind:requirePublications
{eventCounts}
/>

25
src/lib/navigator/EventNetwork/utils/starNetworkBuilder.ts

@ -129,9 +129,15 @@ export function createStarNetworks( @@ -129,9 +129,15 @@ export function createStarNetworks(
referencedIds: new Set<string>()
};
// Find all index events
// Find all index events and non-publication events
const publicationKinds = [30040, 30041, 30818];
const indexEvents = events.filter(event => event.kind === INDEX_EVENT_KIND);
const nonPublicationEvents = events.filter(event =>
event.kind !== undefined && !publicationKinds.includes(event.kind)
);
debug("Found index events", { count: indexEvents.length });
debug("Found non-publication events", { count: nonPublicationEvents.length });
const starNetworks: StarNetwork[] = [];
const processedIndices = new Set<string>();
@ -150,6 +156,23 @@ export function createStarNetworks( @@ -150,6 +156,23 @@ export function createStarNetworks(
});
}
});
// Add non-publication events as standalone nodes (stars with no peripherals)
nonPublicationEvents.forEach(event => {
if (!event.id || !nodeMap.has(event.id)) return;
const node = nodeMap.get(event.id)!;
const star: StarNetwork = {
center: node,
peripheralNodes: [],
links: []
};
starNetworks.push(star);
debug("Created standalone star for non-publication event", {
eventId: event.id,
kind: event.kind
});
});
return starNetworks;
}

26
src/lib/navigator/EventNetwork/utils/tagNetworkBuilder.ts

@ -7,6 +7,7 @@ @@ -7,6 +7,7 @@
import type { NDKEvent } from "@nostr-dev-kit/ndk";
import type { NetworkNode, NetworkLink, GraphData } from "../types";
import { getDisplayNameSync } from "$lib/utils/profileCache";
// Configuration
const TAG_ANCHOR_RADIUS = 15;
@ -59,6 +60,8 @@ export function getTagAnchorColor(tagType: string): string { @@ -59,6 +60,8 @@ export function getTagAnchorColor(tagType: string): string {
return "#F59E0B"; // Yellow for events
case "a":
return "#EF4444"; // Red for articles
case "kind3":
return "#06B6D4"; // Cyan for follow lists
default:
return "#6B7280"; // Gray for others
}
@ -113,12 +116,16 @@ export function createTagAnchorNodes( @@ -113,12 +116,16 @@ export function createTagAnchorNodes(
// Calculate positions for tag anchors randomly within radius
// For single publication view, show all tags. For network view, only show tags with 2+ events
const minEventCount = tagMap.size <= 10 ? 1 : 2;
const validTags = Array.from(tagMap.entries()).filter(
// Exception: for "p" tags, always use minEventCount of 1 to show all people
const minEventCount = tagType === "p" ? 1 : (tagMap.size <= 10 ? 1 : 2);
let validTags = Array.from(tagMap.entries()).filter(
([_, eventIds]) => eventIds.size >= minEventCount,
);
if (validTags.length === 0) return [];
// Sort all tags by number of connections (events) descending
validTags.sort((a, b) => b[1].size - a[1].size);
validTags.forEach(([tagValue, eventIds]) => {
// Position anchors randomly within a radius from the center
@ -142,8 +149,8 @@ export function createTagAnchorNodes( @@ -142,8 +149,8 @@ export function createTagAnchorNodes(
} else if (tagType === "author") {
displayTitle = tagValue;
} else if (tagType === "p") {
// Truncate pubkey for display
displayTitle = `${tagValue.substring(0, 8)}...`;
// Use display name for pubkey
displayTitle = getDisplayNameSync(tagValue);
}
const anchorNode: NetworkNode = {
@ -207,12 +214,21 @@ export function enhanceGraphWithTags( @@ -207,12 +214,21 @@ export function enhanceGraphWithTags(
tagType: string,
width: number,
height: number,
displayLimit?: number,
): GraphData {
// Extract unique tags for the specified type
const tagMap = extractUniqueTagsForType(events, tagType);
// Create tag anchor nodes
const tagAnchors = createTagAnchorNodes(tagMap, tagType, width, height);
let tagAnchors = createTagAnchorNodes(tagMap, tagType, width, height);
// Apply display limit if provided
if (displayLimit && displayLimit > 0 && tagAnchors.length > displayLimit) {
console.log(`[TagBuilder] Limiting display to ${displayLimit} tag anchors out of ${tagAnchors.length}`);
// Sort by connection count (already done in createTagAnchorNodes)
// and take only the top ones up to the limit
tagAnchors = tagAnchors.slice(0, displayLimit);
}
// Create links between anchors and nodes
const tagLinks = createTagLinks(tagAnchors, graphData.nodes);

273
src/lib/stores/visualizationConfig.ts

@ -1,89 +1,234 @@ @@ -1,89 +1,234 @@
import { writable, derived } from 'svelte/store';
import { writable, derived, get } from "svelte/store";
export interface EventKindConfig {
kind: number;
limit: number;
nestedLevels?: number; // Only for kind 30040
depth?: number; // Only for kind 3 (follow lists)
}
export interface VisualizationConfig {
// Event filtering
allowedKinds: number[]; // Using array for ordered display
disabledKinds: number[]; // Kinds that are temporarily disabled but not removed
allowFreeEvents: boolean;
// Display limits (moving from displayLimits store)
maxPublicationIndices: number; // -1 unlimited
maxEventsPerIndex: number; // -1 unlimited
// Event configurations with per-kind limits
eventConfigs: EventKindConfig[];
// Graph traversal
searchThroughFetched: boolean;
// Append mode - add new events to existing graph instead of replacing
appendMode?: boolean;
// Legacy properties for backward compatibility
allowedKinds?: number[];
disabledKinds?: number[];
allowFreeEvents?: boolean;
maxPublicationIndices?: number;
maxEventsPerIndex?: number;
}
// Default configurations for common event kinds
const DEFAULT_EVENT_CONFIGS: EventKindConfig[] = [
{ kind: 0, limit: 50 }, // Metadata events (profiles) - controls how many profiles to fetch
{ kind: 3, limit: 1, depth: 0 }, // Follow lists - limit 1 = just user's, higher = user's + from follows
{ kind: 30040, limit: 20, nestedLevels: 1 },
{ kind: 30041, limit: 20 },
{ kind: 30818, limit: 20 },
];
function createVisualizationConfig() {
const { subscribe, set, update } = writable<VisualizationConfig>({
allowedKinds: [30040, 30041, 30818],
disabledKinds: [30041, 30818], // 30041 and 30818 disabled by default
// Initialize with both new and legacy properties
const initialConfig: VisualizationConfig = {
eventConfigs: DEFAULT_EVENT_CONFIGS,
searchThroughFetched: true,
appendMode: false,
// Legacy properties
allowedKinds: DEFAULT_EVENT_CONFIGS.map(ec => ec.kind),
disabledKinds: [30041, 30818],
allowFreeEvents: false,
maxPublicationIndices: -1,
maxEventsPerIndex: -1,
searchThroughFetched: true
});
};
const { subscribe, set, update } =
writable<VisualizationConfig>(initialConfig);
// Helper to sync legacy properties with eventConfigs
const syncLegacyProperties = (config: VisualizationConfig) => {
config.allowedKinds = config.eventConfigs.map((ec) => ec.kind);
return config;
};
return {
subscribe,
update,
reset: () => set({
allowedKinds: [30040, 30041, 30818],
disabledKinds: [30041, 30818], // 30041 and 30818 disabled by default
allowFreeEvents: false,
maxPublicationIndices: -1,
maxEventsPerIndex: -1,
searchThroughFetched: true
}),
addKind: (kind: number) => update(config => {
if (!config.allowedKinds.includes(kind)) {
return { ...config, allowedKinds: [...config.allowedKinds, kind] };
}
return config;
}),
removeKind: (kind: number) => update(config => ({
...config,
allowedKinds: config.allowedKinds.filter(k => k !== kind)
})),
toggleFreeEvents: () => update(config => ({
...config,
allowFreeEvents: !config.allowFreeEvents
})),
setMaxPublicationIndices: (max: number) => update(config => ({
...config,
maxPublicationIndices: max
})),
setMaxEventsPerIndex: (max: number) => update(config => ({
...config,
maxEventsPerIndex: max
})),
toggleSearchThroughFetched: () => update(config => ({
...config,
searchThroughFetched: !config.searchThroughFetched
})),
toggleKind: (kind: number) => update(config => {
const isDisabled = config.disabledKinds.includes(kind);
if (isDisabled) {
// Re-enable it
return {
reset: () => set(initialConfig),
// Add a new event kind with default limit
addEventKind: (kind: number, limit: number = 10) =>
update((config) => {
// Check if kind already exists
if (config.eventConfigs.some((ec) => ec.kind === kind)) {
return config;
}
const newConfig: EventKindConfig = { kind, limit };
// Add nestedLevels for 30040
if (kind === 30040) {
newConfig.nestedLevels = 1;
}
// Add depth for kind 3
if (kind === 3) {
newConfig.depth = 0;
}
const updated = {
...config,
disabledKinds: config.disabledKinds.filter(k => k !== kind)
eventConfigs: [...config.eventConfigs, newConfig],
};
} else {
// Disable it
return {
return syncLegacyProperties(updated);
}),
// Legacy method for backward compatibility
addKind: (kind: number) =>
update((config) => {
if (config.eventConfigs.some((ec) => ec.kind === kind)) {
return config;
}
const updated = {
...config,
disabledKinds: [...config.disabledKinds, kind]
eventConfigs: [...config.eventConfigs, { kind, limit: 10 }],
};
}
})
return syncLegacyProperties(updated);
}),
// Remove an event kind
removeEventKind: (kind: number) =>
update((config) => {
const updated = {
...config,
eventConfigs: config.eventConfigs.filter((ec) => ec.kind !== kind),
};
return syncLegacyProperties(updated);
}),
// Legacy method for backward compatibility
removeKind: (kind: number) =>
update((config) => {
const updated = {
...config,
eventConfigs: config.eventConfigs.filter((ec) => ec.kind !== kind),
};
return syncLegacyProperties(updated);
}),
// Update limit for a specific kind
updateEventLimit: (kind: number, limit: number) =>
update((config) => ({
...config,
eventConfigs: config.eventConfigs.map((ec) =>
ec.kind === kind ? { ...ec, limit } : ec,
),
})),
// Update nested levels for kind 30040
updateNestedLevels: (levels: number) =>
update((config) => ({
...config,
eventConfigs: config.eventConfigs.map((ec) =>
ec.kind === 30040 ? { ...ec, nestedLevels: levels } : ec,
),
})),
// Update depth for kind 3
updateFollowDepth: (depth: number) =>
update((config) => ({
...config,
eventConfigs: config.eventConfigs.map((ec) =>
ec.kind === 3 ? { ...ec, depth: depth } : ec,
),
})),
// Get config for a specific kind
getEventConfig: (kind: number) => {
let config: EventKindConfig | undefined;
subscribe((c) => {
config = c.eventConfigs.find((ec) => ec.kind === kind);
})();
return config;
},
toggleSearchThroughFetched: () =>
update((config) => ({
...config,
searchThroughFetched: !config.searchThroughFetched,
})),
toggleAppendMode: () =>
update((config) => ({
...config,
appendMode: !config.appendMode,
})),
// Legacy methods for backward compatibility
toggleKind: (kind: number) =>
update((config) => {
const isDisabled = config.disabledKinds?.includes(kind) || false;
if (isDisabled) {
// Re-enable it
return {
...config,
disabledKinds:
config.disabledKinds?.filter((k) => k !== kind) || [],
};
} else {
// Disable it
return {
...config,
disabledKinds: [...(config.disabledKinds || []), kind],
};
}
}),
toggleFreeEvents: () =>
update((config) => ({
...config,
allowFreeEvents: !config.allowFreeEvents,
})),
setMaxPublicationIndices: (max: number) =>
update((config) => ({
...config,
maxPublicationIndices: max,
})),
setMaxEventsPerIndex: (max: number) =>
update((config) => ({
...config,
maxEventsPerIndex: max,
})),
};
}
export const visualizationConfig = createVisualizationConfig();
// Helper to check if a kind is allowed and enabled
// Helper to get all enabled event kinds
export const enabledEventKinds = derived(visualizationConfig, ($config) =>
$config.eventConfigs.map((ec) => ec.kind),
);
// Helper to check if a kind is enabled
export const isKindEnabled = derived(
visualizationConfig,
($config) => (kind: number) =>
$config.eventConfigs.some((ec) => ec.kind === kind),
);
// Legacy helper for backward compatibility
export const isKindAllowed = derived(
visualizationConfig,
$config => (kind: number) => $config.allowedKinds.includes(kind) && !$config.disabledKinds.includes(kind)
);
($config) => (kind: number) => {
const inEventConfigs = $config.eventConfigs.some((ec) => ec.kind === kind);
const notDisabled = !($config.disabledKinds?.includes(kind) || false);
return inEventConfigs && notDisabled;
},
);

56
src/lib/utils/profileCache.ts

@ -1,6 +1,7 @@ @@ -1,6 +1,7 @@
import type { NDKEvent } from "@nostr-dev-kit/ndk";
import { ndkInstance } from "$lib/ndk";
import { get } from "svelte/store";
import { nip19 } from "nostr-tools";
interface ProfileData {
display_name?: string;
@ -72,31 +73,58 @@ export async function getDisplayName(pubkey: string): Promise<string> { @@ -72,31 +73,58 @@ export async function getDisplayName(pubkey: string): Promise<string> {
/**
* Batch fetches profiles for multiple pubkeys
* @param pubkeys - Array of public keys to fetch profiles for
* @param onProgress - Optional callback for progress updates
*/
export async function batchFetchProfiles(pubkeys: string[]): Promise<void> {
export async function batchFetchProfiles(
pubkeys: string[],
onProgress?: (fetched: number, total: number) => void
): Promise<void> {
// Filter out already cached pubkeys
const uncachedPubkeys = pubkeys.filter(pk => !profileCache.has(pk));
if (uncachedPubkeys.length === 0) {
if (onProgress) onProgress(pubkeys.length, pubkeys.length);
return;
}
try {
const ndk = get(ndkInstance);
const profileEvents = await ndk.fetchEvents({
kinds: [0],
authors: uncachedPubkeys
});
// Process each profile event
profileEvents.forEach((event: NDKEvent) => {
try {
const content = JSON.parse(event.content);
profileCache.set(event.pubkey, content as ProfileData);
} catch (e) {
console.error("Failed to parse profile content:", e);
// Report initial progress
const cachedCount = pubkeys.length - uncachedPubkeys.length;
if (onProgress) onProgress(cachedCount, pubkeys.length);
// Batch fetch in chunks to avoid overwhelming relays
const CHUNK_SIZE = 50;
let fetchedCount = cachedCount;
for (let i = 0; i < uncachedPubkeys.length; i += CHUNK_SIZE) {
const chunk = uncachedPubkeys.slice(i, Math.min(i + CHUNK_SIZE, uncachedPubkeys.length));
const profileEvents = await ndk.fetchEvents({
kinds: [0],
authors: chunk
});
// Process each profile event
profileEvents.forEach((event: NDKEvent) => {
try {
const content = JSON.parse(event.content);
profileCache.set(event.pubkey, content as ProfileData);
fetchedCount++;
} catch (e) {
console.error("Failed to parse profile content:", e);
}
});
// Update progress
if (onProgress) {
onProgress(fetchedCount, pubkeys.length);
}
});
}
// Final progress update
if (onProgress) onProgress(pubkeys.length, pubkeys.length);
} catch (e) {
console.error("Failed to batch fetch profiles:", e);
}

322
src/routes/visualize/+page.svelte

@ -13,11 +13,12 @@ @@ -13,11 +13,12 @@
import { filterValidIndexEvents } from "$lib/utils";
import { networkFetchLimit } from "$lib/state";
import { displayLimits } from "$lib/stores/displayLimits";
import { visualizationConfig } from "$lib/stores/visualizationConfig";
import { visualizationConfig, type EventKindConfig } from "$lib/stores/visualizationConfig";
import { filterByDisplayLimits, detectMissingEvents } from "$lib/utils/displayLimits";
import type { PageData } from './$types';
import { getEventKindColor, getEventKindName } from "$lib/utils/eventColors";
import { extractPubkeysFromEvents, batchFetchProfiles } from "$lib/utils/profileCache";
import { activePubkey } from "$lib/ndk";
// Configuration
const DEBUG = false; // Set to true to enable debug logging
@ -47,6 +48,152 @@ @@ -47,6 +48,152 @@
let missingEventIds = $state(new Set<string>()); // Track missing referenced events
let loadingEventKinds = $state<Array<{kind: number, limit: number}>>([]); // Track what kinds are being loaded
let isFetching = false; // Guard against concurrent fetches
let followListEvents = $state<NDKEvent[]>([]); // Store follow list events separately
// Profile loading progress
let profileLoadingProgress = $state<{current: number, total: number} | null>(null);
let profileLoadingMessage = $derived(
profileLoadingProgress
? `Loading profiles: ${profileLoadingProgress.current}/${profileLoadingProgress.total}`
: null
);
/**
* Fetches follow lists (kind 3) with depth expansion
*/
async function fetchFollowLists(config: EventKindConfig): Promise<NDKEvent[]> {
const depth = config.depth || 0;
const allFollowEvents: NDKEvent[] = [];
const processedPubkeys = new Set<string>();
debug(`Fetching kind 3 follow lists with depth ${depth}, addFollowLists: ${config.addFollowLists}`);
// Get the current user's pubkey
const currentUserPubkey = get(activePubkey);
if (!currentUserPubkey) {
console.warn("No logged-in user, cannot fetch user's follow list");
return [];
}
// If limit is 1, only fetch the current user's follow list
if (config.limit === 1) {
const userFollowList = await $ndkInstance.fetchEvents({
kinds: [3],
authors: [currentUserPubkey],
limit: 1
});
if (userFollowList.size === 0) {
console.warn("User has no follow list");
return [];
}
const userFollowEvent = Array.from(userFollowList)[0];
allFollowEvents.push(userFollowEvent);
processedPubkeys.add(currentUserPubkey);
debug(`Fetched user's follow list`);
} else {
// If limit > 1, fetch the user's follow list plus additional ones from people they follow
const userFollowList = await $ndkInstance.fetchEvents({
kinds: [3],
authors: [currentUserPubkey],
limit: 1
});
if (userFollowList.size === 0) {
console.warn("User has no follow list");
return [];
}
const userFollowEvent = Array.from(userFollowList)[0];
allFollowEvents.push(userFollowEvent);
processedPubkeys.add(currentUserPubkey);
// Extract followed pubkeys
const followedPubkeys: string[] = [];
userFollowEvent.tags.forEach(tag => {
if (tag[0] === 'p' && tag[1]) {
followedPubkeys.push(tag[1]);
}
});
debug(`User follows ${followedPubkeys.length} people`);
// Fetch additional follow lists from people you follow
if (followedPubkeys.length > 0) {
const additionalLimit = config.limit - 1; // We already have the user's
const pubkeysToFetch = followedPubkeys.slice(0, additionalLimit);
debug(`Fetching ${pubkeysToFetch.length} additional follow lists (total limit: ${config.limit})`);
const additionalFollowLists = await $ndkInstance.fetchEvents({
kinds: [3],
authors: pubkeysToFetch
});
allFollowEvents.push(...Array.from(additionalFollowLists));
// Mark these as processed
additionalFollowLists.forEach(event => {
processedPubkeys.add(event.pubkey);
});
debug(`Fetched ${additionalFollowLists.size} additional follow lists`);
}
}
// If depth > 0, we need to fetch follow lists of follows (recursively)
if (depth > 0) {
// Start with all pubkeys from fetched follow lists
let currentLevelPubkeys: string[] = [];
allFollowEvents.forEach(event => {
event.tags.forEach(tag => {
if (tag[0] === 'p' && tag[1] && !processedPubkeys.has(tag[1])) {
currentLevelPubkeys.push(tag[1]);
}
});
});
// Fetch additional levels based on depth
for (let level = 1; level <= depth; level++) {
if (currentLevelPubkeys.length === 0) break;
debug(`Fetching level ${level} follow lists for ${currentLevelPubkeys.length} pubkeys`);
// Fetch follow lists for this level
const levelFollowLists = await $ndkInstance.fetchEvents({
kinds: [3],
authors: currentLevelPubkeys
});
const nextLevelPubkeys: string[] = [];
levelFollowLists.forEach(event => {
allFollowEvents.push(event);
processedPubkeys.add(event.pubkey);
// Extract pubkeys for next level
if (level < depth) {
event.tags.forEach(tag => {
if (tag[0] === 'p' && tag[1] && !processedPubkeys.has(tag[1])) {
nextLevelPubkeys.push(tag[1]);
}
});
}
});
currentLevelPubkeys = nextLevelPubkeys;
}
}
debug(`Fetched ${allFollowEvents.length} follow lists total`);
// Store follow lists separately for tag anchor use
followListEvents = [...allFollowEvents];
return allFollowEvents;
}
/**
* Fetches events from the Nostr network
@ -90,25 +237,31 @@ @@ -90,25 +237,31 @@
let allFetchedEvents: NDKEvent[] = [];
// First, fetch non-publication events (like kind 0, 1, etc.)
// First, fetch non-publication events (like kind 0, 1, 3, etc.)
if (otherConfigs.length > 0) {
debug("Fetching non-publication events:", otherConfigs);
for (const config of otherConfigs) {
try {
const fetchedEvents = await $ndkInstance.fetchEvents(
{
kinds: [config.kind],
limit: config.limit
},
{
groupable: true,
skipVerification: false,
skipValidation: false,
}
);
debug(`Fetched ${fetchedEvents.size} events of kind ${config.kind}`);
allFetchedEvents.push(...Array.from(fetchedEvents));
// Special handling for kind 3 (follow lists)
if (config.kind === 3) {
const followEvents = await fetchFollowLists(config);
allFetchedEvents.push(...followEvents);
} else {
const fetchedEvents = await $ndkInstance.fetchEvents(
{
kinds: [config.kind],
limit: config.limit
},
{
groupable: true,
skipVerification: false,
skipValidation: false,
}
);
debug(`Fetched ${fetchedEvents.size} events of kind ${config.kind}`);
allFetchedEvents.push(...Array.from(fetchedEvents));
}
} catch (e) {
console.error(`Error fetching kind ${config.kind}:`, e);
}
@ -306,14 +459,77 @@ @@ -306,14 +459,77 @@
finalEventMap.set(event.id, event);
});
allEvents = Array.from(finalEventMap.values());
// Handle append mode
if ($visualizationConfig.appendMode && allEvents.length > 0) {
// Merge existing events with new events
const existingEventMap = new Map(allEvents.map(e => [e.id, e]));
// Add new events to existing map (new events override old ones)
finalEventMap.forEach((event, id) => {
existingEventMap.set(id, event);
});
allEvents = Array.from(existingEventMap.values());
// Note: followListEvents are already accumulated in fetchFollowLists
} else {
// Replace mode (default)
allEvents = Array.from(finalEventMap.values());
// Clear follow lists in replace mode
if (!$visualizationConfig.appendMode) {
followListEvents = [];
}
}
baseEvents = [...allEvents]; // Store base events for tag expansion
// Step 6: Fetch profiles for all pubkeys in events
debug("Fetching profiles for pubkeys in events");
const pubkeys = extractPubkeysFromEvents(allEvents);
await batchFetchProfiles(Array.from(pubkeys));
debug("Profile fetch complete for", pubkeys.size, "pubkeys");
// Step 6: Fetch profiles (kind 0)
debug("Fetching profiles for events");
// Get kind 0 config to respect its limit
const profileConfig = enabledConfigs.find(ec => ec.kind === 0);
const profileLimit = profileConfig?.limit || 50;
// Collect all pubkeys that need profiles
const allPubkeys = new Set<string>();
// Add event authors (these are the main content creators)
allEvents.forEach(event => {
if (event.pubkey) {
allPubkeys.add(event.pubkey);
}
});
// Add pubkeys from follow lists (for tag anchors)
if (followListEvents.length > 0) {
followListEvents.forEach(event => {
event.tags.forEach(tag => {
if (tag[0] === 'p' && tag[1]) {
allPubkeys.add(tag[1]);
}
});
});
}
// Limit the number of profiles to fetch based on kind 0 limit
const pubkeysArray = Array.from(allPubkeys);
const pubkeysToFetch = profileLimit === -1
? pubkeysArray
: pubkeysArray.slice(0, profileLimit);
debug("Profile fetch strategy:", {
totalPubkeys: allPubkeys.size,
profileLimit,
pubkeysToFetch: pubkeysToFetch.length,
followListsLoaded: followListEvents.length
});
profileLoadingProgress = { current: 0, total: pubkeysToFetch.length };
await batchFetchProfiles(pubkeysToFetch, (fetched, total) => {
profileLoadingProgress = { current: fetched, total };
});
profileLoadingProgress = null; // Clear progress when done
debug("Profile fetch complete for", pubkeysToFetch.length, "pubkeys");
// Step 7: Apply display limits
events = filterByDisplayLimits(allEvents, $displayLimits, $visualizationConfig);
@ -526,7 +742,11 @@ @@ -526,7 +742,11 @@
const newPubkeys = extractPubkeysFromEvents([...newPublications, ...newContentEvents]);
if (newPubkeys.size > 0) {
debug("Fetching profiles for", newPubkeys.size, "new pubkeys from tag expansion");
await batchFetchProfiles(Array.from(newPubkeys));
profileLoadingProgress = { current: 0, total: newPubkeys.size };
await batchFetchProfiles(Array.from(newPubkeys), (fetched, total) => {
profileLoadingProgress = { current: fetched, total };
});
profileLoadingProgress = null;
}
// Apply display limits
@ -587,7 +807,11 @@ @@ -587,7 +807,11 @@
const newPubkeys = extractPubkeysFromEvents(newEvents);
if (newPubkeys.size > 0) {
debug("Fetching profiles for", newPubkeys.size, "pubkeys from missing events");
await batchFetchProfiles(Array.from(newPubkeys));
profileLoadingProgress = { current: 0, total: newPubkeys.size };
await batchFetchProfiles(Array.from(newPubkeys), (fetched, total) => {
profileLoadingProgress = { current: fetched, total };
});
profileLoadingProgress = null;
}
// Add to all events
@ -662,6 +886,20 @@ @@ -662,6 +886,20 @@
// }
// });
/**
* Clears all accumulated events
*/
function clearEvents() {
allEvents = [];
events = [];
baseEvents = [];
followListEvents = [];
missingEventIds = new Set();
// Clear node positions cache in EventNetwork
// This will be handled by the component when events change
}
// Fetch events when component mounts
onMount(() => {
debug("Component mounted");
@ -716,6 +954,21 @@ @@ -716,6 +954,21 @@
</div>
{/each}
</div>
<!-- Profile loading progress bar -->
{#if profileLoadingProgress}
<div class="mt-4">
<p class="text-sm text-gray-600 dark:text-gray-400 mb-2">
{profileLoadingMessage}
</p>
<div class="w-full bg-gray-200 dark:bg-gray-700 rounded-full h-2">
<div
class="bg-blue-600 h-2 rounded-full transition-all duration-300"
style="width: {(profileLoadingProgress.current / profileLoadingProgress.total) * 100}%"
></div>
</div>
</div>
{/if}
</div>
</div>
<!-- Error message -->
@ -736,11 +989,30 @@ @@ -736,11 +989,30 @@
</div>
<!-- Network visualization -->
{:else}
<!-- Profile loading progress bar (overlay when loading profiles after initial load) -->
{#if profileLoadingProgress}
<div class="absolute top-0 left-0 right-0 z-10 p-4">
<div class="bg-white dark:bg-gray-800 rounded-lg shadow-lg p-3">
<p class="text-sm text-gray-600 dark:text-gray-400 mb-2">
{profileLoadingMessage}
</p>
<div class="w-full bg-gray-200 dark:bg-gray-700 rounded-full h-2">
<div
class="bg-blue-600 h-2 rounded-full transition-all duration-300"
style="width: {(profileLoadingProgress.current / profileLoadingProgress.total) * 100}%"
></div>
</div>
</div>
</div>
{/if}
<!-- Event network visualization -->
<EventNetwork
{events}
{events}
{followListEvents}
totalCount={allEvents.length}
onupdate={fetchEvents}
onupdate={fetchEvents}
onclear={clearEvents}
onTagExpansionChange={handleTagExpansion}
onFetchMissing={fetchMissingEvents}
/>

Loading…
Cancel
Save