Compare commits
5 Commits
d5313b2211
...
2e6b7e178a
| Author | SHA1 | Date |
|---|---|---|
|
|
2e6b7e178a | 1 week ago |
|
|
e62e8604a4 | 1 week ago |
|
|
129fc4964b | 1 week ago |
|
|
a9ad84c186 | 1 week ago |
|
|
8353ae8c25 | 1 week ago |
53 changed files with 2568 additions and 626 deletions
@ -0,0 +1,7 @@ |
|||||||
|
# Default compose file from repo root; override with `make prewarm COMPOSE=...` if needed.
|
||||||
|
COMPOSE ?= docker compose
|
||||||
|
|
||||||
|
.PHONY: prewarm |
||||||
|
prewarm: |
||||||
|
@chmod +x scripts/docker-prewarm.sh 2>/dev/null || true
|
||||||
|
@./scripts/docker-prewarm.sh
|
||||||
@ -1,79 +1,130 @@ |
|||||||
# Unfold |
# Unfold: Imwald |
||||||
|
|
||||||
Unfold is a customizable framework for your Nostr-based magazine. |
<p align="center"> |
||||||
|
<img src="assets/laeserin_logo.png" alt="Imwald" width="150"> |
||||||
|
</p> |
||||||
|
|
||||||
(This is the **Imwald** edition of Unfold.) |
A Symfony + FrankenPHP site that **reads Nostr long-form articles (kind 30023)** and related data from relays, stores articles in **MySQL**, and serves pages with Twig. **Comments and profile metadata** are **cache-backed** (not the full source of truth in the DB). |
||||||
|
|
||||||
## Setup |
--- |
||||||
|
|
||||||
### Clone the repository |
## Requirements |
||||||
|
|
||||||
|
| Requirement | Version / notes | |
||||||
|
|------------|-----------------| |
||||||
|
| PHP | **≥ 8.3.13** (see `composer.json`) | |
||||||
|
| Docker | Optional; recommended for local dev and production images | |
||||||
|
| Database | MySQL **8.0** (configurable) | |
||||||
|
|
||||||
|
--- |
||||||
|
|
||||||
|
## Local development (Docker) |
||||||
|
|
||||||
|
1. **Env:** copy `.env.dist` to `.env` and adjust if needed (especially `APP_SECRET` outside dev). |
||||||
|
2. **Start stack** |
||||||
|
|
||||||
|
```bash |
||||||
|
docker compose up -d |
||||||
|
``` |
||||||
|
|
||||||
|
3. **App URL (default):** [http://127.0.0.1:9080](http://127.0.0.1:9080) |
||||||
|
Port comes from `HTTP_PORT` in `.env` and `compose.override.yaml` (loopback only). |
||||||
|
|
||||||
|
4. **First-time DB:** migrations run on **php** container start when `migrations/` contains PHP files (see `frankenphp/docker-entrypoint.sh`). |
||||||
|
|
||||||
|
| Service | Role | |
||||||
|
|--------|------| |
||||||
|
| `php` | FrankenPHP + Caddy, Symfony app, console | |
||||||
|
| `database` | MySQL; dev exposes `127.0.0.1:3307 → 3306` for local clients | |
||||||
|
| `cron` | Runs full **`app:prewarm` every 10 minutes**; repo bind-mounted at `/var/www/html` (see `docker/cron/`) | |
||||||
|
|
||||||
|
--- |
||||||
|
|
||||||
|
## Backfill articles + warm caches (recommended) |
||||||
|
|
||||||
|
To **migrate**, **import articles from Nostr** for a time window, then **prewarm** magazine indices, author metadata, and comment caches: |
||||||
|
|
||||||
```bash |
```bash |
||||||
git clone https://github.com/decent-newsroom/unfold.git |
make prewarm |
||||||
cd unfold |
|
||||||
``` |
``` |
||||||
|
|
||||||
### Create the .env file |
| Step (script order) | Command / effect | |
||||||
|
|---------------------|------------------| |
||||||
|
| 1 | `docker compose up -d --wait` — starts **php**, **database**, and **cron** (the `cron` image runs a full `app:prewarm` on a 10 min schedule) | |
||||||
|
| 2 | `doctrine:migrations:migrate` | |
||||||
|
| 3 | `articles:get -- '-2 month' 'now'` — sync long-form into MySQL for that window | |
||||||
|
| 4 | `app:prewarm` — magazine **30040**, **kind-0** profiles, **comment** cache (default **`--comments-max=20`**, newest by `createdAt`) | |
||||||
|
|
||||||
Copy the example file `.env.dist` and replace placeholders with your actual configuration. |
`make prewarm` brings the stack (including `cron`) up so scheduled prewarm is active. **Optional** extra arguments for the **cron**-scheduled `app:prewarm` go in **`.env`** as **`PREWARM_FLAGS`** (same as you might pass to `php bin/console app:prewarm …`); Compose passes them into the `cron` container. Example: `PREWARM_FLAGS="--metadata-limit=50 --no-magazine"`. **Restart** the `cron` service after changing `PREWARM_FLAGS` so the container reloads the env. Hub / `compose.hub.yaml` has no `cron` service; use a host timer or `exec` if you need the same there. |
||||||
|
|
||||||
If you have your own MySQL database, comment out the database service in `compose.yaml` and skip root password in `.env`. |
--- |
||||||
There are additional comments to that effect in the files. |
|
||||||
|
|
||||||
### Configure `config/unfold.yaml` |
## Console commands (overview) |
||||||
|
|
||||||
Before running the application, review and update `config/unfold.yaml` to match your desired magazine settings, theme, and external links. This file controls: |
| Command | Purpose | |
||||||
- Magazine name, short name, and description |
|---------|---------| |
||||||
- Theme and color settings |
| `articles:get <from> <to>` | Pull long-form articles from Nostr for the time range, persist to DB | |
||||||
- Community articles feature |
| `app:prewarm` | Magazine relay refresh + metadata cache + comment cache warm | |
||||||
- External footer links |
| `doctrine:migrations:migrate` | Apply SQL migrations | |
||||||
- Other project-specific configuration |
| `user:elevate` | (If used) user elevation helper | |
||||||
|
|
||||||
Edit the values in `config/unfold.yaml` as needed for your deployment. |
`php bin/console list` and `… -h` for full options. |
||||||
|
|
||||||
### Customizing Theme and Icons |
### `app:prewarm` (notable options) |
||||||
|
|
||||||
You can override the default theme and icons by adding your own files to `/assets/theme/local/`. To do this: |
| Option | Default | Meaning | |
||||||
- Copy the structure and file names from `/assets/theme/default/`. |
|--------|---------|--------| |
||||||
- Place your custom `theme.css` and icon files in your theme folder. |
| `--no-magazine` | off | Skip magazine 30040 index | |
||||||
- Update your configuration in `config/unfold.yaml` to reference your custom theme if needed. |
| `--no-metadata` | off | Skip Nostr kind-0 / profile cache | |
||||||
|
| `--no-comments` | off | Skip comment thread cache | |
||||||
|
| `--metadata-limit` | `0` (all authors) | Cap distinct author pubkeys | |
||||||
|
| `--metadata-batch` | `50` | Pubkeys per batched Nostr `REQ` | |
||||||
|
| `--comments-max` | `20` | Newest **N** articles (by `createdAt` **DESC**); `0` = all (still bounded by budget) | |
||||||
|
| `--comments-budget` | `120` | Max wall seconds for the comments phase | |
||||||
|
| `--magazine-budget` | `30` | Max wall seconds for magazine refresh | |
||||||
|
|
||||||
This allows you to easily switch or update the look and feel of your magazine without modifying the default assets. |
Prewarm clears the PHP **CLI** execution time limit for that run; relay work can be slow. |
||||||
|
|
||||||
|
### `PREWARM_ON_START` (optional) |
||||||
|
|
||||||
### Build the Docker containers |
| Variable | Set where | Effect | |
||||||
|
|----------|------------|--------| |
||||||
|
| `PREWARM_ON_START=1` | **Compose `environment` on the `php` service** (not only Symfony `.env` inside the container) | After DB is up and migrations run, executes **`app:prewarm` once** on start. **Does not** run `articles:get`. | |
||||||
|
|
||||||
For development: |
For a full **Nostr backfill** + one-shot prewarm, use **`make prewarm`** (or a host **cron** / **systemd** timer) instead of relying on **`PREWARM_ON_START` alone**. |
||||||
```bash |
|
||||||
docker compose build |
|
||||||
``` |
|
||||||
|
|
||||||
For production (using production overrides), set `APP_ENV=prod` in your `.env` file and run: |
--- |
||||||
```bash |
|
||||||
docker compose -f compose.yaml -f compose.prod.yaml build |
|
||||||
``` |
|
||||||
|
|
||||||
|
## Configuration |
||||||
|
|
||||||
### Start the Docker containers |
| What | File | |
||||||
```bash |
|------|------| |
||||||
docker compose up -d |
| Site title, `npub`, `d_tag`, **relays** (`default_relay`, `article_relays`, `profile_relays`), theme | `config/unfold.yaml` (imported as Symfony parameters) | |
||||||
``` |
| `DATABASE_URL`, `APP_SECRET`, `HTTP_PORT`, `MYSQL_*`, optional **`PREWARM_FLAGS`** (for the Docker `cron` service) | `.env` / `.env.local` (see `.env.dist`) | |
||||||
|
| Service wiring (e.g. cache, `NostrClient` args) | `config/services.yaml` | |
||||||
|
|
||||||
|
**Relays (short):** `default_relay` and `article_relays` drive article sync and many queries; `profile_relays` are used **first** for kind-0 / profile fetches, then the merged default + article set (see `NostrClient`). |
||||||
|
|
||||||
### Run Database Migrations |
--- |
||||||
|
|
||||||
Before fetching or displaying articles, make sure your database schema is up to date. Run: |
## Production / Hub image |
||||||
|
|
||||||
```bash |
| Topic | Notes | |
||||||
docker compose exec php php bin/console doctrine:migrations:migrate --no-interaction |
|-------|--------| |
||||||
``` |
| `compose.hub.yaml` | Runs a **pulled** image (default `silberengel/unfold:latest`), no local PHP app build. Override with `UNFOLD_DOCKER_IMAGE`. | |
||||||
|
| HTTP publish | `HTTP_PUBLISH` in `.env` (default **9080** → container **80**). Set `TRUSTED_PROXIES` behind a reverse proxy. | |
||||||
|
| Secrets | Set `APP_SECRET` and DB credentials in **real** env; do not commit production secrets. | |
||||||
|
|
||||||
### Fetching Articles |
File header in `compose.hub.yaml` lists pull, migrate, and optional build/push one-liners. |
||||||
|
|
||||||
To fetch articles from the default relay for the last two months, run: |
--- |
||||||
|
|
||||||
```bash |
## License |
||||||
docker compose exec php php bin/console articles:get -- '-2 month' 'now' |
|
||||||
``` |
**MIT** — see [`LICENSE`](LICENSE). |
||||||
|
|
||||||
|
--- |
||||||
|
|
||||||
|
## Project links (example) |
||||||
|
|
||||||
You can adjust the date range as needed. This command will import articles into the local database. |
Configurable under `parameters.external_links` in `config/unfold.yaml` (e.g. Unfold on GitHub, Decent Newsroom). Adjust for your deployment. |
||||||
|
|||||||
@ -0,0 +1,49 @@ |
|||||||
|
import { Controller } from "@hotwired/stimulus"; |
||||||
|
|
||||||
|
/** |
||||||
|
* After first paint, refreshes Nostr magazine indices (server-side, ≤5s) and swaps header/body HTML. |
||||||
|
*/ |
||||||
|
export default class extends Controller { |
||||||
|
static targets = ["headerNav", "pageBody"]; |
||||||
|
static values = { |
||||||
|
page: String, |
||||||
|
slug: String, |
||||||
|
url: String, |
||||||
|
}; |
||||||
|
|
||||||
|
connect() { |
||||||
|
this.sync(); |
||||||
|
} |
||||||
|
|
||||||
|
async sync() { |
||||||
|
const base = this.urlValue || "/ux/magazine-sync"; |
||||||
|
const params = new URLSearchParams(); |
||||||
|
params.set("page", this.pageValue || "article"); |
||||||
|
const slug = this.slugValue || ""; |
||||||
|
if (slug !== "") { |
||||||
|
params.set("slug", slug); |
||||||
|
} |
||||||
|
const url = `${base}?${params.toString()}`; |
||||||
|
try { |
||||||
|
const res = await fetch(url, { |
||||||
|
headers: { Accept: "application/json" }, |
||||||
|
credentials: "same-origin", |
||||||
|
}); |
||||||
|
if (!res.ok) { |
||||||
|
return; |
||||||
|
} |
||||||
|
const data = await res.json(); |
||||||
|
if (!data.ok) { |
||||||
|
return; |
||||||
|
} |
||||||
|
if (this.hasHeaderNavTarget && data.header) { |
||||||
|
this.headerNavTarget.outerHTML = data.header; |
||||||
|
} |
||||||
|
if (this.hasPageBodyTarget && data.body) { |
||||||
|
this.pageBodyTarget.outerHTML = data.body; |
||||||
|
} |
||||||
|
} catch { |
||||||
|
/* ignore network errors */ |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
@ -0,0 +1,61 @@ |
|||||||
|
# Run a pre-built production image from Docker Hub (no local PHP image build). |
||||||
|
# |
||||||
|
# Usage on the server (copy this file + your .env, no app source required): |
||||||
|
# docker compose -f compose.hub.yaml pull |
||||||
|
# docker compose -f compose.hub.yaml up -d |
||||||
|
# docker compose -f compose.hub.yaml exec php php bin/console doctrine:migrations:migrate --no-interaction |
||||||
|
# |
||||||
|
# Required in .env: APP_SECRET. Set MYSQL_* (or replace DATABASE_URL after editing this file) if you |
||||||
|
# use the bundled database. For TLS in front, set TRUSTED_PROXIES to include your reverse proxy CIDR. |
||||||
|
# |
||||||
|
# Host HTTP port defaults to 9080 (same idea as local dev) so Apache/nginx can keep :80. Override with |
||||||
|
# HTTP_PUBLISH=80 or HTTP_PUBLISH=127.0.0.1:9080 in .env if needed. |
||||||
|
# |
||||||
|
# Build & push (on your machine or CI), e.g.: |
||||||
|
# docker build --platform linux/amd64 --target frankenphp_prod -t silberengel/unfold:latest . |
||||||
|
# docker push silberengel/unfold:latest |
||||||
|
# |
||||||
|
# Override image: UNFOLD_DOCKER_IMAGE=myregistry/unfold:1.0.0 docker compose -f compose.hub.yaml up -d |
||||||
|
|
||||||
|
name: unfold |
||||||
|
|
||||||
|
services: |
||||||
|
php: |
||||||
|
image: ${UNFOLD_DOCKER_IMAGE:-silberengel/unfold:latest} |
||||||
|
pull_policy: always |
||||||
|
restart: unless-stopped |
||||||
|
environment: |
||||||
|
APP_ENV: ${APP_ENV:-prod} |
||||||
|
APP_SECRET: ${APP_SECRET} |
||||||
|
TRUSTED_PROXIES: ${TRUSTED_PROXIES:-127.0.0.0/8,10.0.0.0/8} |
||||||
|
SERVER_NAME: ${SERVER_NAME:-:80} |
||||||
|
DATABASE_URL: mysql://${MYSQL_USER:-unfold_user}:${MYSQL_PASSWORD:-password}@database:3306/${MYSQL_DATABASE:-unfold_db}?serverVersion=${MYSQL_VERSION:-8.0}&charset=${MYSQL_CHARSET:-utf8mb4} |
||||||
|
volumes: |
||||||
|
- caddy_data:/data |
||||||
|
- caddy_config:/config |
||||||
|
ports: |
||||||
|
- "${HTTP_PUBLISH:-9080}:80/tcp" |
||||||
|
depends_on: |
||||||
|
database: |
||||||
|
condition: service_healthy |
||||||
|
|
||||||
|
database: |
||||||
|
image: mysql:${MYSQL_VERSION:-8.0} |
||||||
|
restart: unless-stopped |
||||||
|
environment: |
||||||
|
MYSQL_DATABASE: ${MYSQL_DATABASE:-unfold_db} |
||||||
|
MYSQL_USER: ${MYSQL_USER:-unfold_user} |
||||||
|
MYSQL_PASSWORD: ${MYSQL_PASSWORD:-password} |
||||||
|
MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD:-root_password} |
||||||
|
healthcheck: |
||||||
|
test: ["CMD", "mysqladmin", "ping", "-h", "localhost"] |
||||||
|
timeout: 5s |
||||||
|
retries: 5 |
||||||
|
start_period: 60s |
||||||
|
volumes: |
||||||
|
- database_data:/var/lib/mysql:rw |
||||||
|
|
||||||
|
volumes: |
||||||
|
caddy_data: |
||||||
|
caddy_config: |
||||||
|
database_data: |
||||||
@ -1,30 +1,21 @@ |
|||||||
FROM php:8.2-cli |
FROM php:8.3-cli |
||||||
|
|
||||||
# Install cron and Redis PHP extension dependencies |
RUN apt-get update && apt-get install -y --no-install-recommends \ |
||||||
RUN apt-get update && apt-get install -y \ |
bash \ |
||||||
cron \ |
cron \ |
||||||
libzip-dev \ |
&& rm -rf /var/lib/apt/lists/* |
||||||
libicu-dev \ |
|
||||||
libpq-dev \ |
|
||||||
libonig-dev |
|
||||||
|
|
||||||
|
ADD https://github.com/mlocati/docker-php-extension-installer/releases/latest/download/install-php-extensions /usr/local/bin/ |
||||||
|
RUN chmod +x /usr/local/bin/install-php-extensions \ |
||||||
|
&& install-php-extensions pdo_mysql intl opcache zip gmp |
||||||
|
|
||||||
# Set working directory |
|
||||||
WORKDIR /var/www/html |
WORKDIR /var/www/html |
||||||
|
|
||||||
# Install Symfony CLI tools (optional) |
COPY crontab /etc/cron.d/unfold-prewarm |
||||||
# RUN curl -sS https://get.symfony.com/cli/installer | bash |
COPY prewarm_cron.sh /prewarm_cron.sh |
||||||
|
COPY entry-cron.sh /entry-cron.sh |
||||||
|
|
||||||
# Copy cron and script |
RUN chmod 0644 /etc/cron.d/unfold-prewarm \ |
||||||
COPY crontab /etc/cron.d/app-cron |
&& chmod +x /prewarm_cron.sh /entry-cron.sh |
||||||
COPY index_articles.sh /index_articles.sh |
|
||||||
|
|
||||||
# Set permissions |
CMD ["/entry-cron.sh"] |
||||||
RUN chmod 0644 /etc/cron.d/app-cron && \ |
|
||||||
chmod +x /index_articles.sh |
|
||||||
|
|
||||||
# Apply cron job |
|
||||||
RUN crontab /etc/cron.d/app-cron |
|
||||||
|
|
||||||
# Run cron in the foreground |
|
||||||
CMD ["cron", "-f"] |
|
||||||
|
|||||||
@ -1,94 +1,15 @@ |
|||||||
|
# `cron` service (Docker) |
||||||
|
|
||||||
# 🕒 Cron Job Container |
The `cron` image runs a single job: **`php bin/console app:prewarm` every 10 minutes**, against the app tree bind-mounted at `/var/www/html`. |
||||||
|
|
||||||
This folder contains the Docker configuration to run scheduled Symfony commands via cron inside a separate container. |
- **Flags:** set **`PREWARM_FLAGS`** in the project `.env` (Compose injects it). Example: `PREWARM_FLAGS="--metadata-limit=30 --no-magazine"`. After editing, run `docker compose up -d --force-recreate cron` (or `docker compose up -d cron`) so the container gets the new value. If unset, `app:prewarm` uses its **built-in defaults** (same idea as running the console with no args). |
||||||
|
|
||||||
- Run Symfony console commands periodically using a cron schedule (e.g. every 6 hours) |
- **How env reaches cron:** the entrypoint writes `PREWARM_FLAGS` to `/run/cron-prewarm.env` at boot, because the system `crond` does not pass the container environment into crontab jobs. |
||||||
- Decouple scheduled jobs from the main PHP/FPM container |
|
||||||
- Easily manage and test cron execution in a Dockerized Symfony project |
|
||||||
|
|
||||||
--- |
- **Logs inside the container:** `tail -f /var/log/cron.log` (e.g. `docker compose exec cron tail -f /var/log/cron.log`). |
||||||
|
|
||||||
## Build & Run |
- **PHP 8.3** extensions in the image are limited to what `app:prewarm` needs; the host **vendor** tree is what you mount from the repo. |
||||||
|
|
||||||
1. **Build the cron image** |
- **Not included in** `compose.hub.yaml` (no app source mount). For production images, use host **cron** / **systemd** to `exec` the same command. |
||||||
From the project root: |
|
||||||
```bash |
|
||||||
docker-compose build cron |
|
||||||
``` |
|
||||||
|
|
||||||
2. **Start the cron container** |
Change the schedule: edit `docker/cron/crontab`, then `docker compose build cron && docker compose up -d cron`. |
||||||
```bash |
|
||||||
docker-compose up -d cron |
|
||||||
``` |
|
||||||
|
|
||||||
--- |
|
||||||
|
|
||||||
## Cron Schedule |
|
||||||
|
|
||||||
The default cron schedule is set to run **every 6 hours**: |
|
||||||
|
|
||||||
```cron |
|
||||||
0 */6 * * * root /run_commands.sh >> /var/log/cron.log 2>&1 |
|
||||||
``` |
|
||||||
|
|
||||||
To customize the schedule, edit the `crontab` file and rebuild the container. |
|
||||||
|
|
||||||
--- |
|
||||||
|
|
||||||
## Testing & Debugging |
|
||||||
|
|
||||||
### Manually test the command runner |
|
||||||
|
|
||||||
You can run the script manually to check behavior without waiting for the cron trigger: |
|
||||||
|
|
||||||
```bash |
|
||||||
docker-compose exec cron /run_commands.sh |
|
||||||
``` |
|
||||||
|
|
||||||
### Check the cron output log |
|
||||||
|
|
||||||
```bash |
|
||||||
docker-compose exec cron tail -f /var/log/cron.log |
|
||||||
``` |
|
||||||
|
|
||||||
### Shell into the cron container |
|
||||||
|
|
||||||
```bash |
|
||||||
docker-compose exec cron bash |
|
||||||
``` |
|
||||||
|
|
||||||
Once inside, you can: |
|
||||||
- Check crontab entries: `crontab -l` |
|
||||||
- Manually trigger cron: `cron` or `cron -f` (in another session) |
|
||||||
|
|
||||||
--- |
|
||||||
|
|
||||||
## Customization |
|
||||||
|
|
||||||
- **Add/Remove Symfony Commands:** |
|
||||||
Edit `run_commands.sh` to include the commands you want to run. |
|
||||||
|
|
||||||
- **Change Schedule:** |
|
||||||
Edit `crontab` using standard cron syntax. |
|
||||||
|
|
||||||
- **Logging:** |
|
||||||
Logs are sent to `/var/log/cron.log` inside the container. |
|
||||||
|
|
||||||
--- |
|
||||||
|
|
||||||
## Rebuilding After Changes |
|
||||||
|
|
||||||
If you modify the `crontab` or `run_commands.sh`, make sure to rebuild: |
|
||||||
|
|
||||||
```bash |
|
||||||
docker-compose build cron |
|
||||||
docker-compose up -d cron |
|
||||||
``` |
|
||||||
|
|
||||||
--- |
|
||||||
|
|
||||||
## Notes |
|
||||||
|
|
||||||
- Symfony project source is mounted at `/var/www/html` via volume. |
|
||||||
- Make sure your commands do **not rely on services** (like `php-fpm`) that are not running in this container. |
|
||||||
|
|||||||
@ -1 +1 @@ |
|||||||
*/5 * * * * /index_articles.sh >> /var/log/cron.log 2>&1 |
*/10 * * * * root /prewarm_cron.sh >>/var/log/cron.log 2>&1 |
||||||
|
|||||||
@ -0,0 +1,6 @@ |
|||||||
|
#!/bin/bash |
||||||
|
set -euo pipefail |
||||||
|
# crond does not pass Compose env to job children; write once at boot for prewarm_cron.sh. |
||||||
|
export PREWARM_FLAGS="${PREWARM_FLAGS:-}" |
||||||
|
declare -p PREWARM_FLAGS >/run/cron-prewarm.env |
||||||
|
exec cron -f |
||||||
@ -1,5 +0,0 @@ |
|||||||
#!/bin/bash |
|
||||||
set -e |
|
||||||
export PATH="/usr/local/bin:/usr/bin:/bin" |
|
||||||
|
|
||||||
php /var/www/html/bin/console articles:get -- '-6 min' 'now' |
|
||||||
@ -0,0 +1,9 @@ |
|||||||
|
#!/bin/bash |
||||||
|
set -euo pipefail |
||||||
|
# shellcheck source=/dev/null |
||||||
|
if [[ -f /run/cron-prewarm.env ]]; then |
||||||
|
source /run/cron-prewarm.env |
||||||
|
fi |
||||||
|
cd /var/www/html |
||||||
|
# shellcheck disable=SC2086 |
||||||
|
exec php bin/console app:prewarm ${PREWARM_FLAGS:-} |
||||||
@ -0,0 +1,22 @@ |
|||||||
|
#!/usr/bin/env sh |
||||||
|
# After `docker compose up`, run migrations, backfill long-form articles, and app:prewarm. |
||||||
|
# Usage: from repository root: ./scripts/docker-prewarm.sh |
||||||
|
# Or: make prewarm |
||||||
|
set -eu |
||||||
|
|
||||||
|
ROOT_DIR=$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd) |
||||||
|
cd "$ROOT_DIR" |
||||||
|
|
||||||
|
echo "==> docker compose up -d --wait (php, database, cron: full app:prewarm every 10 min; set PREWARM_FLAGS in .env to match your CLI)" |
||||||
|
docker compose up -d --wait |
||||||
|
|
||||||
|
echo "==> doctrine:migrations:migrate" |
||||||
|
docker compose exec -T php php bin/console doctrine:migrations:migrate --no-interaction |
||||||
|
|
||||||
|
echo "==> articles:get (last 2 months → now)" |
||||||
|
docker compose exec -T php php bin/console articles:get -- '-2 month' 'now' |
||||||
|
|
||||||
|
echo "==> app:prewarm" |
||||||
|
docker compose exec -T php php bin/console app:prewarm |
||||||
|
|
||||||
|
echo "Done." |
||||||
@ -0,0 +1,250 @@ |
|||||||
|
<?php |
||||||
|
|
||||||
|
declare(strict_types=1); |
||||||
|
|
||||||
|
namespace App\Command; |
||||||
|
|
||||||
|
use App\Entity\Article; |
||||||
|
use App\Repository\ArticleRepository; |
||||||
|
use App\Service\ArticleCommentThreadLoader; |
||||||
|
use App\Service\CacheService; |
||||||
|
use App\Service\MagazineRefresher; |
||||||
|
use App\Service\Nip09DeletionApplier; |
||||||
|
use App\Service\NostrClient; |
||||||
|
use Psr\Log\LoggerInterface; |
||||||
|
use swentel\nostr\Key\Key; |
||||||
|
use Symfony\Component\Console\Attribute\AsCommand; |
||||||
|
use Symfony\Component\Console\Command\Command; |
||||||
|
use Symfony\Component\Console\Input\InputInterface; |
||||||
|
use Symfony\Component\Console\Input\InputOption; |
||||||
|
use Symfony\Component\Console\Output\OutputInterface; |
||||||
|
use Symfony\Component\Console\Style\SymfonyStyle; |
||||||
|
use Symfony\Component\DependencyInjection\ParameterBag\ParameterBagInterface; |
||||||
|
|
||||||
|
/** |
||||||
|
* Prewarms magazine index cache, author metadata cache, and optional comment thread cache. |
||||||
|
* Does not persist comments to MySQL; comments are cache-only in this app. |
||||||
|
*/ |
||||||
|
#[AsCommand( |
||||||
|
name: 'app:prewarm', |
||||||
|
description: 'Refresh magazine indices, NIP-09 deletions, profile metadata, and comment caches', |
||||||
|
)] |
||||||
|
final class PrewarmCommand extends Command |
||||||
|
{ |
||||||
|
public function __construct( |
||||||
|
private readonly MagazineRefresher $magazineRefresher, |
||||||
|
private readonly Nip09DeletionApplier $nip09DeletionApplier, |
||||||
|
private readonly CacheService $cacheService, |
||||||
|
private readonly NostrClient $nostrClient, |
||||||
|
private readonly ArticleRepository $articleRepository, |
||||||
|
private readonly ArticleCommentThreadLoader $commentThreadLoader, |
||||||
|
private readonly ParameterBagInterface $params, |
||||||
|
private readonly LoggerInterface $logger, |
||||||
|
) { |
||||||
|
parent::__construct(); |
||||||
|
} |
||||||
|
|
||||||
|
protected function configure(): void |
||||||
|
{ |
||||||
|
$this |
||||||
|
->addOption('no-magazine', null, InputOption::VALUE_NONE, 'Skip magazine 30040 index fetch') |
||||||
|
->addOption('no-deletions', null, InputOption::VALUE_NONE, 'Skip NIP-09 kind 5 deletion sync (30023/30024 DB + 30040 magazine cache)') |
||||||
|
->addOption('deletion-since', null, InputOption::VALUE_REQUIRED, 'strtotime() window start for kind 5 fetch', '-2 month') |
||||||
|
->addOption('no-metadata', null, InputOption::VALUE_NONE, 'Skip Nostr profile metadata cache') |
||||||
|
->addOption('no-comments', null, InputOption::VALUE_NONE, 'Skip comment thread cache') |
||||||
|
->addOption('magazine-budget', null, InputOption::VALUE_REQUIRED, 'Seconds wall time for magazine relay refresh', '30') |
||||||
|
->addOption('metadata-limit', null, InputOption::VALUE_REQUIRED, 'Max distinct author pubkeys to warm (0 = all)', '0') |
||||||
|
->addOption('metadata-batch', null, InputOption::VALUE_REQUIRED, 'Kind-0 metadata: pubkeys per Nostr REQ (batched)', '50') |
||||||
|
->addOption('comments-max', null, InputOption::VALUE_REQUIRED, 'Newest N articles to warm comment cache for (0 = all, order: createdAt DESC)', '20') |
||||||
|
->addOption('comments-budget', null, InputOption::VALUE_REQUIRED, 'Max seconds for the whole comments phase', '120'); |
||||||
|
} |
||||||
|
|
||||||
|
protected function execute(InputInterface $input, OutputInterface $output): int |
||||||
|
{ |
||||||
|
$this->disableCliExecutionTimeLimit(); |
||||||
|
|
||||||
|
$io = new SymfonyStyle($input, $output); |
||||||
|
$keys = new Key(); |
||||||
|
|
||||||
|
if (!$input->getOption('no-magazine')) { |
||||||
|
$budget = max(1, (int) $input->getOption('magazine-budget')); |
||||||
|
$io->section('Magazine index (kinds 30040)'); |
||||||
|
try { |
||||||
|
$this->magazineRefresher->refreshFromRelays($budget, []); |
||||||
|
$io->success('Magazine indices refreshed (within budget).'); |
||||||
|
} catch (\Throwable $e) { |
||||||
|
$this->logger->error('app:prewarm magazine failed', ['e' => $e]); |
||||||
|
$io->warning('Magazine refresh failed: '.$e->getMessage()); |
||||||
|
} |
||||||
|
} else { |
||||||
|
$io->note('Skipping magazine (--no-magazine).'); |
||||||
|
} |
||||||
|
|
||||||
|
// MagazineRefresher sets max_execution_time (e.g. 60 for budget 30); restore before metadata. |
||||||
|
$this->disableCliExecutionTimeLimit(); |
||||||
|
|
||||||
|
if (!$input->getOption('no-deletions')) { |
||||||
|
$io->section('NIP-09 deletions (kind 5 → 30023/30024 / 30040)'); |
||||||
|
$sinceStr = (string) $input->getOption('deletion-since'); |
||||||
|
$since = strtotime($sinceStr); |
||||||
|
if ($since === false) { |
||||||
|
$since = strtotime('-2 month'); |
||||||
|
} |
||||||
|
$until = time(); |
||||||
|
$deletionPubkeys = []; |
||||||
|
foreach ($this->articleRepository->findDistinctAuthorPubkeys() as $pk) { |
||||||
|
if (\is_string($pk) && 64 === \strlen($pk)) { |
||||||
|
$deletionPubkeys[] = $pk; |
||||||
|
} |
||||||
|
} |
||||||
|
$npubParam = (string) $this->params->get('npub'); |
||||||
|
if (str_starts_with($npubParam, 'npub')) { |
||||||
|
try { |
||||||
|
$sitePk = $keys->convertToHex($npubParam); |
||||||
|
if ($sitePk !== '' && 64 === \strlen($sitePk) && !\in_array($sitePk, $deletionPubkeys, true)) { |
||||||
|
$deletionPubkeys[] = $sitePk; |
||||||
|
} |
||||||
|
} catch (\Throwable) { |
||||||
|
} |
||||||
|
} |
||||||
|
if ($deletionPubkeys === []) { |
||||||
|
$io->note('No author pubkeys; skipping kind 5 deletion fetch.'); |
||||||
|
} else { |
||||||
|
try { |
||||||
|
$kind5 = $this->nostrClient->fetchKind5DeletionEventsForAuthors( |
||||||
|
$deletionPubkeys, |
||||||
|
$since, |
||||||
|
$until, |
||||||
|
40 |
||||||
|
); |
||||||
|
$st = $this->nip09DeletionApplier->apply($kind5); |
||||||
|
$io->writeln(sprintf( |
||||||
|
'Kind 5 events: <info>%d</info> (deduped). Articles removed: <info>%d</info>; magazine root/category cache entries removed: <info>%d</info> / <info>%d</info>.', |
||||||
|
\count($kind5), |
||||||
|
$st['articles_removed'], |
||||||
|
$st['magazine_roots'], |
||||||
|
$st['magazine_categories'] |
||||||
|
)); |
||||||
|
} catch (\Throwable $e) { |
||||||
|
$this->logger->error('app:prewarm NIP-09 failed', ['exception' => $e]); |
||||||
|
$io->warning('NIP-09 step failed: '.$e->getMessage()); |
||||||
|
} |
||||||
|
} |
||||||
|
} else { |
||||||
|
$io->note('Skipping NIP-09 deletions (--no-deletions).'); |
||||||
|
} |
||||||
|
|
||||||
|
$this->disableCliExecutionTimeLimit(); |
||||||
|
|
||||||
|
if (!$input->getOption('no-metadata')) { |
||||||
|
$io->section('Author metadata (cache)'); |
||||||
|
$pubkeys = $this->articleRepository->findDistinctAuthorPubkeys(); |
||||||
|
$npubParam = (string) $this->params->get('npub'); |
||||||
|
if (str_starts_with($npubParam, 'npub')) { |
||||||
|
try { |
||||||
|
$sitePk = $keys->convertToHex($npubParam); |
||||||
|
if ($sitePk !== '' && !\in_array($sitePk, $pubkeys, true)) { |
||||||
|
$pubkeys[] = $sitePk; |
||||||
|
} |
||||||
|
} catch (\Throwable) { |
||||||
|
// ignore bad npub |
||||||
|
} |
||||||
|
} |
||||||
|
$limit = (int) $input->getOption('metadata-limit'); |
||||||
|
if ($limit > 0) { |
||||||
|
$pubkeys = \array_slice($pubkeys, 0, $limit); |
||||||
|
} |
||||||
|
$toWarm = []; |
||||||
|
foreach ($pubkeys as $pubkey) { |
||||||
|
if (strlen($pubkey) === 64) { |
||||||
|
$toWarm[] = $pubkey; |
||||||
|
} |
||||||
|
} |
||||||
|
$total = \count($toWarm); |
||||||
|
$n = 0; |
||||||
|
if ($total === 0) { |
||||||
|
$io->note('No valid author pubkeys to warm.'); |
||||||
|
} else { |
||||||
|
$batchSize = max(1, min(200, (int) $input->getOption('metadata-batch'))); |
||||||
|
$io->writeln(sprintf( |
||||||
|
'Fetching kind-0 metadata: <info>%d</info> author(s) in Nostr requests of up to <info>%d</info> pubkeys each.', |
||||||
|
$total, |
||||||
|
$batchSize |
||||||
|
)); |
||||||
|
$bar = $io->createProgressBar($total); |
||||||
|
$bar->start(); |
||||||
|
try { |
||||||
|
foreach (array_chunk($toWarm, $batchSize) as $chunk) { |
||||||
|
$fetched = $this->nostrClient->fetchKind0MetadataForAuthors($chunk, $batchSize); |
||||||
|
$n += $this->cacheService->putPrewarmMetadataBatch($chunk, $fetched, $keys); |
||||||
|
$bar->advance(\count($chunk)); |
||||||
|
} |
||||||
|
} catch (\Throwable $e) { |
||||||
|
$this->logger->error('app:prewarm metadata batch failed', ['exception' => $e]); |
||||||
|
$io->error($e->getMessage()); |
||||||
|
$bar->finish(); |
||||||
|
$io->newLine(2); |
||||||
|
|
||||||
|
return Command::FAILURE; |
||||||
|
} |
||||||
|
$bar->finish(); |
||||||
|
$io->newLine(2); |
||||||
|
} |
||||||
|
$io->success(sprintf('Warmed metadata for %d of %d author(s).', $n, $total)); |
||||||
|
} else { |
||||||
|
$io->note('Skipping metadata (--no-metadata).'); |
||||||
|
} |
||||||
|
|
||||||
|
if ($input->getOption('no-comments')) { |
||||||
|
$io->note('Skipping comments (--no-comments).'); |
||||||
|
|
||||||
|
return Command::SUCCESS; |
||||||
|
} |
||||||
|
|
||||||
|
$maxArticles = (int) $input->getOption('comments-max'); |
||||||
|
|
||||||
|
$io->section('Comment / interaction cache'); |
||||||
|
$deadline = microtime(true) + max(1, (int) $input->getOption('comments-budget')); |
||||||
|
$qb = $this->articleRepository->createQueryBuilder('a') |
||||||
|
->where('a.slug IS NOT NULL') |
||||||
|
->andWhere("a.slug != ''") |
||||||
|
->andWhere('a.pubkey IS NOT NULL') |
||||||
|
->andWhere("a.pubkey != ''") |
||||||
|
->orderBy('a.createdAt', 'DESC'); |
||||||
|
if ($maxArticles > 0) { |
||||||
|
$qb->setMaxResults($maxArticles); |
||||||
|
} |
||||||
|
$articles = $qb->getQuery()->getResult(); |
||||||
|
$w = 0; |
||||||
|
/** @var Article $article */ |
||||||
|
foreach ($articles as $article) { |
||||||
|
if (microtime(true) >= $deadline) { |
||||||
|
$io->warning('Comment phase stopped: comments-budget reached.'); |
||||||
|
break; |
||||||
|
} |
||||||
|
$slug = trim((string) $article->getSlug()); |
||||||
|
$pubkey = (string) $article->getPubkey(); |
||||||
|
if ($slug === '' || strlen($pubkey) !== 64) { |
||||||
|
continue; |
||||||
|
} |
||||||
|
$kind = $article->getKind()?->value ?? 30023; |
||||||
|
$coordinate = $kind.':'.$pubkey.':'.$slug; |
||||||
|
$eventHex = (string) ($article->getEventId() ?? ''); |
||||||
|
try { |
||||||
|
$this->commentThreadLoader->load($coordinate, $eventHex !== '' ? $eventHex : null); |
||||||
|
++$w; |
||||||
|
} catch (\Throwable $e) { |
||||||
|
$this->logger->warning('app:prewarm comment load', ['coord' => $coordinate, 'error' => $e->getMessage()]); |
||||||
|
} |
||||||
|
} |
||||||
|
$io->success(sprintf('Warmed comment cache for %d of %d article(s).', $w, \count($articles))); |
||||||
|
|
||||||
|
return Command::SUCCESS; |
||||||
|
} |
||||||
|
|
||||||
|
private function disableCliExecutionTimeLimit(): void |
||||||
|
{ |
||||||
|
@set_time_limit(0); |
||||||
|
@ini_set('max_execution_time', '0'); |
||||||
|
} |
||||||
|
} |
||||||
@ -0,0 +1,103 @@ |
|||||||
|
<?php |
||||||
|
|
||||||
|
declare(strict_types=1); |
||||||
|
|
||||||
|
namespace App\Controller; |
||||||
|
|
||||||
|
use App\Service\MagazineContentService; |
||||||
|
use App\Service\MagazineRefresher; |
||||||
|
use Psr\Log\LoggerInterface; |
||||||
|
use Symfony\Component\DependencyInjection\ParameterBag\ParameterBagInterface; |
||||||
|
use Symfony\Component\HttpFoundation\JsonResponse; |
||||||
|
use Symfony\Component\HttpFoundation\Request; |
||||||
|
use Symfony\Component\HttpFoundation\Response; |
||||||
|
use Symfony\Component\HttpKernel\Attribute\AsController; |
||||||
|
use Symfony\Component\Routing\Attribute\Route; |
||||||
|
use Twig\Environment; |
||||||
|
|
||||||
|
/** Stale-first: the main request only reads {@see \App\Service\MagazineIndexStore}; this refetches Nostr, updates that store, and returns HTML fragments for Stimulus to patch the document. */ |
||||||
|
#[AsController] |
||||||
|
final class MagazineSyncController |
||||||
|
{ |
||||||
|
public function __construct( |
||||||
|
private readonly Environment $twig, |
||||||
|
private readonly MagazineRefresher $refresher, |
||||||
|
private readonly MagazineContentService $magazineContent, |
||||||
|
private readonly ParameterBagInterface $params, |
||||||
|
private readonly LoggerInterface $logger, |
||||||
|
) { |
||||||
|
} |
||||||
|
|
||||||
|
#[Route('/ux/magazine-sync', name: 'ux_magazine_sync', methods: ['GET'])] |
||||||
|
public function __invoke(Request $request): JsonResponse |
||||||
|
{ |
||||||
|
try { |
||||||
|
$page = (string) $request->query->get('page', 'article'); |
||||||
|
if (!\in_array($page, ['home', 'category', 'article', 'articles'], true)) { |
||||||
|
$page = 'article'; |
||||||
|
} |
||||||
|
$slug = (string) $request->query->get('slug', ''); |
||||||
|
|
||||||
|
$prefer = $slug !== '' ? [$slug] : []; |
||||||
|
|
||||||
|
try { |
||||||
|
$this->refresher->refreshFromRelays(20, $prefer); |
||||||
|
} catch (\Throwable $e) { |
||||||
|
$this->logger->warning('MagazineSyncController: refresh failed', [ |
||||||
|
'message' => $e->getMessage(), |
||||||
|
'exception' => $e, |
||||||
|
]); |
||||||
|
|
||||||
|
return new JsonResponse( |
||||||
|
['ok' => false, 'error' => 'refresh_failed', 'message' => $e->getMessage()], |
||||||
|
Response::HTTP_OK |
||||||
|
); |
||||||
|
} |
||||||
|
|
||||||
|
$community = (bool) $this->params->get('community_articles'); |
||||||
|
$tags = $this->magazineContent->getHomeCategoryAIndexTagsFromStoreOnly(); |
||||||
|
$globals = [ |
||||||
|
'magazine_community_articles' => $community, |
||||||
|
]; |
||||||
|
|
||||||
|
$header = $this->twig->render('ux/magazine/header_ul.html.twig', array_merge($globals, [ |
||||||
|
'cats' => $tags, |
||||||
|
])); |
||||||
|
|
||||||
|
$body = null; |
||||||
|
if ($page === 'home') { |
||||||
|
$body = $this->twig->render('ux/magazine/home_body.html.twig', array_merge($globals, [ |
||||||
|
'indices' => $tags, |
||||||
|
])); |
||||||
|
} elseif ($page === 'category' && $slug !== '') { |
||||||
|
$data = $this->magazineContent->getCategoryPageData($slug); |
||||||
|
$body = $this->twig->render('ux/magazine/category_body.html.twig', array_merge($globals, [ |
||||||
|
'list' => $data['list'], |
||||||
|
'category' => $data['category'], |
||||||
|
])); |
||||||
|
} elseif ($page === 'articles') { |
||||||
|
$body = null; |
||||||
|
} |
||||||
|
|
||||||
|
return new JsonResponse([ |
||||||
|
'ok' => true, |
||||||
|
'header' => $header, |
||||||
|
'body' => $body, |
||||||
|
]); |
||||||
|
} catch (\Throwable $e) { |
||||||
|
$this->logger->error('MagazineSyncController: unexpected failure', [ |
||||||
|
'message' => $e->getMessage(), |
||||||
|
'exception' => $e, |
||||||
|
]); |
||||||
|
|
||||||
|
return new JsonResponse( |
||||||
|
[ |
||||||
|
'ok' => false, |
||||||
|
'error' => 'server_error', |
||||||
|
'message' => 'Magazine UI sync could not be rendered.', |
||||||
|
], |
||||||
|
Response::HTTP_OK |
||||||
|
); |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
@ -0,0 +1,177 @@ |
|||||||
|
<?php |
||||||
|
|
||||||
|
declare(strict_types=1); |
||||||
|
|
||||||
|
namespace App\Service; |
||||||
|
|
||||||
|
use App\Entity\Article; |
||||||
|
use App\Entity\Event; |
||||||
|
use App\Repository\ArticleRepository; |
||||||
|
use Symfony\Component\DependencyInjection\ParameterBag\ParameterBagInterface; |
||||||
|
|
||||||
|
/** |
||||||
|
* Magazine index events for templates. Reads {@see MagazineIndexStore} first; on a cold cache or when |
||||||
|
* the last successful relay sync is older than {@see self::ROOT_REVALIDATE_SECONDS}, the service |
||||||
|
* calls {@see MagazineRefresher} so the root index (and nav) can pick up new categories. |
||||||
|
*/ |
||||||
|
final class MagazineContentService |
||||||
|
{ |
||||||
|
/** Re-fetch root from relays at most this often so new `a` tags appear in the header. */ |
||||||
|
private const ROOT_REVALIDATE_SECONDS = 300; |
||||||
|
|
||||||
|
public function __construct( |
||||||
|
private readonly MagazineIndexStore $store, |
||||||
|
private readonly MagazineRefresher $refresher, |
||||||
|
private readonly ParameterBagInterface $params, |
||||||
|
private readonly ArticleRepository $articleRepository, |
||||||
|
private readonly NostrClient $nostrClient, |
||||||
|
) { |
||||||
|
} |
||||||
|
|
||||||
|
/** |
||||||
|
* "indices" for the home template: Nostr `a` tag rows for each category. |
||||||
|
* |
||||||
|
* @return list<array<int, string>> |
||||||
|
*/ |
||||||
|
public function getHomeCategoryIndexTags(): array |
||||||
|
{ |
||||||
|
$npub = (string) $this->params->get('npub'); |
||||||
|
$dTag = (string) $this->params->get('d_tag'); |
||||||
|
if ($this->store->getRoot($npub, $dTag) === null) { |
||||||
|
$this->refresher->refreshFromRelays(20, []); |
||||||
|
} elseif ($this->shouldRevalidateRootFromRelay()) { |
||||||
|
$this->refresher->refreshFromRelays(20, []); |
||||||
|
} |
||||||
|
|
||||||
|
return $this->getHomeCategoryAIndexTagsFromStoreOnly(); |
||||||
|
} |
||||||
|
|
||||||
|
/** |
||||||
|
* Category `a` tags from the persisted root only (no relay). Used after /ux/magazine-sync |
||||||
|
* has already called {@see MagazineRefresher::refreshFromRelays}. |
||||||
|
* |
||||||
|
* @return list<array<int, string>> |
||||||
|
*/ |
||||||
|
public function getHomeCategoryAIndexTagsFromStoreOnly(): array |
||||||
|
{ |
||||||
|
return $this->categoryATagsFromStoredRoot(); |
||||||
|
} |
||||||
|
|
||||||
|
/** |
||||||
|
* @return list<array<int, string>> |
||||||
|
*/ |
||||||
|
private function categoryATagsFromStoredRoot(): array |
||||||
|
{ |
||||||
|
$npub = (string) $this->params->get('npub'); |
||||||
|
$dTag = (string) $this->params->get('d_tag'); |
||||||
|
$mag = $this->store->getRoot($npub, $dTag); |
||||||
|
|
||||||
|
return $this->categoryATagsFromMag($mag); |
||||||
|
} |
||||||
|
|
||||||
|
/** |
||||||
|
* @return list<array<int, string>> |
||||||
|
*/ |
||||||
|
private function categoryATagsFromMag(?Event $mag): array |
||||||
|
{ |
||||||
|
if ($mag === null) { |
||||||
|
return []; |
||||||
|
} |
||||||
|
$tags = $mag->getTags(); |
||||||
|
$cats = array_filter($tags, static function (mixed $tag): bool { |
||||||
|
return \is_array($tag) && ($tag[0] ?? null) === 'a'; |
||||||
|
}); |
||||||
|
|
||||||
|
return array_values($cats); |
||||||
|
} |
||||||
|
|
||||||
|
private function shouldRevalidateRootFromRelay(): bool |
||||||
|
{ |
||||||
|
$age = $this->refresher->getSecondsSinceLastRelayRun(); |
||||||
|
if ($age === null) { |
||||||
|
return true; |
||||||
|
} |
||||||
|
|
||||||
|
return $age > self::ROOT_REVALIDATE_SECONDS; |
||||||
|
} |
||||||
|
|
||||||
|
/** |
||||||
|
* @return array{list: list<Article>, category: array{title: string, summary: string}} |
||||||
|
*/ |
||||||
|
public function getCategoryPageData(string $slug): array |
||||||
|
{ |
||||||
|
$catIndex = $this->store->getCategory($slug); |
||||||
|
if ($catIndex === null) { |
||||||
|
$this->refresher->refreshFromRelays(20, [$slug]); |
||||||
|
$catIndex = $this->store->getCategory($slug); |
||||||
|
} |
||||||
|
$list = []; |
||||||
|
$coordinates = []; |
||||||
|
$category = []; |
||||||
|
if ($catIndex) { |
||||||
|
foreach ($catIndex->getTags() as $tag) { |
||||||
|
if ($tag[0] === 'title') { |
||||||
|
$category['title'] = (string) $tag[1]; |
||||||
|
} |
||||||
|
if ($tag[0] === 'summary') { |
||||||
|
$category['summary'] = (string) $tag[1]; |
||||||
|
} |
||||||
|
if ($tag[0] === 'a') { |
||||||
|
$coordinates[] = $tag[1]; |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
if (!empty($coordinates)) { |
||||||
|
$pairs = []; |
||||||
|
foreach ($coordinates as $coordinate) { |
||||||
|
$parts = explode(':', (string) $coordinate, 3); |
||||||
|
if (\count($parts) < 3) { |
||||||
|
continue; |
||||||
|
} |
||||||
|
$slugPart = trim((string) $parts[2]); |
||||||
|
if ($slugPart === '') { |
||||||
|
continue; |
||||||
|
} |
||||||
|
$pairs[] = [ |
||||||
|
'pubkey' => (string) $parts[1], |
||||||
|
'slug' => $slugPart, |
||||||
|
]; |
||||||
|
} |
||||||
|
$byAddress = $this->articleRepository->findByAuthorAndSlugIndexed($pairs); |
||||||
|
$missing = []; |
||||||
|
foreach ($coordinates as $coordinate) { |
||||||
|
$parts = explode(':', (string) $coordinate, 3); |
||||||
|
if (\count($parts) < 3) { |
||||||
|
continue; |
||||||
|
} |
||||||
|
$k = (string) $parts[1]."\0".trim((string) $parts[2]); |
||||||
|
if (!isset($byAddress[$k])) { |
||||||
|
$missing[] = (string) $coordinate; |
||||||
|
} |
||||||
|
} |
||||||
|
if ($missing !== []) { |
||||||
|
$this->nostrClient->ingestMissingLongformForCategoryCoordinates($missing); |
||||||
|
$byAddress = $this->articleRepository->findByAuthorAndSlugIndexed($pairs); |
||||||
|
} |
||||||
|
foreach ($coordinates as $coordinate) { |
||||||
|
$parts = explode(':', (string) $coordinate, 3); |
||||||
|
if (\count($parts) < 3) { |
||||||
|
continue; |
||||||
|
} |
||||||
|
$k = (string) $parts[1]."\0".trim((string) $parts[2]); |
||||||
|
if (isset($byAddress[$k])) { |
||||||
|
$list[] = $byAddress[$k]; |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
$category['title'] = $category['title'] ?? ''; |
||||||
|
$category['summary'] = $category['summary'] ?? ''; |
||||||
|
|
||||||
|
return [ |
||||||
|
'list' => $list, |
||||||
|
'category' => $category, |
||||||
|
]; |
||||||
|
} |
||||||
|
} |
||||||
@ -0,0 +1,116 @@ |
|||||||
|
<?php |
||||||
|
|
||||||
|
declare(strict_types=1); |
||||||
|
|
||||||
|
namespace App\Service; |
||||||
|
|
||||||
|
use App\Entity\Event; |
||||||
|
use Psr\Cache\CacheItemPoolInterface; |
||||||
|
use Psr\Cache\InvalidArgumentException; |
||||||
|
|
||||||
|
/** |
||||||
|
* Read/write persisted magazine Nostr index events (kinds 30040) without callback-based relay I/O |
||||||
|
* on the request path. Updated by {@see MagazineRefresher} or the /ux/magazine-sync action. |
||||||
|
*/ |
||||||
|
final class MagazineIndexStore |
||||||
|
{ |
||||||
|
private const ROOT_PREFIX = 'mroot_v1_'; |
||||||
|
private const CAT_PREFIX = 'mcat_v1_'; |
||||||
|
|
||||||
|
/** 30 days — we refresh on page load, TTL is a safety cap if sync stops working. */ |
||||||
|
private const PERSIST_TTL = 2_592_000; |
||||||
|
|
||||||
|
public function __construct( |
||||||
|
private readonly CacheItemPoolInterface $pool, |
||||||
|
) { |
||||||
|
} |
||||||
|
|
||||||
|
public function getRoot(string $npub, string $dTag): ?Event |
||||||
|
{ |
||||||
|
$item = $this->pool->getItem($this->rootKey($npub, $dTag)); |
||||||
|
if (!$item->isHit()) { |
||||||
|
return null; |
||||||
|
} |
||||||
|
|
||||||
|
return $this->unwrap($item->get()); |
||||||
|
} |
||||||
|
|
||||||
|
public function getCategory(string $slug): ?Event |
||||||
|
{ |
||||||
|
if ($slug === '') { |
||||||
|
return null; |
||||||
|
} |
||||||
|
$item = $this->pool->getItem(self::CAT_PREFIX.$slug); |
||||||
|
if (!$item->isHit()) { |
||||||
|
return null; |
||||||
|
} |
||||||
|
|
||||||
|
return $this->unwrap($item->get()); |
||||||
|
} |
||||||
|
|
||||||
|
/** |
||||||
|
* @throws InvalidArgumentException |
||||||
|
*/ |
||||||
|
public function putRoot(string $npub, string $dTag, Event $event): void |
||||||
|
{ |
||||||
|
$item = $this->pool->getItem($this->rootKey($npub, $dTag)); |
||||||
|
$item->set(serialize($event)); |
||||||
|
$item->expiresAfter(self::PERSIST_TTL); |
||||||
|
$this->pool->save($item); |
||||||
|
} |
||||||
|
|
||||||
|
/** |
||||||
|
* @throws InvalidArgumentException |
||||||
|
*/ |
||||||
|
public function putCategory(string $slug, Event $event): void |
||||||
|
{ |
||||||
|
if ($slug === '') { |
||||||
|
return; |
||||||
|
} |
||||||
|
$item = $this->pool->getItem(self::CAT_PREFIX.$slug); |
||||||
|
$item->set(serialize($event)); |
||||||
|
$item->expiresAfter(self::PERSIST_TTL); |
||||||
|
$this->pool->save($item); |
||||||
|
} |
||||||
|
|
||||||
|
/** |
||||||
|
* Remove a cached category index (NIP-09 / local invalidation). |
||||||
|
* |
||||||
|
* @throws InvalidArgumentException |
||||||
|
*/ |
||||||
|
public function deleteCategory(string $slug): void |
||||||
|
{ |
||||||
|
if ($slug === '') { |
||||||
|
return; |
||||||
|
} |
||||||
|
$this->pool->deleteItem(self::CAT_PREFIX.$slug); |
||||||
|
} |
||||||
|
|
||||||
|
/** |
||||||
|
* Remove the cached root magazine index for this npub + d_tag. |
||||||
|
* |
||||||
|
* @throws InvalidArgumentException |
||||||
|
*/ |
||||||
|
public function deleteRoot(string $npub, string $dTag): void |
||||||
|
{ |
||||||
|
$this->pool->deleteItem($this->rootKey($npub, $dTag)); |
||||||
|
} |
||||||
|
|
||||||
|
private function rootKey(string $npub, string $dTag): string |
||||||
|
{ |
||||||
|
return self::ROOT_PREFIX.hash('sha256', $npub."\0".$dTag); |
||||||
|
} |
||||||
|
|
||||||
|
private function unwrap(mixed $value): ?Event |
||||||
|
{ |
||||||
|
if (!\is_string($value) || $value === '') { |
||||||
|
return null; |
||||||
|
} |
||||||
|
$e = unserialize($value, ['allowed_classes' => [Event::class]]); |
||||||
|
if (!$e instanceof Event) { |
||||||
|
return null; |
||||||
|
} |
||||||
|
|
||||||
|
return $e; |
||||||
|
} |
||||||
|
} |
||||||
@ -0,0 +1,177 @@ |
|||||||
|
<?php |
||||||
|
|
||||||
|
declare(strict_types=1); |
||||||
|
|
||||||
|
namespace App\Service; |
||||||
|
|
||||||
|
use App\Entity\Event; |
||||||
|
use Psr\Cache\CacheItemPoolInterface; |
||||||
|
use Psr\Cache\InvalidArgumentException; |
||||||
|
use Psr\Log\LoggerInterface; |
||||||
|
use Symfony\Component\DependencyInjection\ParameterBag\ParameterBagInterface; |
||||||
|
|
||||||
|
/** |
||||||
|
* Pulls magazine indices from relays within a wall-clock budget and persists them to {@see MagazineIndexStore}. |
||||||
|
*/ |
||||||
|
final class MagazineRefresher |
||||||
|
{ |
||||||
|
private const RELAY_STAMP_KEY = 'mag_relay_v1'; |
||||||
|
|
||||||
|
public function __construct( |
||||||
|
private readonly NostrClient $nostrClient, |
||||||
|
private readonly MagazineIndexStore $store, |
||||||
|
private readonly ParameterBagInterface $params, |
||||||
|
private readonly LoggerInterface $logger, |
||||||
|
private readonly CacheItemPoolInterface $appCache, |
||||||
|
) { |
||||||
|
} |
||||||
|
|
||||||
|
/** |
||||||
|
* Fetches the root index then each category index until $budgetSeconds elapses. $preferSlugs |
||||||
|
* are requested first (e.g. current /cat route) so they are less likely to miss the budget. |
||||||
|
*/ |
||||||
|
public function refreshFromRelays(int $budgetSeconds = 8, array $preferSlugs = []): void |
||||||
|
{ |
||||||
|
$budgetSeconds = max(1, min(30, $budgetSeconds)); |
||||||
|
$deadline = microtime(true) + $budgetSeconds; |
||||||
|
$npub = (string) $this->params->get('npub'); |
||||||
|
$dTag = (string) $this->params->get('d_tag'); |
||||||
|
|
||||||
|
// Do not set max_execution_time to the *remaining* soft budget: PHP resets the timer, so |
||||||
|
// after a 6s root fetch, "2s left" would become a 2s hard cap for the *next* relay I/O |
||||||
|
// (e.g. slow TLS) and can fatal. Cap once with headroom; the $deadline loop limits work. |
||||||
|
$this->applyExecutionTimeCap($budgetSeconds); |
||||||
|
|
||||||
|
$defaultRelay = (string) $this->params->get('default_relay'); |
||||||
|
$relayLabel = (string) (parse_url($defaultRelay, \PHP_URL_HOST) ?: $defaultRelay); |
||||||
|
|
||||||
|
$root = $this->nostrClient->getMagazineIndex($npub, $dTag); |
||||||
|
if ($root === null) { |
||||||
|
$this->logger->warning(sprintf( |
||||||
|
'MagazineRefresher: root index not returned (tried from %s)', |
||||||
|
$relayLabel |
||||||
|
), [ |
||||||
|
'd_tag' => $dTag, |
||||||
|
'relay' => $defaultRelay, |
||||||
|
]); |
||||||
|
|
||||||
|
return; |
||||||
|
} |
||||||
|
|
||||||
|
$this->store->putRoot($npub, $dTag, $root); |
||||||
|
|
||||||
|
$slugs = $this->orderedCategorySlugs($this->categorySlugsFromRoot($root), $preferSlugs); |
||||||
|
foreach ($slugs as $slug) { |
||||||
|
if (microtime(true) >= $deadline) { |
||||||
|
$this->logger->notice('MagazineRefresher: stopped at time budget; some categories not fetched', [ |
||||||
|
'unprocessed_from' => $slug, |
||||||
|
]); |
||||||
|
break; |
||||||
|
} |
||||||
|
try { |
||||||
|
$cat = $this->nostrClient->getMagazineIndex($npub, $slug); |
||||||
|
if ($cat !== null) { |
||||||
|
$this->store->putCategory($slug, $cat); |
||||||
|
} |
||||||
|
} catch (\Throwable $e) { |
||||||
|
$this->logger->error(sprintf( |
||||||
|
'MagazineRefresher: category fetch failed (relays from %s): %s', |
||||||
|
$relayLabel, |
||||||
|
$e->getMessage() |
||||||
|
), [ |
||||||
|
'slug' => $slug, |
||||||
|
'message' => $e->getMessage(), |
||||||
|
'relay' => $defaultRelay, |
||||||
|
]); |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
$this->touchLastRelayTime(); |
||||||
|
} |
||||||
|
|
||||||
|
/** |
||||||
|
* @throws InvalidArgumentException |
||||||
|
*/ |
||||||
|
public function getSecondsSinceLastRelayRun(): ?int |
||||||
|
{ |
||||||
|
try { |
||||||
|
$item = $this->appCache->getItem(self::RELAY_STAMP_KEY); |
||||||
|
} catch (InvalidArgumentException) { |
||||||
|
return null; |
||||||
|
} |
||||||
|
if (!$item->isHit()) { |
||||||
|
return null; |
||||||
|
} |
||||||
|
|
||||||
|
return time() - (int) $item->get(); |
||||||
|
} |
||||||
|
|
||||||
|
/** |
||||||
|
* Child category indices are kind 30040; each root "a" tag is a NIP-33 address |
||||||
|
* kind:hexpubkey:d-identifier. The third segment is the child #d (e.g. the long |
||||||
|
* newsroom-…-category-… string), not a shortened title. |
||||||
|
* |
||||||
|
* @return list<string> |
||||||
|
*/ |
||||||
|
private function categorySlugsFromRoot(Event $root): array |
||||||
|
{ |
||||||
|
$slugs = []; |
||||||
|
foreach ($root->getTags() as $tag) { |
||||||
|
if (($tag[0] ?? null) !== 'a' || !isset($tag[1])) { |
||||||
|
continue; |
||||||
|
} |
||||||
|
$parts = explode(':', (string) $tag[1], 3); |
||||||
|
if (\count($parts) < 3) { |
||||||
|
continue; |
||||||
|
} |
||||||
|
$s = trim((string) end($parts)); |
||||||
|
if ($s !== '' && !\in_array($s, $slugs, true)) { |
||||||
|
$slugs[] = $s; |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
return $slugs; |
||||||
|
} |
||||||
|
|
||||||
|
/** |
||||||
|
* @param list<string> $allFromRoot |
||||||
|
* @param list<string> $prefer |
||||||
|
* @return list<string> |
||||||
|
*/ |
||||||
|
private function orderedCategorySlugs(array $allFromRoot, array $prefer): array |
||||||
|
{ |
||||||
|
$prefer = array_values(array_filter($prefer, static function (string $s): bool { |
||||||
|
return $s !== ''; |
||||||
|
})); |
||||||
|
$out = $prefer; |
||||||
|
foreach ($allFromRoot as $s) { |
||||||
|
if (!\in_array($s, $out, true)) { |
||||||
|
$out[] = $s; |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
return $out; |
||||||
|
} |
||||||
|
|
||||||
|
/** |
||||||
|
* @throws InvalidArgumentException |
||||||
|
*/ |
||||||
|
private function touchLastRelayTime(): void |
||||||
|
{ |
||||||
|
$item = $this->appCache->getItem(self::RELAY_STAMP_KEY); |
||||||
|
$item->set((string) time()); |
||||||
|
$item->expiresAfter(86_400); |
||||||
|
$this->appCache->save($item); |
||||||
|
} |
||||||
|
|
||||||
|
/** |
||||||
|
* One generous ceiling for PHP so relay/WebSocket I/O in one Nostr call can outlast the soft |
||||||
|
* $deadline by seconds without a fatal, while the loop still stops *starting* new fetches in time. |
||||||
|
*/ |
||||||
|
private function applyExecutionTimeCap(int $budgetSeconds): void |
||||||
|
{ |
||||||
|
$sec = max(30, min(120, $budgetSeconds + 30)); |
||||||
|
@set_time_limit($sec); |
||||||
|
@ini_set('max_execution_time', (string) $sec); |
||||||
|
} |
||||||
|
} |
||||||
@ -0,0 +1,359 @@ |
|||||||
|
<?php |
||||||
|
|
||||||
|
declare(strict_types=1); |
||||||
|
|
||||||
|
namespace App\Service; |
||||||
|
|
||||||
|
use App\Entity\Event as MagazineNostrEvent; |
||||||
|
use App\Enum\KindsEnum; |
||||||
|
use App\Repository\ArticleRepository; |
||||||
|
use Doctrine\ORM\EntityManagerInterface; |
||||||
|
use Psr\Log\LoggerInterface; |
||||||
|
use swentel\nostr\Key\Key; |
||||||
|
use Symfony\Component\DependencyInjection\ParameterBag\ParameterBagInterface; |
||||||
|
|
||||||
|
/** |
||||||
|
* Applies NIP-09 (kind 5) deletion requests to local MySQL articles and magazine 30040 cache. |
||||||
|
* |
||||||
|
* Relays are not authoritative; we only remove data we can validate (same pubkey as deletion request). |
||||||
|
* For cached 30040 category indices (keyed by `d` only), we require the stored event’s author |
||||||
|
* to match the deletion — not just an `a` tag whose own pubkey matches, so colliding `d` values |
||||||
|
* across authors cannot wipe another author’s cache entry. |
||||||
|
*/ |
||||||
|
final class Nip09DeletionApplier |
||||||
|
{ |
||||||
|
public function __construct( |
||||||
|
private readonly EntityManagerInterface $entityManager, |
||||||
|
private readonly ArticleRepository $articleRepository, |
||||||
|
private readonly MagazineIndexStore $magazineIndexStore, |
||||||
|
private readonly ParameterBagInterface $params, |
||||||
|
private readonly LoggerInterface $logger, |
||||||
|
) { |
||||||
|
} |
||||||
|
|
||||||
|
/** |
||||||
|
* @param list<object> $deletionEvents Kind-5 events from relays (e.g. {@see NostrClient::fetchKind5DeletionEventsForAuthors}) |
||||||
|
* |
||||||
|
* @return array{articles_removed: int, magazine_roots: int, magazine_categories: int} |
||||||
|
*/ |
||||||
|
public function apply(array $deletionEvents): array |
||||||
|
{ |
||||||
|
$articlesRemoved = 0; |
||||||
|
$articlesPendingFlush = 0; |
||||||
|
$roots = 0; |
||||||
|
$cats = 0; |
||||||
|
$seenArticleIds = []; |
||||||
|
|
||||||
|
foreach ($deletionEvents as $ev) { |
||||||
|
if (!\is_object($ev)) { |
||||||
|
continue; |
||||||
|
} |
||||||
|
if ((int) ($ev->kind ?? 0) !== KindsEnum::DELETION_REQUEST->value) { |
||||||
|
continue; |
||||||
|
} |
||||||
|
$deletionPubkey = (string) ($ev->pubkey ?? ''); |
||||||
|
if (64 !== \strlen($deletionPubkey)) { |
||||||
|
continue; |
||||||
|
} |
||||||
|
|
||||||
|
[$eIds, $eKinds] = $this->parseETags($ev); |
||||||
|
$aAddrs = $this->parseATags($ev); |
||||||
|
|
||||||
|
foreach ($eIds as $i => $eId) { |
||||||
|
if (64 !== \strlen($eId)) { |
||||||
|
continue; |
||||||
|
} |
||||||
|
$declared = $eKinds[$i] ?? null; |
||||||
|
if ($declared !== null |
||||||
|
&& !\in_array($declared, [30023, 30024, 30040, 1], true)) { |
||||||
|
// Other kinds: we do not mirror in this app; skip. |
||||||
|
continue; |
||||||
|
} |
||||||
|
if ($declared === 1) { |
||||||
|
continue; |
||||||
|
} |
||||||
|
if ($this->removeArticleByEventIdIfValid($eId, $deletionPubkey, $declared, $seenArticleIds)) { |
||||||
|
++$articlesRemoved; |
||||||
|
++$articlesPendingFlush; |
||||||
|
continue; |
||||||
|
} |
||||||
|
// No article row: 30040 index (or mis-tagged kind); only skip unrelated kinds. |
||||||
|
if ($declared === null || \in_array($declared, [30023, 30024, 30040], true)) { |
||||||
|
$mag = $this->tryRemoveMagazine30040ByEventId($eId, $deletionPubkey); |
||||||
|
if ($mag === 1) { |
||||||
|
++$roots; |
||||||
|
} elseif ($mag === 2) { |
||||||
|
++$cats; |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
foreach ($aAddrs as $addr) { |
||||||
|
$r = $this->removeByNip33Address($addr, $deletionPubkey, $seenArticleIds); |
||||||
|
$articlesRemoved += $r['articles']; |
||||||
|
$articlesPendingFlush += $r['articles']; |
||||||
|
$roots += $r['roots']; |
||||||
|
$cats += $r['cats']; |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
if ($articlesPendingFlush > 0) { |
||||||
|
try { |
||||||
|
$this->entityManager->flush(); |
||||||
|
} catch (\Throwable $e) { |
||||||
|
$this->logger->error('Nip09DeletionApplier: flush failed', ['exception' => $e]); |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
return [ |
||||||
|
'articles_removed' => $articlesRemoved, |
||||||
|
'magazine_roots' => $roots, |
||||||
|
'magazine_categories' => $cats, |
||||||
|
]; |
||||||
|
} |
||||||
|
|
||||||
|
/** 0 = none, 1 = root cache, 2 = category cache */ |
||||||
|
private function tryRemoveMagazine30040ByEventId(string $eventId, string $deletionPubkey): int |
||||||
|
{ |
||||||
|
$npub = (string) $this->params->get('npub'); |
||||||
|
$dTag = (string) $this->params->get('d_tag'); |
||||||
|
if ($npub === '' || $dTag === '') { |
||||||
|
return 0; |
||||||
|
} |
||||||
|
$root = $this->magazineIndexStore->getRoot($npub, $dTag); |
||||||
|
if ($root === null) { |
||||||
|
return 0; |
||||||
|
} |
||||||
|
if ($this->eventIdMatches($root, $eventId) && $this->pubkeyEquals($root->getPubkey(), $deletionPubkey)) { |
||||||
|
$this->magazineIndexStore->deleteRoot($npub, $dTag); |
||||||
|
$this->logger->notice('NIP-09: removed cached magazine root index', [ |
||||||
|
'event_id' => $eventId, |
||||||
|
]); |
||||||
|
|
||||||
|
return 1; |
||||||
|
} |
||||||
|
foreach ($this->categorySlugsFromRoot($root) as $slug) { |
||||||
|
$cat = $this->magazineIndexStore->getCategory($slug); |
||||||
|
if ($cat === null) { |
||||||
|
continue; |
||||||
|
} |
||||||
|
if ($this->eventIdMatches($cat, $eventId) && $this->pubkeyEquals($cat->getPubkey(), $deletionPubkey)) { |
||||||
|
$this->magazineIndexStore->deleteCategory($slug); |
||||||
|
$this->logger->notice('NIP-09: removed cached magazine category index', [ |
||||||
|
'event_id' => $eventId, |
||||||
|
'slug' => $slug, |
||||||
|
]); |
||||||
|
|
||||||
|
return 2; |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
return 0; |
||||||
|
} |
||||||
|
|
||||||
|
private function eventIdMatches(MagazineNostrEvent $e, string $eventId): bool |
||||||
|
{ |
||||||
|
$a = strtolower($e->getId()); |
||||||
|
$b = strtolower($eventId); |
||||||
|
|
||||||
|
return $a === $b; |
||||||
|
} |
||||||
|
|
||||||
|
private function pubkeyEquals(string $a, string $b): bool |
||||||
|
{ |
||||||
|
if (64 !== \strlen($a) || 64 !== \strlen($b)) { |
||||||
|
return $a === $b; |
||||||
|
} |
||||||
|
|
||||||
|
return strtolower($a) === strtolower($b); |
||||||
|
} |
||||||
|
|
||||||
|
/** |
||||||
|
* @return list<string> |
||||||
|
*/ |
||||||
|
private function categorySlugsFromRoot(MagazineNostrEvent $root): array |
||||||
|
{ |
||||||
|
$slugs = []; |
||||||
|
foreach ($root->getTags() as $tag) { |
||||||
|
if (($tag[0] ?? null) !== 'a' || !isset($tag[1])) { |
||||||
|
continue; |
||||||
|
} |
||||||
|
$parts = explode(':', (string) $tag[1], 3); |
||||||
|
if (\count($parts) < 3) { |
||||||
|
continue; |
||||||
|
} |
||||||
|
$s = trim((string) end($parts)); |
||||||
|
if ($s !== '' && !\in_array($s, $slugs, true)) { |
||||||
|
$slugs[] = $s; |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
return $slugs; |
||||||
|
} |
||||||
|
|
||||||
|
/** |
||||||
|
* @param array<string, true> $seenArticleIds |
||||||
|
*/ |
||||||
|
private function removeArticleByEventIdIfValid( |
||||||
|
string $eId, |
||||||
|
string $deletionPubkey, |
||||||
|
?int $declaredKind, |
||||||
|
array &$seenArticleIds, |
||||||
|
): bool { |
||||||
|
if (isset($seenArticleIds[$eId])) { |
||||||
|
return false; |
||||||
|
} |
||||||
|
$article = $this->articleRepository->findOneByEventId($eId); |
||||||
|
if ($article === null) { |
||||||
|
return false; |
||||||
|
} |
||||||
|
if (!$this->pubkeyEquals($article->getPubkey() ?? '', $deletionPubkey)) { |
||||||
|
$this->logger->debug('NIP-09: ignore e tag (pubkey mismatch)', [ |
||||||
|
'event_id' => $eId, |
||||||
|
]); |
||||||
|
|
||||||
|
return false; |
||||||
|
} |
||||||
|
$k = $article->getKind()?->value; |
||||||
|
if ($declaredKind !== null && $k !== null && $declaredKind !== $k) { |
||||||
|
return false; |
||||||
|
} |
||||||
|
if ($k !== null && !\in_array($k, [KindsEnum::LONGFORM->value, KindsEnum::LONGFORM_DRAFT->value], true)) { |
||||||
|
return false; |
||||||
|
} |
||||||
|
$this->entityManager->remove($article); |
||||||
|
$seenArticleIds[$eId] = true; |
||||||
|
$this->logger->notice('NIP-09: removed article from database', [ |
||||||
|
'event_id' => $eId, |
||||||
|
'kind' => $k, |
||||||
|
]); |
||||||
|
|
||||||
|
return true; |
||||||
|
} |
||||||
|
|
||||||
|
/** |
||||||
|
* NIP-33: `kind:pubkeyhex:d-identifier` |
||||||
|
* |
||||||
|
* @param array<string, true> $seenArticleIds |
||||||
|
* |
||||||
|
* @return array{articles: int, roots: int, cats: int} |
||||||
|
*/ |
||||||
|
private function removeByNip33Address(string $addr, string $deletionPubkey, array &$seenArticleIds): array |
||||||
|
{ |
||||||
|
$out = ['articles' => 0, 'roots' => 0, 'cats' => 0]; |
||||||
|
$parts = explode(':', $addr, 3); |
||||||
|
if (\count($parts) < 3) { |
||||||
|
return $out; |
||||||
|
} |
||||||
|
$kind = (int) $parts[0]; |
||||||
|
$pk = (string) $parts[1]; |
||||||
|
$d = trim((string) $parts[2]); |
||||||
|
if ($d === '' || !$this->pubkeyEquals($pk, $deletionPubkey)) { |
||||||
|
return $out; |
||||||
|
} |
||||||
|
|
||||||
|
if ($kind === KindsEnum::LONGFORM->value || $kind === KindsEnum::LONGFORM_DRAFT->value) { |
||||||
|
$article = $this->articleRepository->findOneBy(['pubkey' => $pk, 'slug' => $d]); |
||||||
|
if ($article !== null) { |
||||||
|
$eid = (string) ($article->getEventId() ?? ''); |
||||||
|
$dedupeKey = $eid !== '' ? $eid : 'ps:'.$pk."\0".$d; |
||||||
|
if (!isset($seenArticleIds[$dedupeKey])) { |
||||||
|
$this->entityManager->remove($article); |
||||||
|
$seenArticleIds[$dedupeKey] = true; |
||||||
|
++$out['articles']; |
||||||
|
$this->logger->notice('NIP-09: removed article (a tag)', [ |
||||||
|
'address' => $addr, |
||||||
|
]); |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
return $out; |
||||||
|
} |
||||||
|
|
||||||
|
if ($kind === KindsEnum::PUBLICATION_INDEX->value) { |
||||||
|
$npub = (string) $this->params->get('npub'); |
||||||
|
$siteD = (string) $this->params->get('d_tag'); |
||||||
|
$siteHex = ''; |
||||||
|
if (str_starts_with($npub, 'npub1')) { |
||||||
|
try { |
||||||
|
$h = (new Key())->convertToHex($npub); |
||||||
|
if (64 === \strlen($h)) { |
||||||
|
$siteHex = $h; |
||||||
|
} |
||||||
|
} catch (\Throwable) { |
||||||
|
} |
||||||
|
} |
||||||
|
if ($npub !== '' && $siteD !== '' && $d === $siteD && $siteHex !== '' && $this->pubkeyEquals($pk, $siteHex)) { |
||||||
|
$this->magazineIndexStore->deleteRoot($npub, $siteD); |
||||||
|
++$out['roots']; |
||||||
|
$this->logger->notice('NIP-09: removed magazine root (a tag)', ['address' => $addr]); |
||||||
|
} else { |
||||||
|
// Category cache is keyed by `d` only; the same d string can appear for different |
||||||
|
// authors' 30040 events. Only remove if the cached event was authored by this deletion. |
||||||
|
$cachedCat = $this->magazineIndexStore->getCategory($d); |
||||||
|
if ($cachedCat === null) { |
||||||
|
$this->logger->debug('NIP-09: skip category delete (nothing cached for d)', [ |
||||||
|
'address' => $addr, |
||||||
|
'd' => $d, |
||||||
|
]); |
||||||
|
} elseif (!$this->pubkeyEquals($cachedCat->getPubkey(), $deletionPubkey)) { |
||||||
|
$this->logger->debug('NIP-09: skip category delete (cached index author != deletion author)', [ |
||||||
|
'address' => $addr, |
||||||
|
'd' => $d, |
||||||
|
]); |
||||||
|
} else { |
||||||
|
$this->magazineIndexStore->deleteCategory($d); |
||||||
|
++$out['cats']; |
||||||
|
$this->logger->notice('NIP-09: removed magazine category (a tag)', [ |
||||||
|
'address' => $addr, |
||||||
|
'd' => $d, |
||||||
|
]); |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
return $out; |
||||||
|
} |
||||||
|
|
||||||
|
/** |
||||||
|
* @return array{0: list<string>, 1: list<?int>} e-ids and parallel k kinds (NIP-09 example order) |
||||||
|
*/ |
||||||
|
private function parseETags(object $ev): array |
||||||
|
{ |
||||||
|
$eIds = []; |
||||||
|
$kinds = []; |
||||||
|
foreach ($ev->tags ?? [] as $tag) { |
||||||
|
if (!\is_array($tag) || !isset($tag[0], $tag[1])) { |
||||||
|
continue; |
||||||
|
} |
||||||
|
if ($tag[0] === 'e') { |
||||||
|
$eIds[] = (string) $tag[1]; |
||||||
|
} |
||||||
|
if ($tag[0] === 'k') { |
||||||
|
$kinds[] = (int) $tag[1]; |
||||||
|
} |
||||||
|
} |
||||||
|
$pairs = []; |
||||||
|
for ($i = 0; $i < \count($eIds); ++$i) { |
||||||
|
$pairs[] = $kinds[$i] ?? null; |
||||||
|
} |
||||||
|
|
||||||
|
return [$eIds, $pairs]; |
||||||
|
} |
||||||
|
|
||||||
|
/** |
||||||
|
* @return list<string> NIP-33 addresses |
||||||
|
*/ |
||||||
|
private function parseATags(object $ev): array |
||||||
|
{ |
||||||
|
$a = []; |
||||||
|
foreach ($ev->tags ?? [] as $tag) { |
||||||
|
if (!\is_array($tag) || ($tag[0] ?? null) !== 'a' || !isset($tag[1])) { |
||||||
|
continue; |
||||||
|
} |
||||||
|
$a[] = (string) $tag[1]; |
||||||
|
} |
||||||
|
|
||||||
|
return $a; |
||||||
|
} |
||||||
|
} |
||||||
@ -0,0 +1,3 @@ |
|||||||
|
<div class="category-body" data-magazine-sync-target="pageBody"> |
||||||
|
<twig:Organisms:CardList :list="list" class="article-list" /> |
||||||
|
</div> |
||||||
@ -0,0 +1,10 @@ |
|||||||
|
<ul data-magazine-sync-target="headerNav"> |
||||||
|
{% for category in cats %} |
||||||
|
<li><twig:Molecules:CategoryLink :category="category" /></li> |
||||||
|
{% endfor %} |
||||||
|
{% if magazine_community_articles %} |
||||||
|
<li> |
||||||
|
<a href="{{ path('articles') }}">Latest Articles</a> |
||||||
|
</li> |
||||||
|
{% endif %} |
||||||
|
</ul> |
||||||
Loading…
Reference in new issue