Compare commits
No commits in common. '2e6b7e178a22c2a63c91cfa5302f42b1efd33c65' and 'd5313b2211aa6c0141724c003ed682a4f01788b4' have entirely different histories.
2e6b7e178a
...
d5313b2211
53 changed files with 626 additions and 2568 deletions
@ -1,7 +0,0 @@ |
|||||||
# Default compose file from repo root; override with `make prewarm COMPOSE=...` if needed.
|
|
||||||
COMPOSE ?= docker compose
|
|
||||||
|
|
||||||
.PHONY: prewarm |
|
||||||
prewarm: |
|
||||||
@chmod +x scripts/docker-prewarm.sh 2>/dev/null || true
|
|
||||||
@./scripts/docker-prewarm.sh
|
|
||||||
@ -1,130 +1,79 @@ |
|||||||
# Unfold: Imwald |
# Unfold |
||||||
|
|
||||||
<p align="center"> |
Unfold is a customizable framework for your Nostr-based magazine. |
||||||
<img src="assets/laeserin_logo.png" alt="Imwald" width="150"> |
|
||||||
</p> |
|
||||||
|
|
||||||
A Symfony + FrankenPHP site that **reads Nostr long-form articles (kind 30023)** and related data from relays, stores articles in **MySQL**, and serves pages with Twig. **Comments and profile metadata** are **cache-backed** (not the full source of truth in the DB). |
(This is the **Imwald** edition of Unfold.) |
||||||
|
|
||||||
--- |
## Setup |
||||||
|
|
||||||
## Requirements |
### Clone the repository |
||||||
|
|
||||||
| Requirement | Version / notes | |
|
||||||
|------------|-----------------| |
|
||||||
| PHP | **≥ 8.3.13** (see `composer.json`) | |
|
||||||
| Docker | Optional; recommended for local dev and production images | |
|
||||||
| Database | MySQL **8.0** (configurable) | |
|
||||||
|
|
||||||
--- |
|
||||||
|
|
||||||
## Local development (Docker) |
|
||||||
|
|
||||||
1. **Env:** copy `.env.dist` to `.env` and adjust if needed (especially `APP_SECRET` outside dev). |
|
||||||
2. **Start stack** |
|
||||||
|
|
||||||
```bash |
|
||||||
docker compose up -d |
|
||||||
``` |
|
||||||
|
|
||||||
3. **App URL (default):** [http://127.0.0.1:9080](http://127.0.0.1:9080) |
|
||||||
Port comes from `HTTP_PORT` in `.env` and `compose.override.yaml` (loopback only). |
|
||||||
|
|
||||||
4. **First-time DB:** migrations run on **php** container start when `migrations/` contains PHP files (see `frankenphp/docker-entrypoint.sh`). |
|
||||||
|
|
||||||
| Service | Role | |
|
||||||
|--------|------| |
|
||||||
| `php` | FrankenPHP + Caddy, Symfony app, console | |
|
||||||
| `database` | MySQL; dev exposes `127.0.0.1:3307 → 3306` for local clients | |
|
||||||
| `cron` | Runs full **`app:prewarm` every 10 minutes**; repo bind-mounted at `/var/www/html` (see `docker/cron/`) | |
|
||||||
|
|
||||||
--- |
|
||||||
|
|
||||||
## Backfill articles + warm caches (recommended) |
|
||||||
|
|
||||||
To **migrate**, **import articles from Nostr** for a time window, then **prewarm** magazine indices, author metadata, and comment caches: |
|
||||||
|
|
||||||
```bash |
```bash |
||||||
make prewarm |
git clone https://github.com/decent-newsroom/unfold.git |
||||||
|
cd unfold |
||||||
``` |
``` |
||||||
|
|
||||||
| Step (script order) | Command / effect | |
### Create the .env file |
||||||
|---------------------|------------------| |
|
||||||
| 1 | `docker compose up -d --wait` — starts **php**, **database**, and **cron** (the `cron` image runs a full `app:prewarm` on a 10 min schedule) | |
|
||||||
| 2 | `doctrine:migrations:migrate` | |
|
||||||
| 3 | `articles:get -- '-2 month' 'now'` — sync long-form into MySQL for that window | |
|
||||||
| 4 | `app:prewarm` — magazine **30040**, **kind-0** profiles, **comment** cache (default **`--comments-max=20`**, newest by `createdAt`) | |
|
||||||
|
|
||||||
`make prewarm` brings the stack (including `cron`) up so scheduled prewarm is active. **Optional** extra arguments for the **cron**-scheduled `app:prewarm` go in **`.env`** as **`PREWARM_FLAGS`** (same as you might pass to `php bin/console app:prewarm …`); Compose passes them into the `cron` container. Example: `PREWARM_FLAGS="--metadata-limit=50 --no-magazine"`. **Restart** the `cron` service after changing `PREWARM_FLAGS` so the container reloads the env. Hub / `compose.hub.yaml` has no `cron` service; use a host timer or `exec` if you need the same there. |
|
||||||
|
|
||||||
--- |
Copy the example file `.env.dist` and replace placeholders with your actual configuration. |
||||||
|
|
||||||
## Console commands (overview) |
If you have your own MySQL database, comment out the database service in `compose.yaml` and skip root password in `.env`. |
||||||
|
There are additional comments to that effect in the files. |
||||||
|
|
||||||
| Command | Purpose | |
### Configure `config/unfold.yaml` |
||||||
|---------|---------| |
|
||||||
| `articles:get <from> <to>` | Pull long-form articles from Nostr for the time range, persist to DB | |
|
||||||
| `app:prewarm` | Magazine relay refresh + metadata cache + comment cache warm | |
|
||||||
| `doctrine:migrations:migrate` | Apply SQL migrations | |
|
||||||
| `user:elevate` | (If used) user elevation helper | |
|
||||||
|
|
||||||
`php bin/console list` and `… -h` for full options. |
Before running the application, review and update `config/unfold.yaml` to match your desired magazine settings, theme, and external links. This file controls: |
||||||
|
- Magazine name, short name, and description |
||||||
|
- Theme and color settings |
||||||
|
- Community articles feature |
||||||
|
- External footer links |
||||||
|
- Other project-specific configuration |
||||||
|
|
||||||
### `app:prewarm` (notable options) |
Edit the values in `config/unfold.yaml` as needed for your deployment. |
||||||
|
|
||||||
| Option | Default | Meaning | |
### Customizing Theme and Icons |
||||||
|--------|---------|--------| |
|
||||||
| `--no-magazine` | off | Skip magazine 30040 index | |
|
||||||
| `--no-metadata` | off | Skip Nostr kind-0 / profile cache | |
|
||||||
| `--no-comments` | off | Skip comment thread cache | |
|
||||||
| `--metadata-limit` | `0` (all authors) | Cap distinct author pubkeys | |
|
||||||
| `--metadata-batch` | `50` | Pubkeys per batched Nostr `REQ` | |
|
||||||
| `--comments-max` | `20` | Newest **N** articles (by `createdAt` **DESC**); `0` = all (still bounded by budget) | |
|
||||||
| `--comments-budget` | `120` | Max wall seconds for the comments phase | |
|
||||||
| `--magazine-budget` | `30` | Max wall seconds for magazine refresh | |
|
||||||
|
|
||||||
Prewarm clears the PHP **CLI** execution time limit for that run; relay work can be slow. |
You can override the default theme and icons by adding your own files to `/assets/theme/local/`. To do this: |
||||||
|
- Copy the structure and file names from `/assets/theme/default/`. |
||||||
|
- Place your custom `theme.css` and icon files in your theme folder. |
||||||
|
- Update your configuration in `config/unfold.yaml` to reference your custom theme if needed. |
||||||
|
|
||||||
### `PREWARM_ON_START` (optional) |
This allows you to easily switch or update the look and feel of your magazine without modifying the default assets. |
||||||
|
|
||||||
| Variable | Set where | Effect | |
|
||||||
|----------|------------|--------| |
|
||||||
| `PREWARM_ON_START=1` | **Compose `environment` on the `php` service** (not only Symfony `.env` inside the container) | After DB is up and migrations run, executes **`app:prewarm` once** on start. **Does not** run `articles:get`. | |
|
||||||
|
|
||||||
For a full **Nostr backfill** + one-shot prewarm, use **`make prewarm`** (or a host **cron** / **systemd** timer) instead of relying on **`PREWARM_ON_START` alone**. |
### Build the Docker containers |
||||||
|
|
||||||
--- |
For development: |
||||||
|
```bash |
||||||
## Configuration |
docker compose build |
||||||
|
``` |
||||||
| What | File | |
|
||||||
|------|------| |
|
||||||
| Site title, `npub`, `d_tag`, **relays** (`default_relay`, `article_relays`, `profile_relays`), theme | `config/unfold.yaml` (imported as Symfony parameters) | |
|
||||||
| `DATABASE_URL`, `APP_SECRET`, `HTTP_PORT`, `MYSQL_*`, optional **`PREWARM_FLAGS`** (for the Docker `cron` service) | `.env` / `.env.local` (see `.env.dist`) | |
|
||||||
| Service wiring (e.g. cache, `NostrClient` args) | `config/services.yaml` | |
|
||||||
|
|
||||||
**Relays (short):** `default_relay` and `article_relays` drive article sync and many queries; `profile_relays` are used **first** for kind-0 / profile fetches, then the merged default + article set (see `NostrClient`). |
For production (using production overrides), set `APP_ENV=prod` in your `.env` file and run: |
||||||
|
```bash |
||||||
|
docker compose -f compose.yaml -f compose.prod.yaml build |
||||||
|
``` |
||||||
|
|
||||||
--- |
|
||||||
|
|
||||||
## Production / Hub image |
### Start the Docker containers |
||||||
|
```bash |
||||||
|
docker compose up -d |
||||||
|
``` |
||||||
|
|
||||||
| Topic | Notes | |
|
||||||
|-------|--------| |
|
||||||
| `compose.hub.yaml` | Runs a **pulled** image (default `silberengel/unfold:latest`), no local PHP app build. Override with `UNFOLD_DOCKER_IMAGE`. | |
|
||||||
| HTTP publish | `HTTP_PUBLISH` in `.env` (default **9080** → container **80**). Set `TRUSTED_PROXIES` behind a reverse proxy. | |
|
||||||
| Secrets | Set `APP_SECRET` and DB credentials in **real** env; do not commit production secrets. | |
|
||||||
|
|
||||||
File header in `compose.hub.yaml` lists pull, migrate, and optional build/push one-liners. |
### Run Database Migrations |
||||||
|
|
||||||
--- |
Before fetching or displaying articles, make sure your database schema is up to date. Run: |
||||||
|
|
||||||
## License |
```bash |
||||||
|
docker compose exec php php bin/console doctrine:migrations:migrate --no-interaction |
||||||
|
``` |
||||||
|
|
||||||
**MIT** — see [`LICENSE`](LICENSE). |
### Fetching Articles |
||||||
|
|
||||||
--- |
To fetch articles from the default relay for the last two months, run: |
||||||
|
|
||||||
## Project links (example) |
```bash |
||||||
|
docker compose exec php php bin/console articles:get -- '-2 month' 'now' |
||||||
|
``` |
||||||
|
|
||||||
Configurable under `parameters.external_links` in `config/unfold.yaml` (e.g. Unfold on GitHub, Decent Newsroom). Adjust for your deployment. |
You can adjust the date range as needed. This command will import articles into the local database. |
||||||
|
|||||||
@ -1,49 +0,0 @@ |
|||||||
import { Controller } from "@hotwired/stimulus"; |
|
||||||
|
|
||||||
/** |
|
||||||
* After first paint, refreshes Nostr magazine indices (server-side, ≤5s) and swaps header/body HTML. |
|
||||||
*/ |
|
||||||
export default class extends Controller { |
|
||||||
static targets = ["headerNav", "pageBody"]; |
|
||||||
static values = { |
|
||||||
page: String, |
|
||||||
slug: String, |
|
||||||
url: String, |
|
||||||
}; |
|
||||||
|
|
||||||
connect() { |
|
||||||
this.sync(); |
|
||||||
} |
|
||||||
|
|
||||||
async sync() { |
|
||||||
const base = this.urlValue || "/ux/magazine-sync"; |
|
||||||
const params = new URLSearchParams(); |
|
||||||
params.set("page", this.pageValue || "article"); |
|
||||||
const slug = this.slugValue || ""; |
|
||||||
if (slug !== "") { |
|
||||||
params.set("slug", slug); |
|
||||||
} |
|
||||||
const url = `${base}?${params.toString()}`; |
|
||||||
try { |
|
||||||
const res = await fetch(url, { |
|
||||||
headers: { Accept: "application/json" }, |
|
||||||
credentials: "same-origin", |
|
||||||
}); |
|
||||||
if (!res.ok) { |
|
||||||
return; |
|
||||||
} |
|
||||||
const data = await res.json(); |
|
||||||
if (!data.ok) { |
|
||||||
return; |
|
||||||
} |
|
||||||
if (this.hasHeaderNavTarget && data.header) { |
|
||||||
this.headerNavTarget.outerHTML = data.header; |
|
||||||
} |
|
||||||
if (this.hasPageBodyTarget && data.body) { |
|
||||||
this.pageBodyTarget.outerHTML = data.body; |
|
||||||
} |
|
||||||
} catch { |
|
||||||
/* ignore network errors */ |
|
||||||
} |
|
||||||
} |
|
||||||
} |
|
||||||
@ -1,61 +0,0 @@ |
|||||||
# Run a pre-built production image from Docker Hub (no local PHP image build). |
|
||||||
# |
|
||||||
# Usage on the server (copy this file + your .env, no app source required): |
|
||||||
# docker compose -f compose.hub.yaml pull |
|
||||||
# docker compose -f compose.hub.yaml up -d |
|
||||||
# docker compose -f compose.hub.yaml exec php php bin/console doctrine:migrations:migrate --no-interaction |
|
||||||
# |
|
||||||
# Required in .env: APP_SECRET. Set MYSQL_* (or replace DATABASE_URL after editing this file) if you |
|
||||||
# use the bundled database. For TLS in front, set TRUSTED_PROXIES to include your reverse proxy CIDR. |
|
||||||
# |
|
||||||
# Host HTTP port defaults to 9080 (same idea as local dev) so Apache/nginx can keep :80. Override with |
|
||||||
# HTTP_PUBLISH=80 or HTTP_PUBLISH=127.0.0.1:9080 in .env if needed. |
|
||||||
# |
|
||||||
# Build & push (on your machine or CI), e.g.: |
|
||||||
# docker build --platform linux/amd64 --target frankenphp_prod -t silberengel/unfold:latest . |
|
||||||
# docker push silberengel/unfold:latest |
|
||||||
# |
|
||||||
# Override image: UNFOLD_DOCKER_IMAGE=myregistry/unfold:1.0.0 docker compose -f compose.hub.yaml up -d |
|
||||||
|
|
||||||
name: unfold |
|
||||||
|
|
||||||
services: |
|
||||||
php: |
|
||||||
image: ${UNFOLD_DOCKER_IMAGE:-silberengel/unfold:latest} |
|
||||||
pull_policy: always |
|
||||||
restart: unless-stopped |
|
||||||
environment: |
|
||||||
APP_ENV: ${APP_ENV:-prod} |
|
||||||
APP_SECRET: ${APP_SECRET} |
|
||||||
TRUSTED_PROXIES: ${TRUSTED_PROXIES:-127.0.0.0/8,10.0.0.0/8} |
|
||||||
SERVER_NAME: ${SERVER_NAME:-:80} |
|
||||||
DATABASE_URL: mysql://${MYSQL_USER:-unfold_user}:${MYSQL_PASSWORD:-password}@database:3306/${MYSQL_DATABASE:-unfold_db}?serverVersion=${MYSQL_VERSION:-8.0}&charset=${MYSQL_CHARSET:-utf8mb4} |
|
||||||
volumes: |
|
||||||
- caddy_data:/data |
|
||||||
- caddy_config:/config |
|
||||||
ports: |
|
||||||
- "${HTTP_PUBLISH:-9080}:80/tcp" |
|
||||||
depends_on: |
|
||||||
database: |
|
||||||
condition: service_healthy |
|
||||||
|
|
||||||
database: |
|
||||||
image: mysql:${MYSQL_VERSION:-8.0} |
|
||||||
restart: unless-stopped |
|
||||||
environment: |
|
||||||
MYSQL_DATABASE: ${MYSQL_DATABASE:-unfold_db} |
|
||||||
MYSQL_USER: ${MYSQL_USER:-unfold_user} |
|
||||||
MYSQL_PASSWORD: ${MYSQL_PASSWORD:-password} |
|
||||||
MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD:-root_password} |
|
||||||
healthcheck: |
|
||||||
test: ["CMD", "mysqladmin", "ping", "-h", "localhost"] |
|
||||||
timeout: 5s |
|
||||||
retries: 5 |
|
||||||
start_period: 60s |
|
||||||
volumes: |
|
||||||
- database_data:/var/lib/mysql:rw |
|
||||||
|
|
||||||
volumes: |
|
||||||
caddy_data: |
|
||||||
caddy_config: |
|
||||||
database_data: |
|
||||||
@ -1,21 +1,30 @@ |
|||||||
FROM php:8.3-cli |
FROM php:8.2-cli |
||||||
|
|
||||||
RUN apt-get update && apt-get install -y --no-install-recommends \ |
# Install cron and Redis PHP extension dependencies |
||||||
bash \ |
RUN apt-get update && apt-get install -y \ |
||||||
cron \ |
cron \ |
||||||
&& rm -rf /var/lib/apt/lists/* |
libzip-dev \ |
||||||
|
libicu-dev \ |
||||||
|
libpq-dev \ |
||||||
|
libonig-dev |
||||||
|
|
||||||
ADD https://github.com/mlocati/docker-php-extension-installer/releases/latest/download/install-php-extensions /usr/local/bin/ |
|
||||||
RUN chmod +x /usr/local/bin/install-php-extensions \ |
|
||||||
&& install-php-extensions pdo_mysql intl opcache zip gmp |
|
||||||
|
|
||||||
|
# Set working directory |
||||||
WORKDIR /var/www/html |
WORKDIR /var/www/html |
||||||
|
|
||||||
COPY crontab /etc/cron.d/unfold-prewarm |
# Install Symfony CLI tools (optional) |
||||||
COPY prewarm_cron.sh /prewarm_cron.sh |
# RUN curl -sS https://get.symfony.com/cli/installer | bash |
||||||
COPY entry-cron.sh /entry-cron.sh |
|
||||||
|
|
||||||
RUN chmod 0644 /etc/cron.d/unfold-prewarm \ |
# Copy cron and script |
||||||
&& chmod +x /prewarm_cron.sh /entry-cron.sh |
COPY crontab /etc/cron.d/app-cron |
||||||
|
COPY index_articles.sh /index_articles.sh |
||||||
|
|
||||||
CMD ["/entry-cron.sh"] |
# Set permissions |
||||||
|
RUN chmod 0644 /etc/cron.d/app-cron && \ |
||||||
|
chmod +x /index_articles.sh |
||||||
|
|
||||||
|
# Apply cron job |
||||||
|
RUN crontab /etc/cron.d/app-cron |
||||||
|
|
||||||
|
# Run cron in the foreground |
||||||
|
CMD ["cron", "-f"] |
||||||
|
|||||||
@ -1,15 +1,94 @@ |
|||||||
# `cron` service (Docker) |
|
||||||
|
|
||||||
The `cron` image runs a single job: **`php bin/console app:prewarm` every 10 minutes**, against the app tree bind-mounted at `/var/www/html`. |
# 🕒 Cron Job Container |
||||||
|
|
||||||
- **Flags:** set **`PREWARM_FLAGS`** in the project `.env` (Compose injects it). Example: `PREWARM_FLAGS="--metadata-limit=30 --no-magazine"`. After editing, run `docker compose up -d --force-recreate cron` (or `docker compose up -d cron`) so the container gets the new value. If unset, `app:prewarm` uses its **built-in defaults** (same idea as running the console with no args). |
This folder contains the Docker configuration to run scheduled Symfony commands via cron inside a separate container. |
||||||
|
|
||||||
- **How env reaches cron:** the entrypoint writes `PREWARM_FLAGS` to `/run/cron-prewarm.env` at boot, because the system `crond` does not pass the container environment into crontab jobs. |
- Run Symfony console commands periodically using a cron schedule (e.g. every 6 hours) |
||||||
|
- Decouple scheduled jobs from the main PHP/FPM container |
||||||
|
- Easily manage and test cron execution in a Dockerized Symfony project |
||||||
|
|
||||||
- **Logs inside the container:** `tail -f /var/log/cron.log` (e.g. `docker compose exec cron tail -f /var/log/cron.log`). |
--- |
||||||
|
|
||||||
- **PHP 8.3** extensions in the image are limited to what `app:prewarm` needs; the host **vendor** tree is what you mount from the repo. |
## Build & Run |
||||||
|
|
||||||
- **Not included in** `compose.hub.yaml` (no app source mount). For production images, use host **cron** / **systemd** to `exec` the same command. |
1. **Build the cron image** |
||||||
|
From the project root: |
||||||
|
```bash |
||||||
|
docker-compose build cron |
||||||
|
``` |
||||||
|
|
||||||
Change the schedule: edit `docker/cron/crontab`, then `docker compose build cron && docker compose up -d cron`. |
2. **Start the cron container** |
||||||
|
```bash |
||||||
|
docker-compose up -d cron |
||||||
|
``` |
||||||
|
|
||||||
|
--- |
||||||
|
|
||||||
|
## Cron Schedule |
||||||
|
|
||||||
|
The default cron schedule is set to run **every 6 hours**: |
||||||
|
|
||||||
|
```cron |
||||||
|
0 */6 * * * root /run_commands.sh >> /var/log/cron.log 2>&1 |
||||||
|
``` |
||||||
|
|
||||||
|
To customize the schedule, edit the `crontab` file and rebuild the container. |
||||||
|
|
||||||
|
--- |
||||||
|
|
||||||
|
## Testing & Debugging |
||||||
|
|
||||||
|
### Manually test the command runner |
||||||
|
|
||||||
|
You can run the script manually to check behavior without waiting for the cron trigger: |
||||||
|
|
||||||
|
```bash |
||||||
|
docker-compose exec cron /run_commands.sh |
||||||
|
``` |
||||||
|
|
||||||
|
### Check the cron output log |
||||||
|
|
||||||
|
```bash |
||||||
|
docker-compose exec cron tail -f /var/log/cron.log |
||||||
|
``` |
||||||
|
|
||||||
|
### Shell into the cron container |
||||||
|
|
||||||
|
```bash |
||||||
|
docker-compose exec cron bash |
||||||
|
``` |
||||||
|
|
||||||
|
Once inside, you can: |
||||||
|
- Check crontab entries: `crontab -l` |
||||||
|
- Manually trigger cron: `cron` or `cron -f` (in another session) |
||||||
|
|
||||||
|
--- |
||||||
|
|
||||||
|
## Customization |
||||||
|
|
||||||
|
- **Add/Remove Symfony Commands:** |
||||||
|
Edit `run_commands.sh` to include the commands you want to run. |
||||||
|
|
||||||
|
- **Change Schedule:** |
||||||
|
Edit `crontab` using standard cron syntax. |
||||||
|
|
||||||
|
- **Logging:** |
||||||
|
Logs are sent to `/var/log/cron.log` inside the container. |
||||||
|
|
||||||
|
--- |
||||||
|
|
||||||
|
## Rebuilding After Changes |
||||||
|
|
||||||
|
If you modify the `crontab` or `run_commands.sh`, make sure to rebuild: |
||||||
|
|
||||||
|
```bash |
||||||
|
docker-compose build cron |
||||||
|
docker-compose up -d cron |
||||||
|
``` |
||||||
|
|
||||||
|
--- |
||||||
|
|
||||||
|
## Notes |
||||||
|
|
||||||
|
- Symfony project source is mounted at `/var/www/html` via volume. |
||||||
|
- Make sure your commands do **not rely on services** (like `php-fpm`) that are not running in this container. |
||||||
|
|||||||
@ -1 +1 @@ |
|||||||
*/10 * * * * root /prewarm_cron.sh >>/var/log/cron.log 2>&1 |
*/5 * * * * /index_articles.sh >> /var/log/cron.log 2>&1 |
||||||
|
|||||||
@ -1,6 +0,0 @@ |
|||||||
#!/bin/bash |
|
||||||
set -euo pipefail |
|
||||||
# crond does not pass Compose env to job children; write once at boot for prewarm_cron.sh. |
|
||||||
export PREWARM_FLAGS="${PREWARM_FLAGS:-}" |
|
||||||
declare -p PREWARM_FLAGS >/run/cron-prewarm.env |
|
||||||
exec cron -f |
|
||||||
@ -0,0 +1,5 @@ |
|||||||
|
#!/bin/bash |
||||||
|
set -e |
||||||
|
export PATH="/usr/local/bin:/usr/bin:/bin" |
||||||
|
|
||||||
|
php /var/www/html/bin/console articles:get -- '-6 min' 'now' |
||||||
@ -1,9 +0,0 @@ |
|||||||
#!/bin/bash |
|
||||||
set -euo pipefail |
|
||||||
# shellcheck source=/dev/null |
|
||||||
if [[ -f /run/cron-prewarm.env ]]; then |
|
||||||
source /run/cron-prewarm.env |
|
||||||
fi |
|
||||||
cd /var/www/html |
|
||||||
# shellcheck disable=SC2086 |
|
||||||
exec php bin/console app:prewarm ${PREWARM_FLAGS:-} |
|
||||||
@ -1,22 +0,0 @@ |
|||||||
#!/usr/bin/env sh |
|
||||||
# After `docker compose up`, run migrations, backfill long-form articles, and app:prewarm. |
|
||||||
# Usage: from repository root: ./scripts/docker-prewarm.sh |
|
||||||
# Or: make prewarm |
|
||||||
set -eu |
|
||||||
|
|
||||||
ROOT_DIR=$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd) |
|
||||||
cd "$ROOT_DIR" |
|
||||||
|
|
||||||
echo "==> docker compose up -d --wait (php, database, cron: full app:prewarm every 10 min; set PREWARM_FLAGS in .env to match your CLI)" |
|
||||||
docker compose up -d --wait |
|
||||||
|
|
||||||
echo "==> doctrine:migrations:migrate" |
|
||||||
docker compose exec -T php php bin/console doctrine:migrations:migrate --no-interaction |
|
||||||
|
|
||||||
echo "==> articles:get (last 2 months → now)" |
|
||||||
docker compose exec -T php php bin/console articles:get -- '-2 month' 'now' |
|
||||||
|
|
||||||
echo "==> app:prewarm" |
|
||||||
docker compose exec -T php php bin/console app:prewarm |
|
||||||
|
|
||||||
echo "Done." |
|
||||||
@ -1,250 +0,0 @@ |
|||||||
<?php |
|
||||||
|
|
||||||
declare(strict_types=1); |
|
||||||
|
|
||||||
namespace App\Command; |
|
||||||
|
|
||||||
use App\Entity\Article; |
|
||||||
use App\Repository\ArticleRepository; |
|
||||||
use App\Service\ArticleCommentThreadLoader; |
|
||||||
use App\Service\CacheService; |
|
||||||
use App\Service\MagazineRefresher; |
|
||||||
use App\Service\Nip09DeletionApplier; |
|
||||||
use App\Service\NostrClient; |
|
||||||
use Psr\Log\LoggerInterface; |
|
||||||
use swentel\nostr\Key\Key; |
|
||||||
use Symfony\Component\Console\Attribute\AsCommand; |
|
||||||
use Symfony\Component\Console\Command\Command; |
|
||||||
use Symfony\Component\Console\Input\InputInterface; |
|
||||||
use Symfony\Component\Console\Input\InputOption; |
|
||||||
use Symfony\Component\Console\Output\OutputInterface; |
|
||||||
use Symfony\Component\Console\Style\SymfonyStyle; |
|
||||||
use Symfony\Component\DependencyInjection\ParameterBag\ParameterBagInterface; |
|
||||||
|
|
||||||
/** |
|
||||||
* Prewarms magazine index cache, author metadata cache, and optional comment thread cache. |
|
||||||
* Does not persist comments to MySQL; comments are cache-only in this app. |
|
||||||
*/ |
|
||||||
#[AsCommand( |
|
||||||
name: 'app:prewarm', |
|
||||||
description: 'Refresh magazine indices, NIP-09 deletions, profile metadata, and comment caches', |
|
||||||
)] |
|
||||||
final class PrewarmCommand extends Command |
|
||||||
{ |
|
||||||
public function __construct( |
|
||||||
private readonly MagazineRefresher $magazineRefresher, |
|
||||||
private readonly Nip09DeletionApplier $nip09DeletionApplier, |
|
||||||
private readonly CacheService $cacheService, |
|
||||||
private readonly NostrClient $nostrClient, |
|
||||||
private readonly ArticleRepository $articleRepository, |
|
||||||
private readonly ArticleCommentThreadLoader $commentThreadLoader, |
|
||||||
private readonly ParameterBagInterface $params, |
|
||||||
private readonly LoggerInterface $logger, |
|
||||||
) { |
|
||||||
parent::__construct(); |
|
||||||
} |
|
||||||
|
|
||||||
protected function configure(): void |
|
||||||
{ |
|
||||||
$this |
|
||||||
->addOption('no-magazine', null, InputOption::VALUE_NONE, 'Skip magazine 30040 index fetch') |
|
||||||
->addOption('no-deletions', null, InputOption::VALUE_NONE, 'Skip NIP-09 kind 5 deletion sync (30023/30024 DB + 30040 magazine cache)') |
|
||||||
->addOption('deletion-since', null, InputOption::VALUE_REQUIRED, 'strtotime() window start for kind 5 fetch', '-2 month') |
|
||||||
->addOption('no-metadata', null, InputOption::VALUE_NONE, 'Skip Nostr profile metadata cache') |
|
||||||
->addOption('no-comments', null, InputOption::VALUE_NONE, 'Skip comment thread cache') |
|
||||||
->addOption('magazine-budget', null, InputOption::VALUE_REQUIRED, 'Seconds wall time for magazine relay refresh', '30') |
|
||||||
->addOption('metadata-limit', null, InputOption::VALUE_REQUIRED, 'Max distinct author pubkeys to warm (0 = all)', '0') |
|
||||||
->addOption('metadata-batch', null, InputOption::VALUE_REQUIRED, 'Kind-0 metadata: pubkeys per Nostr REQ (batched)', '50') |
|
||||||
->addOption('comments-max', null, InputOption::VALUE_REQUIRED, 'Newest N articles to warm comment cache for (0 = all, order: createdAt DESC)', '20') |
|
||||||
->addOption('comments-budget', null, InputOption::VALUE_REQUIRED, 'Max seconds for the whole comments phase', '120'); |
|
||||||
} |
|
||||||
|
|
||||||
protected function execute(InputInterface $input, OutputInterface $output): int |
|
||||||
{ |
|
||||||
$this->disableCliExecutionTimeLimit(); |
|
||||||
|
|
||||||
$io = new SymfonyStyle($input, $output); |
|
||||||
$keys = new Key(); |
|
||||||
|
|
||||||
if (!$input->getOption('no-magazine')) { |
|
||||||
$budget = max(1, (int) $input->getOption('magazine-budget')); |
|
||||||
$io->section('Magazine index (kinds 30040)'); |
|
||||||
try { |
|
||||||
$this->magazineRefresher->refreshFromRelays($budget, []); |
|
||||||
$io->success('Magazine indices refreshed (within budget).'); |
|
||||||
} catch (\Throwable $e) { |
|
||||||
$this->logger->error('app:prewarm magazine failed', ['e' => $e]); |
|
||||||
$io->warning('Magazine refresh failed: '.$e->getMessage()); |
|
||||||
} |
|
||||||
} else { |
|
||||||
$io->note('Skipping magazine (--no-magazine).'); |
|
||||||
} |
|
||||||
|
|
||||||
// MagazineRefresher sets max_execution_time (e.g. 60 for budget 30); restore before metadata. |
|
||||||
$this->disableCliExecutionTimeLimit(); |
|
||||||
|
|
||||||
if (!$input->getOption('no-deletions')) { |
|
||||||
$io->section('NIP-09 deletions (kind 5 → 30023/30024 / 30040)'); |
|
||||||
$sinceStr = (string) $input->getOption('deletion-since'); |
|
||||||
$since = strtotime($sinceStr); |
|
||||||
if ($since === false) { |
|
||||||
$since = strtotime('-2 month'); |
|
||||||
} |
|
||||||
$until = time(); |
|
||||||
$deletionPubkeys = []; |
|
||||||
foreach ($this->articleRepository->findDistinctAuthorPubkeys() as $pk) { |
|
||||||
if (\is_string($pk) && 64 === \strlen($pk)) { |
|
||||||
$deletionPubkeys[] = $pk; |
|
||||||
} |
|
||||||
} |
|
||||||
$npubParam = (string) $this->params->get('npub'); |
|
||||||
if (str_starts_with($npubParam, 'npub')) { |
|
||||||
try { |
|
||||||
$sitePk = $keys->convertToHex($npubParam); |
|
||||||
if ($sitePk !== '' && 64 === \strlen($sitePk) && !\in_array($sitePk, $deletionPubkeys, true)) { |
|
||||||
$deletionPubkeys[] = $sitePk; |
|
||||||
} |
|
||||||
} catch (\Throwable) { |
|
||||||
} |
|
||||||
} |
|
||||||
if ($deletionPubkeys === []) { |
|
||||||
$io->note('No author pubkeys; skipping kind 5 deletion fetch.'); |
|
||||||
} else { |
|
||||||
try { |
|
||||||
$kind5 = $this->nostrClient->fetchKind5DeletionEventsForAuthors( |
|
||||||
$deletionPubkeys, |
|
||||||
$since, |
|
||||||
$until, |
|
||||||
40 |
|
||||||
); |
|
||||||
$st = $this->nip09DeletionApplier->apply($kind5); |
|
||||||
$io->writeln(sprintf( |
|
||||||
'Kind 5 events: <info>%d</info> (deduped). Articles removed: <info>%d</info>; magazine root/category cache entries removed: <info>%d</info> / <info>%d</info>.', |
|
||||||
\count($kind5), |
|
||||||
$st['articles_removed'], |
|
||||||
$st['magazine_roots'], |
|
||||||
$st['magazine_categories'] |
|
||||||
)); |
|
||||||
} catch (\Throwable $e) { |
|
||||||
$this->logger->error('app:prewarm NIP-09 failed', ['exception' => $e]); |
|
||||||
$io->warning('NIP-09 step failed: '.$e->getMessage()); |
|
||||||
} |
|
||||||
} |
|
||||||
} else { |
|
||||||
$io->note('Skipping NIP-09 deletions (--no-deletions).'); |
|
||||||
} |
|
||||||
|
|
||||||
$this->disableCliExecutionTimeLimit(); |
|
||||||
|
|
||||||
if (!$input->getOption('no-metadata')) { |
|
||||||
$io->section('Author metadata (cache)'); |
|
||||||
$pubkeys = $this->articleRepository->findDistinctAuthorPubkeys(); |
|
||||||
$npubParam = (string) $this->params->get('npub'); |
|
||||||
if (str_starts_with($npubParam, 'npub')) { |
|
||||||
try { |
|
||||||
$sitePk = $keys->convertToHex($npubParam); |
|
||||||
if ($sitePk !== '' && !\in_array($sitePk, $pubkeys, true)) { |
|
||||||
$pubkeys[] = $sitePk; |
|
||||||
} |
|
||||||
} catch (\Throwable) { |
|
||||||
// ignore bad npub |
|
||||||
} |
|
||||||
} |
|
||||||
$limit = (int) $input->getOption('metadata-limit'); |
|
||||||
if ($limit > 0) { |
|
||||||
$pubkeys = \array_slice($pubkeys, 0, $limit); |
|
||||||
} |
|
||||||
$toWarm = []; |
|
||||||
foreach ($pubkeys as $pubkey) { |
|
||||||
if (strlen($pubkey) === 64) { |
|
||||||
$toWarm[] = $pubkey; |
|
||||||
} |
|
||||||
} |
|
||||||
$total = \count($toWarm); |
|
||||||
$n = 0; |
|
||||||
if ($total === 0) { |
|
||||||
$io->note('No valid author pubkeys to warm.'); |
|
||||||
} else { |
|
||||||
$batchSize = max(1, min(200, (int) $input->getOption('metadata-batch'))); |
|
||||||
$io->writeln(sprintf( |
|
||||||
'Fetching kind-0 metadata: <info>%d</info> author(s) in Nostr requests of up to <info>%d</info> pubkeys each.', |
|
||||||
$total, |
|
||||||
$batchSize |
|
||||||
)); |
|
||||||
$bar = $io->createProgressBar($total); |
|
||||||
$bar->start(); |
|
||||||
try { |
|
||||||
foreach (array_chunk($toWarm, $batchSize) as $chunk) { |
|
||||||
$fetched = $this->nostrClient->fetchKind0MetadataForAuthors($chunk, $batchSize); |
|
||||||
$n += $this->cacheService->putPrewarmMetadataBatch($chunk, $fetched, $keys); |
|
||||||
$bar->advance(\count($chunk)); |
|
||||||
} |
|
||||||
} catch (\Throwable $e) { |
|
||||||
$this->logger->error('app:prewarm metadata batch failed', ['exception' => $e]); |
|
||||||
$io->error($e->getMessage()); |
|
||||||
$bar->finish(); |
|
||||||
$io->newLine(2); |
|
||||||
|
|
||||||
return Command::FAILURE; |
|
||||||
} |
|
||||||
$bar->finish(); |
|
||||||
$io->newLine(2); |
|
||||||
} |
|
||||||
$io->success(sprintf('Warmed metadata for %d of %d author(s).', $n, $total)); |
|
||||||
} else { |
|
||||||
$io->note('Skipping metadata (--no-metadata).'); |
|
||||||
} |
|
||||||
|
|
||||||
if ($input->getOption('no-comments')) { |
|
||||||
$io->note('Skipping comments (--no-comments).'); |
|
||||||
|
|
||||||
return Command::SUCCESS; |
|
||||||
} |
|
||||||
|
|
||||||
$maxArticles = (int) $input->getOption('comments-max'); |
|
||||||
|
|
||||||
$io->section('Comment / interaction cache'); |
|
||||||
$deadline = microtime(true) + max(1, (int) $input->getOption('comments-budget')); |
|
||||||
$qb = $this->articleRepository->createQueryBuilder('a') |
|
||||||
->where('a.slug IS NOT NULL') |
|
||||||
->andWhere("a.slug != ''") |
|
||||||
->andWhere('a.pubkey IS NOT NULL') |
|
||||||
->andWhere("a.pubkey != ''") |
|
||||||
->orderBy('a.createdAt', 'DESC'); |
|
||||||
if ($maxArticles > 0) { |
|
||||||
$qb->setMaxResults($maxArticles); |
|
||||||
} |
|
||||||
$articles = $qb->getQuery()->getResult(); |
|
||||||
$w = 0; |
|
||||||
/** @var Article $article */ |
|
||||||
foreach ($articles as $article) { |
|
||||||
if (microtime(true) >= $deadline) { |
|
||||||
$io->warning('Comment phase stopped: comments-budget reached.'); |
|
||||||
break; |
|
||||||
} |
|
||||||
$slug = trim((string) $article->getSlug()); |
|
||||||
$pubkey = (string) $article->getPubkey(); |
|
||||||
if ($slug === '' || strlen($pubkey) !== 64) { |
|
||||||
continue; |
|
||||||
} |
|
||||||
$kind = $article->getKind()?->value ?? 30023; |
|
||||||
$coordinate = $kind.':'.$pubkey.':'.$slug; |
|
||||||
$eventHex = (string) ($article->getEventId() ?? ''); |
|
||||||
try { |
|
||||||
$this->commentThreadLoader->load($coordinate, $eventHex !== '' ? $eventHex : null); |
|
||||||
++$w; |
|
||||||
} catch (\Throwable $e) { |
|
||||||
$this->logger->warning('app:prewarm comment load', ['coord' => $coordinate, 'error' => $e->getMessage()]); |
|
||||||
} |
|
||||||
} |
|
||||||
$io->success(sprintf('Warmed comment cache for %d of %d article(s).', $w, \count($articles))); |
|
||||||
|
|
||||||
return Command::SUCCESS; |
|
||||||
} |
|
||||||
|
|
||||||
private function disableCliExecutionTimeLimit(): void |
|
||||||
{ |
|
||||||
@set_time_limit(0); |
|
||||||
@ini_set('max_execution_time', '0'); |
|
||||||
} |
|
||||||
} |
|
||||||
@ -1,103 +0,0 @@ |
|||||||
<?php |
|
||||||
|
|
||||||
declare(strict_types=1); |
|
||||||
|
|
||||||
namespace App\Controller; |
|
||||||
|
|
||||||
use App\Service\MagazineContentService; |
|
||||||
use App\Service\MagazineRefresher; |
|
||||||
use Psr\Log\LoggerInterface; |
|
||||||
use Symfony\Component\DependencyInjection\ParameterBag\ParameterBagInterface; |
|
||||||
use Symfony\Component\HttpFoundation\JsonResponse; |
|
||||||
use Symfony\Component\HttpFoundation\Request; |
|
||||||
use Symfony\Component\HttpFoundation\Response; |
|
||||||
use Symfony\Component\HttpKernel\Attribute\AsController; |
|
||||||
use Symfony\Component\Routing\Attribute\Route; |
|
||||||
use Twig\Environment; |
|
||||||
|
|
||||||
/** Stale-first: the main request only reads {@see \App\Service\MagazineIndexStore}; this refetches Nostr, updates that store, and returns HTML fragments for Stimulus to patch the document. */ |
|
||||||
#[AsController] |
|
||||||
final class MagazineSyncController |
|
||||||
{ |
|
||||||
public function __construct( |
|
||||||
private readonly Environment $twig, |
|
||||||
private readonly MagazineRefresher $refresher, |
|
||||||
private readonly MagazineContentService $magazineContent, |
|
||||||
private readonly ParameterBagInterface $params, |
|
||||||
private readonly LoggerInterface $logger, |
|
||||||
) { |
|
||||||
} |
|
||||||
|
|
||||||
#[Route('/ux/magazine-sync', name: 'ux_magazine_sync', methods: ['GET'])] |
|
||||||
public function __invoke(Request $request): JsonResponse |
|
||||||
{ |
|
||||||
try { |
|
||||||
$page = (string) $request->query->get('page', 'article'); |
|
||||||
if (!\in_array($page, ['home', 'category', 'article', 'articles'], true)) { |
|
||||||
$page = 'article'; |
|
||||||
} |
|
||||||
$slug = (string) $request->query->get('slug', ''); |
|
||||||
|
|
||||||
$prefer = $slug !== '' ? [$slug] : []; |
|
||||||
|
|
||||||
try { |
|
||||||
$this->refresher->refreshFromRelays(20, $prefer); |
|
||||||
} catch (\Throwable $e) { |
|
||||||
$this->logger->warning('MagazineSyncController: refresh failed', [ |
|
||||||
'message' => $e->getMessage(), |
|
||||||
'exception' => $e, |
|
||||||
]); |
|
||||||
|
|
||||||
return new JsonResponse( |
|
||||||
['ok' => false, 'error' => 'refresh_failed', 'message' => $e->getMessage()], |
|
||||||
Response::HTTP_OK |
|
||||||
); |
|
||||||
} |
|
||||||
|
|
||||||
$community = (bool) $this->params->get('community_articles'); |
|
||||||
$tags = $this->magazineContent->getHomeCategoryAIndexTagsFromStoreOnly(); |
|
||||||
$globals = [ |
|
||||||
'magazine_community_articles' => $community, |
|
||||||
]; |
|
||||||
|
|
||||||
$header = $this->twig->render('ux/magazine/header_ul.html.twig', array_merge($globals, [ |
|
||||||
'cats' => $tags, |
|
||||||
])); |
|
||||||
|
|
||||||
$body = null; |
|
||||||
if ($page === 'home') { |
|
||||||
$body = $this->twig->render('ux/magazine/home_body.html.twig', array_merge($globals, [ |
|
||||||
'indices' => $tags, |
|
||||||
])); |
|
||||||
} elseif ($page === 'category' && $slug !== '') { |
|
||||||
$data = $this->magazineContent->getCategoryPageData($slug); |
|
||||||
$body = $this->twig->render('ux/magazine/category_body.html.twig', array_merge($globals, [ |
|
||||||
'list' => $data['list'], |
|
||||||
'category' => $data['category'], |
|
||||||
])); |
|
||||||
} elseif ($page === 'articles') { |
|
||||||
$body = null; |
|
||||||
} |
|
||||||
|
|
||||||
return new JsonResponse([ |
|
||||||
'ok' => true, |
|
||||||
'header' => $header, |
|
||||||
'body' => $body, |
|
||||||
]); |
|
||||||
} catch (\Throwable $e) { |
|
||||||
$this->logger->error('MagazineSyncController: unexpected failure', [ |
|
||||||
'message' => $e->getMessage(), |
|
||||||
'exception' => $e, |
|
||||||
]); |
|
||||||
|
|
||||||
return new JsonResponse( |
|
||||||
[ |
|
||||||
'ok' => false, |
|
||||||
'error' => 'server_error', |
|
||||||
'message' => 'Magazine UI sync could not be rendered.', |
|
||||||
], |
|
||||||
Response::HTTP_OK |
|
||||||
); |
|
||||||
} |
|
||||||
} |
|
||||||
} |
|
||||||
@ -1,177 +0,0 @@ |
|||||||
<?php |
|
||||||
|
|
||||||
declare(strict_types=1); |
|
||||||
|
|
||||||
namespace App\Service; |
|
||||||
|
|
||||||
use App\Entity\Article; |
|
||||||
use App\Entity\Event; |
|
||||||
use App\Repository\ArticleRepository; |
|
||||||
use Symfony\Component\DependencyInjection\ParameterBag\ParameterBagInterface; |
|
||||||
|
|
||||||
/** |
|
||||||
* Magazine index events for templates. Reads {@see MagazineIndexStore} first; on a cold cache or when |
|
||||||
* the last successful relay sync is older than {@see self::ROOT_REVALIDATE_SECONDS}, the service |
|
||||||
* calls {@see MagazineRefresher} so the root index (and nav) can pick up new categories. |
|
||||||
*/ |
|
||||||
final class MagazineContentService |
|
||||||
{ |
|
||||||
/** Re-fetch root from relays at most this often so new `a` tags appear in the header. */ |
|
||||||
private const ROOT_REVALIDATE_SECONDS = 300; |
|
||||||
|
|
||||||
public function __construct( |
|
||||||
private readonly MagazineIndexStore $store, |
|
||||||
private readonly MagazineRefresher $refresher, |
|
||||||
private readonly ParameterBagInterface $params, |
|
||||||
private readonly ArticleRepository $articleRepository, |
|
||||||
private readonly NostrClient $nostrClient, |
|
||||||
) { |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* "indices" for the home template: Nostr `a` tag rows for each category. |
|
||||||
* |
|
||||||
* @return list<array<int, string>> |
|
||||||
*/ |
|
||||||
public function getHomeCategoryIndexTags(): array |
|
||||||
{ |
|
||||||
$npub = (string) $this->params->get('npub'); |
|
||||||
$dTag = (string) $this->params->get('d_tag'); |
|
||||||
if ($this->store->getRoot($npub, $dTag) === null) { |
|
||||||
$this->refresher->refreshFromRelays(20, []); |
|
||||||
} elseif ($this->shouldRevalidateRootFromRelay()) { |
|
||||||
$this->refresher->refreshFromRelays(20, []); |
|
||||||
} |
|
||||||
|
|
||||||
return $this->getHomeCategoryAIndexTagsFromStoreOnly(); |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* Category `a` tags from the persisted root only (no relay). Used after /ux/magazine-sync |
|
||||||
* has already called {@see MagazineRefresher::refreshFromRelays}. |
|
||||||
* |
|
||||||
* @return list<array<int, string>> |
|
||||||
*/ |
|
||||||
public function getHomeCategoryAIndexTagsFromStoreOnly(): array |
|
||||||
{ |
|
||||||
return $this->categoryATagsFromStoredRoot(); |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* @return list<array<int, string>> |
|
||||||
*/ |
|
||||||
private function categoryATagsFromStoredRoot(): array |
|
||||||
{ |
|
||||||
$npub = (string) $this->params->get('npub'); |
|
||||||
$dTag = (string) $this->params->get('d_tag'); |
|
||||||
$mag = $this->store->getRoot($npub, $dTag); |
|
||||||
|
|
||||||
return $this->categoryATagsFromMag($mag); |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* @return list<array<int, string>> |
|
||||||
*/ |
|
||||||
private function categoryATagsFromMag(?Event $mag): array |
|
||||||
{ |
|
||||||
if ($mag === null) { |
|
||||||
return []; |
|
||||||
} |
|
||||||
$tags = $mag->getTags(); |
|
||||||
$cats = array_filter($tags, static function (mixed $tag): bool { |
|
||||||
return \is_array($tag) && ($tag[0] ?? null) === 'a'; |
|
||||||
}); |
|
||||||
|
|
||||||
return array_values($cats); |
|
||||||
} |
|
||||||
|
|
||||||
private function shouldRevalidateRootFromRelay(): bool |
|
||||||
{ |
|
||||||
$age = $this->refresher->getSecondsSinceLastRelayRun(); |
|
||||||
if ($age === null) { |
|
||||||
return true; |
|
||||||
} |
|
||||||
|
|
||||||
return $age > self::ROOT_REVALIDATE_SECONDS; |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* @return array{list: list<Article>, category: array{title: string, summary: string}} |
|
||||||
*/ |
|
||||||
public function getCategoryPageData(string $slug): array |
|
||||||
{ |
|
||||||
$catIndex = $this->store->getCategory($slug); |
|
||||||
if ($catIndex === null) { |
|
||||||
$this->refresher->refreshFromRelays(20, [$slug]); |
|
||||||
$catIndex = $this->store->getCategory($slug); |
|
||||||
} |
|
||||||
$list = []; |
|
||||||
$coordinates = []; |
|
||||||
$category = []; |
|
||||||
if ($catIndex) { |
|
||||||
foreach ($catIndex->getTags() as $tag) { |
|
||||||
if ($tag[0] === 'title') { |
|
||||||
$category['title'] = (string) $tag[1]; |
|
||||||
} |
|
||||||
if ($tag[0] === 'summary') { |
|
||||||
$category['summary'] = (string) $tag[1]; |
|
||||||
} |
|
||||||
if ($tag[0] === 'a') { |
|
||||||
$coordinates[] = $tag[1]; |
|
||||||
} |
|
||||||
} |
|
||||||
} |
|
||||||
|
|
||||||
if (!empty($coordinates)) { |
|
||||||
$pairs = []; |
|
||||||
foreach ($coordinates as $coordinate) { |
|
||||||
$parts = explode(':', (string) $coordinate, 3); |
|
||||||
if (\count($parts) < 3) { |
|
||||||
continue; |
|
||||||
} |
|
||||||
$slugPart = trim((string) $parts[2]); |
|
||||||
if ($slugPart === '') { |
|
||||||
continue; |
|
||||||
} |
|
||||||
$pairs[] = [ |
|
||||||
'pubkey' => (string) $parts[1], |
|
||||||
'slug' => $slugPart, |
|
||||||
]; |
|
||||||
} |
|
||||||
$byAddress = $this->articleRepository->findByAuthorAndSlugIndexed($pairs); |
|
||||||
$missing = []; |
|
||||||
foreach ($coordinates as $coordinate) { |
|
||||||
$parts = explode(':', (string) $coordinate, 3); |
|
||||||
if (\count($parts) < 3) { |
|
||||||
continue; |
|
||||||
} |
|
||||||
$k = (string) $parts[1]."\0".trim((string) $parts[2]); |
|
||||||
if (!isset($byAddress[$k])) { |
|
||||||
$missing[] = (string) $coordinate; |
|
||||||
} |
|
||||||
} |
|
||||||
if ($missing !== []) { |
|
||||||
$this->nostrClient->ingestMissingLongformForCategoryCoordinates($missing); |
|
||||||
$byAddress = $this->articleRepository->findByAuthorAndSlugIndexed($pairs); |
|
||||||
} |
|
||||||
foreach ($coordinates as $coordinate) { |
|
||||||
$parts = explode(':', (string) $coordinate, 3); |
|
||||||
if (\count($parts) < 3) { |
|
||||||
continue; |
|
||||||
} |
|
||||||
$k = (string) $parts[1]."\0".trim((string) $parts[2]); |
|
||||||
if (isset($byAddress[$k])) { |
|
||||||
$list[] = $byAddress[$k]; |
|
||||||
} |
|
||||||
} |
|
||||||
} |
|
||||||
|
|
||||||
$category['title'] = $category['title'] ?? ''; |
|
||||||
$category['summary'] = $category['summary'] ?? ''; |
|
||||||
|
|
||||||
return [ |
|
||||||
'list' => $list, |
|
||||||
'category' => $category, |
|
||||||
]; |
|
||||||
} |
|
||||||
} |
|
||||||
@ -1,116 +0,0 @@ |
|||||||
<?php |
|
||||||
|
|
||||||
declare(strict_types=1); |
|
||||||
|
|
||||||
namespace App\Service; |
|
||||||
|
|
||||||
use App\Entity\Event; |
|
||||||
use Psr\Cache\CacheItemPoolInterface; |
|
||||||
use Psr\Cache\InvalidArgumentException; |
|
||||||
|
|
||||||
/** |
|
||||||
* Read/write persisted magazine Nostr index events (kinds 30040) without callback-based relay I/O |
|
||||||
* on the request path. Updated by {@see MagazineRefresher} or the /ux/magazine-sync action. |
|
||||||
*/ |
|
||||||
final class MagazineIndexStore |
|
||||||
{ |
|
||||||
private const ROOT_PREFIX = 'mroot_v1_'; |
|
||||||
private const CAT_PREFIX = 'mcat_v1_'; |
|
||||||
|
|
||||||
/** 30 days — we refresh on page load, TTL is a safety cap if sync stops working. */ |
|
||||||
private const PERSIST_TTL = 2_592_000; |
|
||||||
|
|
||||||
public function __construct( |
|
||||||
private readonly CacheItemPoolInterface $pool, |
|
||||||
) { |
|
||||||
} |
|
||||||
|
|
||||||
public function getRoot(string $npub, string $dTag): ?Event |
|
||||||
{ |
|
||||||
$item = $this->pool->getItem($this->rootKey($npub, $dTag)); |
|
||||||
if (!$item->isHit()) { |
|
||||||
return null; |
|
||||||
} |
|
||||||
|
|
||||||
return $this->unwrap($item->get()); |
|
||||||
} |
|
||||||
|
|
||||||
public function getCategory(string $slug): ?Event |
|
||||||
{ |
|
||||||
if ($slug === '') { |
|
||||||
return null; |
|
||||||
} |
|
||||||
$item = $this->pool->getItem(self::CAT_PREFIX.$slug); |
|
||||||
if (!$item->isHit()) { |
|
||||||
return null; |
|
||||||
} |
|
||||||
|
|
||||||
return $this->unwrap($item->get()); |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* @throws InvalidArgumentException |
|
||||||
*/ |
|
||||||
public function putRoot(string $npub, string $dTag, Event $event): void |
|
||||||
{ |
|
||||||
$item = $this->pool->getItem($this->rootKey($npub, $dTag)); |
|
||||||
$item->set(serialize($event)); |
|
||||||
$item->expiresAfter(self::PERSIST_TTL); |
|
||||||
$this->pool->save($item); |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* @throws InvalidArgumentException |
|
||||||
*/ |
|
||||||
public function putCategory(string $slug, Event $event): void |
|
||||||
{ |
|
||||||
if ($slug === '') { |
|
||||||
return; |
|
||||||
} |
|
||||||
$item = $this->pool->getItem(self::CAT_PREFIX.$slug); |
|
||||||
$item->set(serialize($event)); |
|
||||||
$item->expiresAfter(self::PERSIST_TTL); |
|
||||||
$this->pool->save($item); |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* Remove a cached category index (NIP-09 / local invalidation). |
|
||||||
* |
|
||||||
* @throws InvalidArgumentException |
|
||||||
*/ |
|
||||||
public function deleteCategory(string $slug): void |
|
||||||
{ |
|
||||||
if ($slug === '') { |
|
||||||
return; |
|
||||||
} |
|
||||||
$this->pool->deleteItem(self::CAT_PREFIX.$slug); |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* Remove the cached root magazine index for this npub + d_tag. |
|
||||||
* |
|
||||||
* @throws InvalidArgumentException |
|
||||||
*/ |
|
||||||
public function deleteRoot(string $npub, string $dTag): void |
|
||||||
{ |
|
||||||
$this->pool->deleteItem($this->rootKey($npub, $dTag)); |
|
||||||
} |
|
||||||
|
|
||||||
private function rootKey(string $npub, string $dTag): string |
|
||||||
{ |
|
||||||
return self::ROOT_PREFIX.hash('sha256', $npub."\0".$dTag); |
|
||||||
} |
|
||||||
|
|
||||||
private function unwrap(mixed $value): ?Event |
|
||||||
{ |
|
||||||
if (!\is_string($value) || $value === '') { |
|
||||||
return null; |
|
||||||
} |
|
||||||
$e = unserialize($value, ['allowed_classes' => [Event::class]]); |
|
||||||
if (!$e instanceof Event) { |
|
||||||
return null; |
|
||||||
} |
|
||||||
|
|
||||||
return $e; |
|
||||||
} |
|
||||||
} |
|
||||||
@ -1,177 +0,0 @@ |
|||||||
<?php |
|
||||||
|
|
||||||
declare(strict_types=1); |
|
||||||
|
|
||||||
namespace App\Service; |
|
||||||
|
|
||||||
use App\Entity\Event; |
|
||||||
use Psr\Cache\CacheItemPoolInterface; |
|
||||||
use Psr\Cache\InvalidArgumentException; |
|
||||||
use Psr\Log\LoggerInterface; |
|
||||||
use Symfony\Component\DependencyInjection\ParameterBag\ParameterBagInterface; |
|
||||||
|
|
||||||
/** |
|
||||||
* Pulls magazine indices from relays within a wall-clock budget and persists them to {@see MagazineIndexStore}. |
|
||||||
*/ |
|
||||||
final class MagazineRefresher |
|
||||||
{ |
|
||||||
private const RELAY_STAMP_KEY = 'mag_relay_v1'; |
|
||||||
|
|
||||||
public function __construct( |
|
||||||
private readonly NostrClient $nostrClient, |
|
||||||
private readonly MagazineIndexStore $store, |
|
||||||
private readonly ParameterBagInterface $params, |
|
||||||
private readonly LoggerInterface $logger, |
|
||||||
private readonly CacheItemPoolInterface $appCache, |
|
||||||
) { |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* Fetches the root index then each category index until $budgetSeconds elapses. $preferSlugs |
|
||||||
* are requested first (e.g. current /cat route) so they are less likely to miss the budget. |
|
||||||
*/ |
|
||||||
public function refreshFromRelays(int $budgetSeconds = 8, array $preferSlugs = []): void |
|
||||||
{ |
|
||||||
$budgetSeconds = max(1, min(30, $budgetSeconds)); |
|
||||||
$deadline = microtime(true) + $budgetSeconds; |
|
||||||
$npub = (string) $this->params->get('npub'); |
|
||||||
$dTag = (string) $this->params->get('d_tag'); |
|
||||||
|
|
||||||
// Do not set max_execution_time to the *remaining* soft budget: PHP resets the timer, so |
|
||||||
// after a 6s root fetch, "2s left" would become a 2s hard cap for the *next* relay I/O |
|
||||||
// (e.g. slow TLS) and can fatal. Cap once with headroom; the $deadline loop limits work. |
|
||||||
$this->applyExecutionTimeCap($budgetSeconds); |
|
||||||
|
|
||||||
$defaultRelay = (string) $this->params->get('default_relay'); |
|
||||||
$relayLabel = (string) (parse_url($defaultRelay, \PHP_URL_HOST) ?: $defaultRelay); |
|
||||||
|
|
||||||
$root = $this->nostrClient->getMagazineIndex($npub, $dTag); |
|
||||||
if ($root === null) { |
|
||||||
$this->logger->warning(sprintf( |
|
||||||
'MagazineRefresher: root index not returned (tried from %s)', |
|
||||||
$relayLabel |
|
||||||
), [ |
|
||||||
'd_tag' => $dTag, |
|
||||||
'relay' => $defaultRelay, |
|
||||||
]); |
|
||||||
|
|
||||||
return; |
|
||||||
} |
|
||||||
|
|
||||||
$this->store->putRoot($npub, $dTag, $root); |
|
||||||
|
|
||||||
$slugs = $this->orderedCategorySlugs($this->categorySlugsFromRoot($root), $preferSlugs); |
|
||||||
foreach ($slugs as $slug) { |
|
||||||
if (microtime(true) >= $deadline) { |
|
||||||
$this->logger->notice('MagazineRefresher: stopped at time budget; some categories not fetched', [ |
|
||||||
'unprocessed_from' => $slug, |
|
||||||
]); |
|
||||||
break; |
|
||||||
} |
|
||||||
try { |
|
||||||
$cat = $this->nostrClient->getMagazineIndex($npub, $slug); |
|
||||||
if ($cat !== null) { |
|
||||||
$this->store->putCategory($slug, $cat); |
|
||||||
} |
|
||||||
} catch (\Throwable $e) { |
|
||||||
$this->logger->error(sprintf( |
|
||||||
'MagazineRefresher: category fetch failed (relays from %s): %s', |
|
||||||
$relayLabel, |
|
||||||
$e->getMessage() |
|
||||||
), [ |
|
||||||
'slug' => $slug, |
|
||||||
'message' => $e->getMessage(), |
|
||||||
'relay' => $defaultRelay, |
|
||||||
]); |
|
||||||
} |
|
||||||
} |
|
||||||
|
|
||||||
$this->touchLastRelayTime(); |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* @throws InvalidArgumentException |
|
||||||
*/ |
|
||||||
public function getSecondsSinceLastRelayRun(): ?int |
|
||||||
{ |
|
||||||
try { |
|
||||||
$item = $this->appCache->getItem(self::RELAY_STAMP_KEY); |
|
||||||
} catch (InvalidArgumentException) { |
|
||||||
return null; |
|
||||||
} |
|
||||||
if (!$item->isHit()) { |
|
||||||
return null; |
|
||||||
} |
|
||||||
|
|
||||||
return time() - (int) $item->get(); |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* Child category indices are kind 30040; each root "a" tag is a NIP-33 address |
|
||||||
* kind:hexpubkey:d-identifier. The third segment is the child #d (e.g. the long |
|
||||||
* newsroom-…-category-… string), not a shortened title. |
|
||||||
* |
|
||||||
* @return list<string> |
|
||||||
*/ |
|
||||||
private function categorySlugsFromRoot(Event $root): array |
|
||||||
{ |
|
||||||
$slugs = []; |
|
||||||
foreach ($root->getTags() as $tag) { |
|
||||||
if (($tag[0] ?? null) !== 'a' || !isset($tag[1])) { |
|
||||||
continue; |
|
||||||
} |
|
||||||
$parts = explode(':', (string) $tag[1], 3); |
|
||||||
if (\count($parts) < 3) { |
|
||||||
continue; |
|
||||||
} |
|
||||||
$s = trim((string) end($parts)); |
|
||||||
if ($s !== '' && !\in_array($s, $slugs, true)) { |
|
||||||
$slugs[] = $s; |
|
||||||
} |
|
||||||
} |
|
||||||
|
|
||||||
return $slugs; |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* @param list<string> $allFromRoot |
|
||||||
* @param list<string> $prefer |
|
||||||
* @return list<string> |
|
||||||
*/ |
|
||||||
private function orderedCategorySlugs(array $allFromRoot, array $prefer): array |
|
||||||
{ |
|
||||||
$prefer = array_values(array_filter($prefer, static function (string $s): bool { |
|
||||||
return $s !== ''; |
|
||||||
})); |
|
||||||
$out = $prefer; |
|
||||||
foreach ($allFromRoot as $s) { |
|
||||||
if (!\in_array($s, $out, true)) { |
|
||||||
$out[] = $s; |
|
||||||
} |
|
||||||
} |
|
||||||
|
|
||||||
return $out; |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* @throws InvalidArgumentException |
|
||||||
*/ |
|
||||||
private function touchLastRelayTime(): void |
|
||||||
{ |
|
||||||
$item = $this->appCache->getItem(self::RELAY_STAMP_KEY); |
|
||||||
$item->set((string) time()); |
|
||||||
$item->expiresAfter(86_400); |
|
||||||
$this->appCache->save($item); |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* One generous ceiling for PHP so relay/WebSocket I/O in one Nostr call can outlast the soft |
|
||||||
* $deadline by seconds without a fatal, while the loop still stops *starting* new fetches in time. |
|
||||||
*/ |
|
||||||
private function applyExecutionTimeCap(int $budgetSeconds): void |
|
||||||
{ |
|
||||||
$sec = max(30, min(120, $budgetSeconds + 30)); |
|
||||||
@set_time_limit($sec); |
|
||||||
@ini_set('max_execution_time', (string) $sec); |
|
||||||
} |
|
||||||
} |
|
||||||
@ -1,359 +0,0 @@ |
|||||||
<?php |
|
||||||
|
|
||||||
declare(strict_types=1); |
|
||||||
|
|
||||||
namespace App\Service; |
|
||||||
|
|
||||||
use App\Entity\Event as MagazineNostrEvent; |
|
||||||
use App\Enum\KindsEnum; |
|
||||||
use App\Repository\ArticleRepository; |
|
||||||
use Doctrine\ORM\EntityManagerInterface; |
|
||||||
use Psr\Log\LoggerInterface; |
|
||||||
use swentel\nostr\Key\Key; |
|
||||||
use Symfony\Component\DependencyInjection\ParameterBag\ParameterBagInterface; |
|
||||||
|
|
||||||
/** |
|
||||||
* Applies NIP-09 (kind 5) deletion requests to local MySQL articles and magazine 30040 cache. |
|
||||||
* |
|
||||||
* Relays are not authoritative; we only remove data we can validate (same pubkey as deletion request). |
|
||||||
* For cached 30040 category indices (keyed by `d` only), we require the stored event’s author |
|
||||||
* to match the deletion — not just an `a` tag whose own pubkey matches, so colliding `d` values |
|
||||||
* across authors cannot wipe another author’s cache entry. |
|
||||||
*/ |
|
||||||
final class Nip09DeletionApplier |
|
||||||
{ |
|
||||||
public function __construct( |
|
||||||
private readonly EntityManagerInterface $entityManager, |
|
||||||
private readonly ArticleRepository $articleRepository, |
|
||||||
private readonly MagazineIndexStore $magazineIndexStore, |
|
||||||
private readonly ParameterBagInterface $params, |
|
||||||
private readonly LoggerInterface $logger, |
|
||||||
) { |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* @param list<object> $deletionEvents Kind-5 events from relays (e.g. {@see NostrClient::fetchKind5DeletionEventsForAuthors}) |
|
||||||
* |
|
||||||
* @return array{articles_removed: int, magazine_roots: int, magazine_categories: int} |
|
||||||
*/ |
|
||||||
public function apply(array $deletionEvents): array |
|
||||||
{ |
|
||||||
$articlesRemoved = 0; |
|
||||||
$articlesPendingFlush = 0; |
|
||||||
$roots = 0; |
|
||||||
$cats = 0; |
|
||||||
$seenArticleIds = []; |
|
||||||
|
|
||||||
foreach ($deletionEvents as $ev) { |
|
||||||
if (!\is_object($ev)) { |
|
||||||
continue; |
|
||||||
} |
|
||||||
if ((int) ($ev->kind ?? 0) !== KindsEnum::DELETION_REQUEST->value) { |
|
||||||
continue; |
|
||||||
} |
|
||||||
$deletionPubkey = (string) ($ev->pubkey ?? ''); |
|
||||||
if (64 !== \strlen($deletionPubkey)) { |
|
||||||
continue; |
|
||||||
} |
|
||||||
|
|
||||||
[$eIds, $eKinds] = $this->parseETags($ev); |
|
||||||
$aAddrs = $this->parseATags($ev); |
|
||||||
|
|
||||||
foreach ($eIds as $i => $eId) { |
|
||||||
if (64 !== \strlen($eId)) { |
|
||||||
continue; |
|
||||||
} |
|
||||||
$declared = $eKinds[$i] ?? null; |
|
||||||
if ($declared !== null |
|
||||||
&& !\in_array($declared, [30023, 30024, 30040, 1], true)) { |
|
||||||
// Other kinds: we do not mirror in this app; skip. |
|
||||||
continue; |
|
||||||
} |
|
||||||
if ($declared === 1) { |
|
||||||
continue; |
|
||||||
} |
|
||||||
if ($this->removeArticleByEventIdIfValid($eId, $deletionPubkey, $declared, $seenArticleIds)) { |
|
||||||
++$articlesRemoved; |
|
||||||
++$articlesPendingFlush; |
|
||||||
continue; |
|
||||||
} |
|
||||||
// No article row: 30040 index (or mis-tagged kind); only skip unrelated kinds. |
|
||||||
if ($declared === null || \in_array($declared, [30023, 30024, 30040], true)) { |
|
||||||
$mag = $this->tryRemoveMagazine30040ByEventId($eId, $deletionPubkey); |
|
||||||
if ($mag === 1) { |
|
||||||
++$roots; |
|
||||||
} elseif ($mag === 2) { |
|
||||||
++$cats; |
|
||||||
} |
|
||||||
} |
|
||||||
} |
|
||||||
|
|
||||||
foreach ($aAddrs as $addr) { |
|
||||||
$r = $this->removeByNip33Address($addr, $deletionPubkey, $seenArticleIds); |
|
||||||
$articlesRemoved += $r['articles']; |
|
||||||
$articlesPendingFlush += $r['articles']; |
|
||||||
$roots += $r['roots']; |
|
||||||
$cats += $r['cats']; |
|
||||||
} |
|
||||||
} |
|
||||||
|
|
||||||
if ($articlesPendingFlush > 0) { |
|
||||||
try { |
|
||||||
$this->entityManager->flush(); |
|
||||||
} catch (\Throwable $e) { |
|
||||||
$this->logger->error('Nip09DeletionApplier: flush failed', ['exception' => $e]); |
|
||||||
} |
|
||||||
} |
|
||||||
|
|
||||||
return [ |
|
||||||
'articles_removed' => $articlesRemoved, |
|
||||||
'magazine_roots' => $roots, |
|
||||||
'magazine_categories' => $cats, |
|
||||||
]; |
|
||||||
} |
|
||||||
|
|
||||||
/** 0 = none, 1 = root cache, 2 = category cache */ |
|
||||||
private function tryRemoveMagazine30040ByEventId(string $eventId, string $deletionPubkey): int |
|
||||||
{ |
|
||||||
$npub = (string) $this->params->get('npub'); |
|
||||||
$dTag = (string) $this->params->get('d_tag'); |
|
||||||
if ($npub === '' || $dTag === '') { |
|
||||||
return 0; |
|
||||||
} |
|
||||||
$root = $this->magazineIndexStore->getRoot($npub, $dTag); |
|
||||||
if ($root === null) { |
|
||||||
return 0; |
|
||||||
} |
|
||||||
if ($this->eventIdMatches($root, $eventId) && $this->pubkeyEquals($root->getPubkey(), $deletionPubkey)) { |
|
||||||
$this->magazineIndexStore->deleteRoot($npub, $dTag); |
|
||||||
$this->logger->notice('NIP-09: removed cached magazine root index', [ |
|
||||||
'event_id' => $eventId, |
|
||||||
]); |
|
||||||
|
|
||||||
return 1; |
|
||||||
} |
|
||||||
foreach ($this->categorySlugsFromRoot($root) as $slug) { |
|
||||||
$cat = $this->magazineIndexStore->getCategory($slug); |
|
||||||
if ($cat === null) { |
|
||||||
continue; |
|
||||||
} |
|
||||||
if ($this->eventIdMatches($cat, $eventId) && $this->pubkeyEquals($cat->getPubkey(), $deletionPubkey)) { |
|
||||||
$this->magazineIndexStore->deleteCategory($slug); |
|
||||||
$this->logger->notice('NIP-09: removed cached magazine category index', [ |
|
||||||
'event_id' => $eventId, |
|
||||||
'slug' => $slug, |
|
||||||
]); |
|
||||||
|
|
||||||
return 2; |
|
||||||
} |
|
||||||
} |
|
||||||
|
|
||||||
return 0; |
|
||||||
} |
|
||||||
|
|
||||||
private function eventIdMatches(MagazineNostrEvent $e, string $eventId): bool |
|
||||||
{ |
|
||||||
$a = strtolower($e->getId()); |
|
||||||
$b = strtolower($eventId); |
|
||||||
|
|
||||||
return $a === $b; |
|
||||||
} |
|
||||||
|
|
||||||
private function pubkeyEquals(string $a, string $b): bool |
|
||||||
{ |
|
||||||
if (64 !== \strlen($a) || 64 !== \strlen($b)) { |
|
||||||
return $a === $b; |
|
||||||
} |
|
||||||
|
|
||||||
return strtolower($a) === strtolower($b); |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* @return list<string> |
|
||||||
*/ |
|
||||||
private function categorySlugsFromRoot(MagazineNostrEvent $root): array |
|
||||||
{ |
|
||||||
$slugs = []; |
|
||||||
foreach ($root->getTags() as $tag) { |
|
||||||
if (($tag[0] ?? null) !== 'a' || !isset($tag[1])) { |
|
||||||
continue; |
|
||||||
} |
|
||||||
$parts = explode(':', (string) $tag[1], 3); |
|
||||||
if (\count($parts) < 3) { |
|
||||||
continue; |
|
||||||
} |
|
||||||
$s = trim((string) end($parts)); |
|
||||||
if ($s !== '' && !\in_array($s, $slugs, true)) { |
|
||||||
$slugs[] = $s; |
|
||||||
} |
|
||||||
} |
|
||||||
|
|
||||||
return $slugs; |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* @param array<string, true> $seenArticleIds |
|
||||||
*/ |
|
||||||
private function removeArticleByEventIdIfValid( |
|
||||||
string $eId, |
|
||||||
string $deletionPubkey, |
|
||||||
?int $declaredKind, |
|
||||||
array &$seenArticleIds, |
|
||||||
): bool { |
|
||||||
if (isset($seenArticleIds[$eId])) { |
|
||||||
return false; |
|
||||||
} |
|
||||||
$article = $this->articleRepository->findOneByEventId($eId); |
|
||||||
if ($article === null) { |
|
||||||
return false; |
|
||||||
} |
|
||||||
if (!$this->pubkeyEquals($article->getPubkey() ?? '', $deletionPubkey)) { |
|
||||||
$this->logger->debug('NIP-09: ignore e tag (pubkey mismatch)', [ |
|
||||||
'event_id' => $eId, |
|
||||||
]); |
|
||||||
|
|
||||||
return false; |
|
||||||
} |
|
||||||
$k = $article->getKind()?->value; |
|
||||||
if ($declaredKind !== null && $k !== null && $declaredKind !== $k) { |
|
||||||
return false; |
|
||||||
} |
|
||||||
if ($k !== null && !\in_array($k, [KindsEnum::LONGFORM->value, KindsEnum::LONGFORM_DRAFT->value], true)) { |
|
||||||
return false; |
|
||||||
} |
|
||||||
$this->entityManager->remove($article); |
|
||||||
$seenArticleIds[$eId] = true; |
|
||||||
$this->logger->notice('NIP-09: removed article from database', [ |
|
||||||
'event_id' => $eId, |
|
||||||
'kind' => $k, |
|
||||||
]); |
|
||||||
|
|
||||||
return true; |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* NIP-33: `kind:pubkeyhex:d-identifier` |
|
||||||
* |
|
||||||
* @param array<string, true> $seenArticleIds |
|
||||||
* |
|
||||||
* @return array{articles: int, roots: int, cats: int} |
|
||||||
*/ |
|
||||||
private function removeByNip33Address(string $addr, string $deletionPubkey, array &$seenArticleIds): array |
|
||||||
{ |
|
||||||
$out = ['articles' => 0, 'roots' => 0, 'cats' => 0]; |
|
||||||
$parts = explode(':', $addr, 3); |
|
||||||
if (\count($parts) < 3) { |
|
||||||
return $out; |
|
||||||
} |
|
||||||
$kind = (int) $parts[0]; |
|
||||||
$pk = (string) $parts[1]; |
|
||||||
$d = trim((string) $parts[2]); |
|
||||||
if ($d === '' || !$this->pubkeyEquals($pk, $deletionPubkey)) { |
|
||||||
return $out; |
|
||||||
} |
|
||||||
|
|
||||||
if ($kind === KindsEnum::LONGFORM->value || $kind === KindsEnum::LONGFORM_DRAFT->value) { |
|
||||||
$article = $this->articleRepository->findOneBy(['pubkey' => $pk, 'slug' => $d]); |
|
||||||
if ($article !== null) { |
|
||||||
$eid = (string) ($article->getEventId() ?? ''); |
|
||||||
$dedupeKey = $eid !== '' ? $eid : 'ps:'.$pk."\0".$d; |
|
||||||
if (!isset($seenArticleIds[$dedupeKey])) { |
|
||||||
$this->entityManager->remove($article); |
|
||||||
$seenArticleIds[$dedupeKey] = true; |
|
||||||
++$out['articles']; |
|
||||||
$this->logger->notice('NIP-09: removed article (a tag)', [ |
|
||||||
'address' => $addr, |
|
||||||
]); |
|
||||||
} |
|
||||||
} |
|
||||||
|
|
||||||
return $out; |
|
||||||
} |
|
||||||
|
|
||||||
if ($kind === KindsEnum::PUBLICATION_INDEX->value) { |
|
||||||
$npub = (string) $this->params->get('npub'); |
|
||||||
$siteD = (string) $this->params->get('d_tag'); |
|
||||||
$siteHex = ''; |
|
||||||
if (str_starts_with($npub, 'npub1')) { |
|
||||||
try { |
|
||||||
$h = (new Key())->convertToHex($npub); |
|
||||||
if (64 === \strlen($h)) { |
|
||||||
$siteHex = $h; |
|
||||||
} |
|
||||||
} catch (\Throwable) { |
|
||||||
} |
|
||||||
} |
|
||||||
if ($npub !== '' && $siteD !== '' && $d === $siteD && $siteHex !== '' && $this->pubkeyEquals($pk, $siteHex)) { |
|
||||||
$this->magazineIndexStore->deleteRoot($npub, $siteD); |
|
||||||
++$out['roots']; |
|
||||||
$this->logger->notice('NIP-09: removed magazine root (a tag)', ['address' => $addr]); |
|
||||||
} else { |
|
||||||
// Category cache is keyed by `d` only; the same d string can appear for different |
|
||||||
// authors' 30040 events. Only remove if the cached event was authored by this deletion. |
|
||||||
$cachedCat = $this->magazineIndexStore->getCategory($d); |
|
||||||
if ($cachedCat === null) { |
|
||||||
$this->logger->debug('NIP-09: skip category delete (nothing cached for d)', [ |
|
||||||
'address' => $addr, |
|
||||||
'd' => $d, |
|
||||||
]); |
|
||||||
} elseif (!$this->pubkeyEquals($cachedCat->getPubkey(), $deletionPubkey)) { |
|
||||||
$this->logger->debug('NIP-09: skip category delete (cached index author != deletion author)', [ |
|
||||||
'address' => $addr, |
|
||||||
'd' => $d, |
|
||||||
]); |
|
||||||
} else { |
|
||||||
$this->magazineIndexStore->deleteCategory($d); |
|
||||||
++$out['cats']; |
|
||||||
$this->logger->notice('NIP-09: removed magazine category (a tag)', [ |
|
||||||
'address' => $addr, |
|
||||||
'd' => $d, |
|
||||||
]); |
|
||||||
} |
|
||||||
} |
|
||||||
} |
|
||||||
|
|
||||||
return $out; |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* @return array{0: list<string>, 1: list<?int>} e-ids and parallel k kinds (NIP-09 example order) |
|
||||||
*/ |
|
||||||
private function parseETags(object $ev): array |
|
||||||
{ |
|
||||||
$eIds = []; |
|
||||||
$kinds = []; |
|
||||||
foreach ($ev->tags ?? [] as $tag) { |
|
||||||
if (!\is_array($tag) || !isset($tag[0], $tag[1])) { |
|
||||||
continue; |
|
||||||
} |
|
||||||
if ($tag[0] === 'e') { |
|
||||||
$eIds[] = (string) $tag[1]; |
|
||||||
} |
|
||||||
if ($tag[0] === 'k') { |
|
||||||
$kinds[] = (int) $tag[1]; |
|
||||||
} |
|
||||||
} |
|
||||||
$pairs = []; |
|
||||||
for ($i = 0; $i < \count($eIds); ++$i) { |
|
||||||
$pairs[] = $kinds[$i] ?? null; |
|
||||||
} |
|
||||||
|
|
||||||
return [$eIds, $pairs]; |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* @return list<string> NIP-33 addresses |
|
||||||
*/ |
|
||||||
private function parseATags(object $ev): array |
|
||||||
{ |
|
||||||
$a = []; |
|
||||||
foreach ($ev->tags ?? [] as $tag) { |
|
||||||
if (!\is_array($tag) || ($tag[0] ?? null) !== 'a' || !isset($tag[1])) { |
|
||||||
continue; |
|
||||||
} |
|
||||||
$a[] = (string) $tag[1]; |
|
||||||
} |
|
||||||
|
|
||||||
return $a; |
|
||||||
} |
|
||||||
} |
|
||||||
@ -1,3 +0,0 @@ |
|||||||
<div class="category-body" data-magazine-sync-target="pageBody"> |
|
||||||
<twig:Organisms:CardList :list="list" class="article-list" /> |
|
||||||
</div> |
|
||||||
@ -1,10 +0,0 @@ |
|||||||
<ul data-magazine-sync-target="headerNav"> |
|
||||||
{% for category in cats %} |
|
||||||
<li><twig:Molecules:CategoryLink :category="category" /></li> |
|
||||||
{% endfor %} |
|
||||||
{% if magazine_community_articles %} |
|
||||||
<li> |
|
||||||
<a href="{{ path('articles') }}">Latest Articles</a> |
|
||||||
</li> |
|
||||||
{% endif %} |
|
||||||
</ul> |
|
||||||
Loading…
Reference in new issue