mirror of
https://github.com/Terranom674/Piwigo_Bratonien_Tools.git
synced 2026-09-19 10:34:30 +00:00
Compare commits
24 Commits
add913aab1
...
54a9c97525
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
54a9c97525 | ||
|
|
dfd180c11e | ||
|
|
714c487ef2 | ||
|
|
ccefc72c25 | ||
|
|
7e05e4e8a1 | ||
|
|
fae2daff4b | ||
|
|
ef4741872a | ||
|
|
da74347c9b | ||
|
|
c2408819f0 | ||
|
|
4beb8f8420 | ||
|
|
0ac34e3ad1 | ||
|
|
b81fd3afa7 | ||
|
|
87647f8f16 | ||
|
|
362229efaa | ||
|
|
8571e3ff1f | ||
|
|
7e5d34e245 | ||
|
|
4c73c43c60 | ||
|
|
cc7fa01c10 | ||
|
|
3d768882d2 | ||
|
|
3c822911f8 | ||
|
|
5839d53f90 | ||
|
|
d8690fca95 | ||
|
|
fd1a9f1938 | ||
|
|
665768fb1f |
@@ -259,6 +259,7 @@ $template->assign(array(
|
||||
|
||||
$template->set_filename('admin_tabs_content', BRATONIEN_TOOLS_PATH . 'template/admin_tabs.tpl');
|
||||
$template->set_filename('plugin_admin_content', BRATONIEN_TOOLS_PATH . 'template/admin.tpl');
|
||||
$template->set_filename('webdav_warmup_admin_content', BRATONIEN_TOOLS_PATH . 'template/webdav_warmup_admin.tpl');
|
||||
$template->set_filename('public_selection_admin_content', BRATONIEN_TOOLS_PATH . 'template/public_selection_admin.tpl');
|
||||
$template->set_filename('asset_manager_admin_content', BRATONIEN_TOOLS_PATH . 'template/asset_manager_admin.tpl');
|
||||
$template->set_filename('album_shares_admin_content', BRATONIEN_TOOLS_PATH . 'template/album_shares_admin.tpl');
|
||||
@@ -267,6 +268,7 @@ $template->set_filename('system_admin_content', BRATONIEN_TOOLS_PATH . 'template
|
||||
|
||||
$admin_content = $template->parse('admin_tabs_content', true);
|
||||
$admin_content .= $template->parse('plugin_admin_content', true);
|
||||
$admin_content .= $template->parse('webdav_warmup_admin_content', true);
|
||||
$admin_content .= $template->parse('public_selection_admin_content', true);
|
||||
$admin_content .= $template->parse('asset_manager_admin_content', true);
|
||||
$admin_content .= $template->parse('album_shares_admin_content', true);
|
||||
|
||||
68
docs/WEBDAV_CACHE_WARMUP_0.9.7.1.8.md
Normal file
68
docs/WEBDAV_CACHE_WARMUP_0.9.7.1.8.md
Normal file
@@ -0,0 +1,68 @@
|
||||
# WebDAV-Cache-Warmup – Patch 0.9.7.1.8
|
||||
|
||||
## Ziel
|
||||
|
||||
Fehlende Piwigo-Derivate für neue Nextcloud-Inhalte im Hintergrund vorbereiten, ohne einen zweiten dauerhaften Bildbestand anzulegen und ohne Piwigo-Datenbankeinträge, Albumzuordnungen oder Bildpfade durch Bratonien Tools zu verändern.
|
||||
|
||||
## Zuständigkeiten
|
||||
|
||||
Bratonien Tools übernimmt ausschließlich Orchestrierung:
|
||||
|
||||
1. neue Alben bzw. neue/geänderte Bilder erkennen;
|
||||
2. Originale in konfigurierbaren Batches temporär aus Nextcloud laden;
|
||||
3. die bereits bestehende Piwigo-Quellzuordnung validieren;
|
||||
4. die lokale Quelle für genau ein Bild atomar bereitstellen;
|
||||
5. Piwigos eigenes `i.php` für die gewünschten, von Piwigo definierten Derivate aufrufen;
|
||||
6. den ursprünglichen Placeholder exakt wiederherstellen und verifizieren;
|
||||
7. temporäre Batch-Dateien löschen.
|
||||
|
||||
Piwigo bleibt allein zuständig für Derivatgrößen, Bildverarbeitung, Qualität, Format und Cache-Dateien.
|
||||
|
||||
## Trigger
|
||||
|
||||
- Neues Album: unmittelbar nach vollständig erfolgreichem Connector-Sync dieser Verbindung (`sync`-Modus).
|
||||
- Einzelne neue/geänderte Bilder in bekannten Alben: periodische Eingangsprüfung, Standard 12 Stunden (`periodic`-Modus).
|
||||
- Administration: `Jetzt auf neue Bilder prüfen` (`manual`-Modus).
|
||||
- On-Demand bleibt parallel aktiv und wird nicht durch den Warmup ersetzt.
|
||||
|
||||
## Batches und Stufen
|
||||
|
||||
- Batchgröße ist konfigurierbar, Standard **10 Bilder**.
|
||||
- Ein Batch wird zuerst vollständig aus Nextcloud geladen.
|
||||
- Anschließend werden die Bilder einzeln am bereits bestehenden Piwigo-Pfad bereitgestellt und von Piwigo verarbeitet.
|
||||
- Stufe 1: von Piwigo definierte Standard-/Custom-Derivate bis einschließlich 1920 px längster Kante.
|
||||
- Erst wenn Stufe 1 abgearbeitet ist, folgt Stufe 2 mit den übrigen von Piwigo definierten Derivaten.
|
||||
|
||||
## Schutz produktiver Daten
|
||||
|
||||
Der Warmup darf keine Piwigo-Bild-/Albumdatensätze schreiben oder neue Piwigo-Sites/Zuordnungen anlegen.
|
||||
|
||||
Vor jedem Lauf wird die bestehende Pfadkette von Piwigo aus validiert:
|
||||
|
||||
`Piwigo image.id -> images.path -> realpath() -> nc-webdav-source/connection-ID/root-fileid -> webdav-map.json`
|
||||
|
||||
Unsichere oder widersprüchliche Zuordnungen führen zum Abbruch, nicht zu einer Rekonstruktion auf Verdacht.
|
||||
|
||||
Zusätzliche Sicherungen:
|
||||
|
||||
- vorhandener per-image Lock `upload/bratonien-webdav-materialize/image-ID.lock` wird auch vom Warmup benutzt;
|
||||
- `webdav-sync.lock` wird für den gesamten Warmup-Lauf geteilt gehalten; der Connector-Sync benötigt denselben Lock exklusiv;
|
||||
- zwei Piwigo-IDs auf derselben physischen Connector-Quelle führen zum Sicherheitsabbruch;
|
||||
- Placeholder wird nie in-place überschrieben;
|
||||
- Swap erfolgt über Dateien im selben Verzeichnis und `rename()`;
|
||||
- Größe und SHA-1 des Placeholders werden vor dem Swap erfasst;
|
||||
- nach jedem Restore müssen Größe und SHA-1 exakt dem Ausgangszustand entsprechen;
|
||||
- Restore-Fehler beendet den Worker sofort als fatal;
|
||||
- bestehende produktive Daten werden beim ersten Lauf nur als Baseline erfasst und nicht automatisch neu aufgebaut.
|
||||
|
||||
## Patchphase
|
||||
|
||||
Die automatische Warmup-Funktion ist in 0.9.7.1.8 standardmäßig **deaktiviert**. Vor Aktivierung sind vorgesehen:
|
||||
|
||||
1. Plugin aktualisieren;
|
||||
2. schreibgeschützten `Produktive Pfade prüfen`-Audit ausführen;
|
||||
3. ersten manuellen Lauf durchführen, der den vorhandenen Bestand nur als Baseline erfasst;
|
||||
4. gezielten Test mit neuem Testalbum / neuen Bildern;
|
||||
5. erst nach bestätigtem Restore- und Piwigo-Cache-Verhalten Automatik aktivieren.
|
||||
|
||||
Die Entwicklung bleibt bis zum vollständig validierten Endstand in Patch-Versionen. Die geplante Finalversion ist **0.10.1**.
|
||||
@@ -4,6 +4,8 @@ if (!defined('PHPWG_ROOT_PATH'))
|
||||
die('Hacking attempt!');
|
||||
}
|
||||
|
||||
require_once(BRATONIEN_TOOLS_PATH.'include/webdav_warmup_settings.inc.php');
|
||||
|
||||
function bratonien_tools_cache_worker_settings_file()
|
||||
{
|
||||
return PHPWG_ROOT_PATH.PWG_LOCAL_DIR.'bratonien-tools-cache-workers.json';
|
||||
@@ -136,6 +138,8 @@ function bratonien_tools_get_cache_worker_settings()
|
||||
$settings['auto_workers'] = min($settings['max_workers'], $settings['cpu_count']);
|
||||
$settings['manual_workers'] = min($settings['max_workers'], $settings['manual_workers']);
|
||||
$settings['worker_count'] = $settings['auto'] ? $settings['auto_workers'] : $settings['manual_workers'];
|
||||
$settings['webdav_warmup'] = bratonien_tools_get_webdav_warmup_settings();
|
||||
$settings['webdav_warmup']['status'] = bratonien_tools_get_webdav_warmup_status();
|
||||
return $settings;
|
||||
}
|
||||
|
||||
|
||||
@@ -23,6 +23,7 @@ require_once(BRATONIEN_TOOLS_PATH . 'include/nc_connector_connection_scope.inc.p
|
||||
require_once(BRATONIEN_TOOLS_PATH . 'include/nc_connector_delete_safe.inc.php');
|
||||
require_once(BRATONIEN_TOOLS_PATH . 'include/nc_connector_webdav.inc.php');
|
||||
require_once(BRATONIEN_TOOLS_PATH . 'include/nc_connector_wizard_webdav_flow.inc.php');
|
||||
require_once(BRATONIEN_TOOLS_PATH . 'include/webdav_warmup_settings.inc.php');
|
||||
|
||||
function bratonien_tools_get_tools()
|
||||
{
|
||||
@@ -31,6 +32,9 @@ function bratonien_tools_get_tools()
|
||||
'image_cache_build' => array('handler' => 'bratonien_tools_start_main_cache_build'),
|
||||
'image_cache_cancel' => array('handler' => 'bratonien_tools_cancel_main_cache_build'),
|
||||
'image_cache_worker_settings' => array('handler' => 'bratonien_tools_save_cache_worker_settings'),
|
||||
'image_cache_webdav_warmup_settings' => array('handler' => 'bratonien_tools_save_webdav_warmup_settings'),
|
||||
'image_cache_webdav_warmup_manual' => array('handler' => 'bratonien_tools_start_webdav_warmup_manual'),
|
||||
'image_cache_webdav_warmup_audit' => array('handler' => 'bratonien_tools_run_webdav_warmup_audit'),
|
||||
'watermark_save' => array('handler' => 'bratonien_tools_save_watermark'),
|
||||
'watermark_file_delete' => array('handler' => 'bratonien_tools_delete_watermark_file'),
|
||||
'watermark_engine' => array('handler' => 'bratonien_tools_handle_watermark_engine'),
|
||||
|
||||
150
include/webdav_warmup_settings.inc.php
Normal file
150
include/webdav_warmup_settings.inc.php
Normal file
@@ -0,0 +1,150 @@
|
||||
<?php
|
||||
if (!defined('PHPWG_ROOT_PATH'))
|
||||
{
|
||||
die('Hacking attempt!');
|
||||
}
|
||||
|
||||
function bratonien_tools_webdav_warmup_settings_file()
|
||||
{
|
||||
return PHPWG_ROOT_PATH.PWG_LOCAL_DIR.'bratonien-webdav-warmup.settings.json';
|
||||
}
|
||||
|
||||
function bratonien_tools_get_webdav_warmup_settings()
|
||||
{
|
||||
$settings = array(
|
||||
'enabled'=>false,
|
||||
'batch_size'=>10,
|
||||
'periodic_hours'=>12,
|
||||
);
|
||||
|
||||
$file = bratonien_tools_webdav_warmup_settings_file();
|
||||
if (is_file($file) && is_readable($file))
|
||||
{
|
||||
$raw = @file_get_contents($file);
|
||||
$saved = $raw !== false ? json_decode($raw, true) : null;
|
||||
if (is_array($saved))
|
||||
{
|
||||
if (array_key_exists('enabled', $saved)) $settings['enabled'] = (bool)$saved['enabled'];
|
||||
if (isset($saved['batch_size'])) $settings['batch_size'] = max(1, min(50, (int)$saved['batch_size']));
|
||||
if (isset($saved['periodic_hours'])) $settings['periodic_hours'] = max(1, min(168, (int)$saved['periodic_hours']));
|
||||
}
|
||||
}
|
||||
return $settings;
|
||||
}
|
||||
|
||||
function bratonien_tools_save_webdav_warmup_settings()
|
||||
{
|
||||
$current = bratonien_tools_get_webdav_warmup_settings();
|
||||
$enabled = !empty($_POST['webdav_warmup_enabled']);
|
||||
$batch_size = isset($_POST['webdav_warmup_batch_size']) ? (int)$_POST['webdav_warmup_batch_size'] : (int)$current['batch_size'];
|
||||
$periodic_hours = isset($_POST['webdav_warmup_periodic_hours']) ? (int)$_POST['webdav_warmup_periodic_hours'] : (int)$current['periodic_hours'];
|
||||
$batch_size = max(1, min(50, $batch_size));
|
||||
$periodic_hours = max(1, min(168, $periodic_hours));
|
||||
|
||||
$payload = array(
|
||||
'enabled'=>$enabled,
|
||||
'batch_size'=>$batch_size,
|
||||
'periodic_hours'=>$periodic_hours,
|
||||
);
|
||||
$json = json_encode($payload, JSON_UNESCAPED_UNICODE | JSON_UNESCAPED_SLASHES | JSON_PRETTY_PRINT);
|
||||
if ($json === false) throw new RuntimeException('Warmup-Einstellungen konnten nicht serialisiert werden.');
|
||||
|
||||
$file = bratonien_tools_webdav_warmup_settings_file();
|
||||
$directory = dirname($file);
|
||||
if (!is_dir($directory) && !@mkdir($directory, 0775, true) && !is_dir($directory))
|
||||
{
|
||||
throw new RuntimeException('Warmup-Einstellungsverzeichnis konnte nicht angelegt werden.');
|
||||
}
|
||||
$tmp = $file.'.tmp-'.bin2hex(random_bytes(4));
|
||||
if (@file_put_contents($tmp, $json."\n", LOCK_EX) === false)
|
||||
{
|
||||
throw new RuntimeException('Warmup-Einstellungen konnten nicht gespeichert werden.');
|
||||
}
|
||||
@chmod($tmp, 0664);
|
||||
if (!@rename($tmp, $file))
|
||||
{
|
||||
@unlink($tmp);
|
||||
throw new RuntimeException('Warmup-Einstellungen konnten nicht atomar gespeichert werden.');
|
||||
}
|
||||
|
||||
return array('message'=>sprintf(
|
||||
'WebDAV-Cache-Warmup gespeichert: %s, %d Bilder pro Batch, Eingangsprüfung alle %d Stunden.',
|
||||
$enabled ? 'automatisch aktiv' : 'Automatik deaktiviert',
|
||||
$batch_size,
|
||||
$periodic_hours
|
||||
));
|
||||
}
|
||||
|
||||
function bratonien_tools_webdav_warmup_php_cli()
|
||||
{
|
||||
if (function_exists('bratonien_tools_find_php_cli'))
|
||||
{
|
||||
$php = bratonien_tools_find_php_cli();
|
||||
if ($php) return $php;
|
||||
}
|
||||
foreach (array('/usr/bin/php', '/usr/bin/php8.4', '/usr/bin/php8.3', '/usr/local/bin/php') as $candidate)
|
||||
{
|
||||
if (is_file($candidate) && is_executable($candidate)) return $candidate;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function bratonien_tools_start_webdav_warmup_manual()
|
||||
{
|
||||
if (!function_exists('exec')) throw new RuntimeException('PHP exec() ist deaktiviert; Warmup kann nicht gestartet werden.');
|
||||
$php = bratonien_tools_webdav_warmup_php_cli();
|
||||
if (!$php) throw new RuntimeException('PHP-CLI wurde für den WebDAV-Warmup nicht gefunden.');
|
||||
$dispatcher = realpath(BRATONIEN_TOOLS_PATH.'runtime/webdav-warmup-dispatch.php');
|
||||
if (!$dispatcher || !is_file($dispatcher)) throw new RuntimeException('WebDAV-Warmup-Dispatcher wurde nicht gefunden.');
|
||||
|
||||
$log = PHPWG_ROOT_PATH.PWG_LOCAL_DIR.'bratonien-webdav-warmup-dispatch.log';
|
||||
$command = 'nohup '.escapeshellarg($php).' '.escapeshellarg($dispatcher).' --mode=manual >> '.escapeshellarg($log).' 2>&1 < /dev/null & echo $!';
|
||||
$output = array();
|
||||
$exit = 1;
|
||||
@exec($command, $output, $exit);
|
||||
$pid = isset($output[0]) ? (int)$output[0] : 0;
|
||||
if ($exit !== 0 || $pid <= 0) throw new RuntimeException('Manuelle Warmup-Prüfung konnte nicht gestartet werden.');
|
||||
|
||||
return array('message'=>'WebDAV-Warmup: Prüfung auf neue oder geänderte Bilder wurde gestartet.');
|
||||
}
|
||||
|
||||
function bratonien_tools_run_webdav_warmup_audit()
|
||||
{
|
||||
if (!function_exists('exec')) throw new RuntimeException('PHP exec() ist deaktiviert; Pfadaudit kann nicht gestartet werden.');
|
||||
$php = bratonien_tools_webdav_warmup_php_cli();
|
||||
if (!$php) throw new RuntimeException('PHP-CLI wurde für den Pfadaudit nicht gefunden.');
|
||||
$audit = realpath(BRATONIEN_TOOLS_PATH.'runtime/lib/webdav-warmup-audit.php');
|
||||
if (!$audit || !is_file($audit)) throw new RuntimeException('WebDAV-Warmup-Pfadaudit wurde nicht gefunden.');
|
||||
|
||||
$output = array();
|
||||
$exit = 1;
|
||||
@exec(escapeshellarg($php).' '.escapeshellarg($audit).' 2>&1', $output, $exit);
|
||||
$text = trim(implode(' | ', array_slice($output, -8)));
|
||||
if ($exit !== 0)
|
||||
{
|
||||
throw new RuntimeException('WebDAV-Pfadaudit meldet einen unsicheren Zustand'.($text !== '' ? ': '.$text : '.'));
|
||||
}
|
||||
return array('message'=>'WebDAV-Pfadaudit erfolgreich'.($text !== '' ? ': '.$text : '.'));
|
||||
}
|
||||
|
||||
function bratonien_tools_get_webdav_warmup_status()
|
||||
{
|
||||
$status = array(
|
||||
'state'=>'idle',
|
||||
'message'=>'Noch kein Warmup-Lauf protokolliert.',
|
||||
'updated_at'=>0,
|
||||
);
|
||||
$files = glob(PHPWG_ROOT_PATH.PWG_LOCAL_DIR.'bratonien-webdav-warmup.status-*.json');
|
||||
if (!$files) return $status;
|
||||
|
||||
$latest = null;
|
||||
foreach ($files as $file)
|
||||
{
|
||||
if (!is_readable($file)) continue;
|
||||
$raw = @file_get_contents($file);
|
||||
$data = $raw !== false ? json_decode($raw, true) : null;
|
||||
if (!is_array($data)) continue;
|
||||
if ($latest === null || (int)($data['updated_at'] ?? 0) > (int)($latest['updated_at'] ?? 0)) $latest = $data;
|
||||
}
|
||||
return is_array($latest) ? array_merge($status, $latest) : $status;
|
||||
}
|
||||
@@ -1,7 +1,7 @@
|
||||
<?php
|
||||
/*
|
||||
Plugin Name: Bratonien Tools
|
||||
Version: 0.9.7.1.7
|
||||
Version: 0.9.7.1.8
|
||||
Description: Erweiterbare Administrationswerkzeuge fuer die Bratonien-Piwigo-Installation.
|
||||
Plugin URI: https://github.com/Terranom674/Piwigo_Bratonien_Tools
|
||||
Author: Bratonien
|
||||
|
||||
46
runtime/lib/piwigo-derivative-call.php
Normal file
46
runtime/lib/piwigo-derivative-call.php
Normal file
@@ -0,0 +1,46 @@
|
||||
<?php
|
||||
if (PHP_SAPI !== 'cli')
|
||||
{
|
||||
http_response_code(404);
|
||||
exit;
|
||||
}
|
||||
|
||||
$request = '';
|
||||
foreach ($argv as $arg)
|
||||
{
|
||||
if (strpos($arg, '--request=') === 0)
|
||||
{
|
||||
$request = substr($arg, strlen('--request='));
|
||||
}
|
||||
}
|
||||
if ($request === '' || strpos($request, '/i.php?/') !== 0)
|
||||
{
|
||||
fwrite(STDERR, "Ungültige Piwigo-Derivatanforderung.\n");
|
||||
exit(2);
|
||||
}
|
||||
|
||||
$piwigo_root = realpath(dirname(__DIR__, 3));
|
||||
if ($piwigo_root === false || !is_file($piwigo_root.'/i.php'))
|
||||
{
|
||||
fwrite(STDERR, "Piwigo i.php wurde nicht gefunden.\n");
|
||||
exit(3);
|
||||
}
|
||||
|
||||
chdir($piwigo_root);
|
||||
$_SERVER['REMOTE_ADDR'] = '127.0.0.1';
|
||||
$_SERVER['SERVER_ADDR'] = '127.0.0.1';
|
||||
$_SERVER['SERVER_NAME'] = 'localhost';
|
||||
$_SERVER['HTTP_HOST'] = 'localhost';
|
||||
$_SERVER['SERVER_PORT'] = '80';
|
||||
$_SERVER['REQUEST_METHOD'] = 'GET';
|
||||
$_SERVER['REQUEST_URI'] = $request;
|
||||
$_SERVER['SCRIPT_NAME'] = '/i.php';
|
||||
$_SERVER['PHP_SELF'] = '/i.php';
|
||||
$_SERVER['QUERY_STRING'] = ltrim((string)parse_url($request, PHP_URL_QUERY), '?');
|
||||
$_SERVER['HTTP_USER_AGENT'] = 'Bratonien-WebDAV-Warmup-Piwigo-Caller/0.9.7.1.8';
|
||||
$_SERVER['HTTPS'] = 'off';
|
||||
|
||||
// Piwigo selbst erzeugt das Derivat. Der Warmup benötigt den ausgelieferten
|
||||
// Bild-Body nicht und prüft danach ausschließlich Piwigos Cache-Datei.
|
||||
ob_start(function ($buffer) { return ''; }, 1);
|
||||
include $piwigo_root.'/i.php';
|
||||
58
runtime/lib/run-webdav-cache-warmup.sh
Normal file
58
runtime/lib/run-webdav-cache-warmup.sh
Normal file
@@ -0,0 +1,58 @@
|
||||
#!/usr/bin/env bash
|
||||
set -Eeuo pipefail
|
||||
|
||||
STATE_DIR="${1:-}"
|
||||
shift || true
|
||||
|
||||
[[ -n "$STATE_DIR" ]] || { echo "Connector-State-Verzeichnis fehlt." >&2; exit 2; }
|
||||
[[ $# -gt 0 ]] || { echo "Warmup-Kommando fehlt." >&2; exit 2; }
|
||||
|
||||
mkdir -p -- "$STATE_DIR"
|
||||
LOCK_FILE="$STATE_DIR/webdav-sync.lock"
|
||||
PRIORITY_FILE="$STATE_DIR/webdav-cache-warmup-priority-sync"
|
||||
|
||||
# Der Connector-Sync nutzt denselben Lock exklusiv. Der Warmup hält ihn für
|
||||
# seinen gesamten Lauf nur lesend/geteilt. Dadurch kann der Placeholder- und
|
||||
# Shadow-Tree während Batch-Download, Piwigo-Aufruf und Restore nicht atomar
|
||||
# unter dem Warmup ausgetauscht werden.
|
||||
exec 9>"$LOCK_FILE"
|
||||
flock -s 9
|
||||
|
||||
COMMAND=("$@")
|
||||
|
||||
set +e
|
||||
"${COMMAND[@]}"
|
||||
result=$?
|
||||
set -e
|
||||
|
||||
# Ein Connector-Sync kann während eines bereits laufenden Warmups neue Alben
|
||||
# melden. Der Dispatcher legt dafür nur eine Prioritätsmarke an; er startet
|
||||
# keinen zweiten konkurrierenden Worker. Falls der aktive Lauf die Marke noch
|
||||
# nicht selbst übernehmen konnte, wird sie hier nach seinem sauberen Ende
|
||||
# garantiert nachgeholt. Dadurch geht ein Sofort-Warmup nie verloren.
|
||||
if [[ -f "$PRIORITY_FILE" ]]; then
|
||||
rm -f -- "$PRIORITY_FILE"
|
||||
SYNC_COMMAND=()
|
||||
replaced=0
|
||||
for arg in "${COMMAND[@]}"; do
|
||||
if [[ "$arg" == --mode=* ]]; then
|
||||
SYNC_COMMAND+=("--mode=sync")
|
||||
replaced=1
|
||||
else
|
||||
SYNC_COMMAND+=("$arg")
|
||||
fi
|
||||
done
|
||||
if [[ "$replaced" -eq 0 ]]; then
|
||||
SYNC_COMMAND+=("--mode=sync")
|
||||
fi
|
||||
|
||||
set +e
|
||||
"${SYNC_COMMAND[@]}"
|
||||
sync_result=$?
|
||||
set -e
|
||||
if [[ "$sync_result" -ne 0 ]]; then
|
||||
result="$sync_result"
|
||||
fi
|
||||
fi
|
||||
|
||||
exit "$result"
|
||||
776
runtime/lib/webdav-cache-warmup.php
Normal file
776
runtime/lib/webdav-cache-warmup.php
Normal file
@@ -0,0 +1,776 @@
|
||||
<?php
|
||||
if (PHP_SAPI !== 'cli')
|
||||
{
|
||||
http_response_code(404);
|
||||
exit;
|
||||
}
|
||||
|
||||
$piwigo_root = realpath(dirname(__DIR__, 3));
|
||||
if ($piwigo_root === false)
|
||||
{
|
||||
fwrite(STDERR, "Piwigo-Root wurde nicht gefunden.\n");
|
||||
exit(1);
|
||||
}
|
||||
define('PHPWG_ROOT_PATH', rtrim($piwigo_root, DIRECTORY_SEPARATOR).DIRECTORY_SEPARATOR);
|
||||
|
||||
$_SERVER['REMOTE_ADDR'] = '127.0.0.1';
|
||||
$_SERVER['SERVER_ADDR'] = '127.0.0.1';
|
||||
$_SERVER['SERVER_NAME'] = 'localhost';
|
||||
$_SERVER['HTTP_HOST'] = 'localhost';
|
||||
$_SERVER['SERVER_PORT'] = '80';
|
||||
$_SERVER['REQUEST_METHOD'] = 'GET';
|
||||
$_SERVER['REQUEST_URI'] = '/';
|
||||
$_SERVER['SCRIPT_NAME'] = '/plugins/bratonien_tools/runtime/lib/webdav-cache-warmup.php';
|
||||
$_SERVER['PHP_SELF'] = $_SERVER['SCRIPT_NAME'];
|
||||
$_SERVER['QUERY_STRING'] = '';
|
||||
$_SERVER['HTTP_USER_AGENT'] = 'Bratonien-WebDAV-Cache-Warmup/0.9.7.1.8';
|
||||
$_SERVER['HTTPS'] = 'off';
|
||||
|
||||
require_once(PHPWG_ROOT_PATH.'include/common.inc.php');
|
||||
require_once(PHPWG_ROOT_PATH.'include/derivative.inc.php');
|
||||
if (!defined('BRATONIEN_TOOLS_PATH'))
|
||||
{
|
||||
fwrite(STDERR, "Bratonien Tools ist nicht aktiv.\n");
|
||||
exit(1);
|
||||
}
|
||||
|
||||
require_once(BRATONIEN_TOOLS_PATH.'include/nc_connector.inc.php');
|
||||
require_once(BRATONIEN_TOOLS_PATH.'include/webdav_materialize_runtime.inc.php');
|
||||
require_once(BRATONIEN_TOOLS_PATH.'include/webdav_warmup_settings.inc.php');
|
||||
|
||||
function bratonien_tools_cache_warmup_log($event, array $fields=array())
|
||||
{
|
||||
$parts = array('[BRAT-WARMUP]', $event);
|
||||
foreach ($fields as $key=>$value)
|
||||
{
|
||||
if (is_bool($value)) $value = $value ? '1' : '0';
|
||||
elseif ($value === null) $value = 'NULL';
|
||||
elseif (is_array($value) || is_object($value)) $value = json_encode($value, JSON_UNESCAPED_SLASHES);
|
||||
$parts[] = $key.'='.str_replace(array("\r", "\n"), array('\\r', '\\n'), (string)$value);
|
||||
}
|
||||
fwrite(STDOUT, implode(' ', $parts)."\n");
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_state_file($connection_id)
|
||||
{
|
||||
return PHPWG_ROOT_PATH.PWG_LOCAL_DIR.'bratonien-webdav-warmup.connection-'.(int)$connection_id.'.json';
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_status_file($connection_id)
|
||||
{
|
||||
return PHPWG_ROOT_PATH.PWG_LOCAL_DIR.'bratonien-webdav-warmup.status-'.(int)$connection_id.'.json';
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_write_json($file, array $payload)
|
||||
{
|
||||
$directory = dirname($file);
|
||||
if (!is_dir($directory) && !@mkdir($directory, 0775, true) && !is_dir($directory))
|
||||
{
|
||||
throw new RuntimeException('Warmup-State-Verzeichnis konnte nicht angelegt werden.');
|
||||
}
|
||||
$json = json_encode($payload, JSON_UNESCAPED_UNICODE | JSON_UNESCAPED_SLASHES | JSON_PRETTY_PRINT);
|
||||
if ($json === false) throw new RuntimeException('Warmup-State konnte nicht serialisiert werden.');
|
||||
$tmp = $file.'.tmp-'.bin2hex(random_bytes(4));
|
||||
if (@file_put_contents($tmp, $json."\n", LOCK_EX) === false) throw new RuntimeException('Warmup-State konnte nicht geschrieben werden.');
|
||||
@chmod($tmp, 0664);
|
||||
if (!@rename($tmp, $file))
|
||||
{
|
||||
@unlink($tmp);
|
||||
throw new RuntimeException('Warmup-State konnte nicht atomar gespeichert werden.');
|
||||
}
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_status($connection_id, $state, $message, array $extra=array())
|
||||
{
|
||||
bratonien_tools_cache_warmup_write_json(
|
||||
bratonien_tools_cache_warmup_status_file($connection_id),
|
||||
array_merge(array(
|
||||
'state'=>$state,
|
||||
'message'=>$message,
|
||||
'connection_id'=>(int)$connection_id,
|
||||
'updated_at'=>time(),
|
||||
), $extra)
|
||||
);
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_load_state($connection_id)
|
||||
{
|
||||
$file = bratonien_tools_cache_warmup_state_file($connection_id);
|
||||
if (!is_file($file) || !is_readable($file)) return null;
|
||||
$raw = @file_get_contents($file);
|
||||
$state = $raw !== false ? json_decode($raw, true) : null;
|
||||
return is_array($state) ? $state : null;
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_save_state($connection_id, array $state)
|
||||
{
|
||||
$state['updated_at'] = time();
|
||||
if (!isset($state['albums']) || !is_array($state['albums'])) $state['albums'] = array();
|
||||
if (!isset($state['images']) || !is_array($state['images'])) $state['images'] = array();
|
||||
bratonien_tools_cache_warmup_write_json(bratonien_tools_cache_warmup_state_file($connection_id), $state);
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_credentials($connection_id)
|
||||
{
|
||||
$connection = bratonien_tools_nc_connector_connection((int)$connection_id, true);
|
||||
if (!$connection || empty($connection['enabled'])) return null;
|
||||
$plain = bratonien_tools_nc_connector_decrypt_secret((string)($connection['secret_blob'] ?? ''));
|
||||
$secret = json_decode($plain, true);
|
||||
if (!is_array($secret)) return null;
|
||||
|
||||
$config = isset($connection['config']) && is_array($connection['config']) ? $connection['config'] : array();
|
||||
$base_url = rtrim((string)($config['nextcloud_url'] ?? ''), '/');
|
||||
$user = trim((string)($secret['nextcloud_user'] ?? ''));
|
||||
$password = (string)($secret['nextcloud_password'] ?? '');
|
||||
if ($base_url === '' || $user === '' || $password === '') return null;
|
||||
|
||||
return array(
|
||||
'connection'=>$connection,
|
||||
'config'=>$config,
|
||||
'base_url'=>$base_url,
|
||||
'user'=>$user,
|
||||
'password'=>$password,
|
||||
);
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_absolute_path($path)
|
||||
{
|
||||
$path = (string)$path;
|
||||
if ($path === '') return null;
|
||||
if (strpos($path, '/') === 0) return $path;
|
||||
return PHPWG_ROOT_PATH.ltrim(preg_replace('#^\./#', '', $path), '/');
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_signature(array $source)
|
||||
{
|
||||
return sha1(implode('|', array(
|
||||
(int)($source['connection_id'] ?? 0),
|
||||
(int)($source['root_fileid'] ?? 0),
|
||||
(int)($source['fileid'] ?? 0),
|
||||
trim((string)($source['webdav_path'] ?? ''), '/'),
|
||||
(string)($source['etag'] ?? ''),
|
||||
(int)($source['size'] ?? 0),
|
||||
(int)($source['width'] ?? 0),
|
||||
(int)($source['height'] ?? 0),
|
||||
)));
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_scan($connection_id)
|
||||
{
|
||||
$images = array();
|
||||
$albums = array();
|
||||
$resolved = array();
|
||||
|
||||
$result = pwg_query('SELECT id, path, width, height, rotation, coi FROM '.IMAGES_TABLE.' ORDER BY id');
|
||||
while ($row = pwg_db_fetch_assoc($result))
|
||||
{
|
||||
$image_id = (int)$row['id'];
|
||||
$source = bratonien_tools_webdav_materialize_source_info($image_id);
|
||||
if (!$source || (int)$source['connection_id'] !== (int)$connection_id) continue;
|
||||
|
||||
$logical = bratonien_tools_cache_warmup_absolute_path($row['path'] ?? '');
|
||||
$real = $logical ? realpath($logical) : false;
|
||||
if ($real === false || !is_file($real))
|
||||
{
|
||||
throw new RuntimeException('Bild #'.$image_id.': bestehender Piwigo-Pfad kann nicht aufgelöst werden.');
|
||||
}
|
||||
$normalized = str_replace('\\', '/', $real);
|
||||
$expected = '/nc-webdav-source/connection-'.(int)$source['connection_id'].'/root-'.(int)$source['root_fileid'].'/';
|
||||
if (strpos($normalized, $expected) === false)
|
||||
{
|
||||
throw new RuntimeException('Bild #'.$image_id.': bestehender Piwigo-Pfad zeigt nicht auf die erwartete Connector-Wurzel.');
|
||||
}
|
||||
if (trim((string)($source['webdav_path'] ?? ''), '/') === '')
|
||||
{
|
||||
throw new RuntimeException('Bild #'.$image_id.': WebDAV-Mapping enthält keinen Pfad.');
|
||||
}
|
||||
|
||||
if (isset($resolved[$normalized]) && $resolved[$normalized] !== $image_id)
|
||||
{
|
||||
throw new RuntimeException('Sicherheitsabbruch: Bild #'.$resolved[$normalized].' und Bild #'.$image_id.' zeigen auf dieselbe physische Connector-Quelle.');
|
||||
}
|
||||
$resolved[$normalized] = $image_id;
|
||||
|
||||
$row['id'] = $image_id;
|
||||
$images[$image_id] = array(
|
||||
'row'=>$row,
|
||||
'source'=>$source,
|
||||
'logical'=>$logical,
|
||||
'resolved'=>$real,
|
||||
'signature'=>bratonien_tools_cache_warmup_signature($source),
|
||||
'albums'=>array(),
|
||||
);
|
||||
}
|
||||
|
||||
if ($images)
|
||||
{
|
||||
foreach (array_chunk(array_keys($images), 500) as $chunk)
|
||||
{
|
||||
$result = pwg_query('SELECT image_id, category_id FROM '.IMAGE_CATEGORY_TABLE.' WHERE image_id IN ('.implode(',', array_map('intval', $chunk)).')');
|
||||
while ($rel = pwg_db_fetch_assoc($result))
|
||||
{
|
||||
$image_id = (int)$rel['image_id'];
|
||||
$category_id = (int)$rel['category_id'];
|
||||
if (!isset($images[$image_id]) || $category_id < 1) continue;
|
||||
$images[$image_id]['albums'][$category_id] = true;
|
||||
$albums[$category_id] = true;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
foreach ($images as &$image) $image['albums'] = array_map('intval', array_keys($image['albums']));
|
||||
unset($image);
|
||||
return array('images'=>$images, 'albums'=>array_map('intval', array_keys($albums)));
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_baseline($connection_id, array $scan)
|
||||
{
|
||||
$state = array('albums'=>array(), 'images'=>array(), 'last_periodic_at'=>time());
|
||||
foreach ($scan['albums'] as $album_id) $state['albums'][(string)$album_id] = array('seen_at'=>time());
|
||||
foreach ($scan['images'] as $image_id=>$image)
|
||||
{
|
||||
$state['images'][(string)$image_id] = array(
|
||||
'stage1_signature'=>$image['signature'],
|
||||
'stage2_signature'=>$image['signature'],
|
||||
'baseline'=>true,
|
||||
);
|
||||
}
|
||||
bratonien_tools_cache_warmup_save_state($connection_id, $state);
|
||||
return $state;
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_select(array $scan, array $state, $mode)
|
||||
{
|
||||
$known_albums = array_fill_keys(array_map('intval', array_keys((array)($state['albums'] ?? array()))), true);
|
||||
$new_albums = array();
|
||||
foreach ($scan['albums'] as $album_id)
|
||||
{
|
||||
if (!isset($known_albums[(int)$album_id])) $new_albums[(int)$album_id] = true;
|
||||
}
|
||||
|
||||
$selected = array();
|
||||
foreach ($scan['images'] as $image_id=>$image)
|
||||
{
|
||||
$saved = isset($state['images'][(string)$image_id]) && is_array($state['images'][(string)$image_id])
|
||||
? $state['images'][(string)$image_id]
|
||||
: array();
|
||||
$stage1_done = isset($saved['stage1_signature']) && hash_equals((string)$saved['stage1_signature'], $image['signature']);
|
||||
$stage2_done = isset($saved['stage2_signature']) && hash_equals((string)$saved['stage2_signature'], $image['signature']);
|
||||
|
||||
$in_new_album = false;
|
||||
foreach ($image['albums'] as $album_id)
|
||||
{
|
||||
if (isset($new_albums[(int)$album_id])) { $in_new_album = true; break; }
|
||||
}
|
||||
|
||||
if ($mode === 'sync')
|
||||
{
|
||||
if ($in_new_album && (!$stage1_done || !$stage2_done)) $selected[$image_id] = $image;
|
||||
}
|
||||
else
|
||||
{
|
||||
if (!$stage1_done || !$stage2_done) $selected[$image_id] = $image;
|
||||
}
|
||||
}
|
||||
|
||||
return array('images'=>$selected, 'new_albums'=>array_map('intval', array_keys($new_albums)));
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_variant_rows(array $image, $stage)
|
||||
{
|
||||
$src = new SrcImage($image['row']);
|
||||
$variants = array();
|
||||
foreach (ImageStdParams::get_defined_type_map() as $type=>$params)
|
||||
{
|
||||
$derivative = new DerivativeImage($params, $src);
|
||||
if ($derivative->same_as_source()) continue;
|
||||
$size = $derivative->get_size();
|
||||
$max_edge = max((int)($size[0] ?? 0), (int)($size[1] ?? 0));
|
||||
$priority = $max_edge > 0 && $max_edge <= 1920;
|
||||
if (($stage === 1 && $priority) || ($stage === 2 && !$priority))
|
||||
{
|
||||
$variants[] = array('name'=>'standard:'.$type, 'target'=>$derivative->get_path());
|
||||
}
|
||||
}
|
||||
foreach (ImageStdParams::$custom as $key=>$last_used)
|
||||
{
|
||||
$params = bratonien_tools_webdav_materialize_custom_params($key);
|
||||
if (!$params) continue;
|
||||
$derivative = new DerivativeImage($params, $src);
|
||||
if ($derivative->same_as_source()) continue;
|
||||
$size = $derivative->get_size();
|
||||
$max_edge = max((int)($size[0] ?? 0), (int)($size[1] ?? 0));
|
||||
$priority = $max_edge > 0 && $max_edge <= 1920;
|
||||
if (($stage === 1 && $priority) || ($stage === 2 && !$priority))
|
||||
{
|
||||
$variants[] = array('name'=>'custom:'.$key, 'target'=>$derivative->get_path());
|
||||
}
|
||||
}
|
||||
return $variants;
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_i_request($target)
|
||||
{
|
||||
$root = PHPWG_ROOT_PATH.PWG_DERIVATIVE_DIR;
|
||||
if (strpos($target, $root) !== 0) return null;
|
||||
$relative = ltrim(substr($target, strlen($root)), '/');
|
||||
if ($relative === '') return null;
|
||||
return '/i.php?/'.implode('/', array_map('rawurlencode', explode('/', $relative)));
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_call_piwigo($request, &$detail=null)
|
||||
{
|
||||
$detail = '';
|
||||
$runner = BRATONIEN_TOOLS_PATH.'runtime/lib/piwigo-derivative-call.php';
|
||||
if (!is_file($runner)) { $detail = 'Piwigo-Derivataufruf fehlt.'; return false; }
|
||||
if (!function_exists('exec')) { $detail = 'PHP exec() ist deaktiviert.'; return false; }
|
||||
$output = array();
|
||||
$exit = 1;
|
||||
$command = escapeshellarg(PHP_BINARY).' '.escapeshellarg($runner).' --request='.escapeshellarg($request).' 2>&1';
|
||||
@exec($command, $output, $exit);
|
||||
if ($exit !== 0)
|
||||
{
|
||||
$detail = 'Piwigo-Aufruf Exit '.$exit;
|
||||
if ($output) $detail .= ': '.implode(' ', array_slice($output, -3));
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_quote_path($path)
|
||||
{
|
||||
$parts = array_values(array_filter(explode('/', trim((string)$path, '/')), 'strlen'));
|
||||
return implode('/', array_map('rawurlencode', $parts));
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_download(array $source, array $credentials, $destination, &$detail=null)
|
||||
{
|
||||
$detail = '';
|
||||
$webdav_path = trim((string)($source['webdav_path'] ?? ''), '/');
|
||||
if ($webdav_path === '') { $detail = 'WebDAV-Pfad fehlt.'; return false; }
|
||||
$fp = @fopen($destination, 'xb');
|
||||
if (!$fp) { $detail = 'Temporäre Quelldatei konnte nicht angelegt werden.'; return false; }
|
||||
$url = $credentials['base_url'].'/remote.php/dav/files/'.rawurlencode($credentials['user']).'/'.bratonien_tools_cache_warmup_quote_path($webdav_path);
|
||||
$ch = curl_init($url);
|
||||
curl_setopt_array($ch, array(
|
||||
CURLOPT_FOLLOWLOCATION=>false,
|
||||
CURLOPT_CONNECTTIMEOUT=>10,
|
||||
CURLOPT_TIMEOUT=>300,
|
||||
CURLOPT_HTTPAUTH=>CURLAUTH_BASIC,
|
||||
CURLOPT_USERPWD=>$credentials['user'].':'.$credentials['password'],
|
||||
CURLOPT_RETURNTRANSFER=>false,
|
||||
CURLOPT_FAILONERROR=>false,
|
||||
CURLOPT_FILE=>$fp,
|
||||
CURLOPT_USERAGENT=>'Bratonien-WebDAV-Cache-Warmup/0.9.7.1.8',
|
||||
));
|
||||
$ok = curl_exec($ch);
|
||||
$errno = curl_errno($ch);
|
||||
$error = curl_error($ch);
|
||||
$http = (int)curl_getinfo($ch, CURLINFO_HTTP_CODE);
|
||||
curl_close($ch);
|
||||
fclose($fp);
|
||||
if ($ok === false || $errno !== 0 || $http < 200 || $http >= 300)
|
||||
{
|
||||
@unlink($destination);
|
||||
$detail = 'Nextcloud-Download fehlgeschlagen (HTTP '.$http.', cURL '.$errno.($error !== '' ? ': '.$error : '').').';
|
||||
return false;
|
||||
}
|
||||
if (!is_file($destination) || filesize($destination) < 1 || @getimagesize($destination) === false)
|
||||
{
|
||||
@unlink($destination);
|
||||
$detail = 'Geladene Datei ist kein lesbares vollständiges Bild.';
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_sync_guard(array $credentials, &$handle=null, &$detail=null)
|
||||
{
|
||||
$handle = null;
|
||||
$detail = '';
|
||||
$state_dir = rtrim((string)($credentials['config']['state_dir'] ?? ''), '/');
|
||||
if ($state_dir === '') { $detail = 'Connector-State-Verzeichnis fehlt.'; return false; }
|
||||
$handle = @fopen($state_dir.'/webdav-sync.lock', 'c');
|
||||
if (!$handle) { $detail = 'Connector-Sync-Lock konnte nicht geöffnet werden.'; return false; }
|
||||
if (!@flock($handle, LOCK_SH | LOCK_NB))
|
||||
{
|
||||
fclose($handle);
|
||||
$handle = null;
|
||||
$detail = 'Connector-Synchronisierung läuft.';
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_image_lock($image_id, &$handle=null)
|
||||
{
|
||||
$dir = PHPWG_ROOT_PATH.'upload/bratonien-webdav-materialize';
|
||||
if (!is_dir($dir) && !@mkdir($dir, 0775, true) && !is_dir($dir)) return false;
|
||||
$handle = @fopen($dir.'/image-'.(int)$image_id.'.lock', 'c');
|
||||
if (!$handle) return false;
|
||||
return @flock($handle, LOCK_EX);
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_unlock(&$handle)
|
||||
{
|
||||
if (is_resource($handle))
|
||||
{
|
||||
@flock($handle, LOCK_UN);
|
||||
fclose($handle);
|
||||
}
|
||||
$handle = null;
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_restore($source_path, $backup, $staging, $original_size, $original_hash, &$detail=null)
|
||||
{
|
||||
$detail = '';
|
||||
@unlink($staging);
|
||||
if (is_file($source_path) || is_link($source_path))
|
||||
{
|
||||
if (!@unlink($source_path)) { $detail = 'Temporäre Quelle konnte nicht entfernt werden.'; return false; }
|
||||
}
|
||||
if (!is_file($backup)) { $detail = 'Placeholder-Backup fehlt.'; return false; }
|
||||
if (!@rename($backup, $source_path)) { $detail = 'Placeholder konnte nicht restauriert werden.'; return false; }
|
||||
clearstatcache(true, $source_path);
|
||||
if (!is_file($source_path)) { $detail = 'Restaurierter Placeholder fehlt.'; return false; }
|
||||
if ((int)filesize($source_path) !== (int)$original_size) { $detail = 'Restaurierter Placeholder hat eine unerwartete Größe.'; return false; }
|
||||
$hash = @sha1_file($source_path);
|
||||
if (!is_string($hash) || !hash_equals($original_hash, $hash)) { $detail = 'Restaurierter Placeholder stimmt nicht mit dem Originalzustand überein.'; return false; }
|
||||
return true;
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_process(array $image, $temp_file, array $credentials, $stage, $force=false)
|
||||
{
|
||||
$image_id = (int)$image['row']['id'];
|
||||
$variants = bratonien_tools_cache_warmup_variant_rows($image, $stage);
|
||||
$pending = array();
|
||||
foreach ($variants as $variant)
|
||||
{
|
||||
if (!$force && is_file($variant['target']) && is_readable($variant['target'])) continue;
|
||||
$request = bratonien_tools_cache_warmup_i_request($variant['target']);
|
||||
if ($request !== null) $pending[] = array('name'=>$variant['name'], 'target'=>$variant['target'], 'request'=>$request);
|
||||
}
|
||||
if (!$pending) return array('ok'=>true, 'generated'=>0, 'message'=>'Bereits im Piwigo-Cache vorhanden.');
|
||||
|
||||
$image_lock = null;
|
||||
if (!bratonien_tools_cache_warmup_image_lock($image_id, $image_lock))
|
||||
{
|
||||
return array('ok'=>false, 'defer'=>true, 'message'=>'Bild-Lock konnte nicht gesetzt werden.');
|
||||
}
|
||||
$sync_lock = null;
|
||||
$detail = '';
|
||||
if (!bratonien_tools_cache_warmup_sync_guard($credentials, $sync_lock, $detail))
|
||||
{
|
||||
bratonien_tools_cache_warmup_unlock($image_lock);
|
||||
return array('ok'=>false, 'defer'=>true, 'message'=>$detail);
|
||||
}
|
||||
|
||||
$fresh = bratonien_tools_webdav_materialize_source_info($image_id);
|
||||
if (!$fresh || (int)$fresh['connection_id'] !== (int)$image['source']['connection_id'] || bratonien_tools_cache_warmup_signature($fresh) !== $image['signature'])
|
||||
{
|
||||
bratonien_tools_cache_warmup_unlock($sync_lock);
|
||||
bratonien_tools_cache_warmup_unlock($image_lock);
|
||||
return array('ok'=>false, 'defer'=>true, 'message'=>'WebDAV-Zuordnung hat sich seit dem Batch-Download geändert.');
|
||||
}
|
||||
|
||||
$source_path = realpath($image['logical']);
|
||||
if ($source_path === false || $source_path !== $image['resolved'] || !is_file($source_path))
|
||||
{
|
||||
bratonien_tools_cache_warmup_unlock($sync_lock);
|
||||
bratonien_tools_cache_warmup_unlock($image_lock);
|
||||
return array('ok'=>false, 'fatal'=>true, 'message'=>'Piwigo-Quellpfad hat sich während des Warmups geändert.');
|
||||
}
|
||||
if (!is_file($temp_file) || !is_readable($temp_file) || @getimagesize($temp_file) === false)
|
||||
{
|
||||
bratonien_tools_cache_warmup_unlock($sync_lock);
|
||||
bratonien_tools_cache_warmup_unlock($image_lock);
|
||||
return array('ok'=>false, 'message'=>'Batch-Quelldatei ist nicht vollständig lesbar.');
|
||||
}
|
||||
|
||||
$placeholder_size = filesize($source_path);
|
||||
$placeholder_hash = @sha1_file($source_path);
|
||||
if (!is_string($placeholder_hash))
|
||||
{
|
||||
bratonien_tools_cache_warmup_unlock($sync_lock);
|
||||
bratonien_tools_cache_warmup_unlock($image_lock);
|
||||
return array('ok'=>false, 'fatal'=>true, 'message'=>'Placeholder konnte vor dem Swap nicht verifiziert werden.');
|
||||
}
|
||||
|
||||
$request_id = substr(bin2hex(random_bytes(8)), 0, 8);
|
||||
$backup = $source_path.'.bratonien-placeholder-'.$request_id;
|
||||
$staging = $source_path.'.bratonien-source-'.$request_id;
|
||||
if (file_exists($backup) || file_exists($staging))
|
||||
{
|
||||
bratonien_tools_cache_warmup_unlock($sync_lock);
|
||||
bratonien_tools_cache_warmup_unlock($image_lock);
|
||||
return array('ok'=>false, 'fatal'=>true, 'message'=>'Unerwartete Swap-Datei ist bereits vorhanden.');
|
||||
}
|
||||
if (!@copy($temp_file, $staging) || @getimagesize($staging) === false)
|
||||
{
|
||||
@unlink($staging);
|
||||
bratonien_tools_cache_warmup_unlock($sync_lock);
|
||||
bratonien_tools_cache_warmup_unlock($image_lock);
|
||||
return array('ok'=>false, 'message'=>'Batch-Quelle konnte nicht sicher neben dem Placeholder bereitgestellt werden.');
|
||||
}
|
||||
|
||||
$swapped = false;
|
||||
$generated = 0;
|
||||
try
|
||||
{
|
||||
if (!@rename($source_path, $backup)) throw new RuntimeException('Placeholder konnte nicht atomar gesichert werden.');
|
||||
if (!@rename($staging, $source_path))
|
||||
{
|
||||
@rename($backup, $source_path);
|
||||
throw new RuntimeException('Temporäre Quelle konnte nicht atomar eingesetzt werden.');
|
||||
}
|
||||
$swapped = true;
|
||||
clearstatcache(true, $source_path);
|
||||
if (@getimagesize($source_path) === false) throw new RuntimeException('Piwigo-Quellpfad enthält nach dem Swap kein lesbares Bild.');
|
||||
|
||||
foreach ($pending as $variant)
|
||||
{
|
||||
$call_detail = '';
|
||||
if (!bratonien_tools_cache_warmup_call_piwigo($variant['request'], $call_detail))
|
||||
{
|
||||
throw new RuntimeException($variant['name'].': '.$call_detail);
|
||||
}
|
||||
clearstatcache(true, $variant['target']);
|
||||
if (!is_file($variant['target']) || !is_readable($variant['target']))
|
||||
{
|
||||
throw new RuntimeException($variant['name'].': Piwigo hat kein Derivat im eigenen Cache erzeugt.');
|
||||
}
|
||||
$generated++;
|
||||
}
|
||||
}
|
||||
catch (Throwable $e)
|
||||
{
|
||||
$restore_detail = '';
|
||||
$restored = $swapped
|
||||
? bratonien_tools_cache_warmup_restore($source_path, $backup, $staging, $placeholder_size, $placeholder_hash, $restore_detail)
|
||||
: true;
|
||||
if (!$swapped) @unlink($staging);
|
||||
bratonien_tools_cache_warmup_unlock($sync_lock);
|
||||
bratonien_tools_cache_warmup_unlock($image_lock);
|
||||
if (!$restored) return array('ok'=>false, 'fatal'=>true, 'message'=>$e->getMessage().' RESTORE FEHLGESCHLAGEN: '.$restore_detail);
|
||||
return array('ok'=>false, 'message'=>$e->getMessage());
|
||||
}
|
||||
|
||||
$restore_detail = '';
|
||||
$restored = bratonien_tools_cache_warmup_restore($source_path, $backup, $staging, $placeholder_size, $placeholder_hash, $restore_detail);
|
||||
bratonien_tools_cache_warmup_unlock($sync_lock);
|
||||
bratonien_tools_cache_warmup_unlock($image_lock);
|
||||
if (!$restored) return array('ok'=>false, 'fatal'=>true, 'message'=>'RESTORE FEHLGESCHLAGEN: '.$restore_detail);
|
||||
return array('ok'=>true, 'generated'=>$generated, 'message'=>'Piwigo hat die angeforderten Derivate erzeugt.');
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_stage_pending(array $selected, array $state, $stage)
|
||||
{
|
||||
$pending = array();
|
||||
$key = $stage === 1 ? 'stage1_signature' : 'stage2_signature';
|
||||
foreach ($selected as $image_id=>$image)
|
||||
{
|
||||
$saved = isset($state['images'][(string)$image_id]) && is_array($state['images'][(string)$image_id]) ? $state['images'][(string)$image_id] : array();
|
||||
if (!isset($saved[$key]) || !hash_equals((string)$saved[$key], $image['signature'])) $pending[$image_id] = $image;
|
||||
}
|
||||
return $pending;
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_run_stage($connection_id, $stage, array $selected, array &$state, array $credentials, $batch_size)
|
||||
{
|
||||
$pending = bratonien_tools_cache_warmup_stage_pending($selected, $state, $stage);
|
||||
if (!$pending) return array('ok'=>true, 'success'=>array(), 'failed'=>0);
|
||||
|
||||
$temp_root = PHPWG_ROOT_PATH.'upload/bratonien-webdav-warmup';
|
||||
if (!is_dir($temp_root) && !@mkdir($temp_root, 0775, true) && !is_dir($temp_root)) throw new RuntimeException('Warmup-Temp-Verzeichnis konnte nicht angelegt werden.');
|
||||
$success = array();
|
||||
$failed = 0;
|
||||
$batch_number = 0;
|
||||
|
||||
foreach (array_chunk(array_keys($pending), max(1, (int)$batch_size)) as $ids)
|
||||
{
|
||||
$batch_number++;
|
||||
$dir = $temp_root.'/connection-'.(int)$connection_id.'-stage'.$stage.'-'.date('YmdHis').'-'.bin2hex(random_bytes(4));
|
||||
if (!@mkdir($dir, 0775, true) && !is_dir($dir)) throw new RuntimeException('Warmup-Batch-Verzeichnis konnte nicht angelegt werden.');
|
||||
|
||||
$downloads = array();
|
||||
foreach ($ids as $image_id)
|
||||
{
|
||||
$file = $dir.'/image-'.(int)$image_id.'.img';
|
||||
$detail = '';
|
||||
if (bratonien_tools_cache_warmup_download($pending[$image_id]['source'], $credentials, $file, $detail))
|
||||
{
|
||||
$downloads[$image_id] = $file;
|
||||
}
|
||||
else
|
||||
{
|
||||
$failed++;
|
||||
bratonien_tools_cache_warmup_log('download_failed', array('stage'=>$stage, 'image_id'=>$image_id, 'detail'=>$detail));
|
||||
}
|
||||
}
|
||||
|
||||
bratonien_tools_cache_warmup_status($connection_id, 'running', 'Warmup Stufe '.$stage.' läuft.', array(
|
||||
'stage'=>$stage,
|
||||
'batch'=>$batch_number,
|
||||
'batch_requested'=>count($ids),
|
||||
'batch_downloaded'=>count($downloads),
|
||||
));
|
||||
|
||||
foreach ($downloads as $image_id=>$file)
|
||||
{
|
||||
$saved = isset($state['images'][(string)$image_id]) && is_array($state['images'][(string)$image_id]) ? $state['images'][(string)$image_id] : array();
|
||||
$force = !empty($saved['baseline']) || (
|
||||
isset($saved['stage1_signature']) && !hash_equals((string)$saved['stage1_signature'], $pending[$image_id]['signature'])
|
||||
);
|
||||
if ($stage === 2 && isset($saved['stage2_signature']) && !hash_equals((string)$saved['stage2_signature'], $pending[$image_id]['signature'])) $force = true;
|
||||
|
||||
$result = bratonien_tools_cache_warmup_process($pending[$image_id], $file, $credentials, $stage, $force);
|
||||
bratonien_tools_cache_warmup_log('image', array(
|
||||
'stage'=>$stage,
|
||||
'image_id'=>$image_id,
|
||||
'ok'=>!empty($result['ok']),
|
||||
'generated'=>(int)($result['generated'] ?? 0),
|
||||
'message'=>(string)($result['message'] ?? ''),
|
||||
));
|
||||
if (!empty($result['fatal']))
|
||||
{
|
||||
foreach ($downloads as $cleanup) @unlink($cleanup);
|
||||
@rmdir($dir);
|
||||
throw new RuntimeException('Sicherheitsabbruch bei Bild #'.$image_id.': '.(string)$result['message']);
|
||||
}
|
||||
if (empty($result['ok']))
|
||||
{
|
||||
$failed++;
|
||||
continue;
|
||||
}
|
||||
|
||||
if (!isset($state['images'][(string)$image_id]) || !is_array($state['images'][(string)$image_id])) $state['images'][(string)$image_id] = array();
|
||||
$state['images'][(string)$image_id][$stage === 1 ? 'stage1_signature' : 'stage2_signature'] = $pending[$image_id]['signature'];
|
||||
unset($state['images'][(string)$image_id]['baseline']);
|
||||
bratonien_tools_cache_warmup_save_state($connection_id, $state);
|
||||
$success[$image_id] = true;
|
||||
}
|
||||
|
||||
foreach ($downloads as $file) @unlink($file);
|
||||
@rmdir($dir);
|
||||
}
|
||||
return array('ok'=>$failed === 0, 'success'=>$success, 'failed'=>$failed);
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_run($connection_id, $mode)
|
||||
{
|
||||
$settings = bratonien_tools_get_webdav_warmup_settings();
|
||||
if ($mode !== 'manual' && empty($settings['enabled']))
|
||||
{
|
||||
bratonien_tools_cache_warmup_log('disabled', array('connection_id'=>$connection_id, 'mode'=>$mode));
|
||||
return 0;
|
||||
}
|
||||
|
||||
$credentials = bratonien_tools_cache_warmup_credentials($connection_id);
|
||||
if (!$credentials) throw new RuntimeException('WebDAV-Verbindung oder Zugangsdaten konnten nicht geladen werden.');
|
||||
$state_dir = rtrim((string)($credentials['config']['state_dir'] ?? ''), '/');
|
||||
if ($state_dir === '') throw new RuntimeException('Connector-State-Verzeichnis fehlt.');
|
||||
if (!is_dir($state_dir) && !@mkdir($state_dir, 0750, true) && !is_dir($state_dir)) throw new RuntimeException('Connector-State-Verzeichnis konnte nicht angelegt werden.');
|
||||
|
||||
$process_lock = @fopen($state_dir.'/webdav-cache-warmup.lock', 'c');
|
||||
if (!$process_lock || !@flock($process_lock, LOCK_EX | LOCK_NB))
|
||||
{
|
||||
if (is_resource($process_lock)) fclose($process_lock);
|
||||
bratonien_tools_cache_warmup_log('already_running', array('connection_id'=>$connection_id));
|
||||
return 0;
|
||||
}
|
||||
|
||||
try
|
||||
{
|
||||
$scan = bratonien_tools_cache_warmup_scan($connection_id);
|
||||
$state = bratonien_tools_cache_warmup_load_state($connection_id);
|
||||
if ($state === null)
|
||||
{
|
||||
bratonien_tools_cache_warmup_baseline($connection_id, $scan);
|
||||
bratonien_tools_cache_warmup_status($connection_id, 'baseline', 'Bestehender produktiver Bestand wurde nur als Ausgangszustand erfasst.', array(
|
||||
'images'=>count($scan['images']),
|
||||
'albums'=>count($scan['albums']),
|
||||
));
|
||||
bratonien_tools_cache_warmup_log('baseline', array('connection_id'=>$connection_id, 'images'=>count($scan['images']), 'albums'=>count($scan['albums'])));
|
||||
return 0;
|
||||
}
|
||||
|
||||
if ($mode === 'periodic')
|
||||
{
|
||||
$last = (int)($state['last_periodic_at'] ?? 0);
|
||||
$interval = max(1, (int)$settings['periodic_hours']) * 3600;
|
||||
if ($last > 0 && time() - $last < $interval)
|
||||
{
|
||||
bratonien_tools_cache_warmup_log('periodic_not_due', array('connection_id'=>$connection_id, 'remaining'=>$interval-(time()-$last)));
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
|
||||
$selection = bratonien_tools_cache_warmup_select($scan, $state, $mode);
|
||||
$selected = $selection['images'];
|
||||
bratonien_tools_cache_warmup_status($connection_id, 'scan', 'Warmup-Eingangsprüfung abgeschlossen.', array(
|
||||
'mode'=>$mode,
|
||||
'images'=>count($scan['images']),
|
||||
'selected'=>count($selected),
|
||||
'new_albums'=>count($selection['new_albums']),
|
||||
));
|
||||
|
||||
if (!$selected)
|
||||
{
|
||||
foreach ($scan['albums'] as $album_id) $state['albums'][(string)$album_id] = array('seen_at'=>time());
|
||||
if ($mode === 'periodic' || $mode === 'manual') $state['last_periodic_at'] = time();
|
||||
bratonien_tools_cache_warmup_save_state($connection_id, $state);
|
||||
bratonien_tools_cache_warmup_status($connection_id, 'complete', 'Keine neuen oder geänderten Bilder für den Warmup gefunden.', array('mode'=>$mode));
|
||||
return 0;
|
||||
}
|
||||
|
||||
$stage1 = bratonien_tools_cache_warmup_run_stage($connection_id, 1, $selected, $state, $credentials, $settings['batch_size']);
|
||||
$stage2_candidates = array();
|
||||
foreach ($selected as $image_id=>$image)
|
||||
{
|
||||
$saved = isset($state['images'][(string)$image_id]) && is_array($state['images'][(string)$image_id]) ? $state['images'][(string)$image_id] : array();
|
||||
if (isset($saved['stage1_signature']) && hash_equals((string)$saved['stage1_signature'], $image['signature'])) $stage2_candidates[$image_id] = $image;
|
||||
}
|
||||
$stage2 = bratonien_tools_cache_warmup_run_stage($connection_id, 2, $stage2_candidates, $state, $credentials, $settings['batch_size']);
|
||||
|
||||
foreach ($scan['albums'] as $album_id) $state['albums'][(string)$album_id] = array('seen_at'=>time());
|
||||
if ($mode === 'periodic' || $mode === 'manual') $state['last_periodic_at'] = time();
|
||||
bratonien_tools_cache_warmup_save_state($connection_id, $state);
|
||||
|
||||
$failed = (int)$stage1['failed'] + (int)$stage2['failed'];
|
||||
bratonien_tools_cache_warmup_status($connection_id, $failed ? 'error' : 'complete', $failed ? 'Warmup mit einzelnen Fehlern beendet.' : 'Warmup vollständig beendet.', array(
|
||||
'mode'=>$mode,
|
||||
'stage1_failed'=>(int)$stage1['failed'],
|
||||
'stage2_failed'=>(int)$stage2['failed'],
|
||||
));
|
||||
return $failed ? 2 : 0;
|
||||
}
|
||||
finally
|
||||
{
|
||||
@flock($process_lock, LOCK_UN);
|
||||
fclose($process_lock);
|
||||
}
|
||||
}
|
||||
|
||||
$connection_id = 0;
|
||||
$mode = 'periodic';
|
||||
foreach ($argv as $arg)
|
||||
{
|
||||
if (preg_match('/^--connection-id=(\d+)$/', $arg, $m)) $connection_id = (int)$m[1];
|
||||
elseif (preg_match('/^--mode=(sync|periodic|manual)$/', $arg, $m)) $mode = $m[1];
|
||||
}
|
||||
if ($connection_id < 1)
|
||||
{
|
||||
fwrite(STDERR, "--connection-id fehlt oder ist ungültig.\n");
|
||||
exit(1);
|
||||
}
|
||||
|
||||
try
|
||||
{
|
||||
exit(bratonien_tools_cache_warmup_run($connection_id, $mode));
|
||||
}
|
||||
catch (Throwable $e)
|
||||
{
|
||||
try { bratonien_tools_cache_warmup_status($connection_id, 'fatal', $e->getMessage()); } catch (Throwable $ignored) {}
|
||||
fwrite(STDERR, '[BRAT-WARMUP] fatal '.$e->getMessage()."\n");
|
||||
exit(1);
|
||||
}
|
||||
112
runtime/lib/webdav-warmup-audit.php
Normal file
112
runtime/lib/webdav-warmup-audit.php
Normal file
@@ -0,0 +1,112 @@
|
||||
<?php
|
||||
if (PHP_SAPI !== 'cli')
|
||||
{
|
||||
http_response_code(404);
|
||||
exit;
|
||||
}
|
||||
|
||||
$piwigo_root = realpath(dirname(__DIR__, 3));
|
||||
if ($piwigo_root === false)
|
||||
{
|
||||
fwrite(STDERR, "Piwigo-Root wurde nicht gefunden.\n");
|
||||
exit(1);
|
||||
}
|
||||
define('PHPWG_ROOT_PATH', rtrim($piwigo_root, DIRECTORY_SEPARATOR).DIRECTORY_SEPARATOR);
|
||||
|
||||
$_SERVER['REMOTE_ADDR'] = '127.0.0.1';
|
||||
$_SERVER['SERVER_ADDR'] = '127.0.0.1';
|
||||
$_SERVER['SERVER_NAME'] = 'localhost';
|
||||
$_SERVER['HTTP_HOST'] = 'localhost';
|
||||
$_SERVER['SERVER_PORT'] = '80';
|
||||
$_SERVER['REQUEST_METHOD'] = 'GET';
|
||||
$_SERVER['REQUEST_URI'] = '/';
|
||||
$_SERVER['SCRIPT_NAME'] = '/plugins/bratonien_tools/runtime/lib/webdav-warmup-audit.php';
|
||||
$_SERVER['PHP_SELF'] = $_SERVER['SCRIPT_NAME'];
|
||||
$_SERVER['QUERY_STRING'] = '';
|
||||
$_SERVER['HTTP_USER_AGENT'] = 'Bratonien-WebDAV-Warmup-Audit/0.9.7.1.8';
|
||||
$_SERVER['HTTPS'] = 'off';
|
||||
|
||||
require_once(PHPWG_ROOT_PATH.'include/common.inc.php');
|
||||
if (!defined('BRATONIEN_TOOLS_PATH'))
|
||||
{
|
||||
fwrite(STDERR, "Bratonien Tools ist nicht aktiv.\n");
|
||||
exit(1);
|
||||
}
|
||||
require_once(BRATONIEN_TOOLS_PATH.'include/webdav_materialize_runtime.inc.php');
|
||||
|
||||
$total = 0;
|
||||
$connector = 0;
|
||||
$errors = array();
|
||||
$warnings = array();
|
||||
$resolved_to_ids = array();
|
||||
|
||||
$result = pwg_query('SELECT id, path FROM '.IMAGES_TABLE.' ORDER BY id');
|
||||
while ($row = pwg_db_fetch_assoc($result))
|
||||
{
|
||||
$total++;
|
||||
$image_id = (int)$row['id'];
|
||||
$info = bratonien_tools_webdav_materialize_source_info($image_id);
|
||||
if (!$info) continue;
|
||||
$connector++;
|
||||
|
||||
$logical = (string)$row['path'];
|
||||
if ($logical === '')
|
||||
{
|
||||
$errors[] = 'Bild #'.$image_id.': Piwigo-Pfad ist leer.';
|
||||
continue;
|
||||
}
|
||||
if (strpos($logical, '/') !== 0)
|
||||
{
|
||||
$logical = PHPWG_ROOT_PATH.ltrim(preg_replace('#^\./#', '', $logical), '/');
|
||||
}
|
||||
|
||||
$resolved = realpath($logical);
|
||||
if ($resolved === false || !is_file($resolved))
|
||||
{
|
||||
$errors[] = 'Bild #'.$image_id.': Piwigo-Pfad kann nicht auf eine Datei aufgelöst werden: '.$logical;
|
||||
continue;
|
||||
}
|
||||
|
||||
$normalized = str_replace('\\', '/', $resolved);
|
||||
$expected = '/nc-webdav-source/connection-'.(int)$info['connection_id'].'/root-'.(int)$info['root_fileid'].'/';
|
||||
if (strpos($normalized, $expected) === false)
|
||||
{
|
||||
$errors[] = 'Bild #'.$image_id.': realpath liegt nicht in der erwarteten Connector-Wurzel: '.$normalized;
|
||||
}
|
||||
|
||||
$webdav_path = trim((string)($info['webdav_path'] ?? ''), '/');
|
||||
if ($webdav_path === '')
|
||||
{
|
||||
$errors[] = 'Bild #'.$image_id.': WebDAV-Pfad fehlt.';
|
||||
}
|
||||
if ((int)($info['connection_id'] ?? 0) < 1 || (int)($info['root_fileid'] ?? 0) < 1)
|
||||
{
|
||||
$errors[] = 'Bild #'.$image_id.': Verbindung oder Root-fileid ist ungültig.';
|
||||
}
|
||||
if ((int)($info['fileid'] ?? 0) < 1)
|
||||
{
|
||||
$warnings[] = 'Bild #'.$image_id.': Nextcloud-fileid fehlt; Original-Fallback bleibt möglich.';
|
||||
}
|
||||
|
||||
if (!isset($resolved_to_ids[$normalized])) $resolved_to_ids[$normalized] = array();
|
||||
$resolved_to_ids[$normalized][] = $image_id;
|
||||
}
|
||||
|
||||
foreach ($resolved_to_ids as $path => $ids)
|
||||
{
|
||||
if (count($ids) > 1)
|
||||
{
|
||||
$warnings[] = 'Mehrere Piwigo-Bild-IDs zeigen auf dieselbe physische Connector-Quelle: '.implode(',', $ids).' -> '.$path;
|
||||
}
|
||||
}
|
||||
|
||||
printf("WebDAV-Warmup-Pfadaudit\n");
|
||||
printf("Piwigo-Bilder gesamt: %d\n", $total);
|
||||
printf("Connector-Bilder: %d\n", $connector);
|
||||
printf("Fehler: %d\n", count($errors));
|
||||
printf("Warnungen: %d\n", count($warnings));
|
||||
|
||||
foreach ($errors as $error) fwrite(STDERR, "ERROR: ".$error."\n");
|
||||
foreach ($warnings as $warning) fwrite(STDOUT, "WARN: ".$warning."\n");
|
||||
|
||||
exit($errors ? 2 : 0);
|
||||
@@ -31,9 +31,25 @@ result=0
|
||||
for config in "${webdav_configs[@]}"; do
|
||||
name="$(basename "$config")"
|
||||
echo "NC Connector WebDAV: $name"
|
||||
if ! env PIWIGO_CONFIG="$config" bash "$SCRIPT_DIR/sync-webdav.sh"; then
|
||||
if env PIWIGO_CONFIG="$config" bash "$SCRIPT_DIR/sync-webdav.sh"; then
|
||||
# Ein neues Album soll nicht bis zur periodischen Einzelbildprüfung
|
||||
# warten. Direkt nach dem vollständig erfolgreichen Sync dieser
|
||||
# Verbindung darf der Warmup gezielt nur diese Verbindung prüfen.
|
||||
if [[ -f "$SCRIPT_DIR/webdav-warmup-dispatch.php" && "$name" =~ ^webdav-connection-([0-9]+)\.conf$ ]]; then
|
||||
connection_id="${BASH_REMATCH[1]}"
|
||||
php "$SCRIPT_DIR/webdav-warmup-dispatch.php" --mode=sync --connection-id="$connection_id" || result=1
|
||||
fi
|
||||
else
|
||||
result=1
|
||||
fi
|
||||
done
|
||||
|
||||
# Einzelne neue/geänderte Bilder in bereits bekannten Alben werden nur über
|
||||
# die periodische Eingangsprüfung behandelt. Der Dispatcher liest zunächst nur
|
||||
# den Warmup-State und startet die eigentliche Prüfung erst, wenn das im Plugin
|
||||
# konfigurierte Intervall (Standard: 12 Stunden) wirklich fällig ist.
|
||||
if [[ "$result" -eq 0 && -f "$SCRIPT_DIR/webdav-warmup-dispatch.php" ]]; then
|
||||
php "$SCRIPT_DIR/webdav-warmup-dispatch.php" --mode=periodic || result=1
|
||||
fi
|
||||
|
||||
exit "$result"
|
||||
|
||||
178
runtime/webdav-warmup-dispatch.php
Normal file
178
runtime/webdav-warmup-dispatch.php
Normal file
@@ -0,0 +1,178 @@
|
||||
<?php
|
||||
if (PHP_SAPI !== 'cli')
|
||||
{
|
||||
http_response_code(404);
|
||||
exit;
|
||||
}
|
||||
|
||||
$piwigo_root = realpath(dirname(__DIR__, 2));
|
||||
if ($piwigo_root === false)
|
||||
{
|
||||
fwrite(STDERR, "Piwigo-Root wurde nicht gefunden.\n");
|
||||
exit(1);
|
||||
}
|
||||
define('PHPWG_ROOT_PATH', rtrim($piwigo_root, DIRECTORY_SEPARATOR).DIRECTORY_SEPARATOR);
|
||||
|
||||
$_SERVER['REMOTE_ADDR'] = '127.0.0.1';
|
||||
$_SERVER['SERVER_ADDR'] = '127.0.0.1';
|
||||
$_SERVER['SERVER_NAME'] = 'localhost';
|
||||
$_SERVER['HTTP_HOST'] = 'localhost';
|
||||
$_SERVER['SERVER_PORT'] = '80';
|
||||
$_SERVER['REQUEST_METHOD'] = 'GET';
|
||||
$_SERVER['REQUEST_URI'] = '/';
|
||||
$_SERVER['SCRIPT_NAME'] = '/plugins/bratonien_tools/runtime/webdav-warmup-dispatch.php';
|
||||
$_SERVER['PHP_SELF'] = $_SERVER['SCRIPT_NAME'];
|
||||
$_SERVER['QUERY_STRING'] = '';
|
||||
$_SERVER['HTTP_USER_AGENT'] = 'Bratonien-WebDAV-Warmup-Dispatcher/0.9.7.1.8';
|
||||
$_SERVER['HTTPS'] = 'off';
|
||||
|
||||
require_once(PHPWG_ROOT_PATH.'include/common.inc.php');
|
||||
if (!defined('BRATONIEN_TOOLS_PATH'))
|
||||
{
|
||||
fwrite(STDERR, "Bratonien Tools ist nicht aktiv.\n");
|
||||
exit(1);
|
||||
}
|
||||
require_once(BRATONIEN_TOOLS_PATH.'include/nc_connector.inc.php');
|
||||
require_once(BRATONIEN_TOOLS_PATH.'include/webdav_warmup_settings.inc.php');
|
||||
|
||||
$mode = 'periodic';
|
||||
$wait = false;
|
||||
$connection_filter = 0;
|
||||
foreach ($argv as $arg)
|
||||
{
|
||||
if (preg_match('/^--mode=(sync|periodic|manual)$/', $arg, $m)) $mode = $m[1];
|
||||
elseif (preg_match('/^--connection-id=(\d+)$/', $arg, $m)) $connection_filter = (int)$m[1];
|
||||
elseif ($arg === '--wait') $wait = true;
|
||||
}
|
||||
|
||||
$settings = bratonien_tools_get_webdav_warmup_settings();
|
||||
if ($mode !== 'manual' && empty($settings['enabled']))
|
||||
{
|
||||
fwrite(STDOUT, "WebDAV-Cache-Warmup ist deaktiviert.\n");
|
||||
exit(0);
|
||||
}
|
||||
if (!function_exists('exec'))
|
||||
{
|
||||
fwrite(STDERR, "PHP exec() ist deaktiviert; Warmup kann nicht gestartet werden.\n");
|
||||
exit(1);
|
||||
}
|
||||
|
||||
$worker = realpath(BRATONIEN_TOOLS_PATH.'runtime/lib/webdav-cache-warmup.php');
|
||||
$guard = realpath(BRATONIEN_TOOLS_PATH.'runtime/lib/run-webdav-cache-warmup.sh');
|
||||
if (!$worker || !is_file($worker))
|
||||
{
|
||||
fwrite(STDERR, "WebDAV-Cache-Warmup-Worker wurde nicht gefunden.\n");
|
||||
exit(1);
|
||||
}
|
||||
if (!$guard || !is_file($guard))
|
||||
{
|
||||
fwrite(STDERR, "WebDAV-Cache-Warmup-Schutzskript wurde nicht gefunden.\n");
|
||||
exit(1);
|
||||
}
|
||||
|
||||
$log = PHPWG_ROOT_PATH.PWG_LOCAL_DIR.'bratonien-webdav-warmup.log';
|
||||
$result = 0;
|
||||
$started = 0;
|
||||
$matched = 0;
|
||||
foreach (bratonien_tools_nc_connector_connections() as $connection)
|
||||
{
|
||||
if (empty($connection['enabled']) || !bratonien_tools_nc_connector_is_webdav($connection)) continue;
|
||||
$connection_id = (int)$connection['id'];
|
||||
if ($connection_id < 1) continue;
|
||||
if ($connection_filter > 0 && $connection_id !== $connection_filter) continue;
|
||||
$matched++;
|
||||
|
||||
$config = isset($connection['config']) && is_array($connection['config']) ? $connection['config'] : array();
|
||||
$state_dir = rtrim((string)($config['state_dir'] ?? ''), '/');
|
||||
if ($state_dir === '')
|
||||
{
|
||||
fwrite(STDERR, "Warmup für Verbindung #{$connection_id} übersprungen: Connector-State-Verzeichnis fehlt.\n");
|
||||
$result = 1;
|
||||
continue;
|
||||
}
|
||||
|
||||
if ($mode === 'periodic')
|
||||
{
|
||||
$state_file = PHPWG_ROOT_PATH.PWG_LOCAL_DIR.'bratonien-webdav-warmup.connection-'.$connection_id.'.json';
|
||||
if (is_file($state_file) && is_readable($state_file))
|
||||
{
|
||||
$raw = @file_get_contents($state_file);
|
||||
$state = $raw !== false ? json_decode($raw, true) : null;
|
||||
if (is_array($state))
|
||||
{
|
||||
$last = (int)($state['last_periodic_at'] ?? 0);
|
||||
$interval = max(1, (int)$settings['periodic_hours']) * 3600;
|
||||
if ($last > 0 && time() - $last < $interval)
|
||||
{
|
||||
$remaining = max(0, $interval - (time() - $last));
|
||||
fwrite(STDOUT, "Periodische Warmup-Prüfung für Verbindung #{$connection_id} noch nicht fällig ({$remaining}s verbleibend).\n");
|
||||
continue;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if ($mode === 'sync')
|
||||
{
|
||||
if (!is_dir($state_dir) && !@mkdir($state_dir, 0750, true) && !is_dir($state_dir))
|
||||
{
|
||||
fwrite(STDERR, "Warmup-Priorität für Verbindung #{$connection_id} konnte nicht signalisiert werden: State-Verzeichnis fehlt.\n");
|
||||
$result = 1;
|
||||
continue;
|
||||
}
|
||||
$priority_file = $state_dir.'/webdav-cache-warmup-priority-sync';
|
||||
$priority_payload = json_encode(array(
|
||||
'connection_id'=>$connection_id,
|
||||
'requested_at'=>time(),
|
||||
'reason'=>'connector-sync-complete',
|
||||
), JSON_UNESCAPED_SLASHES);
|
||||
if (!is_string($priority_payload) || @file_put_contents($priority_file, $priority_payload."\n", LOCK_EX) === false)
|
||||
{
|
||||
fwrite(STDERR, "Warmup-Priorität für Verbindung #{$connection_id} konnte nicht geschrieben werden.\n");
|
||||
$result = 1;
|
||||
continue;
|
||||
}
|
||||
@chmod($priority_file, 0664);
|
||||
}
|
||||
|
||||
// Der Guard hält webdav-sync.lock für den gesamten Worker-Lauf geteilt.
|
||||
// Dadurch kann der produktive Source-/Shadow-Tree während Download, Swap,
|
||||
// Piwigo-Aufruf und Restore nicht von einem Connector-Sync ersetzt werden.
|
||||
$base = escapeshellarg('/bin/bash').' '.escapeshellarg($guard).' '.escapeshellarg($state_dir)
|
||||
.' '.escapeshellarg(PHP_BINARY).' '.escapeshellarg($worker)
|
||||
.' --connection-id='.$connection_id
|
||||
.' --mode='.escapeshellarg($mode);
|
||||
|
||||
if ($wait)
|
||||
{
|
||||
$output = array();
|
||||
$exit = 1;
|
||||
@exec($base.' 2>&1', $output, $exit);
|
||||
foreach ($output as $line) fwrite(STDOUT, $line."\n");
|
||||
if ($exit !== 0) $result = $exit;
|
||||
}
|
||||
else
|
||||
{
|
||||
$command = 'nohup '.$base.' >> '.escapeshellarg($log).' 2>&1 < /dev/null & echo $!';
|
||||
$output = array();
|
||||
$exit = 1;
|
||||
@exec($command, $output, $exit);
|
||||
$pid = isset($output[0]) ? (int)$output[0] : 0;
|
||||
if ($exit !== 0 || $pid <= 0)
|
||||
{
|
||||
fwrite(STDERR, "Warmup für Verbindung #{$connection_id} konnte nicht gestartet werden.\n");
|
||||
$result = 1;
|
||||
continue;
|
||||
}
|
||||
$started++;
|
||||
fwrite(STDOUT, "Warmup {$mode} für Verbindung #{$connection_id} gestartet (PID {$pid}).\n");
|
||||
}
|
||||
}
|
||||
|
||||
if ($connection_filter > 0 && $matched === 0)
|
||||
{
|
||||
fwrite(STDERR, "WebDAV-Verbindung #{$connection_filter} wurde nicht als aktive Connector-Verbindung gefunden.\n");
|
||||
exit(1);
|
||||
}
|
||||
if (!$wait) fwrite(STDOUT, "Gestartete Warmup-Prozesse: {$started}.\n");
|
||||
exit($result);
|
||||
46
template/webdav_warmup_admin.tpl
Normal file
46
template/webdav_warmup_admin.tpl
Normal file
@@ -0,0 +1,46 @@
|
||||
<div id="bratonien-webdav-warmup-card" class="bratonien-card" style="margin-top:16px;">
|
||||
<h4>WebDAV-Cache-Warmup</h4>
|
||||
<p>Bereitet fehlende Piwigo-Bildgrößen für neue Nextcloud-Inhalte vor. Bratonien Tools lädt die Quellen nur temporär in Batches; die eigentlichen Derivate erzeugt ausschließlich Piwigo über seinen normalen <code>i.php</code>-Pfad.</p>
|
||||
<p class="bratonien-main-cache__warning"><strong>Patchphase 0.9.7.1.8:</strong> Die Automatik ist standardmäßig deaktiviert. Bestehende produktive Bilder werden beim ersten Lauf nur als Ausgangsbestand erfasst und nicht automatisch neu aufgebaut.</p>
|
||||
|
||||
<form method="post" class="bratonien-worker-settings">
|
||||
<input type="hidden" name="pwg_token" value="{$PWG_TOKEN|escape:html}">
|
||||
<label><input type="checkbox" name="webdav_warmup_enabled" value="1" {if $CACHE_WORKERS.webdav_warmup.enabled}checked{/if}> Automatischen WebDAV-Warmup aktivieren</label>
|
||||
<label>Bilder pro Batch: <input type="number" name="webdav_warmup_batch_size" value="{$CACHE_WORKERS.webdav_warmup.batch_size}" min="1" max="50" step="1"></label>
|
||||
<label>Eingangsprüfung: <input type="number" name="webdav_warmup_periodic_hours" value="{$CACHE_WORKERS.webdav_warmup.periodic_hours}" min="1" max="168" step="1"> Stunden</label>
|
||||
<button class="buttonLike" type="submit" name="bratonien_tool" value="image_cache_webdav_warmup_settings">Warmup-Einstellungen speichern</button>
|
||||
</form>
|
||||
|
||||
<div class="bratonien-form-grid" style="margin-top:12px;">
|
||||
<span class="bratonien-label">Ablauf</span><span>Neues Album: direkt nach erfolgreichem Connector-Sync · einzelne neue/geänderte Bilder: periodisch oder manuell</span>
|
||||
<span class="bratonien-label">Stufe 1</span><span>Von Piwigo definierte Derivate bis einschließlich 1920 px längster Kante</span>
|
||||
<span class="bratonien-label">Stufe 2</span><span>Danach alle übrigen von Piwigo definierten Derivate</span>
|
||||
<span class="bratonien-label">Letzter Status</span><span><strong>{$CACHE_WORKERS.webdav_warmup.status.state|default:'idle'|escape:html}</strong>{if $CACHE_WORKERS.webdav_warmup.status.message} · {$CACHE_WORKERS.webdav_warmup.status.message|escape:html}{/if}</span>
|
||||
</div>
|
||||
|
||||
<div class="bratonien-actions">
|
||||
<form method="post">
|
||||
<input type="hidden" name="pwg_token" value="{$PWG_TOKEN|escape:html}">
|
||||
<button class="buttonLike" type="submit" name="bratonien_tool" value="image_cache_webdav_warmup_manual">Jetzt auf neue Bilder prüfen</button>
|
||||
</form>
|
||||
<form method="post">
|
||||
<input type="hidden" name="pwg_token" value="{$PWG_TOKEN|escape:html}">
|
||||
<button class="buttonLike" type="submit" name="bratonien_tool" value="image_cache_webdav_warmup_audit">Produktive Pfade prüfen</button>
|
||||
</form>
|
||||
</div>
|
||||
<p class="bratonien-base-note">Der Pfadaudit ist schreibgeschützt. Warmup und On-Demand verwenden denselben Bild-Lock; während eines Warmups wird zusätzlich der Connector-Sync-Lock geteilt gehalten, damit Source- und Shadow-Tree nicht ausgetauscht werden können.</p>
|
||||
</div>
|
||||
|
||||
{literal}
|
||||
<script>
|
||||
(function(){
|
||||
function moveWarmupCard(){
|
||||
var card=document.getElementById('bratonien-webdav-warmup-card');
|
||||
var section=document.getElementById('wartung');
|
||||
if(!card||!section||card.parentNode===section)return;
|
||||
section.appendChild(card);
|
||||
}
|
||||
if(document.readyState==='loading')document.addEventListener('DOMContentLoaded',moveWarmupCard);else moveWarmupCard();
|
||||
})();
|
||||
</script>
|
||||
{/literal}
|
||||
Reference in New Issue
Block a user