mirror of
https://github.com/Terranom674/Piwigo_Bratonien_Tools.git
synced 2026-09-19 10:34:30 +00:00
Compare commits
21 Commits
e2f687237e
...
345ece63f3
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
345ece63f3 | ||
|
|
b2d87d4033 | ||
|
|
424dff783f | ||
|
|
621b4d8bfc | ||
|
|
5b89b453e4 | ||
|
|
2a774dfb75 | ||
|
|
2f4345a9b6 | ||
|
|
b4fef28869 | ||
|
|
006c62d154 | ||
|
|
db5f330e8e | ||
|
|
6d0df4acaa | ||
|
|
1dec85bbd0 | ||
|
|
e9cb1ce1f3 | ||
|
|
2535040557 | ||
|
|
4f4fd88a6a | ||
|
|
dbcceb05c6 | ||
|
|
ae5bc5c45b | ||
|
|
acfebe07fc | ||
|
|
bab54eccb9 | ||
|
|
81e0a9e185 | ||
|
|
116b710d5d |
111
docs/HANDOFF_2026-09-03_CACHE_WORKER.md
Normal file
111
docs/HANDOFF_2026-09-03_CACHE_WORKER.md
Normal file
@@ -0,0 +1,111 @@
|
||||
# HANDOFF – Piwigo / WebDAV Cache-Worker – 2026-09-03
|
||||
|
||||
## Warum hier gestoppt wurde
|
||||
|
||||
Die aktuelle Cache-Worker-Entwicklung ist in einen Architektur-/Debugging-Kreis geraten. Der beobachtete produktive Zustand ist eindeutig falsch: Nach dem Leeren des Bildcaches meldet der manuelle Cache-Aufbau praktisch sofort "fertig", obwohl der eigentliche Piwigo-Bildcache nicht aufgebaut wurde. Weitere Patchversuche an Anzeige, Statusaggregation oder Legacy-Worker-Zählung würden den Fehler nur weiter überdecken.
|
||||
|
||||
Ab hier **nicht weiterpatchen**, bevor der reale Ausführungspfad von Button bis Piwigo-Derivaterzeugung vollständig nachvollzogen ist.
|
||||
|
||||
## Gesicherte Architekturvorgaben
|
||||
|
||||
- Nextcloud ist die autoritative Quelle der Originalbilder.
|
||||
- Piwigo arbeitet produktiv mit einem Shadow-Tree und 1x1-Placeholdern.
|
||||
- Der Worker darf Piwigo nicht nachbauen und selbst keine Derivate rendern.
|
||||
- Richtiger Ablauf: Worker erkennt Arbeit -> Original temporär materialisieren -> Piwigo nativen Derivatpfad aufrufen -> Ergebnis prüfen -> Original wieder entfernen / Placeholder restaurieren.
|
||||
- Piwigo allein bestimmt Dimensionen, Crop, Sharpening, Wasserzeichen, Qualität, Format und Cachepfad.
|
||||
- On-Demand bleibt Fallback.
|
||||
- Materialisierung erfolgt batchweise, Standard 10 Originale pro Batch.
|
||||
- Stage 1 vor Stage 2; neue Sync-Arbeit darf Stage 2 zwischen Batches priorisieren.
|
||||
- Connector/Shadow-Tree bleibt Wahrheit darüber, welche Quellen existieren.
|
||||
|
||||
## Neuer Quellenindex – Konzept weiterhin sinnvoll
|
||||
|
||||
Die Trennung des Worker-Entscheidungszustands vom Piwigo-Cache ist weiterhin sinnvoll:
|
||||
|
||||
- eigener kleiner Quellenindex je WebDAV-Verbindung
|
||||
- Identität primär über `fileid`, Fallback Root/Pfad
|
||||
- Vergleich über `etag`, Dateigröße, MIME/Format, Pfad, Breite/Höhe
|
||||
- Quelle neu/geändert -> Worker darf Piwigo beauftragen
|
||||
- Quelle unverändert -> normaler periodischer Lauf tut nichts
|
||||
- gelöschte Quelle -> beim Abgleich aus Worker-Index entfernen; eigentliche Quellenentfernung bleibt Connector/Shadow-Tree-Aufgabe
|
||||
- manueller Full-Rebuild darf den Quellenindex bewusst ignorieren und alle aktuellen Quellen erneut an Piwigo reichen
|
||||
|
||||
Datei dafür aktuell:
|
||||
`include/webdav_source_index.inc.php`
|
||||
|
||||
## Aktueller problematischer Stand
|
||||
|
||||
### Version
|
||||
|
||||
Aktuell zuletzt auf `0.9.7.1.16` erhöht.
|
||||
|
||||
### Relevante Dateien
|
||||
|
||||
- `runtime/lib/webdav-cache-warmup.php`
|
||||
- `runtime/webdav-warmup-dispatch.php`
|
||||
- `include/webdav_source_index.inc.php`
|
||||
- `include/webdav_warmup_settings.inc.php`
|
||||
- `main-cache-build.php`
|
||||
- `main-cache-status.php`
|
||||
- `tools/image_cache.inc.php`
|
||||
- `template/admin.tpl`
|
||||
- `include/tool_registry.inc.php`
|
||||
|
||||
### Letzte Architekturänderungen
|
||||
|
||||
1. Legacy `main-cache-build.php` zählt WebDAV-Bilder nicht mehr als normale lokale Cachearbeit.
|
||||
2. Eigener WebDAV-Quellenindex wurde eingeführt.
|
||||
3. Warmup-Auswahl wurde auf diesen Quellenindex umgestellt.
|
||||
4. Kombinierter Admin-Button startet lokalen Piwigo-Teil plus WebDAV-Rebuild.
|
||||
5. Statusanzeige wurde zusammengeführt.
|
||||
6. Bei reiner WebDAV-Installation sollte der lokale Legacy-Worker gar nicht mehr gestartet werden.
|
||||
|
||||
Trotzdem beobachtet der Nutzer nach Cache-Leeren weiterhin einen praktisch sofort fertigen Aufbau. Das beweist: **Der tatsächliche WebDAV-Rebuild-Pfad erzeugt die erwarteten Piwigo-Derivate nicht bzw. läuft nicht so, wie die Architektur es annimmt.**
|
||||
|
||||
## Wichtige letzte Commits
|
||||
|
||||
- `ac6353b050cdf30db79ab20716e66e6a6c841abb` – eigener WebDAV-Quellenindex
|
||||
- `6783974c7dfab9d11a0f1e43419de84e92141fd5` – Warmup auf Quellenindex umgestellt
|
||||
- `9f80c4aa974da01d606292c479487bce13b403fa` – Quelle erst nach beiden Stufen vollständig
|
||||
- `030d0c1be7ef8cafdab3f3d46e01ec90d5fd3bdc` – kombinierte Statusanzeige
|
||||
- `e831301650ee2a80c969e6c9b783d37a3c4f593d` – Syntaxkorrektur Status
|
||||
- `479cdbc4734605db3861908286426c76468d12a3` – Quellenindex-Baseline blockiert Aktivierung nicht mehr
|
||||
- `116b710d5d35311eeae261e5ae3cf405c5f51ce3` – Legacy-Worker bei reiner WebDAV-Installation nicht mehr starten
|
||||
- `81e0a9e185a14c08230b4f8fa0f36809c5b2b9a7` – Version 0.9.7.1.16
|
||||
|
||||
## NICHT weiter tun
|
||||
|
||||
- Keine weiteren UI-/Statuspatches, bevor der echte Workerpfad bewiesen ist.
|
||||
- Nicht wieder Cache-Dateiexistenz als Quelle für "neu/geändert" verwenden.
|
||||
- Nicht den Placeholder selbst rendern.
|
||||
- Nicht den Legacy-Full-Cache-Builder für WebDAV reaktivieren.
|
||||
- Nicht weiter an Symptomen wie 45-Sekunden-Stall, "übersprungen" oder Prozentanzeige herumoptimieren.
|
||||
|
||||
## Nächster Einstieg – zwingend
|
||||
|
||||
Der nächste Chat/Entwickler soll **zuerst den tatsächlichen Rebuild-Ausführungspfad beweisen**:
|
||||
|
||||
1. Admin-Handler `image_cache_build` verfolgen.
|
||||
2. `bratonien_tools_start_combined_image_cache_build()` verfolgen.
|
||||
3. Start des `runtime/webdav-warmup-dispatch.php --mode=rebuild` beweisen.
|
||||
4. Verifizieren, dass Dispatcher die aktive WebDAV-Verbindung findet.
|
||||
5. Verifizieren, dass `runtime/lib/webdav-cache-warmup.php --mode=rebuild` wirklich gestartet wird.
|
||||
6. Verifizieren, wie viele Quellen `bratonien_tools_cache_warmup_scan()` tatsächlich liefert.
|
||||
7. Verifizieren, wie viele Varianten Stage 1 und Stage 2 pro Bild tatsächlich liefern.
|
||||
8. Erst an **einem einzigen Bild** nachweisen:
|
||||
- Original wird aus Nextcloud heruntergeladen.
|
||||
- Placeholder wird korrekt gesichert.
|
||||
- Original wird am produktiven Quellpfad eingesetzt.
|
||||
- `piwigo-derivative-call.php` ruft wirklich Piwigos nativen `i.php`-Pfad auf.
|
||||
- Piwigo erzeugt tatsächlich eine konkrete Datei unter `_data/i`.
|
||||
- strikter Validator erkennt sie als gültig.
|
||||
- Placeholder wird exakt restauriert.
|
||||
9. Erst wenn dieser Ein-Bild-Test nachweislich funktioniert, Batchbetrieb und UI wieder anfassen.
|
||||
|
||||
## Entscheidender Verdacht
|
||||
|
||||
Der derzeit wichtigste Verdacht ist **nicht mehr die Anzeige**, sondern dass der Rebuild intern sehr früh keine Arbeit findet oder Piwigo über den aufgerufenen `i.php`-Pfad keine Derivate erzeugt, obwohl der Worker dies erwartet. Genau diese Stelle muss mit realem Lauf/Log belegt werden.
|
||||
|
||||
## Produktionsvalidierung
|
||||
|
||||
Von ChatGPT wurde kein echter Serverlauf durchgeführt. Ohne reale Serverausgaben darf nicht behauptet werden, dass Pfadaudit, Materialisierung, Piwigo-Aufruf oder Cache-Erzeugung funktionieren.
|
||||
@@ -136,6 +136,16 @@ function bratonien_tools_clear_image_cache_atomic()
|
||||
@chmod($real_cache_root, $mode);
|
||||
@file_put_contents($real_cache_root.'/index.htm', 'Not allowed!');
|
||||
|
||||
// Der Quellenindex selbst bleibt bestehen, weil sich die Quellen durch ein
|
||||
// Cache-Leeren nicht ändern. Seine Fertigmarkierungen sind danach aber
|
||||
// zwingend ungültig. Der Worker darf sonst einen leeren Piwigo-Cache als
|
||||
// bereits verarbeitet ansehen, ohne den Cache selbst anzuschauen.
|
||||
$webdav_invalidated = 0;
|
||||
if (function_exists('bratonien_tools_invalidate_webdav_cache_completion'))
|
||||
{
|
||||
$webdav_invalidated = bratonien_tools_invalidate_webdav_cache_completion('Piwigo-Bildcache wurde atomar geleert.');
|
||||
}
|
||||
|
||||
$failed = array();
|
||||
bratonien_tools_atomic_cache_remove_tree($detached, $failed);
|
||||
|
||||
@@ -162,6 +172,11 @@ function bratonien_tools_clear_image_cache_atomic()
|
||||
$before['custom']
|
||||
);
|
||||
|
||||
if ($webdav_invalidated > 0)
|
||||
{
|
||||
$message .= sprintf(' Die Cache-Fertigmarkierungen von %d WebDAV-Worker-Index(en) wurden verworfen; die Quellenindizes selbst bleiben erhalten.', $webdav_invalidated);
|
||||
}
|
||||
|
||||
if ($active['files'] > 0)
|
||||
{
|
||||
$message .= sprintf(
|
||||
|
||||
@@ -25,6 +25,7 @@ require_once(BRATONIEN_TOOLS_PATH . 'include/nc_connector_delete_safe.inc.php');
|
||||
require_once(BRATONIEN_TOOLS_PATH . 'include/nc_connector_webdav.inc.php');
|
||||
require_once(BRATONIEN_TOOLS_PATH . 'include/nc_connector_wizard_webdav_flow.inc.php');
|
||||
require_once(BRATONIEN_TOOLS_PATH . 'include/webdav_warmup_settings.inc.php');
|
||||
require_once(BRATONIEN_TOOLS_PATH . 'include/webdav_scan_diagnostic.inc.php');
|
||||
|
||||
function bratonien_tools_get_tools()
|
||||
{
|
||||
@@ -36,6 +37,7 @@ function bratonien_tools_get_tools()
|
||||
'image_cache_webdav_warmup_settings' => array('handler' => 'bratonien_tools_save_webdav_warmup_settings'),
|
||||
'image_cache_webdav_warmup_manual' => array('handler' => 'bratonien_tools_start_webdav_warmup_manual'),
|
||||
'image_cache_webdav_warmup_audit' => array('handler' => 'bratonien_tools_run_webdav_warmup_audit'),
|
||||
'image_cache_webdav_scan_diagnostic' => array('handler' => 'bratonien_tools_run_webdav_scan_diagnostic'),
|
||||
'watermark_save' => array('handler' => 'bratonien_tools_save_watermark'),
|
||||
'watermark_file_delete' => array('handler' => 'bratonien_tools_delete_watermark_file'),
|
||||
'watermark_engine' => array('handler' => 'bratonien_tools_handle_watermark_engine'),
|
||||
|
||||
79
include/webdav_scan_diagnostic.inc.php
Normal file
79
include/webdav_scan_diagnostic.inc.php
Normal file
@@ -0,0 +1,79 @@
|
||||
<?php
|
||||
if (!defined('PHPWG_ROOT_PATH'))
|
||||
{
|
||||
die('Hacking attempt!');
|
||||
}
|
||||
|
||||
function bratonien_tools_run_webdav_scan_diagnostic()
|
||||
{
|
||||
if (!function_exists('exec')) throw new RuntimeException('PHP exec() ist deaktiviert; der Scan-Test kann nicht gestartet werden.');
|
||||
if (!function_exists('bratonien_tools_nc_connector_connections') || !function_exists('bratonien_tools_nc_connector_is_webdav'))
|
||||
{
|
||||
throw new RuntimeException('Connector-Verbindungen koennen nicht gelesen werden.');
|
||||
}
|
||||
|
||||
$php = function_exists('bratonien_tools_webdav_warmup_php_cli') ? bratonien_tools_webdav_warmup_php_cli() : null;
|
||||
if (!$php) throw new RuntimeException('PHP-CLI wurde fuer den Scan-Test nicht gefunden.');
|
||||
|
||||
$script = realpath(BRATONIEN_TOOLS_PATH.'runtime/lib/webdav-scan-diagnostic.php');
|
||||
if (!$script || !is_file($script)) throw new RuntimeException('WebDAV-Scan-Diagnose wurde nicht gefunden.');
|
||||
|
||||
$reports = array();
|
||||
$failed = false;
|
||||
foreach (bratonien_tools_nc_connector_connections() as $connection)
|
||||
{
|
||||
if (empty($connection['enabled']) || !bratonien_tools_nc_connector_is_webdav($connection)) continue;
|
||||
$connection_id = (int)($connection['id'] ?? 0);
|
||||
if ($connection_id < 1) continue;
|
||||
|
||||
$output = array();
|
||||
$exit = 1;
|
||||
@exec(
|
||||
escapeshellarg($php).' '.escapeshellarg($script).' --connection-id='.$connection_id.' 2>&1',
|
||||
$output,
|
||||
$exit
|
||||
);
|
||||
$text = trim(implode("\n", $output));
|
||||
$decoded = $text !== '' ? json_decode($text, true) : null;
|
||||
if (!is_array($decoded))
|
||||
{
|
||||
$decoded = array(
|
||||
'ok'=>false,
|
||||
'connection_id'=>$connection_id,
|
||||
'fatal'=>'Diagnose lieferte kein gueltiges JSON.',
|
||||
'raw'=>$text,
|
||||
);
|
||||
}
|
||||
$decoded['exit_code'] = (int)$exit;
|
||||
$reports[] = $decoded;
|
||||
if ($exit !== 0 || empty($decoded['ok'])) $failed = true;
|
||||
}
|
||||
|
||||
if (!$reports) throw new RuntimeException('Keine aktive WebDAV-Verbindung gefunden.');
|
||||
|
||||
$parts = array();
|
||||
foreach ($reports as $report)
|
||||
{
|
||||
$id = (int)($report['connection_id'] ?? 0);
|
||||
if (!empty($report['fatal']))
|
||||
{
|
||||
$parts[] = 'WebDAV #'.$id.': FEHLER: '.(string)$report['fatal'].' (Exit '.(int)$report['exit_code'].')';
|
||||
continue;
|
||||
}
|
||||
$parts[] = sprintf(
|
||||
'WebDAV #%d: Mapping %d Dateien, Shadow %d Links, %d passend, %d defekt, %d ohne Mapping, %d doppelt (Exit %d).',
|
||||
$id,
|
||||
(int)($report['mapping_files'] ?? 0),
|
||||
(int)($report['shadow_links'] ?? 0),
|
||||
(int)($report['matched'] ?? 0),
|
||||
(int)($report['broken'] ?? 0),
|
||||
(int)($report['unmapped'] ?? 0),
|
||||
(int)($report['duplicates'] ?? 0),
|
||||
(int)$report['exit_code']
|
||||
);
|
||||
}
|
||||
|
||||
return array(
|
||||
'message'=>($failed ? 'Scan-Test fehlgeschlagen. ' : 'Scan-Test erfolgreich. ').implode(' ', $parts),
|
||||
);
|
||||
}
|
||||
@@ -4,15 +4,17 @@ if (!defined('PHPWG_ROOT_PATH'))
|
||||
die('Hacking attempt!');
|
||||
}
|
||||
|
||||
function bratonien_tools_webdav_source_index_file($connection_id)
|
||||
function bratonien_tools_webdav_source_index_file($state_dir)
|
||||
{
|
||||
return PHPWG_ROOT_PATH.PWG_LOCAL_DIR.'bratonien-webdav-source-index.connection-'.(int)$connection_id.'.json';
|
||||
$state_dir = rtrim((string)$state_dir, '/');
|
||||
if ($state_dir === '') throw new RuntimeException('Connector-State-Verzeichnis fehlt für den WebDAV-Quellenindex.');
|
||||
return $state_dir.'/webdav-cache-index.json';
|
||||
}
|
||||
|
||||
function bratonien_tools_webdav_source_index_empty($connection_id)
|
||||
{
|
||||
return array(
|
||||
'schema_version'=>1,
|
||||
'schema_version'=>2,
|
||||
'connection_id'=>(int)$connection_id,
|
||||
'updated_at'=>0,
|
||||
'last_periodic_at'=>0,
|
||||
@@ -20,9 +22,9 @@ function bratonien_tools_webdav_source_index_empty($connection_id)
|
||||
);
|
||||
}
|
||||
|
||||
function bratonien_tools_webdav_source_index_load($connection_id)
|
||||
function bratonien_tools_webdav_source_index_load($state_dir, $connection_id)
|
||||
{
|
||||
$file = bratonien_tools_webdav_source_index_file($connection_id);
|
||||
$file = bratonien_tools_webdav_source_index_file($state_dir);
|
||||
if (!is_file($file) || !is_readable($file)) return null;
|
||||
|
||||
$raw = @file_get_contents($file);
|
||||
@@ -36,16 +38,16 @@ function bratonien_tools_webdav_source_index_load($connection_id)
|
||||
return $index;
|
||||
}
|
||||
|
||||
function bratonien_tools_webdav_source_index_save($connection_id, array $index)
|
||||
function bratonien_tools_webdav_source_index_save($state_dir, $connection_id, array $index)
|
||||
{
|
||||
$file = bratonien_tools_webdav_source_index_file($connection_id);
|
||||
$file = bratonien_tools_webdav_source_index_file($state_dir);
|
||||
$directory = dirname($file);
|
||||
if (!is_dir($directory) && !@mkdir($directory, 0775, true) && !is_dir($directory))
|
||||
{
|
||||
throw new RuntimeException('WebDAV-Quellenindex-Verzeichnis konnte nicht angelegt werden.');
|
||||
}
|
||||
|
||||
$index['schema_version'] = 1;
|
||||
$index['schema_version'] = 2;
|
||||
$index['connection_id'] = (int)$connection_id;
|
||||
$index['updated_at'] = time();
|
||||
if (!isset($index['sources']) || !is_array($index['sources'])) $index['sources'] = array();
|
||||
@@ -71,10 +73,7 @@ function bratonien_tools_webdav_source_index_key(array $source)
|
||||
{
|
||||
$connection_id = (int)($source['connection_id'] ?? 0);
|
||||
$fileid = (int)($source['fileid'] ?? 0);
|
||||
if ($fileid > 0)
|
||||
{
|
||||
return 'c'.$connection_id.':f'.$fileid;
|
||||
}
|
||||
if ($fileid > 0) return 'c'.$connection_id.':f'.$fileid;
|
||||
|
||||
$root_fileid = (int)($source['root_fileid'] ?? 0);
|
||||
$path = trim((string)($source['webdav_path'] ?? ''), '/');
|
||||
@@ -96,11 +95,10 @@ function bratonien_tools_webdav_source_index_signature(array $source)
|
||||
)));
|
||||
}
|
||||
|
||||
function bratonien_tools_webdav_source_index_metadata(array $source, $image_id)
|
||||
function bratonien_tools_webdav_source_index_metadata(array $source, $shadow_relative)
|
||||
{
|
||||
$path = trim((string)($source['webdav_path'] ?? ''), '/');
|
||||
return array(
|
||||
'image_id'=>(int)$image_id,
|
||||
'fileid'=>(int)($source['fileid'] ?? 0),
|
||||
'root_fileid'=>(int)($source['root_fileid'] ?? 0),
|
||||
'webdav_path'=>$path,
|
||||
@@ -110,6 +108,7 @@ function bratonien_tools_webdav_source_index_metadata(array $source, $image_id)
|
||||
'content_type'=>(string)($source['content_type'] ?? $source['mime'] ?? ''),
|
||||
'width'=>(int)($source['width'] ?? 0),
|
||||
'height'=>(int)($source['height'] ?? 0),
|
||||
'shadow_relative'=>ltrim((string)$shadow_relative, '/'),
|
||||
'source_signature'=>bratonien_tools_webdav_source_index_signature($source),
|
||||
'last_seen_at'=>time(),
|
||||
);
|
||||
|
||||
@@ -4,6 +4,8 @@ if (!defined('PHPWG_ROOT_PATH'))
|
||||
die('Hacking attempt!');
|
||||
}
|
||||
|
||||
require_once(BRATONIEN_TOOLS_PATH.'include/webdav_source_index.inc.php');
|
||||
|
||||
function bratonien_tools_webdav_warmup_settings_file()
|
||||
{
|
||||
return PHPWG_ROOT_PATH.PWG_LOCAL_DIR.'bratonien-webdav-warmup.settings.json';
|
||||
@@ -32,23 +34,78 @@ function bratonien_tools_get_webdav_warmup_settings()
|
||||
return $settings;
|
||||
}
|
||||
|
||||
function bratonien_tools_webdav_warmup_missing_baselines()
|
||||
function bratonien_tools_webdav_warmup_connections()
|
||||
{
|
||||
if (!function_exists('bratonien_tools_nc_connector_connections') || !function_exists('bratonien_tools_nc_connector_is_webdav'))
|
||||
{
|
||||
return array('Connector-Verbindungen konnten nicht geprüft werden.');
|
||||
}
|
||||
$result = array();
|
||||
if (!function_exists('bratonien_tools_nc_connector_connections') || !function_exists('bratonien_tools_nc_connector_is_webdav')) return $result;
|
||||
|
||||
$missing = array();
|
||||
foreach (bratonien_tools_nc_connector_connections() as $connection)
|
||||
{
|
||||
if (empty($connection['enabled']) || !bratonien_tools_nc_connector_is_webdav($connection)) continue;
|
||||
$connection_id = (int)($connection['id'] ?? 0);
|
||||
if ($connection_id < 1) continue;
|
||||
$index_file = PHPWG_ROOT_PATH.PWG_LOCAL_DIR.'bratonien-webdav-source-index.connection-'.$connection_id.'.json';
|
||||
if (!is_file($index_file) || !is_readable($index_file)) $missing[] = '#'.$connection_id;
|
||||
$config = isset($connection['config']) && is_array($connection['config']) ? $connection['config'] : array();
|
||||
$state_dir = rtrim((string)($config['state_dir'] ?? ''), '/');
|
||||
if ($state_dir === '') continue;
|
||||
$result[$connection_id] = array('id'=>$connection_id, 'state_dir'=>$state_dir, 'connection'=>$connection);
|
||||
}
|
||||
return $missing;
|
||||
return $result;
|
||||
}
|
||||
|
||||
function bratonien_tools_webdav_warmup_status_file_for_connection($connection_id)
|
||||
{
|
||||
return PHPWG_ROOT_PATH.PWG_LOCAL_DIR.'bratonien-webdav-warmup.status-'.(int)$connection_id.'.json';
|
||||
}
|
||||
|
||||
function bratonien_tools_webdav_warmup_write_status($connection_id, $state, $message, array $extra=array())
|
||||
{
|
||||
$file = bratonien_tools_webdav_warmup_status_file_for_connection($connection_id);
|
||||
$payload = array_merge(array(
|
||||
'state'=>(string)$state,
|
||||
'message'=>(string)$message,
|
||||
'connection_id'=>(int)$connection_id,
|
||||
'updated_at'=>time(),
|
||||
), $extra);
|
||||
$json = json_encode($payload, JSON_UNESCAPED_UNICODE | JSON_UNESCAPED_SLASHES | JSON_PRETTY_PRINT);
|
||||
if ($json === false) throw new RuntimeException('WebDAV-Worker-Status konnte nicht serialisiert werden.');
|
||||
$directory = dirname($file);
|
||||
if (!is_dir($directory) && !@mkdir($directory, 0775, true) && !is_dir($directory)) throw new RuntimeException('WebDAV-Worker-Statusverzeichnis konnte nicht angelegt werden.');
|
||||
$tmp = $file.'.tmp-'.bin2hex(random_bytes(4));
|
||||
if (@file_put_contents($tmp, $json."\n", LOCK_EX) === false) throw new RuntimeException('WebDAV-Worker-Status konnte nicht geschrieben werden.');
|
||||
@chmod($tmp, 0664);
|
||||
if (!@rename($tmp, $file))
|
||||
{
|
||||
@unlink($tmp);
|
||||
throw new RuntimeException('WebDAV-Worker-Status konnte nicht atomar gespeichert werden.');
|
||||
}
|
||||
}
|
||||
|
||||
function bratonien_tools_invalidate_webdav_cache_completion($reason='Piwigo-Bildcache wurde geleert.')
|
||||
{
|
||||
$invalidated = 0;
|
||||
foreach (bratonien_tools_webdav_warmup_connections() as $connection_id=>$runtime)
|
||||
{
|
||||
$index = bratonien_tools_webdav_source_index_load($runtime['state_dir'], $connection_id);
|
||||
if (is_array($index))
|
||||
{
|
||||
foreach ($index['sources'] as &$entry)
|
||||
{
|
||||
if (!is_array($entry)) continue;
|
||||
unset($entry['stage1_signature'], $entry['stage2_signature']);
|
||||
}
|
||||
unset($entry);
|
||||
bratonien_tools_webdav_source_index_save($runtime['state_dir'], $connection_id, $index);
|
||||
$invalidated++;
|
||||
}
|
||||
|
||||
bratonien_tools_webdav_warmup_write_status(
|
||||
$connection_id,
|
||||
'idle',
|
||||
$reason.' Der Quellenindex bleibt erhalten; seine Cache-Fertigmarkierungen wurden verworfen.',
|
||||
array('cache_invalidated'=>true)
|
||||
);
|
||||
}
|
||||
return $invalidated;
|
||||
}
|
||||
|
||||
function bratonien_tools_save_webdav_warmup_settings()
|
||||
@@ -60,8 +117,6 @@ function bratonien_tools_save_webdav_warmup_settings()
|
||||
$batch_size = max(1, min(50, $batch_size));
|
||||
$periodic_hours = max(1, min(168, $periodic_hours));
|
||||
|
||||
$missing = $enabled ? bratonien_tools_webdav_warmup_missing_baselines() : array();
|
||||
|
||||
$payload = array(
|
||||
'enabled'=>$enabled,
|
||||
'batch_size'=>$batch_size,
|
||||
@@ -88,18 +143,12 @@ function bratonien_tools_save_webdav_warmup_settings()
|
||||
throw new RuntimeException('Warmup-Einstellungen konnten nicht atomar gespeichert werden.');
|
||||
}
|
||||
|
||||
$message = sprintf(
|
||||
'WebDAV-Cache-Warmup gespeichert: %s, %d Bilder pro Batch, Quellenindex-Abgleich alle %d Stunden.',
|
||||
return array('message'=>sprintf(
|
||||
'WebDAV-Worker gespeichert: %s, %d Bilder pro Batch, Shadow-Tree-Abgleich alle %d Stunden. Der Worker-Index wird bei Bedarf automatisch aus dem Shadow-Tree aufgebaut.',
|
||||
$enabled ? 'automatisch aktiv' : 'Automatik deaktiviert',
|
||||
$batch_size,
|
||||
$periodic_hours
|
||||
);
|
||||
if ($enabled && $missing)
|
||||
{
|
||||
$message .= ' Für '.implode(', ', $missing).' existiert noch kein Quellenindex; dieser wird beim ersten Worker-Lauf automatisch als Ausgangsbestand angelegt.';
|
||||
}
|
||||
|
||||
return array('message'=>$message);
|
||||
));
|
||||
}
|
||||
|
||||
function bratonien_tools_webdav_warmup_php_cli()
|
||||
@@ -118,29 +167,52 @@ function bratonien_tools_webdav_warmup_php_cli()
|
||||
|
||||
function bratonien_tools_start_webdav_warmup_mode($mode, $message)
|
||||
{
|
||||
if (!function_exists('exec')) throw new RuntimeException('PHP exec() ist deaktiviert; Warmup kann nicht gestartet werden.');
|
||||
if (!function_exists('exec')) throw new RuntimeException('PHP exec() ist deaktiviert; Worker kann nicht gestartet werden.');
|
||||
$php = bratonien_tools_webdav_warmup_php_cli();
|
||||
if (!$php) throw new RuntimeException('PHP-CLI wurde für den WebDAV-Warmup nicht gefunden.');
|
||||
if (!$php) throw new RuntimeException('PHP-CLI wurde für den WebDAV-Worker nicht gefunden.');
|
||||
$dispatcher = realpath(BRATONIEN_TOOLS_PATH.'runtime/webdav-warmup-dispatch.php');
|
||||
if (!$dispatcher || !is_file($dispatcher)) throw new RuntimeException('WebDAV-Warmup-Dispatcher wurde nicht gefunden.');
|
||||
if (!$dispatcher || !is_file($dispatcher)) throw new RuntimeException('WebDAV-Worker-Dispatcher wurde nicht gefunden.');
|
||||
if (!in_array($mode, array('manual','rebuild'), true)) throw new RuntimeException('Ungültiger manueller Worker-Modus.');
|
||||
|
||||
$connections = bratonien_tools_webdav_warmup_connections();
|
||||
if (!$connections) throw new RuntimeException('Keine aktive WebDAV-Verbindung mit vollständiger Runtime gefunden.');
|
||||
|
||||
$run_id = date('YmdHis').'-'.bin2hex(random_bytes(4));
|
||||
foreach ($connections as $connection_id=>$runtime)
|
||||
{
|
||||
bratonien_tools_webdav_warmup_write_status(
|
||||
$connection_id,
|
||||
'queued',
|
||||
$mode === 'rebuild'
|
||||
? 'Cache-Rebuild wurde angefordert. Der Worker wartet auf den Start und gleicht danach Shadow-Tree und eigenen Index ab.'
|
||||
: 'Quellenprüfung wurde angefordert. Der Worker wartet auf den Start und gleicht danach Shadow-Tree und eigenen Index ab.',
|
||||
array('mode'=>$mode, 'run_id'=>$run_id)
|
||||
);
|
||||
}
|
||||
|
||||
if (!in_array($mode, array('manual','rebuild'), true)) throw new RuntimeException('Ungültiger manueller Warmup-Modus.');
|
||||
$log = PHPWG_ROOT_PATH.PWG_LOCAL_DIR.'bratonien-webdav-warmup-dispatch.log';
|
||||
$command = 'nohup '.escapeshellarg($php).' '.escapeshellarg($dispatcher).' --mode='.escapeshellarg($mode).' >> '.escapeshellarg($log).' 2>&1 < /dev/null & echo $!';
|
||||
$output = array();
|
||||
$exit = 1;
|
||||
@exec($command, $output, $exit);
|
||||
$pid = isset($output[0]) ? (int)$output[0] : 0;
|
||||
if ($exit !== 0 || $pid <= 0) throw new RuntimeException('WebDAV-Warmup konnte nicht gestartet werden.');
|
||||
if ($exit !== 0 || $pid <= 0)
|
||||
{
|
||||
foreach ($connections as $connection_id=>$runtime)
|
||||
{
|
||||
bratonien_tools_webdav_warmup_write_status($connection_id, 'error', 'WebDAV-Worker konnte nicht gestartet werden.', array('mode'=>$mode, 'run_id'=>$run_id));
|
||||
}
|
||||
throw new RuntimeException('WebDAV-Worker konnte nicht gestartet werden.');
|
||||
}
|
||||
|
||||
return array('message'=>$message, 'pid'=>$pid);
|
||||
return array('message'=>$message, 'pid'=>$pid, 'run_id'=>$run_id);
|
||||
}
|
||||
|
||||
function bratonien_tools_start_webdav_warmup_manual()
|
||||
{
|
||||
return bratonien_tools_start_webdav_warmup_mode(
|
||||
'manual',
|
||||
'WebDAV-Quellenindex: Prüfung auf neue oder geänderte Bilder wurde gestartet.'
|
||||
'WebDAV-Worker: Shadow-Tree wird mit dem eigenen Index auf neue oder geänderte Quellen abgeglichen.'
|
||||
);
|
||||
}
|
||||
|
||||
@@ -148,14 +220,52 @@ function bratonien_tools_start_webdav_cache_rebuild()
|
||||
{
|
||||
return bratonien_tools_start_webdav_warmup_mode(
|
||||
'rebuild',
|
||||
'WebDAV-Bildcache: vollständiger Wiederaufbau durch Piwigo wurde gestartet.'
|
||||
'WebDAV-Bildcache: der vollständige aktuelle Shadow-Bestand wird erneut an Piwigo übergeben.'
|
||||
);
|
||||
}
|
||||
|
||||
function bratonien_tools_has_local_cache_sources()
|
||||
{
|
||||
if (!function_exists('bratonien_tools_webdav_materialize_source_info')) return true;
|
||||
|
||||
$result = pwg_query('SELECT id FROM '.IMAGES_TABLE.' ORDER BY id');
|
||||
while ($row = pwg_db_fetch_assoc($result))
|
||||
{
|
||||
$image_id = (int)($row['id'] ?? 0);
|
||||
if ($image_id < 1) continue;
|
||||
if (!bratonien_tools_webdav_materialize_source_info($image_id)) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
function bratonien_tools_start_combined_image_cache_build()
|
||||
{
|
||||
$main = bratonien_tools_start_main_cache_build();
|
||||
// Der WebDAV-Status muss vor jedem asynchronen Start auf queued wechseln.
|
||||
// Sonst kann die Admin-Seite den complete-Status des vorigen Laufs sofort
|
||||
// als Ergebnis des neuen Klicks interpretieren und die Abfrage beenden.
|
||||
$webdav = bratonien_tools_start_webdav_cache_rebuild();
|
||||
$main = array('started'=>false, 'message'=>'');
|
||||
|
||||
if (bratonien_tools_has_local_cache_sources())
|
||||
{
|
||||
$main = bratonien_tools_start_main_cache_build();
|
||||
}
|
||||
else
|
||||
{
|
||||
bratonien_tools_write_main_cache_status(array(
|
||||
'state'=>'complete',
|
||||
'message'=>'Lokaler Piwigo-Teil nicht erforderlich. Der WebDAV-Worker arbeitet ausschließlich mit Shadow-Tree und eigenem Index.',
|
||||
'total'=>0,
|
||||
'completed'=>0,
|
||||
'generated'=>0,
|
||||
'cached'=>0,
|
||||
'skipped'=>0,
|
||||
'errors'=>0,
|
||||
'current'=>'',
|
||||
'worker_count'=>0,
|
||||
));
|
||||
$main['message'] = 'Lokaler Piwigo-Teil nicht gestartet, weil keine lokalen Bildquellen vorhanden sind.';
|
||||
}
|
||||
|
||||
$parts = array();
|
||||
if (!empty($main['message'])) $parts[] = $main['message'];
|
||||
@@ -173,16 +283,13 @@ function bratonien_tools_run_webdav_warmup_audit()
|
||||
$php = bratonien_tools_webdav_warmup_php_cli();
|
||||
if (!$php) throw new RuntimeException('PHP-CLI wurde für den Pfadaudit nicht gefunden.');
|
||||
$audit = realpath(BRATONIEN_TOOLS_PATH.'runtime/lib/webdav-warmup-audit.php');
|
||||
if (!$audit || !is_file($audit)) throw new RuntimeException('WebDAV-Warmup-Pfadaudit wurde nicht gefunden.');
|
||||
if (!$audit || !is_file($audit)) throw new RuntimeException('WebDAV-Pfadaudit wurde nicht gefunden.');
|
||||
|
||||
$output = array();
|
||||
$exit = 1;
|
||||
@exec(escapeshellarg($php).' '.escapeshellarg($audit).' 2>&1', $output, $exit);
|
||||
$text = trim(implode(' | ', array_slice($output, -8)));
|
||||
if ($exit !== 0)
|
||||
{
|
||||
throw new RuntimeException('WebDAV-Pfadaudit meldet einen unsicheren Zustand'.($text !== '' ? ': '.$text : '.'));
|
||||
}
|
||||
if ($exit !== 0) throw new RuntimeException('WebDAV-Pfadaudit meldet einen unsicheren Zustand'.($text !== '' ? ': '.$text : '.'));
|
||||
return array('message'=>'WebDAV-Pfadaudit erfolgreich'.($text !== '' ? ': '.$text : '.'));
|
||||
}
|
||||
|
||||
@@ -190,7 +297,7 @@ function bratonien_tools_get_webdav_warmup_status()
|
||||
{
|
||||
$status = array(
|
||||
'state'=>'idle',
|
||||
'message'=>'Noch kein Quellenindex-/Warmup-Lauf protokolliert.',
|
||||
'message'=>'Noch kein Shadow-Tree-/Worker-Lauf protokolliert.',
|
||||
'updated_at'=>0,
|
||||
);
|
||||
$files = glob(PHPWG_ROOT_PATH.PWG_LOCAL_DIR.'bratonien-webdav-warmup.status-*.json');
|
||||
@@ -206,4 +313,4 @@ function bratonien_tools_get_webdav_warmup_status()
|
||||
if ($latest === null || (int)($data['updated_at'] ?? 0) > (int)($latest['updated_at'] ?? 0)) $latest = $data;
|
||||
}
|
||||
return is_array($latest) ? array_merge($status, $latest) : $status;
|
||||
}
|
||||
}
|
||||
@@ -40,11 +40,14 @@ function bratonien_tools_status_webdav_label(array $status)
|
||||
$state = (string)($status['state'] ?? 'idle');
|
||||
$prefix = $connection > 0 ? 'WebDAV #'.$connection.': ' : 'WebDAV: ';
|
||||
|
||||
if ($state === 'queued') return $prefix.((string)($status['message'] ?? '') !== '' ? (string)$status['message'] : 'Worker wartet auf den Start.');
|
||||
if ($state === 'scan')
|
||||
{
|
||||
$images = (int)($status['images'] ?? 0);
|
||||
$sources = (int)($status['shadow_sources'] ?? 0);
|
||||
$selected = (int)($status['selected'] ?? 0);
|
||||
return $prefix.'Quellenindex wird mit dem aktuellen Connector-Bestand verglichen · '.$images.' Quellen gefunden · '.$selected.' neu/geändert bzw. für Rebuild ausgewählt.';
|
||||
$removed = (int)($status['removed_from_index'] ?? 0);
|
||||
$created = !empty($status['index_created']);
|
||||
return $prefix.'Shadow-Tree wird mit dem eigenen Worker-Index verglichen · '.$sources.' Quellen im Shadow-Tree · '.$selected.' zu verarbeiten'.($removed > 0 ? ' · '.$removed.' gelöschte Quelle(n) aus dem Worker-Index entfernt' : '').($created ? ' · Worker-Index wurde neu angelegt' : '').'.';
|
||||
}
|
||||
if ($state === 'running')
|
||||
{
|
||||
@@ -52,28 +55,18 @@ function bratonien_tools_status_webdav_label(array $status)
|
||||
$batch = (int)($status['batch'] ?? 0);
|
||||
$requested = (int)($status['batch_requested'] ?? 0);
|
||||
$downloaded = (int)($status['batch_downloaded'] ?? 0);
|
||||
$parts = array($prefix.'Piwigo verarbeitet temporär bereitgestellte WebDAV-Originale');
|
||||
$completed = (int)($status['stage_completed'] ?? 0);
|
||||
$total = (int)($status['selected_total'] ?? 0);
|
||||
$parts = array($prefix.'Worker stellt Originale temporär bereit; Piwigo erzeugt die Derivate');
|
||||
if ($stage > 0) $parts[] = 'Stufe '.$stage;
|
||||
if ($batch > 0) $parts[] = 'Batch '.$batch;
|
||||
if ($requested > 0) $parts[] = $downloaded.' / '.$requested.' Quellen dieses Batches geladen';
|
||||
if ($total > 0) $parts[] = $completed.' / '.$total.' Quellen dieser Stufe abgeschlossen';
|
||||
return implode(' · ', $parts).'.';
|
||||
}
|
||||
if ($state === 'preempted')
|
||||
{
|
||||
return $prefix.'Stufe 2 wurde nach einem vollständigen Batch unterbrochen, damit neue Connector-Inhalte zuerst verarbeitet werden können.';
|
||||
}
|
||||
if ($state === 'baseline')
|
||||
{
|
||||
return $prefix.'Quellenindex wurde als Ausgangsbestand angelegt; noch keine Derivate wurden deshalb erzeugt.';
|
||||
}
|
||||
if ($state === 'complete')
|
||||
{
|
||||
return $prefix.((string)($status['message'] ?? '') !== '' ? (string)$status['message'] : 'Verarbeitung abgeschlossen.');
|
||||
}
|
||||
if ($state === 'error' || $state === 'fatal')
|
||||
{
|
||||
return $prefix.'FEHLER: '.((string)($status['message'] ?? '') !== '' ? (string)$status['message'] : 'WebDAV-Verarbeitung fehlgeschlagen.');
|
||||
}
|
||||
if ($state === 'preempted') return $prefix.'Stufe 2 wurde nach einem vollständigen Batch unterbrochen, damit neue Connector-Inhalte zuerst verarbeitet werden können.';
|
||||
if ($state === 'complete') return $prefix.((string)($status['message'] ?? '') !== '' ? (string)$status['message'] : 'Verarbeitung abgeschlossen.');
|
||||
if ($state === 'error' || $state === 'fatal') return $prefix.'FEHLER: '.((string)($status['message'] ?? '') !== '' ? (string)$status['message'] : 'WebDAV-Verarbeitung fehlgeschlagen.');
|
||||
return $prefix.((string)($status['message'] ?? '') !== '' ? (string)$status['message'] : 'wartet.');
|
||||
}
|
||||
|
||||
@@ -104,23 +97,31 @@ $webdav_error = false;
|
||||
$webdav_pending = false;
|
||||
$webdav_latest = 0;
|
||||
$webdav_lines = array();
|
||||
foreach ($webdav as $status)
|
||||
$webdav_all_complete = !empty($webdav);
|
||||
foreach ($webdav as &$status)
|
||||
{
|
||||
$state = (string)($status['state'] ?? 'idle');
|
||||
$webdav_latest = max($webdav_latest, (int)($status['updated_at'] ?? 0));
|
||||
if (in_array($state, array('scan','running'), true)) $webdav_active = true;
|
||||
$updated = (int)($status['updated_at'] ?? 0);
|
||||
$webdav_latest = max($webdav_latest, $updated);
|
||||
|
||||
if ($state === 'queued' && $updated > 0 && (time() - $updated) > 45)
|
||||
{
|
||||
$state = 'error';
|
||||
$status['state'] = 'error';
|
||||
$status['message'] = 'Worker wurde angefordert, hat aber innerhalb von 45 Sekunden keinen Scan gestartet.';
|
||||
}
|
||||
|
||||
if (in_array($state, array('queued','scan','running'), true)) $webdav_active = true;
|
||||
if ($state === 'preempted') $webdav_pending = true;
|
||||
if (in_array($state, array('error','fatal'), true)) $webdav_error = true;
|
||||
if ($state !== 'complete') $webdav_all_complete = false;
|
||||
$webdav_lines[] = bratonien_tools_status_webdav_label($status);
|
||||
}
|
||||
unset($status);
|
||||
|
||||
$local_state = (string)($local['state'] ?? 'idle');
|
||||
$local_updated = (int)($local['updated_at'] ?? 0);
|
||||
$local_stale = in_array($local_state, array('running','queued'), true) && $local_updated > 0 && (time() - $local_updated) > 45;
|
||||
|
||||
// Ein ruhiger lokaler Worker ist kein Stall des Gesamtaufbaus, solange der
|
||||
// getrennte WebDAV-Worker sichtbar weiterarbeitet. Der 45-Sekunden-Fehler darf
|
||||
// deshalb nur ausgelöst werden, wenn auch kein WebDAV-Teil aktiv ist.
|
||||
if ($local_stale && !$webdav_active)
|
||||
{
|
||||
$local['state'] = 'error';
|
||||
@@ -129,27 +130,11 @@ if ($local_stale && !$webdav_active)
|
||||
$local_state = 'error';
|
||||
}
|
||||
|
||||
$local_label = '';
|
||||
if ($local_state === 'running' || $local_state === 'queued')
|
||||
{
|
||||
$local_label = 'Lokaler Piwigo-Teil läuft: '.((string)($local['message'] ?? '') !== '' ? (string)$local['message'] : 'Cache-Varianten werden verarbeitet.');
|
||||
}
|
||||
elseif ($local_state === 'complete')
|
||||
{
|
||||
$local_label = 'Lokaler Piwigo-Teil fertig.';
|
||||
}
|
||||
elseif ($local_state === 'cancelled')
|
||||
{
|
||||
$local_label = 'Lokaler Piwigo-Teil wurde abgebrochen.';
|
||||
}
|
||||
elseif ($local_state === 'error')
|
||||
{
|
||||
$local_label = 'Lokaler Piwigo-Teil mit Fehler: '.((string)($local['message'] ?? '') !== '' ? (string)$local['message'] : 'unbekannter Fehler');
|
||||
}
|
||||
else
|
||||
{
|
||||
$local_label = 'Lokaler Piwigo-Teil: keine relevanten lokalen Bildquellen aktiv.';
|
||||
}
|
||||
if ($local_state === 'running' || $local_state === 'queued') $local_label = 'Lokaler Piwigo-Teil läuft: '.((string)($local['message'] ?? '') !== '' ? (string)$local['message'] : 'Cache-Varianten werden verarbeitet.');
|
||||
elseif ($local_state === 'complete') $local_label = 'Lokaler Piwigo-Teil fertig.';
|
||||
elseif ($local_state === 'cancelled') $local_label = 'Lokaler Piwigo-Teil wurde abgebrochen.';
|
||||
elseif ($local_state === 'error') $local_label = 'Lokaler Piwigo-Teil mit Fehler: '.((string)($local['message'] ?? '') !== '' ? (string)$local['message'] : 'unbekannter Fehler');
|
||||
else $local_label = 'Lokaler Piwigo-Teil: keine relevanten lokalen Bildquellen aktiv.';
|
||||
|
||||
$overall = $local;
|
||||
$overall['local'] = $local;
|
||||
@@ -159,9 +144,6 @@ $overall['updated_at'] = max($local_updated, $webdav_latest);
|
||||
if ($webdav_active)
|
||||
{
|
||||
$overall['state'] = 'running';
|
||||
// Während der WebDAV-Verarbeitung wären die lokalen Variantenzahlen als
|
||||
// Gesamtfortschritt irreführend. Die Oberfläche zeigt deshalb bewusst die
|
||||
// aktuelle Arbeitsphase statt eines falschen Prozentwerts.
|
||||
$overall['total'] = 0;
|
||||
$overall['completed'] = 0;
|
||||
$overall['generated'] = 0;
|
||||
@@ -185,7 +167,7 @@ elseif ($webdav_pending)
|
||||
$overall['message'] = 'Cache-Aufbau wartet auf priorisierte Connector-Inhalte. '.$local_label;
|
||||
$overall['current'] = implode(' ', $webdav_lines);
|
||||
}
|
||||
elseif ($webdav && $local_state === 'complete')
|
||||
elseif ($webdav_all_complete && $local_state === 'complete')
|
||||
{
|
||||
$overall['state'] = 'complete';
|
||||
$overall['message'] = 'Cache-Aufbau abgeschlossen. '.$local_label;
|
||||
@@ -193,8 +175,12 @@ elseif ($webdav && $local_state === 'complete')
|
||||
}
|
||||
else
|
||||
{
|
||||
// Ein alter complete-Status darf niemals einen neuen Lauf als fertig markieren.
|
||||
// Idle/unklare WebDAV-Zustände bleiben sichtbar, bis ein echter Lauf einen
|
||||
// aktuellen queued/scan/running/complete/error-Zustand geschrieben hat.
|
||||
$overall['state'] = $local_state === 'complete' ? 'idle' : $local_state;
|
||||
$overall['message'] = $local_label;
|
||||
if ($webdav_lines) $overall['current'] = implode(' ', $webdav_lines);
|
||||
}
|
||||
|
||||
echo json_encode($overall, JSON_UNESCAPED_UNICODE | JSON_UNESCAPED_SLASHES);
|
||||
echo json_encode($overall, JSON_UNESCAPED_UNICODE | JSON_UNESCAPED_SLASHES);
|
||||
@@ -1,7 +1,7 @@
|
||||
<?php
|
||||
/*
|
||||
Plugin Name: Bratonien Tools
|
||||
Version: 0.9.7.1.15
|
||||
Version: 0.9.7.1.20
|
||||
Description: Erweiterbare Administrationswerkzeuge fuer die Bratonien-Piwigo-Installation.
|
||||
Plugin URI: https://github.com/Terranom674/Piwigo_Bratonien_Tools
|
||||
Author: Bratonien
|
||||
|
||||
194
runtime/lib/piwigo-cache-request-by-shadow.php
Normal file
194
runtime/lib/piwigo-cache-request-by-shadow.php
Normal file
@@ -0,0 +1,194 @@
|
||||
<?php
|
||||
if (PHP_SAPI !== 'cli')
|
||||
{
|
||||
http_response_code(404);
|
||||
exit;
|
||||
}
|
||||
|
||||
$piwigo_root = realpath(dirname(__DIR__, 4));
|
||||
if ($piwigo_root === false)
|
||||
{
|
||||
fwrite(STDERR, "Piwigo-Root wurde nicht gefunden.\n");
|
||||
exit(1);
|
||||
}
|
||||
define('PHPWG_ROOT_PATH', rtrim($piwigo_root, DIRECTORY_SEPARATOR).DIRECTORY_SEPARATOR);
|
||||
|
||||
$_SERVER['REMOTE_ADDR'] = '127.0.0.1';
|
||||
$_SERVER['SERVER_ADDR'] = '127.0.0.1';
|
||||
$_SERVER['SERVER_NAME'] = 'localhost';
|
||||
$_SERVER['HTTP_HOST'] = 'localhost';
|
||||
$_SERVER['SERVER_PORT'] = '80';
|
||||
$_SERVER['REQUEST_METHOD'] = 'GET';
|
||||
$_SERVER['REQUEST_URI'] = '/';
|
||||
$_SERVER['SCRIPT_NAME'] = '/plugins/bratonien_tools/runtime/lib/piwigo-cache-request-by-shadow.php';
|
||||
$_SERVER['PHP_SELF'] = $_SERVER['SCRIPT_NAME'];
|
||||
$_SERVER['QUERY_STRING'] = '';
|
||||
$_SERVER['HTTP_USER_AGENT'] = 'Bratonien-Piwigo-Shadow-Cache-Bridge/0.9.7.1.17';
|
||||
$_SERVER['HTTPS'] = 'off';
|
||||
|
||||
require_once(PHPWG_ROOT_PATH.'include/common.inc.php');
|
||||
require_once(PHPWG_ROOT_PATH.'include/derivative.inc.php');
|
||||
if (!defined('BRATONIEN_TOOLS_PATH'))
|
||||
{
|
||||
fwrite(STDERR, "Bratonien Tools ist nicht aktiv.\n");
|
||||
exit(1);
|
||||
}
|
||||
require_once(BRATONIEN_TOOLS_PATH.'include/nc_connector.inc.php');
|
||||
require_once(BRATONIEN_TOOLS_PATH.'include/webdav_cache_validation.inc.php');
|
||||
require_once(BRATONIEN_TOOLS_PATH.'include/webdav_materialize_runtime.inc.php');
|
||||
|
||||
function bratonien_tools_shadow_bridge_fail($message, $code=1)
|
||||
{
|
||||
fwrite(STDERR, $message."\n");
|
||||
exit((int)$code);
|
||||
}
|
||||
|
||||
function bratonien_tools_shadow_bridge_i_request($target)
|
||||
{
|
||||
$root = PHPWG_ROOT_PATH.PWG_DERIVATIVE_DIR;
|
||||
$target = str_replace('\\', '/', (string)$target);
|
||||
$root = str_replace('\\', '/', (string)$root);
|
||||
if (strpos($target, $root) !== 0) return null;
|
||||
$relative = ltrim(substr($target, strlen($root)), '/');
|
||||
if ($relative === '') return null;
|
||||
return '/i.php?/'.implode('/', array_map('rawurlencode', explode('/', $relative)));
|
||||
}
|
||||
|
||||
function bratonien_tools_shadow_bridge_call_i($request, &$detail=null)
|
||||
{
|
||||
$detail = '';
|
||||
$runner = BRATONIEN_TOOLS_PATH.'runtime/lib/piwigo-derivative-call.php';
|
||||
if (!is_file($runner)) { $detail = 'Piwigo-Derivataufruf fehlt.'; return false; }
|
||||
if (!function_exists('exec')) { $detail = 'PHP exec() ist deaktiviert.'; return false; }
|
||||
$output = array();
|
||||
$exit = 1;
|
||||
@exec(escapeshellarg(PHP_BINARY).' '.escapeshellarg($runner).' --request='.escapeshellarg($request).' 2>&1', $output, $exit);
|
||||
if ($exit !== 0)
|
||||
{
|
||||
$detail = 'Piwigo-Aufruf Exit '.$exit;
|
||||
if ($output) $detail .= ': '.implode(' ', array_slice($output, -3));
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
function bratonien_tools_shadow_bridge_variants(array $row, $stage)
|
||||
{
|
||||
$src = new SrcImage($row);
|
||||
$variants = array();
|
||||
|
||||
foreach (ImageStdParams::get_defined_type_map() as $type=>$params)
|
||||
{
|
||||
$derivative = new DerivativeImage($params, $src);
|
||||
if ($derivative->same_as_source()) continue;
|
||||
$size = $derivative->get_size();
|
||||
$max_edge = max((int)($size[0] ?? 0), (int)($size[1] ?? 0));
|
||||
$priority = $max_edge > 0 && $max_edge <= 1920;
|
||||
if (($stage === 1 && $priority) || ($stage === 2 && !$priority))
|
||||
{
|
||||
$variants[] = array('name'=>'standard:'.$type, 'derivative'=>$derivative, 'params'=>$params, 'target'=>$derivative->get_path());
|
||||
}
|
||||
}
|
||||
|
||||
foreach (ImageStdParams::$custom as $key=>$last_used)
|
||||
{
|
||||
$params = bratonien_tools_webdav_materialize_custom_params($key);
|
||||
if (!$params) continue;
|
||||
$derivative = new DerivativeImage($params, $src);
|
||||
if ($derivative->same_as_source()) continue;
|
||||
$size = $derivative->get_size();
|
||||
$max_edge = max((int)($size[0] ?? 0), (int)($size[1] ?? 0));
|
||||
$priority = $max_edge > 0 && $max_edge <= 1920;
|
||||
if (($stage === 1 && $priority) || ($stage === 2 && !$priority))
|
||||
{
|
||||
$variants[] = array('name'=>'custom:'.$key, 'derivative'=>$derivative, 'params'=>$params, 'target'=>$derivative->get_path());
|
||||
}
|
||||
}
|
||||
|
||||
return $variants;
|
||||
}
|
||||
|
||||
$connection_id = 0;
|
||||
$shadow_relative = '';
|
||||
$stage = 0;
|
||||
foreach ($argv as $arg)
|
||||
{
|
||||
if (preg_match('/^--connection-id=(\d+)$/', $arg, $m)) $connection_id = (int)$m[1];
|
||||
elseif (strpos($arg, '--shadow-relative=') === 0) $shadow_relative = ltrim(substr($arg, strlen('--shadow-relative=')), '/');
|
||||
elseif (preg_match('/^--stage=([12])$/', $arg, $m)) $stage = (int)$m[1];
|
||||
}
|
||||
if ($connection_id < 1 || $shadow_relative === '' || !in_array($stage, array(1,2), true))
|
||||
{
|
||||
bratonien_tools_shadow_bridge_fail('Parameter --connection-id, --shadow-relative und --stage=1|2 werden benötigt.', 2);
|
||||
}
|
||||
if (strpos($shadow_relative, '..') !== false || strpos($shadow_relative, "\0") !== false)
|
||||
{
|
||||
bratonien_tools_shadow_bridge_fail('Unsicherer Shadow-Pfad.', 2);
|
||||
}
|
||||
|
||||
$connection = bratonien_tools_nc_connector_connection($connection_id, false);
|
||||
if (!$connection || empty($connection['enabled'])) bratonien_tools_shadow_bridge_fail('WebDAV-Verbindung wurde nicht gefunden.');
|
||||
$config = isset($connection['config']) && is_array($connection['config']) ? $connection['config'] : array();
|
||||
$gallery_root = rtrim((string)($config['parallel_gallery_root'] ?? ''), '/');
|
||||
if ($gallery_root === '' || !is_dir($gallery_root)) bratonien_tools_shadow_bridge_fail('Shadow-Tree der Verbindung fehlt.');
|
||||
|
||||
$shadow_absolute = $gallery_root.'/'.$shadow_relative;
|
||||
if (!is_link($shadow_absolute)) bratonien_tools_shadow_bridge_fail('Shadow-Datei ist kein Symlink: '.$shadow_relative);
|
||||
$shadow_real = realpath($shadow_absolute);
|
||||
if ($shadow_real === false || !is_file($shadow_real)) bratonien_tools_shadow_bridge_fail('Shadow-Ziel ist nicht lesbar: '.$shadow_relative);
|
||||
|
||||
$root = rtrim(str_replace('\\', '/', PHPWG_ROOT_PATH), '/');
|
||||
$gallery = rtrim(str_replace('\\', '/', $gallery_root), '/');
|
||||
if (strpos($gallery.'/', $root.'/') !== 0) bratonien_tools_shadow_bridge_fail('Shadow-Tree liegt außerhalb von Piwigo.');
|
||||
$relative_gallery = ltrim(substr($gallery, strlen($root)), '/');
|
||||
$db_path = './'.$relative_gallery.'/'.$shadow_relative;
|
||||
|
||||
$result = pwg_query("SELECT * FROM ".IMAGES_TABLE." WHERE path='".pwg_db_real_escape_string($db_path)."' LIMIT 2");
|
||||
$rows = array();
|
||||
while ($row = pwg_db_fetch_assoc($result)) $rows[] = $row;
|
||||
if (count($rows) !== 1)
|
||||
{
|
||||
bratonien_tools_shadow_bridge_fail('Piwigo kann den Shadow-Pfad nicht eindeutig einem Bild zuordnen: '.$db_path);
|
||||
}
|
||||
$row = $rows[0];
|
||||
|
||||
$variants = bratonien_tools_shadow_bridge_variants($row, $stage);
|
||||
$generated = 0;
|
||||
$already = 0;
|
||||
foreach ($variants as $variant)
|
||||
{
|
||||
$reason = '';
|
||||
if (bratonien_tools_webdav_derivative_cache_valid($variant['target'], $variant['derivative'], $variant['params'], $reason))
|
||||
{
|
||||
$already++;
|
||||
continue;
|
||||
}
|
||||
|
||||
$request = bratonien_tools_shadow_bridge_i_request($variant['target']);
|
||||
if ($request === null) bratonien_tools_shadow_bridge_fail($variant['name'].': ungültiger Piwigo-Derivatpfad.');
|
||||
|
||||
$detail = '';
|
||||
if (!bratonien_tools_shadow_bridge_call_i($request, $detail))
|
||||
{
|
||||
bratonien_tools_shadow_bridge_fail($variant['name'].': '.$detail);
|
||||
}
|
||||
clearstatcache(true, $variant['target']);
|
||||
$reason = '';
|
||||
if (!bratonien_tools_webdav_derivative_cache_valid($variant['target'], $variant['derivative'], $variant['params'], $reason))
|
||||
{
|
||||
bratonien_tools_shadow_bridge_fail($variant['name'].': Piwigo hat kein gültiges Derivat erzeugt ('.$reason.').');
|
||||
}
|
||||
$generated++;
|
||||
}
|
||||
|
||||
echo json_encode(array(
|
||||
'ok'=>true,
|
||||
'connection_id'=>$connection_id,
|
||||
'shadow_relative'=>$shadow_relative,
|
||||
'stage'=>$stage,
|
||||
'piwigo_image_id'=>(int)$row['id'],
|
||||
'variants'=>count($variants),
|
||||
'generated'=>$generated,
|
||||
'already_valid'=>$already,
|
||||
), JSON_UNESCAPED_UNICODE | JSON_UNESCAPED_SLASHES)."\n";
|
||||
exit(0);
|
||||
@@ -23,26 +23,22 @@ $_SERVER['REQUEST_URI'] = '/';
|
||||
$_SERVER['SCRIPT_NAME'] = '/plugins/bratonien_tools/runtime/lib/webdav-cache-warmup.php';
|
||||
$_SERVER['PHP_SELF'] = $_SERVER['SCRIPT_NAME'];
|
||||
$_SERVER['QUERY_STRING'] = '';
|
||||
$_SERVER['HTTP_USER_AGENT'] = 'Bratonien-WebDAV-Cache-Warmup/0.9.7.1.13';
|
||||
$_SERVER['HTTP_USER_AGENT'] = 'Bratonien-WebDAV-Cache-Worker/0.9.7.1.17';
|
||||
$_SERVER['HTTPS'] = 'off';
|
||||
|
||||
require_once(PHPWG_ROOT_PATH.'include/common.inc.php');
|
||||
require_once(PHPWG_ROOT_PATH.'include/derivative.inc.php');
|
||||
if (!defined('BRATONIEN_TOOLS_PATH'))
|
||||
{
|
||||
fwrite(STDERR, "Bratonien Tools ist nicht aktiv.\n");
|
||||
exit(1);
|
||||
}
|
||||
|
||||
require_once(BRATONIEN_TOOLS_PATH.'include/nc_connector.inc.php');
|
||||
require_once(BRATONIEN_TOOLS_PATH.'include/webdav_cache_validation.inc.php');
|
||||
require_once(BRATONIEN_TOOLS_PATH.'include/webdav_materialize_runtime.inc.php');
|
||||
require_once(BRATONIEN_TOOLS_PATH.'include/webdav_source_index.inc.php');
|
||||
require_once(BRATONIEN_TOOLS_PATH.'include/webdav_warmup_settings.inc.php');
|
||||
|
||||
function bratonien_tools_cache_warmup_log($event, array $fields=array())
|
||||
{
|
||||
$parts = array('[BRAT-WARMUP]', $event);
|
||||
$parts = array('[BRAT-WORKER]', $event);
|
||||
foreach ($fields as $key=>$value)
|
||||
{
|
||||
if (is_bool($value)) $value = $value ? '1' : '0';
|
||||
@@ -73,17 +69,17 @@ function bratonien_tools_cache_warmup_write_json($file, array $payload)
|
||||
$directory = dirname($file);
|
||||
if (!is_dir($directory) && !@mkdir($directory, 0775, true) && !is_dir($directory))
|
||||
{
|
||||
throw new RuntimeException('Warmup-Statusverzeichnis konnte nicht angelegt werden.');
|
||||
throw new RuntimeException('Worker-Statusverzeichnis konnte nicht angelegt werden.');
|
||||
}
|
||||
$json = json_encode($payload, JSON_UNESCAPED_UNICODE | JSON_UNESCAPED_SLASHES | JSON_PRETTY_PRINT);
|
||||
if ($json === false) throw new RuntimeException('Warmup-Status konnte nicht serialisiert werden.');
|
||||
if ($json === false) throw new RuntimeException('Worker-Status konnte nicht serialisiert werden.');
|
||||
$tmp = $file.'.tmp-'.bin2hex(random_bytes(4));
|
||||
if (@file_put_contents($tmp, $json."\n", LOCK_EX) === false) throw new RuntimeException('Warmup-Status konnte nicht geschrieben werden.');
|
||||
if (@file_put_contents($tmp, $json."\n", LOCK_EX) === false) throw new RuntimeException('Worker-Status konnte nicht geschrieben werden.');
|
||||
@chmod($tmp, 0664);
|
||||
if (!@rename($tmp, $file))
|
||||
{
|
||||
@unlink($tmp);
|
||||
throw new RuntimeException('Warmup-Status konnte nicht atomar gespeichert werden.');
|
||||
throw new RuntimeException('Worker-Status konnte nicht atomar gespeichert werden.');
|
||||
}
|
||||
}
|
||||
|
||||
@@ -112,7 +108,9 @@ function bratonien_tools_cache_warmup_credentials($connection_id)
|
||||
$base_url = rtrim((string)($config['nextcloud_url'] ?? ''), '/');
|
||||
$user = trim((string)($secret['nextcloud_user'] ?? ''));
|
||||
$password = (string)($secret['nextcloud_password'] ?? '');
|
||||
if ($base_url === '' || $user === '' || $password === '') return null;
|
||||
$state_dir = rtrim((string)($config['state_dir'] ?? ''), '/');
|
||||
$gallery_root = rtrim((string)($config['parallel_gallery_root'] ?? ''), '/');
|
||||
if ($base_url === '' || $user === '' || $password === '' || $state_dir === '' || $gallery_root === '') return null;
|
||||
|
||||
return array(
|
||||
'connection'=>$connection,
|
||||
@@ -120,163 +118,111 @@ function bratonien_tools_cache_warmup_credentials($connection_id)
|
||||
'base_url'=>$base_url,
|
||||
'user'=>$user,
|
||||
'password'=>$password,
|
||||
'state_dir'=>$state_dir,
|
||||
'gallery_root'=>$gallery_root,
|
||||
);
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_absolute_path($path)
|
||||
function bratonien_tools_cache_warmup_load_json($file, $label)
|
||||
{
|
||||
$path = (string)$path;
|
||||
if ($path === '') return null;
|
||||
if (strpos($path, '/') === 0) return $path;
|
||||
return PHPWG_ROOT_PATH.ltrim(preg_replace('#^\./#', '', $path), '/');
|
||||
if (!is_file($file) || !is_readable($file)) throw new RuntimeException($label.' fehlt oder ist nicht lesbar: '.$file);
|
||||
$raw = @file_get_contents($file);
|
||||
$data = $raw !== false ? json_decode($raw, true) : null;
|
||||
if (!is_array($data)) throw new RuntimeException($label.' enthält kein gültiges JSON.');
|
||||
return $data;
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_scan($connection_id)
|
||||
function bratonien_tools_cache_warmup_normalize_path($path)
|
||||
{
|
||||
$images = array();
|
||||
$resolved = array();
|
||||
return rtrim(str_replace('\\', '/', (string)$path), '/');
|
||||
}
|
||||
|
||||
$result = pwg_query('SELECT id, path, width, height, rotation, coi FROM '.IMAGES_TABLE.' ORDER BY id');
|
||||
while ($row = pwg_db_fetch_assoc($result))
|
||||
function bratonien_tools_cache_warmup_scan($connection_id, array $credentials)
|
||||
{
|
||||
$state_dir = $credentials['state_dir'];
|
||||
$gallery_root = $credentials['gallery_root'];
|
||||
$mapping = bratonien_tools_cache_warmup_load_json($state_dir.'/webdav-map.json', 'WebDAV-Mapping');
|
||||
$files = isset($mapping['files']) && is_array($mapping['files']) ? $mapping['files'] : array();
|
||||
if (!$files) throw new RuntimeException('WebDAV-Mapping enthält keine Quellen.');
|
||||
if (!is_dir($gallery_root)) throw new RuntimeException('Shadow-Tree fehlt: '.$gallery_root);
|
||||
|
||||
$by_source = array();
|
||||
foreach ($files as $path=>$meta)
|
||||
{
|
||||
$image_id = (int)$row['id'];
|
||||
$source = bratonien_tools_webdav_materialize_source_info($image_id);
|
||||
if (!$source || (int)$source['connection_id'] !== (int)$connection_id) continue;
|
||||
if (!is_array($meta) || (string)($meta['kind'] ?? '') !== 'file') continue;
|
||||
$normalized = bratonien_tools_cache_warmup_normalize_path($path);
|
||||
$by_source[$normalized] = $meta;
|
||||
}
|
||||
|
||||
$logical = bratonien_tools_cache_warmup_absolute_path($row['path'] ?? '');
|
||||
$real = $logical ? realpath($logical) : false;
|
||||
if ($real === false || !is_file($real))
|
||||
{
|
||||
throw new RuntimeException('Bild #'.$image_id.': bestehender Piwigo-Pfad kann nicht aufgelöst werden.');
|
||||
}
|
||||
$sources = array();
|
||||
$seen_targets = array();
|
||||
$gallery_prefix = bratonien_tools_cache_warmup_normalize_path(realpath($gallery_root) ?: $gallery_root).'/';
|
||||
$iterator = new RecursiveIteratorIterator(
|
||||
new RecursiveDirectoryIterator($gallery_root, FilesystemIterator::SKIP_DOTS),
|
||||
RecursiveIteratorIterator::LEAVES_ONLY
|
||||
);
|
||||
|
||||
$normalized = str_replace('\\', '/', $real);
|
||||
$expected = '/nc-webdav-source/connection-'.(int)$source['connection_id'].'/root-'.(int)$source['root_fileid'].'/';
|
||||
if (strpos($normalized, $expected) === false)
|
||||
{
|
||||
throw new RuntimeException('Bild #'.$image_id.': bestehender Piwigo-Pfad zeigt nicht auf die erwartete Connector-Wurzel.');
|
||||
}
|
||||
if (trim((string)($source['webdav_path'] ?? ''), '/') === '')
|
||||
{
|
||||
throw new RuntimeException('Bild #'.$image_id.': WebDAV-Mapping enthält keinen Pfad.');
|
||||
}
|
||||
if (isset($resolved[$normalized]) && $resolved[$normalized] !== $image_id)
|
||||
{
|
||||
throw new RuntimeException('Sicherheitsabbruch: Bild #'.$resolved[$normalized].' und Bild #'.$image_id.' zeigen auf dieselbe physische Connector-Quelle.');
|
||||
}
|
||||
$resolved[$normalized] = $image_id;
|
||||
foreach ($iterator as $item)
|
||||
{
|
||||
$shadow_absolute = $item->getPathname();
|
||||
if (!is_link($shadow_absolute)) continue;
|
||||
$resolved = realpath($shadow_absolute);
|
||||
if ($resolved === false || !is_file($resolved)) throw new RuntimeException('Shadow-Link zeigt auf keine lesbare Quelle: '.$shadow_absolute);
|
||||
$resolved = bratonien_tools_cache_warmup_normalize_path($resolved);
|
||||
if (!isset($by_source[$resolved])) throw new RuntimeException('Shadow-Link besitzt keinen Eintrag im Connector-Mapping: '.$shadow_absolute);
|
||||
if (isset($seen_targets[$resolved])) throw new RuntimeException('Sicherheitsabbruch: dieselbe Connector-Quelle ist mehrfach im Shadow-Tree verknüpft: '.$resolved);
|
||||
$seen_targets[$resolved] = true;
|
||||
|
||||
$row['id'] = $image_id;
|
||||
$index_key = bratonien_tools_webdav_source_index_key($source);
|
||||
$images[$image_id] = array(
|
||||
'row'=>$row,
|
||||
'source'=>$source,
|
||||
'logical'=>$logical,
|
||||
'resolved'=>$real,
|
||||
'index_key'=>$index_key,
|
||||
$shadow_normalized = bratonien_tools_cache_warmup_normalize_path($shadow_absolute);
|
||||
if (strpos($shadow_normalized.'/', $gallery_prefix) !== 0) throw new RuntimeException('Shadow-Datei liegt außerhalb der erwarteten Galerie-Wurzel.');
|
||||
$shadow_relative = ltrim(substr($shadow_normalized, strlen(rtrim($gallery_prefix, '/'))), '/');
|
||||
if ($shadow_relative === '') throw new RuntimeException('Leerer relativer Shadow-Pfad erkannt.');
|
||||
|
||||
$meta = $by_source[$resolved];
|
||||
$root_fileid = 0;
|
||||
if (preg_match('#/root-([0-9]+)/#', $resolved.'/', $m)) $root_fileid = (int)$m[1];
|
||||
if ($root_fileid < 1) throw new RuntimeException('Connector-Wurzel konnte aus der Placeholder-Quelle nicht bestimmt werden: '.$resolved);
|
||||
|
||||
$source = array(
|
||||
'connection_id'=>(int)$connection_id,
|
||||
'root_fileid'=>$root_fileid,
|
||||
'fileid'=>(int)($meta['fileid'] ?? 0),
|
||||
'webdav_path'=>trim((string)($meta['webdav_path'] ?? ''), '/'),
|
||||
'content_type'=>(string)($meta['content_type'] ?? ''),
|
||||
'size'=>(int)($meta['size'] ?? 0),
|
||||
'etag'=>(string)($meta['etag'] ?? ''),
|
||||
'width'=>(int)($meta['width'] ?? 0),
|
||||
'height'=>(int)($meta['height'] ?? 0),
|
||||
);
|
||||
if ($source['webdav_path'] === '') throw new RuntimeException('Connector-Mapping enthält für '.$shadow_relative.' keinen WebDAV-Pfad.');
|
||||
|
||||
$key = bratonien_tools_webdav_source_index_key($source);
|
||||
if (isset($sources[$key])) throw new RuntimeException('Doppelte Quellenidentität im Shadow-Tree: '.$key);
|
||||
$sources[$key] = array(
|
||||
'index_key'=>$key,
|
||||
'signature'=>bratonien_tools_webdav_source_index_signature($source),
|
||||
'source'=>$source,
|
||||
'source_path'=>$resolved,
|
||||
'shadow_absolute'=>$shadow_absolute,
|
||||
'shadow_relative'=>$shadow_relative,
|
||||
);
|
||||
}
|
||||
|
||||
return array('images'=>$images);
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_build_baseline($connection_id, array $scan)
|
||||
{
|
||||
$index = bratonien_tools_webdav_source_index_empty($connection_id);
|
||||
foreach ($scan['images'] as $image_id=>$image)
|
||||
{
|
||||
$entry = bratonien_tools_webdav_source_index_metadata($image['source'], $image_id);
|
||||
$entry['stage1_signature'] = $image['signature'];
|
||||
$entry['stage2_signature'] = $image['signature'];
|
||||
$entry['baseline'] = true;
|
||||
$index['sources'][$image['index_key']] = $entry;
|
||||
}
|
||||
$index['last_periodic_at'] = time();
|
||||
bratonien_tools_webdav_source_index_save($connection_id, $index);
|
||||
return $index;
|
||||
return array('sources'=>$sources);
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_select(array $scan, array $index, $mode)
|
||||
{
|
||||
if ($mode === 'rebuild') return $scan['images'];
|
||||
|
||||
if ($mode === 'rebuild') return $scan['sources'];
|
||||
$selected = array();
|
||||
foreach ($scan['images'] as $image_id=>$image)
|
||||
foreach ($scan['sources'] as $key=>$item)
|
||||
{
|
||||
$entry = isset($index['sources'][$image['index_key']]) && is_array($index['sources'][$image['index_key']])
|
||||
? $index['sources'][$image['index_key']]
|
||||
: null;
|
||||
|
||||
if ($entry === null || !bratonien_tools_webdav_source_index_is_current($entry, $image['signature']))
|
||||
{
|
||||
$selected[$image_id] = $image;
|
||||
}
|
||||
$entry = isset($index['sources'][$key]) && is_array($index['sources'][$key]) ? $index['sources'][$key] : null;
|
||||
if ($entry === null || !bratonien_tools_webdav_source_index_is_current($entry, $item['signature'])) $selected[$key] = $item;
|
||||
}
|
||||
return $selected;
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_variant_rows(array $image, $stage)
|
||||
{
|
||||
$src = new SrcImage($image['row']);
|
||||
$variants = array();
|
||||
foreach (ImageStdParams::get_defined_type_map() as $type=>$params)
|
||||
{
|
||||
$derivative = new DerivativeImage($params, $src);
|
||||
if ($derivative->same_as_source()) continue;
|
||||
$size = $derivative->get_size();
|
||||
$max_edge = max((int)($size[0] ?? 0), (int)($size[1] ?? 0));
|
||||
$priority = $max_edge > 0 && $max_edge <= 1920;
|
||||
if (($stage === 1 && $priority) || ($stage === 2 && !$priority))
|
||||
{
|
||||
$variants[] = array('name'=>'standard:'.$type, 'target'=>$derivative->get_path(), 'derivative'=>$derivative, 'params'=>$params);
|
||||
}
|
||||
}
|
||||
foreach (ImageStdParams::$custom as $key=>$last_used)
|
||||
{
|
||||
$params = bratonien_tools_webdav_materialize_custom_params($key);
|
||||
if (!$params) continue;
|
||||
$derivative = new DerivativeImage($params, $src);
|
||||
if ($derivative->same_as_source()) continue;
|
||||
$size = $derivative->get_size();
|
||||
$max_edge = max((int)($size[0] ?? 0), (int)($size[1] ?? 0));
|
||||
$priority = $max_edge > 0 && $max_edge <= 1920;
|
||||
if (($stage === 1 && $priority) || ($stage === 2 && !$priority))
|
||||
{
|
||||
$variants[] = array('name'=>'custom:'.$key, 'target'=>$derivative->get_path(), 'derivative'=>$derivative, 'params'=>$params);
|
||||
}
|
||||
}
|
||||
return $variants;
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_i_request($target)
|
||||
{
|
||||
$root = PHPWG_ROOT_PATH.PWG_DERIVATIVE_DIR;
|
||||
if (strpos($target, $root) !== 0) return null;
|
||||
$relative = ltrim(substr($target, strlen($root)), '/');
|
||||
if ($relative === '') return null;
|
||||
return '/i.php?/'.implode('/', array_map('rawurlencode', explode('/', $relative)));
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_call_piwigo($request, &$detail=null)
|
||||
{
|
||||
$detail = '';
|
||||
$runner = BRATONIEN_TOOLS_PATH.'runtime/lib/piwigo-derivative-call.php';
|
||||
if (!is_file($runner)) { $detail = 'Piwigo-Derivataufruf fehlt.'; return false; }
|
||||
if (!function_exists('exec')) { $detail = 'PHP exec() ist deaktiviert.'; return false; }
|
||||
$output = array();
|
||||
$exit = 1;
|
||||
$command = escapeshellarg(PHP_BINARY).' '.escapeshellarg($runner).' --request='.escapeshellarg($request).' 2>&1';
|
||||
@exec($command, $output, $exit);
|
||||
if ($exit !== 0)
|
||||
{
|
||||
$detail = 'Piwigo-Aufruf Exit '.$exit;
|
||||
if ($output) $detail .= ': '.implode(' ', array_slice($output, -3));
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_quote_path($path)
|
||||
{
|
||||
$parts = array_values(array_filter(explode('/', trim((string)$path, '/')), 'strlen'));
|
||||
@@ -301,7 +247,7 @@ function bratonien_tools_cache_warmup_download(array $source, array $credentials
|
||||
CURLOPT_RETURNTRANSFER=>false,
|
||||
CURLOPT_FAILONERROR=>false,
|
||||
CURLOPT_FILE=>$fp,
|
||||
CURLOPT_USERAGENT=>'Bratonien-WebDAV-Cache-Warmup/0.9.7.1.13',
|
||||
CURLOPT_USERAGENT=>'Bratonien-WebDAV-Cache-Worker/0.9.7.1.17',
|
||||
));
|
||||
$ok = curl_exec($ch);
|
||||
$errno = curl_errno($ch);
|
||||
@@ -328,9 +274,7 @@ function bratonien_tools_cache_warmup_sync_guard(array $credentials, &$handle=nu
|
||||
{
|
||||
$handle = null;
|
||||
$detail = '';
|
||||
$state_dir = rtrim((string)($credentials['config']['state_dir'] ?? ''), '/');
|
||||
if ($state_dir === '') { $detail = 'Connector-State-Verzeichnis fehlt.'; return false; }
|
||||
$handle = @fopen($state_dir.'/webdav-sync.lock', 'c');
|
||||
$handle = @fopen($credentials['state_dir'].'/webdav-sync.lock', 'c');
|
||||
if (!$handle) { $detail = 'Connector-Sync-Lock konnte nicht geöffnet werden.'; return false; }
|
||||
if (!@flock($handle, LOCK_SH | LOCK_NB))
|
||||
{
|
||||
@@ -342,11 +286,11 @@ function bratonien_tools_cache_warmup_sync_guard(array $credentials, &$handle=nu
|
||||
return true;
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_image_lock($image_id, &$handle=null)
|
||||
function bratonien_tools_cache_warmup_source_lock($index_key, &$handle=null)
|
||||
{
|
||||
$dir = PHPWG_ROOT_PATH.'upload/bratonien-webdav-materialize';
|
||||
if (!is_dir($dir) && !@mkdir($dir, 0775, true) && !is_dir($dir)) return false;
|
||||
$handle = @fopen($dir.'/image-'.(int)$image_id.'.lock', 'c');
|
||||
$handle = @fopen($dir.'/source-'.sha1((string)$index_key).'.lock', 'c');
|
||||
if (!$handle) return false;
|
||||
return @flock($handle, LOCK_EX);
|
||||
}
|
||||
@@ -361,6 +305,30 @@ function bratonien_tools_cache_warmup_unlock(&$handle)
|
||||
$handle = null;
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_fresh_source(array $item, array $credentials)
|
||||
{
|
||||
$mapping = bratonien_tools_cache_warmup_load_json($credentials['state_dir'].'/webdav-map.json', 'WebDAV-Mapping');
|
||||
$files = isset($mapping['files']) && is_array($mapping['files']) ? $mapping['files'] : array();
|
||||
$wanted = bratonien_tools_cache_warmup_normalize_path($item['source_path']);
|
||||
foreach ($files as $path=>$meta)
|
||||
{
|
||||
if (bratonien_tools_cache_warmup_normalize_path($path) !== $wanted || !is_array($meta)) continue;
|
||||
$source = array(
|
||||
'connection_id'=>(int)$item['source']['connection_id'],
|
||||
'root_fileid'=>(int)$item['source']['root_fileid'],
|
||||
'fileid'=>(int)($meta['fileid'] ?? 0),
|
||||
'webdav_path'=>trim((string)($meta['webdav_path'] ?? ''), '/'),
|
||||
'content_type'=>(string)($meta['content_type'] ?? ''),
|
||||
'size'=>(int)($meta['size'] ?? 0),
|
||||
'etag'=>(string)($meta['etag'] ?? ''),
|
||||
'width'=>(int)($meta['width'] ?? 0),
|
||||
'height'=>(int)($meta['height'] ?? 0),
|
||||
);
|
||||
return $source;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_restore($source_path, $backup, $staging, $original_size, $original_hash, &$detail=null)
|
||||
{
|
||||
$detail = '';
|
||||
@@ -379,51 +347,63 @@ function bratonien_tools_cache_warmup_restore($source_path, $backup, $staging, $
|
||||
return true;
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_process(array $image, $temp_file, array $credentials, $stage)
|
||||
function bratonien_tools_cache_warmup_call_piwigo(array $item, $stage, &$detail=null)
|
||||
{
|
||||
$image_id = (int)$image['row']['id'];
|
||||
$variants = bratonien_tools_cache_warmup_variant_rows($image, $stage);
|
||||
$requests = array();
|
||||
foreach ($variants as $variant)
|
||||
$detail = '';
|
||||
$bridge = BRATONIEN_TOOLS_PATH.'runtime/lib/piwigo-cache-request-by-shadow.php';
|
||||
if (!is_file($bridge)) { $detail = 'Piwigo-Shadow-Bridge fehlt.'; return false; }
|
||||
if (!function_exists('exec')) { $detail = 'PHP exec() ist deaktiviert.'; return false; }
|
||||
$output = array();
|
||||
$exit = 1;
|
||||
$command = escapeshellarg(PHP_BINARY).' '.escapeshellarg($bridge)
|
||||
.' --connection-id='.(int)$item['source']['connection_id']
|
||||
.' --shadow-relative='.escapeshellarg($item['shadow_relative'])
|
||||
.' --stage='.(int)$stage.' 2>&1';
|
||||
@exec($command, $output, $exit);
|
||||
if ($exit !== 0)
|
||||
{
|
||||
$request = bratonien_tools_cache_warmup_i_request($variant['target']);
|
||||
if ($request !== null) $requests[] = $variant + array('request'=>$request);
|
||||
$detail = $output ? implode(' ', array_slice($output, -4)) : 'Piwigo-Shadow-Bridge Exit '.$exit;
|
||||
return false;
|
||||
}
|
||||
if (!$requests) return array('ok'=>true, 'generated'=>0, 'message'=>'Für diese Stufe sind keine Piwigo-Derivate definiert.');
|
||||
$detail = $output ? (string)end($output) : '';
|
||||
return true;
|
||||
}
|
||||
|
||||
$image_lock = null;
|
||||
if (!bratonien_tools_cache_warmup_image_lock($image_id, $image_lock))
|
||||
function bratonien_tools_cache_warmup_process(array $item, $temp_file, array $credentials, $stage)
|
||||
{
|
||||
$source_lock = null;
|
||||
if (!bratonien_tools_cache_warmup_source_lock($item['index_key'], $source_lock))
|
||||
{
|
||||
return array('ok'=>false, 'defer'=>true, 'message'=>'Bild-Lock konnte nicht gesetzt werden.');
|
||||
return array('ok'=>false, 'defer'=>true, 'message'=>'Quellen-Lock konnte nicht gesetzt werden.');
|
||||
}
|
||||
$sync_lock = null;
|
||||
$detail = '';
|
||||
if (!bratonien_tools_cache_warmup_sync_guard($credentials, $sync_lock, $detail))
|
||||
{
|
||||
bratonien_tools_cache_warmup_unlock($image_lock);
|
||||
bratonien_tools_cache_warmup_unlock($source_lock);
|
||||
return array('ok'=>false, 'defer'=>true, 'message'=>$detail);
|
||||
}
|
||||
|
||||
$fresh = bratonien_tools_webdav_materialize_source_info($image_id);
|
||||
if (!$fresh || (int)$fresh['connection_id'] !== (int)$image['source']['connection_id'] || bratonien_tools_webdav_source_index_signature($fresh) !== $image['signature'])
|
||||
$fresh = bratonien_tools_cache_warmup_fresh_source($item, $credentials);
|
||||
if (!$fresh || bratonien_tools_webdav_source_index_signature($fresh) !== $item['signature'])
|
||||
{
|
||||
bratonien_tools_cache_warmup_unlock($sync_lock);
|
||||
bratonien_tools_cache_warmup_unlock($image_lock);
|
||||
return array('ok'=>false, 'defer'=>true, 'message'=>'WebDAV-Zuordnung hat sich seit dem Batch-Download geändert.');
|
||||
bratonien_tools_cache_warmup_unlock($source_lock);
|
||||
return array('ok'=>false, 'defer'=>true, 'message'=>'Connector-Mapping hat sich seit dem Batch-Download geändert.');
|
||||
}
|
||||
if (!is_link($item['shadow_absolute']) || bratonien_tools_cache_warmup_normalize_path(realpath($item['shadow_absolute']) ?: '') !== bratonien_tools_cache_warmup_normalize_path($item['source_path']))
|
||||
{
|
||||
bratonien_tools_cache_warmup_unlock($sync_lock);
|
||||
bratonien_tools_cache_warmup_unlock($source_lock);
|
||||
return array('ok'=>false, 'defer'=>true, 'message'=>'Shadow-Tree hat sich seit dem Batch-Download geändert.');
|
||||
}
|
||||
|
||||
$source_path = realpath($image['logical']);
|
||||
if ($source_path === false || $source_path !== $image['resolved'] || !is_file($source_path))
|
||||
$source_path = $item['source_path'];
|
||||
if (!is_file($source_path) || !is_readable($source_path) || !is_file($temp_file) || @getimagesize($temp_file) === false)
|
||||
{
|
||||
bratonien_tools_cache_warmup_unlock($sync_lock);
|
||||
bratonien_tools_cache_warmup_unlock($image_lock);
|
||||
return array('ok'=>false, 'fatal'=>true, 'message'=>'Piwigo-Quellpfad hat sich während des Warmups geändert.');
|
||||
}
|
||||
if (!is_file($temp_file) || !is_readable($temp_file) || @getimagesize($temp_file) === false)
|
||||
{
|
||||
bratonien_tools_cache_warmup_unlock($sync_lock);
|
||||
bratonien_tools_cache_warmup_unlock($image_lock);
|
||||
return array('ok'=>false, 'message'=>'Batch-Quelldatei ist nicht vollständig lesbar.');
|
||||
bratonien_tools_cache_warmup_unlock($source_lock);
|
||||
return array('ok'=>false, 'message'=>'Placeholder oder Batch-Quelle ist nicht vollständig lesbar.');
|
||||
}
|
||||
|
||||
$placeholder_size = filesize($source_path);
|
||||
@@ -431,7 +411,7 @@ function bratonien_tools_cache_warmup_process(array $image, $temp_file, array $c
|
||||
if (!is_string($placeholder_hash))
|
||||
{
|
||||
bratonien_tools_cache_warmup_unlock($sync_lock);
|
||||
bratonien_tools_cache_warmup_unlock($image_lock);
|
||||
bratonien_tools_cache_warmup_unlock($source_lock);
|
||||
return array('ok'=>false, 'fatal'=>true, 'message'=>'Placeholder konnte vor dem Swap nicht verifiziert werden.');
|
||||
}
|
||||
|
||||
@@ -441,19 +421,18 @@ function bratonien_tools_cache_warmup_process(array $image, $temp_file, array $c
|
||||
if (file_exists($backup) || file_exists($staging))
|
||||
{
|
||||
bratonien_tools_cache_warmup_unlock($sync_lock);
|
||||
bratonien_tools_cache_warmup_unlock($image_lock);
|
||||
bratonien_tools_cache_warmup_unlock($source_lock);
|
||||
return array('ok'=>false, 'fatal'=>true, 'message'=>'Unerwartete Swap-Datei ist bereits vorhanden.');
|
||||
}
|
||||
if (!@copy($temp_file, $staging) || @getimagesize($staging) === false)
|
||||
{
|
||||
@unlink($staging);
|
||||
bratonien_tools_cache_warmup_unlock($sync_lock);
|
||||
bratonien_tools_cache_warmup_unlock($image_lock);
|
||||
bratonien_tools_cache_warmup_unlock($source_lock);
|
||||
return array('ok'=>false, 'message'=>'Batch-Quelle konnte nicht sicher neben dem Placeholder bereitgestellt werden.');
|
||||
}
|
||||
|
||||
$swapped = false;
|
||||
$generated = 0;
|
||||
try
|
||||
{
|
||||
if (!@rename($source_path, $backup)) throw new RuntimeException('Placeholder konnte nicht atomar gesichert werden.');
|
||||
@@ -464,22 +443,12 @@ function bratonien_tools_cache_warmup_process(array $image, $temp_file, array $c
|
||||
}
|
||||
$swapped = true;
|
||||
clearstatcache(true, $source_path);
|
||||
if (@getimagesize($source_path) === false) throw new RuntimeException('Piwigo-Quellpfad enthält nach dem Swap kein lesbares Bild.');
|
||||
if (@getimagesize($source_path) === false) throw new RuntimeException('Shadow-Quelle enthält nach dem Swap kein lesbares Bild.');
|
||||
|
||||
foreach ($requests as $variant)
|
||||
$bridge_detail = '';
|
||||
if (!bratonien_tools_cache_warmup_call_piwigo($item, $stage, $bridge_detail))
|
||||
{
|
||||
$call_detail = '';
|
||||
if (!bratonien_tools_cache_warmup_call_piwigo($variant['request'], $call_detail))
|
||||
{
|
||||
throw new RuntimeException($variant['name'].': '.$call_detail);
|
||||
}
|
||||
clearstatcache(true, $variant['target']);
|
||||
$verify_reason = '';
|
||||
if (!bratonien_tools_webdav_derivative_cache_valid($variant['target'], $variant['derivative'], $variant['params'], $verify_reason))
|
||||
{
|
||||
throw new RuntimeException($variant['name'].': Piwigo hat kein gültiges Derivat erzeugt ('.$verify_reason.').');
|
||||
}
|
||||
$generated++;
|
||||
throw new RuntimeException('Piwigo-Verarbeitung fehlgeschlagen: '.$bridge_detail);
|
||||
}
|
||||
}
|
||||
catch (Throwable $e)
|
||||
@@ -490,7 +459,7 @@ function bratonien_tools_cache_warmup_process(array $image, $temp_file, array $c
|
||||
: true;
|
||||
if (!$swapped) @unlink($staging);
|
||||
bratonien_tools_cache_warmup_unlock($sync_lock);
|
||||
bratonien_tools_cache_warmup_unlock($image_lock);
|
||||
bratonien_tools_cache_warmup_unlock($source_lock);
|
||||
if (!$restored) return array('ok'=>false, 'fatal'=>true, 'message'=>$e->getMessage().' RESTORE FEHLGESCHLAGEN: '.$restore_detail);
|
||||
return array('ok'=>false, 'message'=>$e->getMessage());
|
||||
}
|
||||
@@ -498,115 +467,75 @@ function bratonien_tools_cache_warmup_process(array $image, $temp_file, array $c
|
||||
$restore_detail = '';
|
||||
$restored = bratonien_tools_cache_warmup_restore($source_path, $backup, $staging, $placeholder_size, $placeholder_hash, $restore_detail);
|
||||
bratonien_tools_cache_warmup_unlock($sync_lock);
|
||||
bratonien_tools_cache_warmup_unlock($image_lock);
|
||||
bratonien_tools_cache_warmup_unlock($source_lock);
|
||||
if (!$restored) return array('ok'=>false, 'fatal'=>true, 'message'=>'RESTORE FEHLGESCHLAGEN: '.$restore_detail);
|
||||
return array('ok'=>true, 'generated'=>$generated, 'message'=>'Piwigo hat die angeforderten Derivate verarbeitet.');
|
||||
return array('ok'=>true, 'message'=>'Piwigo hat den Shadow-Pfad verarbeitet.');
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_stage_pending(array $selected, array $index, $stage, $force_all=false)
|
||||
function bratonien_tools_cache_warmup_stage_pending(array $selected, array $index, $stage)
|
||||
{
|
||||
if ($force_all) return $selected;
|
||||
|
||||
$pending = array();
|
||||
$stage_key = $stage === 1 ? 'stage1_signature' : 'stage2_signature';
|
||||
foreach ($selected as $image_id=>$image)
|
||||
$field = $stage === 1 ? 'stage1_signature' : 'stage2_signature';
|
||||
foreach ($selected as $key=>$item)
|
||||
{
|
||||
$entry = isset($index['sources'][$image['index_key']]) && is_array($index['sources'][$image['index_key']])
|
||||
? $index['sources'][$image['index_key']]
|
||||
: array();
|
||||
if (!isset($entry[$stage_key]) || !hash_equals((string)$entry[$stage_key], $image['signature']))
|
||||
{
|
||||
$pending[$image_id] = $image;
|
||||
}
|
||||
$entry = isset($index['sources'][$key]) && is_array($index['sources'][$key]) ? $index['sources'][$key] : array();
|
||||
if (!isset($entry[$field]) || !hash_equals((string)$entry[$field], $item['signature'])) $pending[$key] = $item;
|
||||
}
|
||||
return $pending;
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_mark_stage_success($connection_id, array &$index, array $image, $image_id, $stage)
|
||||
function bratonien_tools_cache_warmup_run_stage($connection_id, $stage, array $selected, array &$index, array $credentials, $batch_size, $priority_file='')
|
||||
{
|
||||
$key = $image['index_key'];
|
||||
$previous = isset($index['sources'][$key]) && is_array($index['sources'][$key]) ? $index['sources'][$key] : array();
|
||||
$previous_signature = (string)($previous['source_signature'] ?? '');
|
||||
$entry = array_merge($previous, bratonien_tools_webdav_source_index_metadata($image['source'], $image_id));
|
||||
|
||||
if ($stage === 1)
|
||||
{
|
||||
$entry['stage1_signature'] = $image['signature'];
|
||||
if ($previous_signature === '' || !hash_equals($previous_signature, $image['signature'])) unset($entry['stage2_signature']);
|
||||
}
|
||||
else
|
||||
{
|
||||
$entry['stage2_signature'] = $image['signature'];
|
||||
}
|
||||
unset($entry['baseline']);
|
||||
$entry['last_processed_at'] = time();
|
||||
$index['sources'][$key] = $entry;
|
||||
bratonien_tools_webdav_source_index_save($connection_id, $index);
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_run_stage($connection_id, $stage, array $selected, array &$index, array $credentials, $batch_size, $priority_file='', $force_all=false)
|
||||
{
|
||||
$pending = bratonien_tools_cache_warmup_stage_pending($selected, $index, $stage, $force_all);
|
||||
if (!$pending) return array('ok'=>true, 'success'=>array(), 'failed'=>0, 'preempted'=>false);
|
||||
|
||||
$pending = bratonien_tools_cache_warmup_stage_pending($selected, $index, $stage);
|
||||
if (!$pending) return array('failed'=>0, 'preempted'=>false, 'completed'=>0);
|
||||
if ($stage === 2 && bratonien_tools_cache_warmup_priority_pending($priority_file))
|
||||
{
|
||||
bratonien_tools_cache_warmup_log('stage2_preempted', array('connection_id'=>$connection_id, 'point'=>'before_first_batch'));
|
||||
return array('ok'=>true, 'success'=>array(), 'failed'=>0, 'preempted'=>true);
|
||||
return array('failed'=>0, 'preempted'=>true, 'completed'=>0);
|
||||
}
|
||||
|
||||
$temp_root = PHPWG_ROOT_PATH.'upload/bratonien-webdav-warmup';
|
||||
if (!is_dir($temp_root) && !@mkdir($temp_root, 0775, true) && !is_dir($temp_root)) throw new RuntimeException('Warmup-Temp-Verzeichnis konnte nicht angelegt werden.');
|
||||
$success = array();
|
||||
if (!is_dir($temp_root) && !@mkdir($temp_root, 0775, true) && !is_dir($temp_root)) throw new RuntimeException('Worker-Temp-Verzeichnis konnte nicht angelegt werden.');
|
||||
$failed = 0;
|
||||
$completed = 0;
|
||||
$batch_number = 0;
|
||||
|
||||
foreach (array_chunk(array_keys($pending), max(1, (int)$batch_size)) as $ids)
|
||||
foreach (array_chunk(array_keys($pending), max(1, (int)$batch_size)) as $keys)
|
||||
{
|
||||
$batch_number++;
|
||||
$dir = $temp_root.'/connection-'.(int)$connection_id.'-stage'.$stage.'-'.date('YmdHis').'-'.bin2hex(random_bytes(4));
|
||||
if (!@mkdir($dir, 0775, true) && !is_dir($dir)) throw new RuntimeException('Warmup-Batch-Verzeichnis konnte nicht angelegt werden.');
|
||||
if (!@mkdir($dir, 0775, true) && !is_dir($dir)) throw new RuntimeException('Worker-Batch-Verzeichnis konnte nicht angelegt werden.');
|
||||
|
||||
$downloads = array();
|
||||
foreach ($ids as $image_id)
|
||||
foreach ($keys as $key)
|
||||
{
|
||||
$file = $dir.'/image-'.(int)$image_id.'.img';
|
||||
$file = $dir.'/'.sha1($key).'.img';
|
||||
$detail = '';
|
||||
if (bratonien_tools_cache_warmup_download($pending[$image_id]['source'], $credentials, $file, $detail))
|
||||
{
|
||||
$downloads[$image_id] = $file;
|
||||
}
|
||||
if (bratonien_tools_cache_warmup_download($pending[$key]['source'], $credentials, $file, $detail)) $downloads[$key] = $file;
|
||||
else
|
||||
{
|
||||
$failed++;
|
||||
bratonien_tools_cache_warmup_log('download_failed', array('stage'=>$stage, 'image_id'=>$image_id, 'detail'=>$detail));
|
||||
bratonien_tools_cache_warmup_log('download_failed', array('stage'=>$stage, 'source'=>$key, 'detail'=>$detail));
|
||||
}
|
||||
}
|
||||
|
||||
bratonien_tools_cache_warmup_status($connection_id, 'running', 'Warmup Stufe '.$stage.' läuft.', array(
|
||||
bratonien_tools_cache_warmup_status($connection_id, 'running', 'Piwigo verarbeitet WebDAV-Quellen aus dem Worker-Index.', array(
|
||||
'stage'=>$stage,
|
||||
'batch'=>$batch_number,
|
||||
'batch_requested'=>count($ids),
|
||||
'batch_requested'=>count($keys),
|
||||
'batch_downloaded'=>count($downloads),
|
||||
'source_index'=>count($index['sources']),
|
||||
'selected_total'=>count($selected),
|
||||
'stage_completed'=>$completed,
|
||||
));
|
||||
|
||||
foreach ($downloads as $image_id=>$file)
|
||||
foreach ($downloads as $key=>$file)
|
||||
{
|
||||
$result = bratonien_tools_cache_warmup_process($pending[$image_id], $file, $credentials, $stage);
|
||||
bratonien_tools_cache_warmup_log('image', array(
|
||||
'stage'=>$stage,
|
||||
'image_id'=>$image_id,
|
||||
'source_key'=>$pending[$image_id]['index_key'],
|
||||
'ok'=>!empty($result['ok']),
|
||||
'generated'=>(int)($result['generated'] ?? 0),
|
||||
'message'=>(string)($result['message'] ?? ''),
|
||||
));
|
||||
$result = bratonien_tools_cache_warmup_process($pending[$key], $file, $credentials, $stage);
|
||||
bratonien_tools_cache_warmup_log('source', array('stage'=>$stage, 'source'=>$key, 'ok'=>!empty($result['ok']), 'message'=>(string)($result['message'] ?? '')));
|
||||
if (!empty($result['fatal']))
|
||||
{
|
||||
foreach ($downloads as $cleanup) @unlink($cleanup);
|
||||
@rmdir($dir);
|
||||
throw new RuntimeException('Sicherheitsabbruch bei Bild #'.$image_id.': '.(string)$result['message']);
|
||||
throw new RuntimeException('Sicherheitsabbruch bei '.$key.': '.(string)$result['message']);
|
||||
}
|
||||
if (empty($result['ok']))
|
||||
{
|
||||
@@ -614,8 +543,13 @@ function bratonien_tools_cache_warmup_run_stage($connection_id, $stage, array $s
|
||||
continue;
|
||||
}
|
||||
|
||||
bratonien_tools_cache_warmup_mark_stage_success($connection_id, $index, $pending[$image_id], $image_id, $stage);
|
||||
$success[$image_id] = true;
|
||||
$item = $pending[$key];
|
||||
$existing = isset($index['sources'][$key]) && is_array($index['sources'][$key]) ? $index['sources'][$key] : array();
|
||||
$metadata = bratonien_tools_webdav_source_index_metadata($item['source'], $item['shadow_relative']);
|
||||
$index['sources'][$key] = array_merge($existing, $metadata);
|
||||
$index['sources'][$key][$stage === 1 ? 'stage1_signature' : 'stage2_signature'] = $item['signature'];
|
||||
bratonien_tools_webdav_source_index_save($credentials['state_dir'], $connection_id, $index);
|
||||
$completed++;
|
||||
}
|
||||
|
||||
foreach ($downloads as $file) @unlink($file);
|
||||
@@ -623,26 +557,21 @@ function bratonien_tools_cache_warmup_run_stage($connection_id, $stage, array $s
|
||||
|
||||
if ($stage === 2 && bratonien_tools_cache_warmup_priority_pending($priority_file))
|
||||
{
|
||||
bratonien_tools_cache_warmup_log('stage2_preempted', array('connection_id'=>$connection_id, 'point'=>'after_batch', 'batch'=>$batch_number));
|
||||
return array('ok'=>$failed === 0, 'success'=>$success, 'failed'=>$failed, 'preempted'=>true);
|
||||
return array('failed'=>$failed, 'preempted'=>true, 'completed'=>$completed);
|
||||
}
|
||||
}
|
||||
return array('ok'=>$failed === 0, 'success'=>$success, 'failed'=>$failed, 'preempted'=>false);
|
||||
|
||||
return array('failed'=>$failed, 'preempted'=>false, 'completed'=>$completed);
|
||||
}
|
||||
|
||||
function bratonien_tools_cache_warmup_run($connection_id, $mode)
|
||||
{
|
||||
$settings = bratonien_tools_get_webdav_warmup_settings();
|
||||
if (!in_array($mode, array('manual','rebuild'), true) && empty($settings['enabled']))
|
||||
{
|
||||
bratonien_tools_cache_warmup_log('disabled', array('connection_id'=>$connection_id, 'mode'=>$mode));
|
||||
return 0;
|
||||
}
|
||||
if (!in_array($mode, array('manual','rebuild'), true) && empty($settings['enabled'])) return 0;
|
||||
|
||||
$credentials = bratonien_tools_cache_warmup_credentials($connection_id);
|
||||
if (!$credentials) throw new RuntimeException('WebDAV-Verbindung oder Zugangsdaten konnten nicht geladen werden.');
|
||||
$state_dir = rtrim((string)($credentials['config']['state_dir'] ?? ''), '/');
|
||||
if ($state_dir === '') throw new RuntimeException('Connector-State-Verzeichnis fehlt.');
|
||||
if (!$credentials) throw new RuntimeException('WebDAV-Verbindung, Runtime-Pfade oder Zugangsdaten konnten nicht geladen werden.');
|
||||
$state_dir = $credentials['state_dir'];
|
||||
if (!is_dir($state_dir) && !@mkdir($state_dir, 0750, true) && !is_dir($state_dir)) throw new RuntimeException('Connector-State-Verzeichnis konnte nicht angelegt werden.');
|
||||
$priority_file = bratonien_tools_cache_warmup_priority_file($state_dir);
|
||||
|
||||
@@ -658,76 +587,62 @@ function bratonien_tools_cache_warmup_run($connection_id, $mode)
|
||||
{
|
||||
if ($mode === 'sync' && is_file($priority_file)) @unlink($priority_file);
|
||||
|
||||
$scan = bratonien_tools_cache_warmup_scan($connection_id);
|
||||
$index = bratonien_tools_webdav_source_index_load($connection_id);
|
||||
$index = bratonien_tools_webdav_source_index_load($state_dir, $connection_id);
|
||||
$index_created = false;
|
||||
if ($index === null)
|
||||
{
|
||||
if ($mode === 'rebuild')
|
||||
{
|
||||
$index = bratonien_tools_webdav_source_index_empty($connection_id);
|
||||
}
|
||||
else
|
||||
{
|
||||
$index = bratonien_tools_cache_warmup_build_baseline($connection_id, $scan);
|
||||
bratonien_tools_cache_warmup_status($connection_id, 'baseline', 'Bestehender Connector-Bestand wurde als Quellenindex erfasst.', array(
|
||||
'images'=>count($scan['images']),
|
||||
'indexed'=>count($index['sources']),
|
||||
));
|
||||
bratonien_tools_cache_warmup_log('baseline', array('connection_id'=>$connection_id, 'images'=>count($scan['images'])));
|
||||
return 0;
|
||||
}
|
||||
$index = bratonien_tools_webdav_source_index_empty($connection_id);
|
||||
$index_created = true;
|
||||
}
|
||||
|
||||
if ($mode === 'periodic')
|
||||
{
|
||||
$last = (int)($index['last_periodic_at'] ?? 0);
|
||||
$interval = max(1, (int)$settings['periodic_hours']) * 3600;
|
||||
if ($last > 0 && time() - $last < $interval)
|
||||
{
|
||||
bratonien_tools_cache_warmup_log('periodic_not_due', array('connection_id'=>$connection_id, 'remaining'=>$interval-(time()-$last)));
|
||||
return 0;
|
||||
}
|
||||
if ($last > 0 && time() - $last < $interval) return 0;
|
||||
}
|
||||
|
||||
$current_keys = array();
|
||||
foreach ($scan['images'] as $image) $current_keys[] = $image['index_key'];
|
||||
$removed = bratonien_tools_webdav_source_index_prune($index, $current_keys);
|
||||
|
||||
$scan = bratonien_tools_cache_warmup_scan($connection_id, $credentials);
|
||||
$removed = bratonien_tools_webdav_source_index_prune($index, array_keys($scan['sources']));
|
||||
$selected = bratonien_tools_cache_warmup_select($scan, $index, $mode);
|
||||
bratonien_tools_cache_warmup_status($connection_id, 'scan', 'Quellenindex-Abgleich abgeschlossen.', array(
|
||||
|
||||
bratonien_tools_cache_warmup_status($connection_id, 'scan', 'Worker-Index wurde ausschließlich mit dem aktuellen Shadow-Tree abgeglichen.', array(
|
||||
'mode'=>$mode,
|
||||
'images'=>count($scan['images']),
|
||||
'indexed'=>count($index['sources']),
|
||||
'shadow_sources'=>count($scan['sources']),
|
||||
'selected'=>count($selected),
|
||||
'removed'=>$removed,
|
||||
'removed_from_index'=>$removed,
|
||||
'index_created'=>$index_created,
|
||||
));
|
||||
|
||||
if (!$selected)
|
||||
{
|
||||
if ($mode === 'periodic' || $mode === 'manual') $index['last_periodic_at'] = time();
|
||||
bratonien_tools_webdav_source_index_save($connection_id, $index);
|
||||
bratonien_tools_cache_warmup_status($connection_id, 'complete', 'Quellenindex unverändert; keine Arbeit für Piwigo.', array(
|
||||
'mode'=>$mode,
|
||||
'images'=>count($scan['images']),
|
||||
'removed'=>$removed,
|
||||
));
|
||||
bratonien_tools_webdav_source_index_save($state_dir, $connection_id, $index);
|
||||
bratonien_tools_cache_warmup_status($connection_id, 'complete', 'Keine neuen oder geänderten Shadow-Quellen gefunden.', array('mode'=>$mode, 'shadow_sources'=>count($scan['sources'])));
|
||||
return 0;
|
||||
}
|
||||
|
||||
$force_all = $mode === 'rebuild';
|
||||
$stage1 = bratonien_tools_cache_warmup_run_stage($connection_id, 1, $selected, $index, $credentials, $settings['batch_size'], $priority_file, $force_all);
|
||||
|
||||
$stage2_candidates = array();
|
||||
foreach ($selected as $image_id=>$image)
|
||||
if ($mode === 'rebuild')
|
||||
{
|
||||
$entry = isset($index['sources'][$image['index_key']]) && is_array($index['sources'][$image['index_key']]) ? $index['sources'][$image['index_key']] : array();
|
||||
if (isset($entry['stage1_signature']) && hash_equals((string)$entry['stage1_signature'], $image['signature'])) $stage2_candidates[$image_id] = $image;
|
||||
foreach ($selected as $key=>$item)
|
||||
{
|
||||
if (!isset($index['sources'][$key]) || !is_array($index['sources'][$key])) $index['sources'][$key] = array();
|
||||
unset($index['sources'][$key]['stage1_signature'], $index['sources'][$key]['stage2_signature']);
|
||||
}
|
||||
}
|
||||
$stage2 = bratonien_tools_cache_warmup_run_stage($connection_id, 2, $stage2_candidates, $index, $credentials, $settings['batch_size'], $priority_file, $force_all);
|
||||
|
||||
$stage1 = bratonien_tools_cache_warmup_run_stage($connection_id, 1, $selected, $index, $credentials, $settings['batch_size'], $priority_file);
|
||||
$stage2_candidates = array();
|
||||
foreach ($selected as $key=>$item)
|
||||
{
|
||||
$entry = isset($index['sources'][$key]) && is_array($index['sources'][$key]) ? $index['sources'][$key] : array();
|
||||
if (isset($entry['stage1_signature']) && hash_equals((string)$entry['stage1_signature'], $item['signature'])) $stage2_candidates[$key] = $item;
|
||||
}
|
||||
$stage2 = bratonien_tools_cache_warmup_run_stage($connection_id, 2, $stage2_candidates, $index, $credentials, $settings['batch_size'], $priority_file);
|
||||
|
||||
if (!empty($stage2['preempted']))
|
||||
{
|
||||
bratonien_tools_cache_warmup_status($connection_id, 'preempted', 'Warmup Stufe 2 wurde nach einem vollständigen Batch für neue Sync-Priorität freigegeben.', array(
|
||||
bratonien_tools_cache_warmup_status($connection_id, 'preempted', 'Stufe 2 wurde nach einem vollständigen Batch für neue Connector-Inhalte freigegeben.', array(
|
||||
'mode'=>$mode,
|
||||
'stage1_failed'=>(int)$stage1['failed'],
|
||||
'stage2_failed'=>(int)$stage2['failed'],
|
||||
@@ -736,16 +651,17 @@ function bratonien_tools_cache_warmup_run($connection_id, $mode)
|
||||
}
|
||||
|
||||
if ($mode === 'periodic' || $mode === 'manual') $index['last_periodic_at'] = time();
|
||||
bratonien_tools_webdav_source_index_save($connection_id, $index);
|
||||
bratonien_tools_webdav_source_index_save($state_dir, $connection_id, $index);
|
||||
|
||||
$failed = (int)$stage1['failed'] + (int)$stage2['failed'];
|
||||
bratonien_tools_cache_warmup_status($connection_id, $failed ? 'error' : 'complete', $failed ? 'Warmup mit einzelnen Fehlern beendet.' : ($mode === 'rebuild' ? 'WebDAV-Bildcache durch Piwigo vollständig neu angefordert.' : 'Quellenänderungen durch Piwigo verarbeitet.'), array(
|
||||
bratonien_tools_cache_warmup_status($connection_id, $failed ? 'error' : 'complete', $failed ? 'Worker-Lauf mit einzelnen Fehlern beendet.' : ($mode === 'rebuild' ? 'Piwigo hat den vollständigen Shadow-Bestand für den Cache-Rebuild verarbeitet.' : 'Worker-Lauf vollständig beendet.'), array(
|
||||
'mode'=>$mode,
|
||||
'shadow_sources'=>count($scan['sources']),
|
||||
'selected'=>count($selected),
|
||||
'removed'=>$removed,
|
||||
'stage1_completed'=>(int)$stage1['completed'],
|
||||
'stage2_completed'=>(int)$stage2['completed'],
|
||||
'stage1_failed'=>(int)$stage1['failed'],
|
||||
'stage2_failed'=>(int)$stage2['failed'],
|
||||
'indexed'=>count($index['sources']),
|
||||
));
|
||||
return $failed ? 2 : 0;
|
||||
}
|
||||
@@ -776,6 +692,6 @@ try
|
||||
catch (Throwable $e)
|
||||
{
|
||||
try { bratonien_tools_cache_warmup_status($connection_id, 'fatal', $e->getMessage()); } catch (Throwable $ignored) {}
|
||||
fwrite(STDERR, '[BRAT-WARMUP] fatal '.$e->getMessage()."\n");
|
||||
fwrite(STDERR, '[BRAT-WORKER] fatal '.$e->getMessage()."\n");
|
||||
exit(1);
|
||||
}
|
||||
|
||||
152
runtime/lib/webdav-scan-diagnostic.php
Normal file
152
runtime/lib/webdav-scan-diagnostic.php
Normal file
@@ -0,0 +1,152 @@
|
||||
#!/usr/bin/env php
|
||||
<?php
|
||||
if (PHP_SAPI !== 'cli')
|
||||
{
|
||||
fwrite(STDERR, "CLI only\n");
|
||||
exit(1);
|
||||
}
|
||||
|
||||
$connection_id = 0;
|
||||
foreach ($argv as $arg)
|
||||
{
|
||||
if (preg_match('/^--connection-id=(\d+)$/', $arg, $m)) $connection_id = (int)$m[1];
|
||||
}
|
||||
if ($connection_id < 1)
|
||||
{
|
||||
fwrite(STDERR, "--connection-id fehlt oder ist ungueltig.\n");
|
||||
exit(2);
|
||||
}
|
||||
|
||||
$piwigo_root = realpath(dirname(__DIR__, 4));
|
||||
if ($piwigo_root === false)
|
||||
{
|
||||
fwrite(STDERR, "Piwigo-Root wurde nicht gefunden.\n");
|
||||
exit(3);
|
||||
}
|
||||
define('PHPWG_ROOT_PATH', rtrim($piwigo_root, DIRECTORY_SEPARATOR).DIRECTORY_SEPARATOR);
|
||||
|
||||
$_SERVER['REMOTE_ADDR'] = '127.0.0.1';
|
||||
$_SERVER['SERVER_ADDR'] = '127.0.0.1';
|
||||
$_SERVER['SERVER_NAME'] = 'localhost';
|
||||
$_SERVER['HTTP_HOST'] = 'localhost';
|
||||
$_SERVER['SERVER_PORT'] = '80';
|
||||
$_SERVER['REQUEST_METHOD'] = 'GET';
|
||||
$_SERVER['REQUEST_URI'] = '/';
|
||||
$_SERVER['SCRIPT_NAME'] = '/plugins/bratonien_tools/runtime/lib/webdav-scan-diagnostic.php';
|
||||
$_SERVER['PHP_SELF'] = $_SERVER['SCRIPT_NAME'];
|
||||
$_SERVER['QUERY_STRING'] = '';
|
||||
$_SERVER['HTTP_USER_AGENT'] = 'Bratonien-WebDAV-Scan-Diagnostic/0.9.7.1.20';
|
||||
$_SERVER['HTTPS'] = 'off';
|
||||
|
||||
try
|
||||
{
|
||||
require_once(PHPWG_ROOT_PATH.'include/common.inc.php');
|
||||
if (!defined('BRATONIEN_TOOLS_PATH')) throw new RuntimeException('Bratonien Tools ist nicht aktiv.');
|
||||
require_once(BRATONIEN_TOOLS_PATH.'include/nc_connector.inc.php');
|
||||
|
||||
$connection = bratonien_tools_nc_connector_connection($connection_id, true);
|
||||
if (!$connection || empty($connection['enabled'])) throw new RuntimeException('WebDAV-Verbindung wurde nicht gefunden oder ist deaktiviert.');
|
||||
if (!bratonien_tools_nc_connector_is_webdav($connection)) throw new RuntimeException('Verbindung ist keine WebDAV-Placeholder-Verbindung.');
|
||||
|
||||
$config = isset($connection['config']) && is_array($connection['config']) ? $connection['config'] : array();
|
||||
$state_dir = rtrim((string)($config['state_dir'] ?? ''), '/');
|
||||
$gallery_root = rtrim((string)($config['parallel_gallery_root'] ?? ''), '/');
|
||||
if ($state_dir === '') throw new RuntimeException('Connector-State-Verzeichnis fehlt.');
|
||||
if ($gallery_root === '') throw new RuntimeException('Shadow-Tree-Pfad fehlt in der Connector-Konfiguration.');
|
||||
|
||||
$mapping_file = $state_dir.'/webdav-map.json';
|
||||
if (!is_file($mapping_file) || !is_readable($mapping_file)) throw new RuntimeException('WebDAV-Mapping fehlt oder ist nicht lesbar: '.$mapping_file);
|
||||
$mapping = json_decode((string)file_get_contents($mapping_file), true);
|
||||
if (!is_array($mapping) || !isset($mapping['files']) || !is_array($mapping['files'])) throw new RuntimeException('WebDAV-Mapping ist ungueltig.');
|
||||
if (!is_dir($gallery_root)) throw new RuntimeException('Shadow-Tree fehlt: '.$gallery_root);
|
||||
|
||||
$normalize = static function ($path) {
|
||||
return rtrim(str_replace('\\', '/', (string)$path), '/');
|
||||
};
|
||||
|
||||
$mapped_files = array();
|
||||
foreach ($mapping['files'] as $path=>$meta)
|
||||
{
|
||||
if (!is_array($meta) || (string)($meta['kind'] ?? '') !== 'file') continue;
|
||||
$mapped_files[$normalize($path)] = $meta;
|
||||
}
|
||||
|
||||
$shadow_links = 0;
|
||||
$matched = 0;
|
||||
$broken = 0;
|
||||
$unmapped = 0;
|
||||
$duplicates = 0;
|
||||
$seen = array();
|
||||
$examples = array();
|
||||
|
||||
$iterator = new RecursiveIteratorIterator(
|
||||
new RecursiveDirectoryIterator($gallery_root, FilesystemIterator::SKIP_DOTS),
|
||||
RecursiveIteratorIterator::LEAVES_ONLY
|
||||
);
|
||||
foreach ($iterator as $item)
|
||||
{
|
||||
$shadow = $item->getPathname();
|
||||
if (!is_link($shadow)) continue;
|
||||
$shadow_links++;
|
||||
$resolved = realpath($shadow);
|
||||
if ($resolved === false || !is_file($resolved))
|
||||
{
|
||||
$broken++;
|
||||
if (count($examples) < 8) $examples[] = array('state'=>'broken','shadow'=>$shadow);
|
||||
continue;
|
||||
}
|
||||
$resolved = $normalize($resolved);
|
||||
if (!isset($mapped_files[$resolved]))
|
||||
{
|
||||
$unmapped++;
|
||||
if (count($examples) < 8) $examples[] = array('state'=>'unmapped','shadow'=>$shadow,'target'=>$resolved);
|
||||
continue;
|
||||
}
|
||||
if (isset($seen[$resolved]))
|
||||
{
|
||||
$duplicates++;
|
||||
if (count($examples) < 8) $examples[] = array('state'=>'duplicate','shadow'=>$shadow,'target'=>$resolved);
|
||||
continue;
|
||||
}
|
||||
$seen[$resolved] = true;
|
||||
$matched++;
|
||||
if (count($examples) < 8)
|
||||
{
|
||||
$meta = $mapped_files[$resolved];
|
||||
$examples[] = array(
|
||||
'state'=>'matched',
|
||||
'shadow'=>$shadow,
|
||||
'target'=>$resolved,
|
||||
'fileid'=>(int)($meta['fileid'] ?? 0),
|
||||
'webdav_path'=>(string)($meta['webdav_path'] ?? ''),
|
||||
'etag'=>(string)($meta['etag'] ?? ''),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
$payload = array(
|
||||
'ok'=>($shadow_links > 0 && $matched === $shadow_links && $broken === 0 && $unmapped === 0 && $duplicates === 0),
|
||||
'connection_id'=>$connection_id,
|
||||
'state_dir'=>$state_dir,
|
||||
'gallery_root'=>$gallery_root,
|
||||
'mapping_file'=>$mapping_file,
|
||||
'mapping_files'=>count($mapped_files),
|
||||
'shadow_links'=>$shadow_links,
|
||||
'matched'=>$matched,
|
||||
'broken'=>$broken,
|
||||
'unmapped'=>$unmapped,
|
||||
'duplicates'=>$duplicates,
|
||||
'examples'=>$examples,
|
||||
);
|
||||
echo json_encode($payload, JSON_UNESCAPED_UNICODE | JSON_UNESCAPED_SLASHES | JSON_PRETTY_PRINT)."\n";
|
||||
exit($payload['ok'] ? 0 : 4);
|
||||
}
|
||||
catch (Throwable $e)
|
||||
{
|
||||
echo json_encode(array(
|
||||
'ok'=>false,
|
||||
'connection_id'=>$connection_id,
|
||||
'fatal'=>$e->getMessage(),
|
||||
), JSON_UNESCAPED_UNICODE | JSON_UNESCAPED_SLASHES | JSON_PRETTY_PRINT)."\n";
|
||||
exit(1);
|
||||
}
|
||||
@@ -23,7 +23,7 @@ $_SERVER['REQUEST_URI'] = '/';
|
||||
$_SERVER['SCRIPT_NAME'] = '/plugins/bratonien_tools/runtime/webdav-warmup-dispatch.php';
|
||||
$_SERVER['PHP_SELF'] = $_SERVER['SCRIPT_NAME'];
|
||||
$_SERVER['QUERY_STRING'] = '';
|
||||
$_SERVER['HTTP_USER_AGENT'] = 'Bratonien-WebDAV-Warmup-Dispatcher/0.9.7.1.12';
|
||||
$_SERVER['HTTP_USER_AGENT'] = 'Bratonien-WebDAV-Worker-Dispatcher/0.9.7.1.19';
|
||||
$_SERVER['HTTPS'] = 'off';
|
||||
|
||||
require_once(PHPWG_ROOT_PATH.'include/common.inc.php');
|
||||
@@ -48,12 +48,12 @@ foreach ($argv as $arg)
|
||||
$settings = bratonien_tools_get_webdav_warmup_settings();
|
||||
if (!in_array($mode, array('manual','rebuild'), true) && empty($settings['enabled']))
|
||||
{
|
||||
fwrite(STDOUT, "WebDAV-Cache-Warmup ist deaktiviert.\n");
|
||||
fwrite(STDOUT, "WebDAV-Worker ist deaktiviert.\n");
|
||||
exit(0);
|
||||
}
|
||||
if (!function_exists('exec'))
|
||||
{
|
||||
fwrite(STDERR, "PHP exec() ist deaktiviert; Warmup kann nicht gestartet werden.\n");
|
||||
fwrite(STDERR, "PHP exec() ist deaktiviert; Worker kann nicht gestartet werden.\n");
|
||||
exit(1);
|
||||
}
|
||||
|
||||
@@ -61,7 +61,7 @@ $worker = realpath(BRATONIEN_TOOLS_PATH.'runtime/lib/webdav-cache-warmup.php');
|
||||
$priority_waiter = realpath(BRATONIEN_TOOLS_PATH.'runtime/lib/run-webdav-priority-wait.sh');
|
||||
if (!$worker || !is_file($worker))
|
||||
{
|
||||
fwrite(STDERR, "WebDAV-Cache-Warmup-Worker wurde nicht gefunden.\n");
|
||||
fwrite(STDERR, "WebDAV-Worker wurde nicht gefunden.\n");
|
||||
exit(1);
|
||||
}
|
||||
if (!$priority_waiter || !is_file($priority_waiter))
|
||||
@@ -74,6 +74,14 @@ $log = PHPWG_ROOT_PATH.PWG_LOCAL_DIR.'bratonien-webdav-warmup.log';
|
||||
$result = 0;
|
||||
$started = 0;
|
||||
$matched = 0;
|
||||
|
||||
// Ein manueller Lauf wird bereits vom Admin-Handler als eigener nohup-
|
||||
// Dispatcher gestartet. Innerhalb dieses Hintergrundprozesses darf nicht noch
|
||||
// einmal ein zweiter, abgekoppelter nohup-Prozess erzeugt werden: sonst kann
|
||||
// der Dispatcher erfolgreich enden, obwohl der eigentliche Worker nie bis zum
|
||||
// Scan kommt. Manual/Rebuild laufen deshalb im Dispatcher-Prozess selbst.
|
||||
$run_inline = $wait || in_array($mode, array('manual','rebuild'), true);
|
||||
|
||||
foreach (bratonien_tools_nc_connector_connections() as $connection)
|
||||
{
|
||||
if (empty($connection['enabled']) || !bratonien_tools_nc_connector_is_webdav($connection)) continue;
|
||||
@@ -86,37 +94,16 @@ foreach (bratonien_tools_nc_connector_connections() as $connection)
|
||||
$state_dir = rtrim((string)($config['state_dir'] ?? ''), '/');
|
||||
if ($state_dir === '')
|
||||
{
|
||||
fwrite(STDERR, "Warmup für Verbindung #{$connection_id} übersprungen: Connector-State-Verzeichnis fehlt.\n");
|
||||
fwrite(STDERR, "Worker für Verbindung #{$connection_id} übersprungen: Connector-State-Verzeichnis fehlt.\n");
|
||||
$result = 1;
|
||||
continue;
|
||||
}
|
||||
|
||||
if ($mode === 'periodic')
|
||||
{
|
||||
$state_file = PHPWG_ROOT_PATH.PWG_LOCAL_DIR.'bratonien-webdav-warmup.connection-'.$connection_id.'.json';
|
||||
if (is_file($state_file) && is_readable($state_file))
|
||||
{
|
||||
$raw = @file_get_contents($state_file);
|
||||
$state = $raw !== false ? json_decode($raw, true) : null;
|
||||
if (is_array($state))
|
||||
{
|
||||
$last = (int)($state['last_periodic_at'] ?? 0);
|
||||
$interval = max(1, (int)$settings['periodic_hours']) * 3600;
|
||||
if ($last > 0 && time() - $last < $interval)
|
||||
{
|
||||
$remaining = max(0, $interval - (time() - $last));
|
||||
fwrite(STDOUT, "Periodische Warmup-Prüfung für Verbindung #{$connection_id} noch nicht fällig ({$remaining}s verbleibend).\n");
|
||||
continue;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if ($mode === 'sync')
|
||||
{
|
||||
if (!is_dir($state_dir) && !@mkdir($state_dir, 0750, true) && !is_dir($state_dir))
|
||||
{
|
||||
fwrite(STDERR, "Warmup-Priorität für Verbindung #{$connection_id} konnte nicht signalisiert werden: State-Verzeichnis fehlt.\n");
|
||||
fwrite(STDERR, "Worker-Priorität für Verbindung #{$connection_id} konnte nicht signalisiert werden: State-Verzeichnis fehlt.\n");
|
||||
$result = 1;
|
||||
continue;
|
||||
}
|
||||
@@ -128,7 +115,7 @@ foreach (bratonien_tools_nc_connector_connections() as $connection)
|
||||
), JSON_UNESCAPED_SLASHES);
|
||||
if (!is_string($priority_payload) || @file_put_contents($priority_file, $priority_payload."\n", LOCK_EX) === false)
|
||||
{
|
||||
fwrite(STDERR, "Warmup-Priorität für Verbindung #{$connection_id} konnte nicht geschrieben werden.\n");
|
||||
fwrite(STDERR, "Worker-Priorität für Verbindung #{$connection_id} konnte nicht geschrieben werden.\n");
|
||||
$result = 1;
|
||||
continue;
|
||||
}
|
||||
@@ -138,23 +125,38 @@ foreach (bratonien_tools_nc_connector_connections() as $connection)
|
||||
$worker_command = escapeshellarg(PHP_BINARY).' '.escapeshellarg($worker)
|
||||
.' --connection-id='.$connection_id
|
||||
.' --mode='.escapeshellarg($mode);
|
||||
$base = $mode === 'sync'
|
||||
? escapeshellarg('/bin/bash').' '.escapeshellarg($priority_waiter).' '.escapeshellarg($state_dir).' '.$worker_command
|
||||
: $worker_command;
|
||||
|
||||
if ($mode === 'sync')
|
||||
{
|
||||
$base = escapeshellarg('/bin/bash').' '.escapeshellarg($priority_waiter).' '.escapeshellarg($state_dir).' '.$worker_command;
|
||||
}
|
||||
else
|
||||
{
|
||||
$base = $worker_command;
|
||||
}
|
||||
|
||||
if ($wait)
|
||||
if ($run_inline)
|
||||
{
|
||||
$output = array();
|
||||
$exit = 1;
|
||||
@exec($base.' 2>&1', $output, $exit);
|
||||
foreach ($output as $line) fwrite(STDOUT, $line."\n");
|
||||
if ($exit !== 0) $result = $exit;
|
||||
foreach ($output as $line)
|
||||
{
|
||||
fwrite(STDOUT, $line."\n");
|
||||
@file_put_contents($log, $line."\n", FILE_APPEND | LOCK_EX);
|
||||
}
|
||||
if ($exit !== 0)
|
||||
{
|
||||
$result = $exit;
|
||||
if (function_exists('bratonien_tools_webdav_warmup_write_status'))
|
||||
{
|
||||
$detail = $output ? implode(' ', array_slice($output, -4)) : 'Worker Exit '.$exit;
|
||||
bratonien_tools_webdav_warmup_write_status(
|
||||
$connection_id,
|
||||
'error',
|
||||
'WebDAV-Worker wurde gestartet, ist aber vor erfolgreichem Abschluss beendet worden: '.$detail,
|
||||
array('mode'=>$mode)
|
||||
);
|
||||
}
|
||||
}
|
||||
else
|
||||
{
|
||||
$started++;
|
||||
}
|
||||
}
|
||||
else
|
||||
{
|
||||
@@ -165,12 +167,12 @@ foreach (bratonien_tools_nc_connector_connections() as $connection)
|
||||
$pid = isset($output[0]) ? (int)$output[0] : 0;
|
||||
if ($exit !== 0 || $pid <= 0)
|
||||
{
|
||||
fwrite(STDERR, "Warmup für Verbindung #{$connection_id} konnte nicht gestartet werden.\n");
|
||||
fwrite(STDERR, "Worker für Verbindung #{$connection_id} konnte nicht gestartet werden.\n");
|
||||
$result = 1;
|
||||
continue;
|
||||
}
|
||||
$started++;
|
||||
fwrite(STDOUT, "Warmup {$mode} für Verbindung #{$connection_id} gestartet (PID {$pid}).\n");
|
||||
fwrite(STDOUT, "WebDAV-Worker {$mode} für Verbindung #{$connection_id} gestartet (PID {$pid}).\n");
|
||||
}
|
||||
}
|
||||
|
||||
@@ -179,5 +181,5 @@ if ($connection_filter > 0 && $matched === 0)
|
||||
fwrite(STDERR, "WebDAV-Verbindung #{$connection_filter} wurde nicht als aktive Connector-Verbindung gefunden.\n");
|
||||
exit(1);
|
||||
}
|
||||
if (!$wait) fwrite(STDOUT, "Gestartete Warmup-Prozesse: {$started}.\n");
|
||||
if (!$run_inline) fwrite(STDOUT, "Gestartete WebDAV-Worker: {$started}.\n");
|
||||
exit($result);
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
<div id="bratonien-webdav-warmup-card" class="bratonien-card" style="margin-top:16px;">
|
||||
<h4>WebDAV-Quellenindex & Cache-Warmup</h4>
|
||||
<p>Bratonien Tools vergleicht den aktuellen Nextcloud-/Connector-Bestand mit einem eigenen kompakten Quellenindex. Nur neue, geänderte oder noch nicht vollständig verarbeitete Quellen werden an Piwigo übergeben. Der Piwigo-Bildcache selbst entscheidet nicht, ob eine Quelle neu ist.</p>
|
||||
<p class="bratonien-main-cache__warning"><strong>Patchphase 0.9.7.1.13:</strong> Der Quellenindex speichert unter anderem File-ID, Pfad/Name, ETag, Größe, Format und Abmessungen. Die eigentlichen Derivate erzeugt ausschließlich Piwigo über seinen normalen <code>i.php</code>-Pfad.</p>
|
||||
<p>Bratonien Tools vergleicht den aktuellen Shadow-Tree mit einem eigenen kompakten Quellenindex. Nur neue, geänderte oder noch nicht vollständig verarbeitete Quellen werden an Piwigo übergeben. Der Piwigo-Bildcache selbst entscheidet nicht, ob eine Quelle neu ist.</p>
|
||||
<p class="bratonien-main-cache__warning"><strong>Diagnose:</strong> Der Scan-Test startet synchron eine eigene PHP-CLI-Prüfung und liest ausschließlich Connector-Konfiguration, <code>webdav-map.json</code> und den Shadow-Tree. Er schreibt keinen Index, lädt keine Originale und erzeugt keine Derivate. Damit sehen wir direkt, ob der Scan-Prozess überhaupt starten kann und wie viele Shadow-Links er tatsächlich findet.</p>
|
||||
|
||||
<form method="post" class="bratonien-worker-settings">
|
||||
<input type="hidden" name="pwg_token" value="{$PWG_TOKEN|escape:html}">
|
||||
@@ -12,13 +12,17 @@
|
||||
</form>
|
||||
|
||||
<div class="bratonien-form-grid" style="margin-top:12px;">
|
||||
<span class="bratonien-label">Änderungserkennung</span><span>Connector-Bestand ↔ eigener Quellenindex; Piwigo-Placeholder und Cache-Dateien sind dafür nicht maßgeblich</span>
|
||||
<span class="bratonien-label">Änderungserkennung</span><span>Shadow-Tree ↔ eigener Quellenindex; Piwigo-Index und Cache-Dateien sind dafür nicht maßgeblich</span>
|
||||
<span class="bratonien-label">Stufe 1</span><span>Von Piwigo definierte Derivate bis einschließlich 1920 px längster Kante</span>
|
||||
<span class="bratonien-label">Stufe 2</span><span>Danach alle übrigen von Piwigo definierten Derivate</span>
|
||||
<span class="bratonien-label">Letzter Status</span><span><strong>{$CACHE_WORKERS.webdav_warmup.status.state|default:'idle'|escape:html}</strong>{if $CACHE_WORKERS.webdav_warmup.status.message} · {$CACHE_WORKERS.webdav_warmup.status.message|escape:html}{/if}</span>
|
||||
</div>
|
||||
|
||||
<div class="bratonien-actions">
|
||||
<form method="post">
|
||||
<input type="hidden" name="pwg_token" value="{$PWG_TOKEN|escape:html}">
|
||||
<button class="buttonLike" type="submit" name="bratonien_tool" value="image_cache_webdav_scan_diagnostic">Scan jetzt testen</button>
|
||||
</form>
|
||||
<form method="post">
|
||||
<input type="hidden" name="pwg_token" value="{$PWG_TOKEN|escape:html}">
|
||||
<button class="buttonLike" type="submit" name="bratonien_tool" value="image_cache_webdav_warmup_manual">Quellenindex jetzt abgleichen</button>
|
||||
@@ -28,7 +32,7 @@
|
||||
<button class="buttonLike" type="submit" name="bratonien_tool" value="image_cache_webdav_warmup_audit">Produktive Pfade prüfen</button>
|
||||
</form>
|
||||
</div>
|
||||
<p class="bratonien-base-note">Der Pfadaudit ist schreibgeschützt. Warmup und On-Demand verwenden denselben Bild-Lock; während der produktiven Materialisierung wird zusätzlich der Connector-Sync-Lock geteilt gehalten, damit Source- und Shadow-Tree nicht ausgetauscht werden können.</p>
|
||||
<p class="bratonien-base-note">Der Scan-Test und der Pfadaudit sind schreibgeschützt. Warmup und On-Demand verwenden denselben Bild-Lock; während der produktiven Materialisierung wird zusätzlich der Connector-Sync-Lock geteilt gehalten, damit Source- und Shadow-Tree nicht ausgetauscht werden können.</p>
|
||||
</div>
|
||||
|
||||
{literal}
|
||||
|
||||
Reference in New Issue
Block a user