feat: #3636593 Add Drush commands to create and cancel archive jobs (!7) · Merge requests · project / earkiv

What this changes

drush earkiv:create

Creates a job with the default archiver, storage and their default settings from earkiv.settings.
Options override the settings: --url, --connections, --sitemap (--sitemap=0 turns it off), --exclude and --start-url (repeatable, replace the defaults), --storage-path, --archiver, --storage.
The settings are validated and normalised by the plugin itself (buildConfigurationForm(), validateConfigurationForm(), submitConfigurationForm() with a FormState), so the rules are the same as in the form. Settings that are not given get the form default.
--process creates and stores the new job right away, under the processing lock, retrying failed steps like cron. Other queued jobs are not touched. If another process holds the lock, the job is left for the next earkiv:process.

drush earkiv:cancel ID

New status ArchiveStatusEnum::CANCELLED ("Cancelled"). Cancelled jobs are never picked up again.
A running job: HTTrackCommands::waitFor() checks the status with loadUnchanged() every 5 seconds. When the job is cancelled (or deleted) it stops HTTrack or tar and cleans up. The command waits up to 15 seconds for the process to confirm through the progress record. This also works when the job runs on another web server.
If no process confirms, it is assumed dead: the command cleans up itself and releases the lock. The lock API cannot release a lock held by another process, so with the database lock backend the row is deleted from semaphore. (The lock service is a proxy class, so this checks for the table instead of instanceof DatabaseLockBackend.)
A job that is not running is cleaned up directly. A job being moved to storage cannot be cancelled, so the stored archive is never half copied.
Clean-up is a new cancel() method on EArkivDrushCommandsInterface. For HTTrack it removes httrack_ID, any earkiv_ID_*.tar.gz in the temporary directory and a file entity that still points to temporary://. Local storage has nothing to clean up.

Other

EArkivCommands::processJob() processes one step of a given job; earkiv:process and earkiv:create --process both use it. The job is reloaded afterwards, so a cancellation is recorded and logged.
ArchiveProgress::get() can bypass the static State cache, needed to see the confirmation from the other process.
README: documents both commands.

In DDEV:

earkiv:create with defaults, and with all options: the stored settings match, normalised like the form (integer connections, one pattern per line). Invalid start URL and unknown plugin are rejected with the plugin message.
earkiv:create --process on a small site section: both steps ran, job ended as Done.
Cancel a new job: Cancelled; cancelling again reports it as finished; unknown ID is rejected.
Cancel a running crawl: the running process stopped HTTrack and cleaned up, the command got the confirmation after 7 s. No HTTrack process, working directory or lock left.
Kill Drush and HTTrack with SIGKILL, then cancel: no confirmation, the command cleaned up after 15 s and removed the stale lock (1 row before, 0 after).
Cancel a job waiting for storage: the temporary archive and its file entity were removed, the file reference cleared.

If the process running a job died but HTTrack itself is still running (an orphan), earkiv:cancel removes the working directory but does not kill HTTrack, since it cannot know which process belongs to the job on which server.
--connections is validated by the plugin, which does not check the form #max; values above 32 are capped when the crawl starts.

AI-Generated: Yes (Used Claude Code to write the code changes, run the tests in DDEV and draft this description.)