Issue #3240978 by victoria-marina, yusufhm, joshua1234511, lucienchalom,...

# drupal-purger

This module allows Section’s global, distributed caching layer to quickly respond to invalidation events from a Drupal instance in exactly the same way that Drupal’s internal cache or a local varnish cache running on the host machine does, ensuring that the content in Section’s global caching layer is always up to date.

This module allows Section’s global, distributed caching layer to quickly

respond to invalidation events from a Drupal instance in exactly the same way

that Drupal’s internal cache or a local varnish cache running on the host

machine does, ensuring that the content in Section’s global caching layer is

always up to date.

## Installation

Notes on Architecture:

- Assuming you have the purge module installed, navigate to the Purger's UI at `/admin/config/development/performance/purge`

- This module depends on the key module to store your password for aperture. Make sure your password is correct because the purger will send hundreds of requests to the API which could potentially lock out your account if the authentication attempts fail.

- Drupal will queue bans with the Core tags queuer, but make sure you have a processor installed (cron, or LateRuntimeProcessor)

- The bundled purger will combine tag purges into one request per every 250 tag invalidations in the queue. Bundled URL purging is not supported but we are open to feedback – after consulting with the back-end team, the opinion is that this would be too resource intensive.

- Tags are hashed to preserve privacy. If varnish caches a page while logged in, you will need to clear the cache. Consider updating your VCL to not cache requests that have either tags specific to authentication, or a session cookie. More info on this in the installation instructions linked above.

- There are many other invalidation types that you can use with other processors and queuers. See the table below

- The actual purger functions are performed by code inside `src/Plugin/Purge/Purger/SectionPurgerBase` as well as the two child classes of it: `SectionPurger` and `SectionBundledPurger`.

- The configuration form is controlled by code in `src/Form`. Data input via this form is stored in `/src/Entity/SectionPurgerSettings.php`. If you want to create a hardcoded variable value that is not customer facing, simply include this variable in this file and make no reference to it in the form. If you'd like a default value that is subsequently adjustable by the user, add that variable with the default value to `SectionPurgerSettings.php` and add its field to the user input form. There are examples of both of these in the existing codebase. Note that in order for a variable to be overwritten by the form input, it needs to be enumerated in the `section_purger.schema.yml`

- This module was developed as a fork of the generic http purger. Any questions on the development history can be answered by comparing the current state to that project — the git history is rather mangled due to moving between repos.

- The generic HTTP purger also contained code for a Bundled Purger (which lived in the Purger directory alongside the Section Purgers). The functionality of this has not been comprehensively understood, but in the abstract it coalesces multiple ban requests into a single API call. Because the choke point for this operation was believed to be Varnish Cache's ability to process bans rather than the API's ability to accept requests, this mode was determined to be unnecessary at the time. If this functionality is desired, the relevant code (it was not adapted for Section in any way before deletion) can be recovered from version control or from the source HTTP Purger.

- The plugin currently supports an optional sitename feature designed to support Drupal multisites. If no sitename is specified, then the plugin clears cache for any pages with the relevant cache tags. If the sitename is filled in, the plugin appends a check for the specified hostname. Each Drupal site within a multisite has its own admin dashboard and its own UI in which to enable the purger.

- The plugin is able to handle alternate varnish proxy names (as written in your section config). Usually the name for the proxy is 'varnish' but some configurations have multiple instances of varnish running for different tasks. Note that if using multiple varnish instances, you will have to enable another purger in order to handle multiple instances of varnish. If using multi-site as well as multiple varnish proxies, you will need a purger instance per-varnish instance per-site.

- If you wish to uninstall the Plugin from a live site for the purposes of testing, the most efficient way to do so is to uninstall the plugin from the admin console. Once it has been uninstalled there, you can delete it from the filesystem of the live box (Drupal may overwrite old files of the same name if you try and upload the same module again — this has not been proven). Deleting the module from the filesystem while it is still installed will cause a total failure of the site. The only known fix for this is a reinstall of Drupal as a whole, although there are likely easier ways to fix it. 

- Assuming you have the purge module installed, navigate to the Purger's UI at

`/admin/config/development/performance/purge`

- This module depends on the key module to store your password for aperture.

Make sure your password is correct because the purger will send hundreds of

requests to the API which could potentially lock out your account if the

authentication attempts fail.

- Drupal will queue bans with the Core tags queuer, but make sure you have a

processor installed (cron, or LateRuntimeProcessor)

- The bundled purger will combine tag purges into one request per every 250 tag

invalidations in the queue. Bundled URL purging is not supported but we are open

to feedback – after consulting with the back-end team, the opinion is that this

would be too resource intensive.

- Tags are hashed to preserve privacy. If varnish caches a page while logged in,

you will need to clear the cache. Consider updating your VCL to not cache

requests that have either tags specific to authentication, or a session cookie.

More info on this in the installation instructions linked above.

- There are many other invalidation types that you can use with other processors

and queuers. See the table below.

- The actual purger functions are performed by code inside

`src/Plugin/Purge/Purger/SectionPurgerBase` as well as the two child classes of

it: `SectionPurger` and `SectionBundledPurger`.

- The configuration form is controlled by code in `src/Form`. Data input via

this form is stored in `/src/Entity/SectionPurgerSettings.php`. If you want to

create a hardcoded variable value that is not customer facing, simply include

this variable in this file and make no reference to it in the form.

If you'd like a default value that is subsequently adjustable by the user, add

that variable with the default value to `SectionPurgerSettings.php` and add its

field to the user input form. There are examples of both of these in the

existing codebase. Note that in order for a variable to be overwritten by the

form input, it needs to be enumerated in the `section_purger.schema.yml`

- This module was developed as a fork of the generic http purger. Any questions

on the development history can be answered by comparing the current state to

that project — the git history is rather mangled due to moving between repos.

- The generic HTTP purger also contained code for a Bundled Purger (which lived

in the Purger directory alongside the Section Purgers). The functionality of

this has not been comprehensively understood, but in the abstract it coalesces

multiple ban requests into a single API call. Because the choke point for this

operation was believed to be Varnish Cache's ability to process bans rather than

the API's ability to accept requests, this mode was determined to be unnecessary

at the time. If this functionality is desired, the relevant code (it was not

adapted for Section in any way before deletion) can be recovered from version

control or from the source HTTP Purger.

- The plugin currently supports an optional sitename feature designed to

support Drupal multisites. If no sitename is specified, then the plugin clears

cache for any pages with the relevant cache tags. If the sitename is filled in,

the plugin appends a check for the specified hostname. Each Drupal site within

a multisite has its own admin dashboard and its own UI in which to enable the

purger.

- The plugin is able to handle alternate varnish proxy names (as written in your

section config). Usually the name for the proxy is 'varnish' but some

configurations have multiple instances of varnish running for different tasks.

Note that if using multiple varnish instances, you will have to enable another

purger in order to handle multiple instances of varnish. If using multi-site

as well as multiple varnish proxies, you will need a purger instance per-varnish

instance per-site.

- If you wish to uninstall the Plugin from a live site for the purposes of

testing, the most efficient way to do so is to uninstall the plugin from the

admin console. Once it has been uninstalled there, you can delete it from the

filesystem of the live box (Drupal may overwrite old files of the same name if

you try and upload the same module again — this has not been proven). Deleting

the module from the filesystem while it is still installed will cause a total

failure of the site. The only known fix for this is a reinstall of Drupal as a

whole, although there are likely easier ways to fix it.

| Invalidation Type | Description                                                                                                                                            |

namespace Drupal\section_purger\Tests;

use Drupal\section_purger\Tests\SectionPurgerFormTestBase;

/**

 * Tests \Drupal\section_purger\Form\SectionPurgerForm.

 *

 * @group section_purger

 */

class SectionPurgerFormTest extends SectionPurgerFormTestBase

{

class SectionPurgerFormTest extends SectionPurgerFormTestBase {

  /**

   * The full class of the form being tested.

   * @see purge_tokens_token_info()

   */

  protected $tokenGroups = ['invalidation'];

}

/**

 * Testbase for testing \Drupal\section_purger\Form\SectionPurgerFormBase.

 */

abstract class SectionPurgerFormTestBase extends PurgerConfigFormTestBase

{

abstract class SectionPurgerFormTestBase extends PurgerConfigFormTestBase {

  /**

   * Modules to enable.

  /**

   * Verify that the form contains all fields we require.

   */

    public function testFieldExistence()

    {

  public function testFieldExistence() {

    $this->drupalLogin($this->admin_user);

    $this->drupalGet($this->route);

    $fields = [

      'edit-path' => '/',

      'edit-request-method' => 0,

      'edit-scheme' => 0,

      'edit-verify' => true,

      'edit-verify' => TRUE,

      'edit-headers-0-field' => '',

      'edit-headers-0-value' => '',

      'edit-show-body-form' => '1',

  }

  /**

     * Tests \Drupal\section_purger\Form\SectionPurgerFormBase::buildFormTokensHelp().

   * Tests SectionPurgerFormBase::buildFormTokensHelp().

   */

    public function testTokensHelp()

    {

  public function testTokensHelp() {

    $this->drupalLogin($this->admin_user);

    $this->drupalGet($this->route);

    $this->assertText('Tokens');

  /**

   * Test validating the data.

   */

    public function testFormValidation()

    {

  public function testFormValidation() {

    // Assert that valid timeout values don't cause validation errors.

    $form_state = $this->getFormStateInstance();

    $form_state->addBuildInfo('args', [$this->formArgs]);

    $form_state->setValues(

      [

        'connect_timeout' => 0.3,

        'connectTimeout' => 0.3,

        'timeout' => 0.1,

        'name' => 'foobar',

      ]

    $form_state->addBuildInfo('args', [$this->formArgs]);

    $form_state->setValues(

      [

        'connect_timeout' => 2.3,

        'connectTimeout' => 2.3,

        'timeout' => 7.7,

        'name' => 'foobar',

      ]

    $form_state->addBuildInfo('args', [$this->formArgs]);

    $form_state->setValues(

      [

        'connect_timeout' => 0.0,

        'connectTimeout' => 0.0,

        'timeout' => 0.0,

        'name' => 'foobar',

      ]

    $errors = $form_state->getErrors();

    $this->assertEqual(2, count($errors));

    $this->assertTrue(isset($errors['timeout']));

        $this->assertTrue(isset($errors['connect_timeout']));

    $this->assertTrue(isset($errors['connectTimeout']));

    // Submit timeout values that are too high and confirm the validation error.

    $form_state = $this->getFormStateInstance();

    $form_state->addBuildInfo('args', [$this->formArgs]);

    $form_state->setValues(

      [

        'connect_timeout' => 2.4,

        'connectTimeout' => 2.4,

        'timeout' => 7.7,

        'name' => 'foobar',

      ]

    $errors = $form_state->getErrors();

    $this->assertEqual(2, count($errors));

    $this->assertTrue(isset($errors['timeout']));

        $this->assertTrue(isset($errors['connect_timeout']));

    $this->assertTrue(isset($errors['connectTimeout']));

  }

  /**

   * Test posting data to the HTTP Purger settings form.

   */

    public function testFormSubmit()

    {

  public function testFormSubmit() {

    // Assert that all (simple) fields submit as intended.

    $this->drupalLogin($this->admin_user);

    $edit = [

      'password' => 'password',

      'port' => 8080,

      'path' => 'node/1',

      'request_method' => 1,

      'requestMethod' => 1,

      'scheme' => 0,

      'verify' => true,

      'verify' => TRUE,

      'show_body_form' => 1,

      'body_content_type' => 'foo/bar',

      'bodyContentType' => 'foo/bar',

      'body' => 'baz',

      'timeout' => 6,

      'runtime_measurement' => 1,

      'connect_timeout' => 0.5,

      'cooldown_time' => 0.8,

      'max_requests' => 25,

      'http_errors' => 1,

      'runtimeMeasurement' => 1,

      'connectTimeout' => 0.5,

      'cooldownTime' => 0.8,

      'maxRequests' => 25,

      'httpErrors' => 1,

    ];

        $this->drupalPostForm($this->route, $edit, t('Save configuration'));

    $this->drupalPostForm($this->route, $edit, 'Save configuration');

    $this->drupalGet($this->route);

    foreach ($edit as $field => $value) {

      $this->assertFieldById('edit-' . str_replace('_', '-', $field), $value);

    $this->assertFieldById('edit-headers-0-field', 'foo');

    $this->assertFieldById('edit-headers-0-value', 'bar');

  }

}

    password:

      type: key_select

      translatable: false

    request_method:

    requestMethod:

      type: string

      translatable: false

    scheme:

    body:

      type: text

      translatable: false

    body_content_type:

    bodyContentType:

      type: string

      translatable: false

    #

    # Performance settings:

    #

    runtime_measurement:

    runtimeMeasurement:

      type: boolean

      translatable: false

    timeout:

      type: float

      translatable: false

    connect_timeout:

    connectTimeout:

      type: float

      translatable: false

    cooldown_time:

    cooldownTime:

      type: float

      translatable: false

    max_requests:

    maxRequests:

      type: integer

      translatable: false

    #

    # Success resolution:

    #

    http_errors:

    httpErrors:

      type: boolean

      translatable: false

description: 'Purger for clearing cache on the Section Platform'

package: "Purge - reverse proxies & CDNs"

dependencies:

  - purge

  - purge_tokens

  - key

  - purge:purge

  - purge:purge_tokens

  - key:key

core_version_requirement: ^8.9 || ^9