Commit 438a1b60 authored by C_Logemann's avatar C_Logemann

Issue #2009898 by swim: Port Boost for Drupal 8

parent f98cbf4f
## About
Drupal 8 port of the fantastic D7 Boost module.
## Installation
Download and install like any D8 module. Once installed visit a page as an anonymous user to
generate a local cache file. Check for the X-Boost response header to confirm it's working. There
are two X-Boost response headers to look for; partial and full. Patial means that PHP is still executing
and that the apache or nginx config has not been applied correctly.
## Legend
X-Boost-Cache: partial - bad
X-Boost-Cache: full - good
## Nginx config
The following is a very basic example of an nginx configuration for use with Boost.
```
server {
server_name mydomain.com;
access_log /srv/www/mydomain.com/logs/access.log;
error_log /srv/www/mydomain.com/logs/error.log;
root /srv/www/mydomain.com/public_html;
fastcgi_param SCRIPT_NAME $fastcgi_script_name;
location ~ (^|/)\. {
return 403;
}
location / {
index index.html index.php;
expires max;
set $request_url $request_uri;
if ($request_uri ~ ^/admin/(.*)$) {
rewrite ^ /index.php;
}
location ~* ^(?:.+\.(?:htaccess|make|txt|engine|inc|info|install|module|profile|po|pot|sh|.*sql|test|theme|tpl(?:\.php)?|xtmpl)|code-style\.pl|/Entries.*|/Repository|/Root|/Tag|/Template)$ {
return 404;
}
add_header X-Boost-Cache "full";
try_files $uri @rewrite;
}
location @rewrite {
gzip_static on;
if ($request_method = POST) {
rewrite ^ /index.php;
}
set $boost_uri "${request_uri}.html";
try_files ^ /sites/default/files/boost$boost_uri @drupal;
}
location @drupal {
rewrite ^ /index.php;
}
location ~ \.php$ {
include /etc/nginx/fastcgi_params;
fastcgi_pass 127.0.0.1:9000;
fastcgi_index index.php;
fastcgi_pass_header Set-Cookie;
fastcgi_pass_header Cookie;
fastcgi_ignore_headers Cache-Control Expires Set-Cookie;
fastcgi_param SCRIPT_FILENAME /srv/www/mydomain.com/public_html$fastcgi_script_name;
}
}
```
## Apache config
@todo
## WRK HTTP benchmarks
#### Boost with nginx config
Running 30s test @ http://testsite.dev/testpage <br>
12 threads and 100 connections
| Thread Stats | Avg | Stdev | Max | +/- Stdev |
| ------------- | ------------- | ------ | -------- | ---------- |
| Latency | 38.12ms | 64.66ms | 343.24ms | 82.64% |
| Req/Sec | 1.68k | 755.46 | 16.98k | 69.10% |
595713 requests in 30.10s, 6.84GB read <br>
Requests/sec: 19791.44 <br>
Transfer/sec: 232.74MB
#### Boost without nginx config
Running 30s test @ http://testsite.dev/testpage <br>
12 threads and 100 connections
| Thread Stats | Avg | Stdev | Max | +/- Stdev |
| ------------- | ------------- | -------- | ------- | --------- |
| Latency | 1.68s | 212.61ms | 2.00s | 93.69% |
| Req/Sec | 9.01 | 7.65 | 60.00 | 68.12% |
1670 requests in 30.05s, 20.13MB read <br>
Socket errors: connect 0, read 0, write 0, timeout 7<br>
Requests/sec: 55.58 <br>
Transfer/sec: 685.99KB
#### No cache
Running 30s test @ http://testsite.dev/testpage <br>
12 threads and 100 connections
| Thread Stats | Avg | Stdev | Max | +/- Stdev |
| ------------- | ------------- | ------- | ------- | --------- |
| Latency | 0.00us | 0.00us | 0.00us | -nan% |
| Req/Sec | 6.37 | 8.09 | 40.00 | 86.45% |
236 requests in 30.05s, 2.85MB read <br>
Socket errors: connect 0, read 0, write 0, timeout 236 <br>
Requests/sec: 7.85 <br>
Transfer/sec: 97.04KB
## The going on's
1. Page is requested for the first time and is built dynamically by PHP.
2. Page is cached on the local file system; if accessed by an anonymous user.
3. When the route is requested again the page is served from the file system.
4. For this module to do anything worth while the apache or nginx config must be applied.
## Roadmap
1. Sub-module Boost crawler; implementing Guzzle.
2. Generate file cache via batch function.
3. Implement cron to invalidate and re-generate cache.
\ No newline at end of file
BOOST MODULE FOR DRUPAL 7.x
---------------------------
CONTENTS OF THIS README
-----------------------
* Description
* Requirements
* Installation
* Support
* Credits
DESCRIPTION
-----------
This module provides static page caching for Drupal websites.
It provides a significant performance increase as well as
scalability for sites that receive mostly anonymous traffic.
Web pages load very fast from the cache instead of waiting on
PHP and Drupal to serve them from the database. If the page is
not found in the cache, then the request is passed to Drupal.
More information: http://drupal.org/project/boost
For information on supported features (as well as those deprecated
since 7.x-1.x), please read: https://drupal.org/node/1434362
REQUIREMENTS
------------
Drupal's clean URLs MUST be enabled and working properly.
INSTALLATION
------------
Handbook page: http://drupal.org/node/1459690
1. Goto: [Administer > Configuration > Search and metadata > Clean URLs]
and ensure that Drupal's clean URLs are enabled and working correctly
on your site.
2. Unzip and upload the module folder (as is) to the sites/all/modules
folder in your Drupal installation directory.
3. Goto: [Administer > Configuration > Development > Performance] and disable
the Drupal core cache for anonymous users. Boost will not be able to
generate its cache if a page is already in the Drupal core cache.
This is the only core setting you must disable, others can be left enabled.
4. Goto: [Administer > Configuration > System > Boost > Boost Settings]
and review the default settings.
5. Goto: [Administer > Configuration > System > Boost > File System]
Make sure that the cache directory is writeable by the web server:
you may need to create the directory, and set the permissions.
Ideally, the cache directory should be owned by your user and be in
the group of your web server ("www-data" on Debian/Ubuntu), with a
unix permission of 0775 (read/write/exec owner, read/write/exec group,
read/exec others).
6. Review the other default Boost settings.
7. IMPORTANT - This step is easy and required for Boost to work!
Backup the original .htaccess file from your Drupal installation
directory for safe keeping.
Copy the custom generated htaccess rule from [Administer > Configuration
> System > Boost > .htaccess > .htaccess Generation] page and paste
the rules into the Drupal htaccess file as shown below.
# RewriteBase /
-------paste the rules right here--------
# Rewrite URLs of the form 'x' to the form 'index.php?q=x'.
# Pass all requests not referring directly to files in the filesystem to
# index.php. Clean URLs are handled in drupal_environment_initialize().
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_URI} !=/favicon.ico
RewriteRule ^ index.php [L]
Note: If you get "400 Bad Request" responses from Apache server, make sure you have configured the RewriteBase. For example when using VirtualHost configurations it is necessary to define as:
RewriteBase /
For more information, please read the handbook page:
http://drupal.org/node/1459690
SUPPORT
-------
Project issue queue:
http://drupal.org/project/issues/boost
Please take the time to review other support issues before posting a new one.
It can be hard to debug boost support issues without access to the server.
Provide as much information as possible in order to reproduce the issue.
Feel free to post install tips in the installation handbook page:
https://drupal.org/node/1459690
Paid support is possible by contacting the module maintainers via their
user contact page (see below). We also accept donations, see:
https://drupal.org/node/1434362 (support section).
CREDITS
-------
4.7 Originally developed by Arto Bendiken.
5.x Port by Alexander I. Grafov, developed by Mike Carper.
6.x Port by Ben Lavender.
6.x Developed & Maintained by
Mike Carper https://drupal.org/user/282446
since 2012, minimal maintenance by Mathieu Lutfy
7.x Port by Mike Carper
7.x Developed & Maintained by
Mike Carper (mikeytown2) https://drupal.org/user/282446
Mathieu Lutfy (bgm) https://drupal.org/user/89461
<?php
/**
* @file
* Admin page callbacks for the boost module.
*/
/**
* Form builder; Configure boost settings.
*
* @ingroup forms
* @see system_settings_form()
*/
function boost_admin_debug_settings() {
$form['boost_message_debug'] = array(
'#type' => 'checkbox',
'#title' => t('Send debug info for each request to watchdog.'),
'#default_value' => variable_get('boost_message_debug', BOOST_MESSAGE_DEBUG),
'#description' => t('Only use for debugging purposes as this can fill up watchdog fairly quickly.'),
);
// reset htaccess on submit;
$form['#submit'][] = 'boost_form_submit_handler';
return system_settings_form($form);
}
<?php
/**
* @file
* Admin page callbacks for the boost module.
*/
/**
* Form builder; Configure boost settings.
*
* @ingroup forms
* @see system_settings_form()
*/
function boost_admin_expiration_settings() {
$form['boost_ignore_flush'] = array(
'#type' => 'checkbox',
'#title' => t('Ignore a cache flush command if cron issued the request.'),
'#default_value' => variable_get('boost_ignore_flush', BOOST_IGNORE_FLUSH),
'#description' => t('Drupal will flush all caches when cron is executed, depending on the <a href="!urlcore">core minimum cache lifetime</a> setting. To ignore the request to flush the cache on cron runs, enable this option. If enabled, pages in the Boost cache will be flushed only when their <a href="!urlboost">Boost maximum cache lifetime</a> expires.',
array(
'!urlcore' => url('admin/config/development/performance'),
'!urlboost' => url('admin/config/system/boost'),
)),
);
$form['boost_expire_cron'] = array(
'#type' => 'checkbox',
'#title' => t('Remove old cache files on cron.'),
'#default_value' => variable_get('boost_expire_cron', BOOST_EXPIRE_CRON),
'#description' => t('If enabled, each time cron runs Boost will check each cached page and delete those that have expired (maximum cache lifetime). The expiration time is displayed in the comment that Boost adds to the bottom of the html pages it creates. This setting is recommended for most sites.'),
);
// reset htaccess on submit;
$form['#submit'][] = 'boost_form_submit_handler';
return system_settings_form($form);
}
<?php
/**
* @file
* Admin page callbacks for the boost module.
*/
/**
* Form builder; Configure boost settings.
*
* @ingroup forms
* @see system_settings_form()
*/
function boost_admin_filesystem_settings() {
$form['boost_root_cache_dir'] = array(
'#type' => 'textfield',
'#title' => t('Root cache directory'),
'#default_value' => variable_get('boost_root_cache_dir', BOOST_ROOT_CACHE_DIR),
);
$form['boost_normal_dir'] = array(
'#type' => 'textfield',
'#title' => t('Normal cache directory'),
'#default_value' => variable_get('boost_normal_dir', BOOST_NORMAL_DIR),
);
$form['boost_char'] = array(
'#type' => 'textfield',
'#title' => t('Character replacement for "?" in the URL'),
'#default_value' => variable_get('boost_char', BOOST_CHAR),
);
// reset htaccess on submit;
$form['#submit'][] = 'boost_form_submit_handler';
return system_settings_form($form);
}
This diff is collapsed.
<?php
/**
* @file
* Admin page callbacks for the boost module.
*/
/**
* Form builder; Configure boost settings.
*
* @ingroup forms
* @see system_settings_form()
*/
function boost_admin_settings() {
drupal_add_js(drupal_get_path('module', 'system') . '/system.js');
$form['cacheability'] = array(
'#type' => 'fieldset',
'#title' => t('Boost cacheability settings'),
);
// See http://api.drupal.org/api/function/block_admin_configure/7
$access = user_access('use PHP for settings');
$options = array(
BOOST_VISIBILITY_NOTLISTED => t('All pages except those listed'),
BOOST_VISIBILITY_LISTED => t('Only the listed pages'),
);
$description = t("Specify pages by using their paths. Enter one path per line. The '*' character is a wildcard. Example paths are %blog for the blog page and %blog-wildcard for every personal blog. %front is the front page.", array('%blog' => 'blog', '%blog-wildcard' => 'blog/*', '%front' => '<front>'));
if (module_exists('php') && $access) {
$options += array(BOOST_VISIBILITY_PHP => t('Pages on which this PHP code returns <code>TRUE</code> (experts only)'));
$title = t('Pages or PHP code');
$description .= ' ' . t('If the PHP option is chosen, enter PHP code between %php. Note that executing incorrect PHP code can break your Drupal site.', array('%php' => '<?php ?>'));
}
else {
$title = t('Pages');
}
$form['cacheability']['boost_cacheability_option'] = array(
'#type' => 'radios',
'#title' => t('Cache specific pages'),
'#options' => $options,
'#default_value' => variable_get('boost_cacheability_option', BOOST_VISIBILITY_NOTLISTED),
);
$form['cacheability']['boost_cacheability_pages'] = array(
'#type' => 'textarea',
'#title' => '<span class="element-invisible">' . $title . '</span>',
'#default_value' => variable_get('boost_cacheability_pages', BOOST_CACHEABILITY_PAGES),
'#description' => $description,
);
$types = boost_get_storage_types();
$period = drupal_map_assoc(array(0, 60, 180, 300, 600, 900, 1800, 2700, 3600, 10800, 21600, 32400, 43200, 64800, 86400, 2*86400, 3*86400, 4*86400, 5*86400, 6*86400, 604800, 2*604800, 3*604800, 4*604800, 8*604800, 16*604800, 52*604800), 'format_interval');
$form['cache_types'] = array(
'#type' => 'fieldset',
'#title' => t('Boost cache type settings'),
);
foreach ($types as $title => $content_types) {
$form['cache_types'][$title] = array(
'#type' => 'fieldset',
'#title' => t('@title settings', array('@title' => $title)),
'#collapsible' => TRUE,
);
$collapsed = TRUE;
foreach ($content_types as $type => $values) {
$form['cache_types'][$title][$type] = array(
'#type' => 'fieldset',
'#title' => t('@type settings', array('@type' => $type)),
'#description' => t('Cache @description of type @type',
array(
'@description' => $values['description'],
'@type' => $type,
)
),
);
// This content type enabled?
$form['cache_types'][$title][$type]['boost_enabled_' . $type] = array(
'#type' => 'checkbox',
'#title' => t('Cache Enabled'),
'#default_value' => $values['enabled'],
);
// https://drupal.org/node/1416214#comment-7225650
// // Enable gzip?
// $form['cache_types'][$title][$type]['boost_gzip_' . $type] = array(
// '#type' => 'checkbox',
// '#title' => t('Enable gzip compression'),
// '#description' => (BOOST_GZIP ? t('Avoids having to compress the content by the web server on every request (recommended).') : t('Your host does not support zlib. See: !url', array('!url' => 'http://www.php.net/manual/en/zlib.installation.php'))),
// '#default_value' => (BOOST_GZIP ? $values['gzip'] : 0),
// '#disabled' => ! BOOST_GZIP,
// );
// Content type extension
$form['cache_types'][$title][$type]['boost_extension_' . $type] = array(
'#type' => 'textfield',
'#title' => t('Filename Extension',
array(
'@title' => $title,
'@description' => $values['description'],
'@type' => $type,
)
),
'#default_value' => $values['extension'],
);
// Maximum cache lifetime
$form['cache_types'][$title][$type]['boost_lifetime_max_' . $type] = array(
'#type' => 'select',
'#options' => $period,
'#title' => t('@type - Maximum Cache Lifetime',
array(
'@title' => $title,
'@description' => $values['description'],
'@type' => $type,
)
),
'#default_value' => $values['lifetime_max'],
);
// Minimum cache lifetime
$form['cache_types'][$title][$type]['boost_lifetime_min_' . $type] = array(
'#type' => 'select',
'#options' => $period,
'#title' => t('@type - Minimum Cache Lifetime',
array(
'@title' => $title,
'@description' => $values['description'],
'@type' => $type,
)
),
'#default_value' => $values['lifetime_min'],
);
if ($values['enabled']) {
$collapsed = !$values['enabled'];
}
}
$form['cache_types'][$title]['#collapsed'] = $collapsed;
}
// reset htaccess on submit;
$form['#submit'][] = 'boost_form_submit_handler';
return system_settings_form($form);
}
<?php
/**
* @file
* Displays compatible modules for boost, descriptions of functionality and
* links if not installed.
*/
/**
* Returns a table listing of the status of recommended modules.
*/
function boost_compatible_output() {
$modules_enabled = array();
$modules_disabled = array();
// Query from the database in case modules are already installed but not enabled
$result = db_query("SELECT name, status FROM {system} WHERE type = 'module' ORDER BY weight ASC, name ASC");
foreach ($result as $record) {
if ($record->status == 1) {
$modules_enabled[] = $record->name;
}
else {
$modules_disabled[] = $record->name;
}
}
// list of compatible modules
$recommended = array(
'boost_crawler' => array(
'title' => 'Boost Crawler',
'link' => 'http://drupal.org/project/boost' ,
'description' => t('Minimal crawler - expires and regenerates pages only when content is edited and on next cron run. If this module is disabled, the content will be regenerated by the next visitor (who may have a slow page load). Only recommended for sites with low traffic, since the page cache will probably have been regenerated by the time the crawler accesses it. This is a Boost sub-module.'),
),
'fast_404' => array(
'title' => 'Fast 404',
'link' => 'http://drupal.org/project/fast_404' ,
'description' => t('Produces a faster "page not found" (404) page, reducing server load. Boost does not cache 404 pages since it would increase the size of the cache for pages which should not be accessed anyway. However, the default 404 mechanism of Drupal can be expensive in terms of performance. If you have a large amount of 404, consider adding a redirection for them in your .htaccess file.'),
),
'globalredirect' => array(
'title' => 'Global Redirect',
'link' => 'http://drupal.org/project/globalredirect',
'description' => t('Redirects pages to their canonical URL. Avoids caching two pages for the same content.'),
),
'httprl' => array(
'title' => 'HTTP Parallel Request & Threading Library',
'link' => 'http://drupal.org/project/httprl',
'description' => t('Required by Boost crawler, performs HTTP requests to regenerate expired pages.'),
),
'expire' => array(
'title' => 'Cache Expiration',
'link' => 'http://drupal.org/project/expire',
'description' => t('Expire specific pages of the cache immediately when content is added or modified. Also required by the Boost crawler, to crawl expired content after expiration.'),
),
);
ksort($recommended);
// uses array_intersect as array_intersect_key only came in PHP > 5.1
$installed = array_intersect($modules_enabled, array_keys($recommended));
$disabled = array_intersect($modules_disabled, array_keys($recommended));
$not_installed = array_diff( array_diff( array_keys($recommended), $installed), $disabled );
// prepare the table
$header = array(t('Module'), t('Status'), t('Description'));
$rows = array();
foreach ($recommended as $module => $data) {
$status = in_array($module, $not_installed) ? t('Not Installed') : (in_array($module, $disabled) ? t('Disabled') : t('Enabled'));
$trclass = in_array($module, $not_installed) ? 'info' : (in_array($module, $disabled) ? 'warning' : 'ok' ) ;
$module_link = l($data['title'], $data['link']);
$rows[] = array(
'data' => array(
$module_link,
$status,
$data['description'],
),
'class' => array($trclass),
'no_striping' => TRUE,
);
}
$output = '<p class="boost-listmodules-intro">' . t("The following modules can be useful for most common use cases. For more information, please read and contribute to the <a href='!url'>Boost handbook</a>.", array('!url' => 'http://drupal.org/node/1434362')) . '</p>';
$output .= theme('table', array('attributes' => array('class' => array('system-status-report')), 'header' => $header, 'rows' => $rows));
return array(
'my_page' => array(
'#markup' => $output,
),
);
}
<?php
/**
* @file
* Prints the cache status of the currently displayed page.
*
* see @boost_block_view()
*/
function boost_block_view_status() {
global $user;
$block = array();
$block['subject'] = '';
// Don't show the block to anonymous users
if (! $user->uid) {
return $block;
}
// Do not use the global $_boost to not confuse hook_exit()
$_boost = boost_transform_url();
// Unset these variables otherwise boost_is_cacheable() will quickly bail out.
unset($_boost['is_cacheable']);
unset($_boost['is_cacheable_reason']);
$_boost = boost_is_cacheable($_boost, 'status');
if (! $_boost['is_cacheable']) {
$reason = ($_boost['is_cacheable_reason'] ? $_boost['is_cacheable_reason'] : 'reason unknown');
$block['content']['is_not_cacheable'] = array(
'#markup' => '<p>' . t('This page will not be cached: %reason', array('%reason' => $reason)) . '</p>',
);
return $block;
}
// We need the extention for the filename
$_boost['header_info'] = boost_get_header_info();
$_boost['matched_header_info'] = boost_match_header_attributes($_boost['header_info']);
$filename = (isset($_boost['filename']) ? $_boost['filename'] . '.' . $_boost['matched_header_info']['extension'] : 'n/a');
if (file_exists($filename)) {
// be precise on the time (seconds and timezone)
$generated = date('c', filemtime($filename));
}
else {
$generated = 'not cached yet (either no one has visited the page recently, or something is preventing the cache from being generated).';
}
$block['content'] = array(
'filename' => array(
'#markup' => '<p>' . t('File: %filename', array('%filename' => $filename)) . '</p>',
),
'generated' => array(
'#markup' => '<p>' . t('Generated: %generated', array('%generated' => $generated)) . '</p>',
),
);
if (file_exists($filename) && user_access('boost flush pages')) {
// 1922532 - Variable required to remove warning under PHP 5.4
$form = drupal_get_form('boost_block_flush_form');
$block['content']['flush'] = array(
'#markup' => drupal_render($form),
);
}
return $block;
}
function boost_block_flush_form() {
$form = array();
$form['boost_cache']['clear'] = array(
'#type' => 'submit',
'#value' => t('Flush Page'),
);
return $form;
}
function boost_block_flush_form_submit() {
$_boost = boost_transform_url();
// We need the extention for the filename
$_boost['header_info'] = boost_get_header_info();
$_boost['matched_header_info'] = boost_match_header_attributes($_boost['header_info']);
$filename = $_boost['filename'] . '.' . $_boost['matched_header_info']['extension'];
if (is_file($filename)) {
if (unlink($filename)) {
drupal_set_message(t('%filename was deleted from the Boost cache', array('%filename' => $filename)));
}
else {
drupal_set_message(t('%filename could not be deleted, check file permissions on disk to see if the web server can write/delete the file.', array('%filename' => $filename)));
}
}
else {
drupal_set_message(t('%filename was not found in the Boost cache and could not be deleted.', array('%filename' => $filename)));
}
}