Unverified Commit 446586b6 authored by alexpott's avatar alexpott
Browse files

Issue #2949715 by jhodgdon, ankitjain28may, Lendude, mohit1604, dww, alexpott:...

Issue #2949715 by jhodgdon, ankitjain28may, Lendude, mohit1604, dww, alexpott: Automated Tests 8.x topic needs an update
parent 21c35049
......@@ -1062,7 +1062,7 @@
/**
* @defgroup testing Automated tests
* @{
* Overview of PHPUnit tests and Simpletest tests.
* Overview of PHPUnit and Nightwatch automated tests.
*
* The Drupal project has embraced a philosophy of using automated tests,
* consisting of both unit tests (which test the functionality of classes at a
......@@ -1083,123 +1083,70 @@
* code actually works, and ensure that later changes do not break the new
* functionality.
*
* @section write_unit Writing PHPUnit tests for classes
* PHPUnit tests for classes are written using the industry-standard PHPUnit
* framework. Use a PHPUnit test to test functionality of a class if the Drupal
* environment (database, settings, etc.) and web browser are not needed for the
* test, or if the Drupal environment can be replaced by a "mock" object. To
* write a PHPUnit test:
* - Define a class that extends \Drupal\Tests\UnitTestCase.
* - The class name needs to end in the word Test.
* - The namespace must be a subspace/subdirectory of \Drupal\yourmodule\Tests,
* where yourmodule is your module's machine name.
* - The test class file must be named and placed under the
* yourmodule/tests/src/Unit directory, according to the PSR-4 standard.
* - Your test class needs a phpDoc comment block with a description and
* a @group annotation, which gives information about the test.
* - Add test cases by adding method names that start with 'test' and have no
* arguments, for example testYourTestCase(). Each one should test a logical
* subset of the functionality.
* For more details, see:
* - https://www.drupal.org/phpunit for full documentation on how to write
* PHPUnit tests for Drupal.
* - http://phpunit.de for general information on the PHPUnit framework.
* - @link oo_conventions Object-oriented programming topic @endlink for more
* on PSR-4, namespaces, and where to place classes.
*
* @section write_functional Writing functional tests
* Functional tests are written using a Drupal-specific framework that is, for
* historical reasons, known as "Simpletest". Use a Simpletest test to test the
* functionality of sub-system of Drupal, if the functionality depends on the
* Drupal database and settings, or to test the web output of Drupal. To
* write a Simpletest test:
* - For functional tests of the web output of Drupal, define a class that
* extends \Drupal\simpletest\WebTestBase, which contains an internal web
* browser and defines many helpful test assertion methods that you can use
* in your tests. You can specify modules to be enabled by defining a
* $modules member variable -- keep in mind that by default, WebTestBase uses
* a "testing" install profile, with a minimal set of modules enabled.
* - For functional tests that do not test web output, define a class that
* extends \Drupal\KernelTests\KernelTestBase. This class is much faster
* than WebTestBase, because instead of making a full install of Drupal, it
* uses an in-memory pseudo-installation (similar to what the installer and
* update scripts use). To use this test class, you will need to create the
* database tables you need and install needed modules manually.
* - The namespace must be a subspace/subdirectory of \Drupal\yourmodule\Tests,
* where yourmodule is your module's machine name.
* - The test class file must be named and placed under the yourmodule/src/Tests
* directory, according to the PSR-4 standard.
* - Your test class needs a phpDoc comment block with a description and
* a @group annotation, which gives information about the test.
* - You may also override the default setUp() method, which can set be used to
* set up content types and similar procedures.
* @section write_test Writing tests
* All PHP-based tests for Drupal core are written using the industry-standard
* PHPUnit framework, with Drupal extensions. There are several categories of
* tests; each has its own purpose, base class, namespace, and directory:
* - Unit tests:
* - Purpose: Test functionality of a class if the Drupal environment
* (database, settings, etc.) and web browser are not needed for the test,
* or if the Drupal environment can be replaced by a "mock" object.
* - Base class: \Drupal\Tests\UnitTestCase
* - Namespace: \Drupal\Tests\yourmodule\Unit (or a subdirectory)
* - Directory location: yourmodule/tests/src/Unit (or a subdirectory)
* - Kernel tests:
* - Purpose: Test functionality of a class if the full Drupal environment
* and web browser are not needed for the test, but the functionality has
* significant Drupal dependencies that cannot easily be mocked. Kernel
* tests can access services, the database, and a minimal mocked file
* system, and they use an in-memory pseudo-installation. However, modules
* are only installed to the point of having services and hooks, unless you
* install them explicitly.
* - Base class: \Drupal\KernelTests\KernelTestBase
* - Namespace: \Drupal\Tests\yourmodule\Kernel (or a subdirectory)
* - Directory location: yourmodule/tests/src/Kernel (or a subdirectory)
* - Browser tests:
* - Purpose: Test functionality with the full Drupal environment and an
* internal simulated web browser, if JavaScript is not needed.
* - Base class: \Drupal\Tests\BrowserTestBase
* - Namespace: \Drupal\Tests\yourmodule\Functional (or a subdirectory)
* - Directory location: yourmodule/tests/src/Functional (or a subdirectory)
* - Browser tests with JavaScript:
* - Purpose: Test functionality with the full Drupal environment and an
* internal web browser that includes JavaScript execution.
* - Base class: \Drupal\FunctionalJavascriptTests\WebDriverTestBase
* - Namespace: \Drupal\Tests\yourmodule\FunctionalJavascript (or a
* subdirectory)
* - Directory location: yourmodule/tests/src/FunctionalJavascript (or a
* subdirectory)
*
* Some notes about writing PHP test classes:
* - The class needs a phpDoc comment block with a description and
* @group annotation, which gives information about the test.
* - For unit tests, this comment block should also have @coversDefaultClass
* annotation.
* - When writing tests, put the test code into public methods, each covering a
* logical subset of the functionality that is being tested.
* - The test methods must have names starting with 'test'. For unit tests, the
* test methods need to have a phpDoc block with @covers annotation telling
* which class method they are testing.
* - In some cases, you may need to write a test module to support your test;
* put such modules under the yourmodule/tests/modules directory.
* - Add test cases by adding method names that start with 'test' and have no
* arguments, for example testYourTestCase(). Each one should test a logical
* subset of the functionality. Each method runs in a new, isolated test
* environment, so it can only rely on the setUp() method, not what has
* been set up by other test methods.
*
* Besides the PHPUnit tests described above, Drupal Core also includes a few
* JavaScript-only tests, which use the Nightwatch.js framework to test
* JavaScript code using only JavaScript. These are located in
* core/tests/Drupal/Nightwatch.
*
* For more details, see:
* - https://www.drupal.org/simpletest for full documentation on how to write
* functional tests for Drupal.
* - core/tests/README.md for instructions on running tests
* - https://www.drupal.org/phpunit for full documentation on how to write
* and run PHPUnit tests for Drupal.
* - http://phpunit.de for general information on the PHPUnit framework.
* - @link oo_conventions Object-oriented programming topic @endlink for more
* on PSR-4, namespaces, and where to place classes.
*
* @section write_functional_phpunit Write functional PHP tests (phpunit)
* Functional tests extend the BrowserTestBase base class, and use PHPUnit as
* their underlying framework. They use a simulated browser, in which the test
* can click links, visit URLs, post to forms, etc. To write a functional test:
* - Extend \Drupal\Tests\BrowserTestBase.
* - Place the test in the yourmodule/tests/src/Functional/ directory and use
* the \Drupal\Tests\yourmodule\Functional namespace.
* - Add a @group annotation. For example, if the test is for a Drupal 6
* migration process, the group core uses is migrate_drupal_6. Use yourmodule
* as the group name if the test does not belong to another larger group.
* - You may also override the default setUp() method, which can be used to set
* up content types and similar procedures. Don't forget to call the parent
* method.
* - In some cases, you may need to write a test module to support your test;
* put such modules under the yourmodule/tests/modules directory.
* - Add test cases by adding method names that start with 'test' and have no
* arguments, for example testYourTestCase(). Each one should test a logical
* subset of the functionality. Each method runs in a new, isolated test
* environment, so it can only rely on the setUp() method, not what has
* been set up by other test methods.
* For more details, see:
* - https://www.drupal.org/docs/8/phpunit/phpunit-browser-test-tutorial for
* a full tutorial on how to write functional PHPUnit tests for Drupal.
* - https://www.drupal.org/phpunit for the full documentation on how to write
* PHPUnit tests for Drupal.
*
* @section write_jsfunctional_phpunit Write functional JavaScript tests (phpunit)
* To write a functional test that relies on JavaScript:
* - Extend \Drupal\FunctionalJavaScriptTests\JavascriptTestBase.
* - Place the test into the yourmodule/tests/src/FunctionalJavascript/
* directory and use the \Drupal\Tests\yourmodule\FunctionalJavascript
* namespace.
* - Add a @group annotation. Use yourmodule as the group name if the test does
* not belong to another larger group.
* - Set up PhantomJS; see http://phantomjs.org/download.html.
* - To run tests, see core/tests/README.md.
* - When clicking a link/button with Ajax behavior attached, keep in mind that
* the underlying browser might take time to deliver changes to the HTML. Use
* $this->assertSession()->assertWaitOnAjaxRequest() to wait for the Ajax
* request to finish.
* For more details, see:
* - https://www.drupal.org/docs/8/phpunit/phpunit-javascript-testing-tutorial
* for a full tutorial on how to write PHPUnit JavaScript tests for Drupal.
* - https://www.drupal.org/phpunit for the full documentation on how to write
* PHPUnit tests for Drupal.
*
* @section running Running tests
* You can run both Simpletest and PHPUnit tests by enabling the core Testing
* module (core/modules/simpletest). Once that module is enabled, tests can be
* run on through the Testing module's user interface or on the command line.
*
* See @link https://www.drupal.org/node/2116263 Running tests with PHPUnit
* binary @endlink or @link https://www.drupal.org/node/645286 Running tests
* with run-tests.sh @endlink for more information.
* - http://nightwatchjs.org/ for information about Nightwatch testing for
* JavaScript
* @}
*/
......
# Running tests
## Functional tests
## Setting up
* Run the functional tests:
### PHP dependencies
You need the Drupal core development dependencies installed, in order to run
any tests. You can install them using Composer by running
```
export SIMPLETEST_DB='mysql://root@localhost/dev_d8';
export SIMPLETEST_BASE_URL='http://d8.dev';
./vendor/bin/phpunit -c core --testsuite functional;
composer install
```
in the Drupal root directory. These dependencies should not be installed on a
production site.
Note: functional tests have to be invoked with a user in the same group as the
web server user. You can either configure Apache (or nginx) to run as your own
system user or run tests as a privileged user instead.
Invoking tests with a user that is in the same group as the web server will
require you to ensure Drupal keeps gid stickybits when creating new directories.
`$settings['file_chmod_directory'] = 02775;`
To develop locally, a straightforward - but also less secure - approach is to
run tests as your own system user. To achieve that, change the default Apache
user to run as your system user. Typically, you'd need to modify
`/etc/apache2/envvars` on Linux or `/etc/apache2/httpd.conf` on Mac.
Example for Linux:
```
export APACHE_RUN_USER=<your-user>
export APACHE_RUN_GROUP=<your-group>
```
Example for Mac:
```
User <your-user>
Group <your-group>
```
## Functional javascript tests
Javascript tests use the Selenium2Driver which allows you to control a
big range of browsers. By default Drupal uses chromedriver to run tests.
For help installing and starting selenium, see http://mink.behat.org/en/latest/drivers/selenium2.html
* Make sure you have a recent version of chrome installed
* Install selenium-server-standalone and chromedriver
Example for Mac:
```
brew install selenium-server-standalone;
brew install chromedriver;
```
### Test directory
* Before running tests make sure that selenium-server is running
```
selenium-server -port 4444
```
Create a directory called sites/simpletest and make sure that it is writable by
the web server and/or all users.
* Set the correct driver args and run the tests:
```
export MINK_DRIVER_ARGS_WEBDRIVER='["chrome", null, "http://localhost:4444/wd/hub"]';
./vendor/bin/phpunit -c core --testsuite functional-javascript;
```
### PHPUnit configuration
* It is possible to use alternate browsers if the required dependencies are
installed. For example to use Firefox:
Copy the core/phpunit.xml.dist file to phpunit.xml, and place it somewhere
convenient (inside the core directory may not be the best spot, since that
directory may be managed by Composer or Git). You can use the -c option on the
command line to tell PHPUnit where this file is (use the full path).
```
export MINK_DRIVER_ARGS_WEBDRIVER='["firefox", null, "http://localhost:4444/wd/hub"]';
./vendor/bin/phpunit -c core --testsuite functional-javascript;
```
Settings to change in this file:
* SIMPLETEST_BASE_URL: The URL of your site
* SIMPLETEST_DB: The URL of your Drupal database
* The bootstrap attribute of the top-level phpunit tag, to take into account
the location of the file
* BROWSERTEST_OUTPUT_DIRECTORY: Set to sites/simpletest/browser_output;
you will also want to uncomment the printerClass attribute of the
top-level phpunit tag.
* To force all BrowserTestBase (including legacy JavascriptTestBase) tests to use
webdriver:
### Additional setup for JavaScript tests
```
export MINK_DRIVER_CLASS='Drupal\FunctionalJavascriptTests\DrupalSelenium2Driver';
./vendor/bin/phpunit -c core --testsuite functional-javascript;
```
To run JavaScript tests based on the
\Drupal\FunctionalJavascriptTests\WebDriverTestBase base class, you will need to
install the following additional software:
## Running legacy javascript tests
* Google Chrome or Chromium browser
* chromedriver (tested with version 2.45) -- see
https://sites.google.com/a/chromium.org/chromedriver/
* PHP 7.1 or higher
Older javascript test may use the PhantomJSDriver. To run these tests you will
have to install and start PhantomJS.
## Running unit, functional, and kernel tests
* Start PhantomJS:
```
phantomjs --ssl-protocol=any --ignore-ssl-errors=true ./vendor/jcalderonzumba/gastonjs/src/Client/main.js 8510 1024 768 2>&1 >> /dev/null &
```
The PHPUnit executable is vendor/bin/phpunit -- you will need to locate your
vendor directory (which may be outside the Drupal root).
* Then you can run the test:
Here are commands to run one test class, list groups, and run all the tests in a
particular group:
```
./vendor/bin/phpunit -c core --testsuite functional-javascript;
./vendor/bin/phpunit -c /path/to/your/phpunit.xml path/to/your/class/file.php
./vendor/bin/phpunit --list-groups
./vendor/bin/phpunit -c /path/to/your/phpunit.xml --group Groupname
```
## Running tests with a different user
More information on running tests can be found at
https://www.drupal.org/docs/8/phpunit/running-phpunit-tests
If the default user is e.g. `www-data`, the above functional tests will have to
be invoked with sudo instead:
## Running Functional JavaScript tests
You can run JavaScript tests that are based on the
\Drupal\FunctionalJavascriptTests\WebDriverTestBase base class in the same way
as other PHPUnit tests, except that before you start, you will need to start
chromedriver using port 4444, and keep it running:
```
export SIMPLETEST_DB='mysql://root@localhost/dev_d8';
export SIMPLETEST_BASE_URL='http://d8.dev';
sudo -u www-data -E ./vendor/bin/phpunit -c core --testsuite functional;
sudo -u www-data -E ./vendor/bin/phpunit -c core --testsuite functional-javascript;
/path/to/chromedriver --port=4444
```
## Nightwatch tests
## Running Nightwatch tests
- Ensure your vendor directory is populated (e.g. by running `composer install`)
- If you're running PHP 7.0 or greater you will need to upgrade PHPUnit with `composer run-script drupal-phpunit-upgrade`
......@@ -140,3 +102,22 @@ If your core directory is located in a subfolder (e.g. `docroot`), then you can
Tests outside of the `core` folder will run in the version of node you have installed. If you want to transpile with babel (e.g. to use `import` statements) outside of core,
then add your own babel config to the root of your project. For example, if core is located under `docroot/core`, then you could run `yarn add babel-preset-env` inside
`docroot`, then copy the babel settings from `docroot/core/package.json` into `docroot/package.json`.
## Troubleshooting test running
If you run into file permission problems while running tests, you may need to
invoke the phpunit executable with a user in the same group as the web server
user, or with access to files owned by the web server user. For example:
```
sudo -u www-data ./vendor/bin/phpunit -c /path/to/your/phpunit.xml --group Groupname
```
If you have permission problems accessing files after running tests, try
putting
```
$settings['file_chmod_directory'] = 02775;
```
in your settings.php or local.settings.php file.
You may need to use absolute paths in your phpunit.xml file, and/or in your
phpunit command arguments.
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment