Skip to content
Snippets Groups Projects
README.md 9.04 KiB
Newer Older
Cloud is a set of modules to enable Cloud orchestration: A Drupal-based
Cloud dashboard for _AWS_, _Kubernetes_, _OpenStack_, _VMware_ and _Terraform
Cloud_ as a one-stop portal to manage multiple cloud service providers.
Yas Naoi's avatar
Yas Naoi committed
  - PHP `8.0` or higher (`128 MB` memory or higher)
  - MySQL `5.7` or higher _OR_ MariaDB `10.3` or higher
  - Drupal `9.5` or higher
  - Cloud `6.x`
    - This branch includes support for _AWS_, _Kubernetes_, _OpenStack_,
      _VMware_ and _Terraform Cloud_
    - Future support includes _GCP_ and _MS Azure_
- The `aws_cloud` module does **not** support *Classic EC2 instances*
  (_Non-VPC_).
  **Note:** Classic instances (_Non-VPC_) are available for AWS accounts
  created before *2013-12-03*.
  `aws_cloud` module is only tested for _EC2-VPC_ instances.

  See also:
  - [Default VPC and Default Subnets](
      https://docs.aws.amazon.com/vpc/latest/userguide/default-vpc.html
    )
  - [Discussion Forums: Launch a NON-VPC EC2 instance?](
      https://forums.aws.amazon.com/thread.jspa?threadID=182773
    )

- There are the following options to install `cloud` module as a [Cloud
  Orchestrator](https://www.drupal.org/project/cloud_orchestrator) distribution:

  1.  Use PHP composer: `composer create-project
      docomoinnovations/cloud_orchestrator cloud_orchestrator`
  2. Launch Cloud [Cloud Orchestrator](
       https://www.drupal.org/project/cloud_orchestrator
     ) `6.x-dev` from [AWS Marketplace](
       https://aws.amazon.com/marketplace/pp/B08PN2Z6X6?ref_=srh_res_product_title
     )
  3. Launch an Amazon EC2 instance of Cloud Orchestrator using
    [AWS CloudFormation Template](
      https://cloud-orchestrator.s3.amazonaws.com/cfn/cloud_orchestrator_single.yaml
    )
  4. [Download the module](
      https://www.drupal.org/project/cloud
    ) directly to an existing Drupal installation
IAM Policy (AWS)
----------------

- Before you create a new _AWS Cloud service provider_, you may want to
  configure _IAM Policy_ as follows:
  - Mandatory _IAM policy_ for creating an _AWS Cloud service provider_
    are:
    ```
    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Sid": "Mandatory",
                "Effect": "Allow",
                "Action": [
                    "ec2:DescribeAddresses",
                    "ec2:DescribeAvailabilityZones",
                    "ec2:DescribeIamInstanceProfileAssociations",
                    "ec2:DescribeCarrierGateways",
                    "ec2:DescribeFlowLogs",
                    "ec2:DescribeImageAttribute",
                    "ec2:DescribeImages",
                    "ec2:DescribeInstanceAttribute",
                    "ec2:DescribeInstances",
                    "ec2:DescribeInternetGateways",
                    "ec2:DescribeKeyPairs",
                    "ec2:DescribeLaunchTemplates",
                    "ec2:DescribeLaunchTemplateVersions",
                    "ec2:DescribeNetworkInterfaces",
                    "ec2:DescribeSecurityGroups",
                    "ec2:DescribeSnapshots",
                    "ec2:DescribeSubnets",
                    "ec2:DescribeTransitGateways",
                    "ec2:DescribeVolumes",
                    "ec2:DescribeVpcPeeringConnections",
                    "ec2:DescribeVpcs",
                    "iam:ListInstanceProfiles"
                ],
                "Resource": "*"
            }
        ]
    }
    ```
  - Optional _IAM policy_ for creating an _AWS Cloud service provider_ are:
    ```
    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Sid": "Optional",
                "Effect": "Allow",
                "Action": [
                    "cloudwatch:GetMetricData",
                    "ec2:AuthorizeSecurityGroupEgress",
                    "ec2:AuthorizeSecurityGroupIngress",
                    "ec2:CreateKeyPair",
                    "ec2:CreateLaunchTemplate",
                    "ec2:CreateNetworkInterface",
                    "ec2:CreateSecurityGroup",
                    "ec2:CreateSnapshot",
                    "ec2:CreateSubnet",
                    "ec2:CreateTags",
                    "ec2:CreateVpc",
                    "ec2:GetConsoleOutput",
                    "ec2:RevokeSecurityGroupEgress",
                    "ec2:RevokeSecurityGroupIngress"
                ],
                "Resource": "*"
            }
        ]
    }
    ```
  - The resource field should limit to the AWS account number and target
    resource e.g. `"Resource": "arn:aws:ec2:*:049876562615:key-pair/*"`.
  - See also: [Example policies for working in the Amazon EC2 console](
      https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-policies-ec2-console.html
    )

Basic Setup (AWS)
-----------------
1. Add a new _AWS Cloud service provider_ based on your needs.
   Go to `Structure` > `Cloud service providers` and click
   `+ Add cloud service provider` for _AWS Cloud_
2. Enter all required configuration parameters.  The system will automatically
   set up all regions from your AWS account.  There are three options for
   specifying AWS credentials:
   1. Instance credentials - If cloud module is running on an EC2 instance and
      the EC2 instance has an IAM role attached, you have the option to check
      `Use Instance Credentials`.  Doing so is secure and does not require
      `Access key ID` and `Secret access key` to be entered into Drupal.
      Please refer to this AWS tutorial about IAM role and EC2 Instance:
      https://aws.amazon.com/blogs/security/easily-replace-or-attach-an-iam-role-to-an-existing-ec2-instance-by-using-the-ec2-console/
   2. Simple access - Specify `Access key ID` and `Secret access key` to access
      a particular account's EC2 instances.
   3. Assume role - Specify `Access key ID`, `Secret access key` and the
      `Assume role` section.  With this combination, the `cloud` module can
      assume the role of another AWS account and access their EC2 instances.
      To learn more about setting up assume role setup, please read this AWS
      tutorial:

      https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_permissions-to-switch.html
3. Run cron to update your specific Cloud region.
4. Use the links under `Cloud service providers` > `[Cloud service provider]`
   `Cloud service providers` > `[Cloud service provider]` | `Images`
   - Search for images by AMI name.  For example, to import `Anaconda` images
   based on Ubuntu, type in `anaconda*ubuntu*`.
   Use the AWS Console on `aws.amazon.com` to search for images to import
6. `Import` or `Add AWS Cloud key pair`.  The key pair is used to log into any
   system you launch.  Use the links under the tab:
   `Cloud service providers` > `[Cloud service provider]` | `Key pair`
   - Use the `+ Import AWS Cloud key pair` button to import an existing key
     pair.  You will be uploading your public key.
   - Use `+ Add AWS Cloud key pair` to have AWS generate a new private key.
     You will be prompted to download the key after it is created.
7. Setup `Security groups`, `Network interfaces` as needed.
1. Create a launch template under
   `Design` > `Launch template` > `[Cloud service provider]`
2. After creating a template, change the workflow status to `Approved`.
3. Click the `Launch` tab to launch it.
- Configure permissions per your requirements.
  - [The detail about permissions is here.](
      https://www.drupal.org/docs/8/modules/cloud/configuration
    )

Module Structure
================

`cloud` is the heart of modules of the `Cloud` and `Cloud service provider`
packages.
cloud
`-- modules
    |-- cloud_budget
    |-- cloud_cluster_worker
    |-- cloud_service_providers
    |   |-- aws_cloud
    |   |-- cloud_cluster
    |   |-- docker
    |   |-- k8s
    |   |-- openstack
    |   |-- terraform
    |   `-- vmware
    |-- gapps
    `-- tools
        |-- k8s_to_s3
        `-- s3_to_k8s
- When adding a Metrics Server enabled Kubernetes cluster, the metrics
  importing operation can potentially take a long time to complete.
  During this process, there might be database corruption if the `aws_cloud`
  module is enabled.
- As a workaround, enable `aws_cloud` when the server is idle and not processing
  an `Add cloud service provider` for _Kubernetes_ operation.
License
=======

Cloud Orchestrator is released under the terms of the [GPLv2 license](https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html#SEC1).


- `yas` (https://drupal.org/u/yas)
- `baldwinlouie` (https://www.drupal.org/u/baldwinlouie)
- `Kumiko Ono (kumikoono)` (https://www.drupal.org/u/kumikoono)
- `Masami Suzuki (Masami)` (https://www.drupal.org/u/Masami)
- `Ryo Yamashita` (https://www.drupal.org/u/ryo-yamashita)
- `Takumaru Sekine (sekinet)` (https://www.drupal.org/u/sekinet)
- `Xiaohua Guan` (https://www.drupal.org/u/xiaohua-guan)