Skip to content
Snippets Groups Projects
Forked from project / cloud
1813 commits behind the upstream repository.
guanxiaohua's avatar
Issue #3221182 by Xiaohua Guan, yas: Add Cloud Orchestrator cloud service...
xiaohua guan authored
Issue #3221182 by Xiaohua Guan, yas: Add Cloud Orchestrator cloud service provider (QueueWorker) (Convert payload) (OpenStack entities)
cac42fcb
History

INTRODUCTION

Cloud is a set of modules to realize Cloud management: Drupal-based Cloud dashboard such as Amazon Management Console, RightScale, ElasticFox, etc. The module aims to support not only public Cloud like Amazon EC2 but also private Cloud like Kubernetes, VMware or OpenStack since the system is highly modularized by Drupal architecture.

REQUIREMENTS

  • PHP 7.4 or Higher (128 MB Memory or higher)

  • MySQL 5.7 or higher OR

  • MariaDB 10.3 or higher

  • Drupal 9.2.x or higher (The latest version of Drupal 9.2.x)

  • Cloud 4.x

    • This branch includes support for AWS, OpenStack, Kubernetes, VMware and Terraform Cloud
    • Future support includes GCP and MS Azure.
  • AWS

    • Mandatory IAM permissions for creating a cloud service provider are:
      • "ec2:DescribeAddresses",
      • "ec2:DescribeAvailabilityZones",
      • "ec2:DescribeIamInstanceProfileAssociations",
      • "ec2:DescribeCarrierGateways",
      • "ec2:DescribeFlowLogs",
      • "ec2:DescribeImageAttribute",
      • "ec2:DescribeImages",
      • "ec2:DescribeInstanceAttribute",
      • "ec2:DescribeInstances",
      • "ec2:DescribeInternetGateways",
      • "ec2:DescribeKeyPairs",
      • "ec2:DescribeLaunchTemplates",
      • "ec2:DescribeLaunchTemplateVersions",
      • "ec2:DescribeNetworkInterfaces",
      • "ec2:DescribeSecurityGroups",
      • "ec2:DescribeSnapshots",
      • "ec2:DescribeSubnets",
      • "ec2:DescribeTransitGateways",
      • "ec2:DescribeVolumes",
      • "ec2:DescribeVpcPeeringConnections",
      • "ec2:DescribeVpcs",
      • "iam:ListInstanceProfiles"
    • Optional IAM permissions for creating a cloud service provider are:
      • "cloudwatch:GetMetricData",
      • "ec2:AuthorizeSecurityGroupEgress",
      • "ec2:AuthorizeSecurityGroupIngress",
      • "ec2:CreateKeyPair",
      • "ec2:CreateLaunchTemplate",
      • "ec2:CreateNetworkInterface",
      • "ec2:CreateSecurityGroup",
      • "ec2:CreateSnapshot",
      • "ec2:CreateSubnet",
      • "ec2:CreateTags",
      • "ec2:CreateVpc",
      • "ec2:GetConsoleOutput",
      • "ec2:RevokeSecurityGroupEgress",
      • "ec2:RevokeSecurityGroupIngress"

Limitations

INSTALLATION

  1. Download aws-sdk from: https://docs.aws.amazon.com/aws-sdk-php/v3/download/aws.zip and unzip it into the vendor directory.

  2. Download cloud module.

  3. Enable the aws_cloud module. This will also enable the required modules.

    OR (using composer)

  • composer require drupal/cloud

CONFIGURATION

Basic Setup (AWS)

  1. Create a new cloud service provider based on your needs. Go to Structure > Cloud service providers and + Add cloud service provider

  2. Enter all required configuration parameters. The system will automatically setup all regions from your AWS account. There are three options for specifying AWS credentials:

    a. Instance credentials - If cloud module is running on an EC2 instance and the EC2 instance has an IAM role attached, you have the option to check "Use Instance Credentials". Doing so is secure and does not require Access Key ID and Secret Access Key to be entered into Drupal. Please refer to this AWS tutorial about IAM role and EC2 Instance:

    https://aws.amazon.com/blogs/security/easily-replace-or-attach-an-iam-role-to-an-existing-ec2-instance-by-using-the-ec2-console/

    b. Simple access - Specify Access Key ID and Secret Access Key to access a particular account's EC2 instances.

    c. Assume role - Specify Access Key ID, Secret Access Key and the Assume Role section. With this combination, the cloud module can assume the role of another AWS account and access their EC2 instances. To learn more about setting up assume role setup, please read this AWS tutorial:

    https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_permissions-to-switch.html

  3. Run cron to update your specific Cloud region.

  4. Use the links under Cloud service providers > [Cloud service provider] to manage your Amazon EC2 entities.

  5. Import Images using the tab: Cloud service providers > [Cloud service provider] | Images

    • Click on + Import AWS Cloud Image
    • Search for images by AMI name. For example, to import Anaconda images based on Ubuntu, type in anaconda*ubuntu*. Use the AWS Console on aws.amazon.com to search for images to import
  6. Import or Add AWS Cloud Key Pair. The key pair is used to log into any system you launch. Use the links under the tab: Cloud service providers > [Cloud service provider] | Key Pair

    • Use the + Import AWS Cloud Key Pair button to import an existing key pair. You will be uploading your public key.
    • Use + Add AWS Cloud Key Pair to have AWS generate a new private key. You will be prompted to download the key after it is created.
  7. Setup Security groups, Network Interfaces as needed from AWS Management Console.

Launching Instance

  1. Create a launch template under Design > Launch template > [Cloud service provider]
  2. Once template is created, click the Launch tab to launch it.

Permissions

Directory Structure

cloud (Cloud is a core module for Cloud package)
└── modules
    └── cloud_budget
    └── cloud_service_providers
        └── aws_cloud
        └── k8s
        └── docker
        └── openstack
        └── terraform
        └── vmware
    └── gapps
    └── tools
        └── k8s_to_s3
        └── s3_to_k8s

Known Issues

When adding a Metrics Server enabled Kubernetes cluster, the metrics importing operation can potentially take a long time to complete. During this process, there might be database corruption if the aws_cloud module is enabled.

As a workaround, enable aws_cloud when the server is idle and not processing a Add Kubernetes Cloud Service Provider operation.

Active Maintainers