Using the AWS Asset Manager
The AWS Asset Manager can be used in multiple operational scenarios. For example, it can launch a Broker and its Engines in EC2. It’s also possible to launch Engines in EC2 from a Broker that is not launched in EC2. To use the AWS Asset Manager, perform the following tasks.
Create a Broker AMI
Procedure
- Launch an OS AMI (such as SLES 12 64-bit) making sure public IP is enabled, so yum updates to the OS will work. An m4.large AMI with 200 GB of disk space is ideal. Make sure there is enough swap space (about 8 GB) and enough file descriptors set on Broker for the fabric user (at least 200000).
- Install the Broker and configure it normally, starting with the JDK. Install relevant distributions, Enablers, and Skyway templates as needed. See the Installing Brokers section of the Silver Fabric Installation Guide for more information.
- Install an rc script to start and stop the Broker, as shown in Example Unix Broker Startup Script.
- Install any other items you may want in the Broker AMI, such as a reporting database, Maven repository, or LDAP server.
- Optionally, you can configure and run the NFS server on the Broker and export NFS directories. This makes it convenient to export the Grid Library location and a data directory used by Components when they need a shared file system. Refer to the Silver Fabric Cloud Administration Guide for more information on configuring an NFS mount as an alternate shared Grid Library location.
- Clean up the /tmp directory and various system logs.
- Create the AMI with the Broker shut down and snapshot the AMI without system reboot.
Create an Engine AMI
Procedure
- Launch an OS AMI (such as Centos 7 64-bit) making sure public IP is enabled so yum updates to the OS will work. An m4.large AMI with 200 GB of disk space is ideal.
- Extract the Engine TAR archive under /opt, but don't configure or run the Engine.
- Use the /opt/sfengine script as the start stop script and configure start/stop with the systemd service. See the Example AWS Sfengine Script for an example centos/RedHat start script.
- Enable the systemd sfengine.service. See the Example Sfengine.service Script for an example script.
- The standard engine.sh and configure.sh scripts may not always work for various OSes, and may require modification. For example, see the Example Engine.sh Script and Example Configure.sh Script for centos7/RedHat7.
- Create a user account named fabric.
- Clean up the /tmp directory and various system logs.
- Create the AMI with the Engine shut down and snapshot the AMI without system reboot.
Configure EC2 Security
Procedure
- Log in to the AWS Management Console and create a new Key Pair (See the AWS documentation on details), or use an existing AWS Key Pair.
- If you do not plan to use AWS VPC, create an AWS Security Group for the Silver Fabric Broker, named, for example, FSBroker. In this security group, grant access to ports 22, 8000 and 8443, and add more ports as needed.
- If you want your Silver Fabric Engine AWS instances to be in a different security group than the Silver Fabric Broker, create a separate Security Group for Silver Fabric Engines, named, for example, FSEngine. Allow public access to ports you want to access for running applications. It is easiest to grant access to all ports in the range 1-65000 from your Public Host IP address, but you may choose a narrower range.
- Ensure that the one or more security groups you created can access AWS instances within each other’s groups across all TCP/IP and UDP ports (1-65000) and also access instances within the same security group across all TCP/IP and UDP ports (1-65000).
- Go to the AWS Management Console and under , ensure you have the Silver Fabric Broker and Engine AMIs.
Configure Asset Manager
Procedure
- Go to Silver Fabric Asset Manager for AWS with the following configuration properties: Access Key, Secret Key, and Key Pair. and update the
- For the Key Pair field, create the key pair at and upload it to your Broker at the path given in this value.
-
The
Access key and
Secret key values can also be provided in a file by doing the following:
- If you are not using an AWS VPC (note that for all AWS accounts created after Dec. 4, 2013, AWS EC2 includes a default VPC, so you may be in a VPC without being aware of it), edit the Security Groups and set it to the Security Groups you created earlier, such as FSBroker, if you want to use a single security group for Broker and Engines or FSEngine, if you want to use a different security group for the Broker and Engines. More than one Security Group can be provided in a comma-separated list.
- To use Silver Fabric manager AMI in an AWS VPC (note that EC2 includes a default VPC for all accounts created after Dec. 4, 2013) configure the SubnetId and Security group Ids fields in the AWS Asset Manager configuration, but do not configure the Security Groups field. When using the security group ID of a VPC, configure the security group associated with the VPC appropriately to allow the Silver Fabric Broker to be accessible to Engines, and vice-versa. More than one Security Group ID can be provided through a comma-separated list.
- Select the Sweep Volumes option if you want detached volumes to be automatically deleted. Be careful while using this option because it will delete all detached volumes with status "available" in your AWS account.
- Select the Terminate option if you want idle Silver Fabric Engine AWS instances (the default minimum idle timeout is 15 minutes, and can be set with the Engine Idle Age variable) to be automatically terminated, otherwise, they are stopped, but not terminated.
- Set the Ec2 Url property to the relevant EC2 endpoint URL shown at http://docs.aws.amazon.com/general/latest/gr/rande.html#ec2_region
- Go to and select if it is shown. If at any time your Engines turn red on the page, you will need to do this step, which will restart all Engines and running components.
- Log into EC2 Management Console with your account, go to https://aws.amazon.com/marketplace/pp/B00O7WM7QW and launch an instance of the Centos 7 x86_64 AMI in your AWS account. The only purpose of this launch is to accept the Centos 7 AWS Marketplace license. You only have to do this once. You can terminate the instance as soon as you successfully launch it.
- Go to and enable the Silver Fabric Asset Manager for AWS.
- Each Stack running within the Silver Fabric Broker can be configured to use its own specific Silver Fabric Engine AMI and a compatible AWS instance type.
- If the Require Policy Rule is selected, AssetManagerId is a must Component rule.
- If the AssetManagerId component rule is selected, it will always be validated against the value given in the Asset Manager configuration irrespective of the Require Policy Rule value.
- Resource preference Rules can be set with an equal operator for the Engine properties AssetManagerId, ec2AmiId, ec2InstanceType, ec2Zone, or vpcSubnetId. If there is a match, the respective properties values from the Asset Manager configuration will be overriden with the values defined in the Component policy rules.
- Initially launched Engines will have the Engine Group property value set to initial. In order to use the initially launched Engines for allocation, the Component Policy rule Group should be set to initial with the equal operator, or else the new Engine would be launched with the Engine Group property value as null.
- Select Aws Standard Image if the Engine installation and provisioning needs to be done from the Asset Manager on the launched instances.
- Provide Volume Size along with Device Name if the default root volume size needs to be updated. Note that volume size has to be greater than the original AMI volume size. Device name is the root device name of the AMI.
-
If the Broker is SSL-enabled, make sure that the required certificates are available on the Broker. If you are using a custom AMI, make sure that certificates are available with the AMI. If the Broker is SSL-enabled and a standard AMI is used, SSL certificates need to be copied to Engines. They can be specified in the
AwsConfig.properties file. If these values are not provided, the Asset Manager will copy the certificates from the default location (SF_HOME/webapps/livecluster/engineUpdate/shared). To specify them, add the following to the
AwsConfig.properties file:
After Configuration
If changes needs to be done to an enabled Asset Manager, disable the respective Asset Manager first, make changes and then enable it again. If several configuration changes needs to be done, it is advisable to create another Asset Manager instance with the different configuration.
After your Broker is running, you can:
Important General Notes
The following notes apply to the AWS Asset Manager:
- Do not use an Elastic IP for the Broker.
- You do not need to manually launch a Silver Fabric Engine AWS instance. The Silver Fabric Engine AMI is automatically used by the Broker AWS Asset Manager to create AWS instances, as needed.
- You can connect to any running Silver Fabric Engine AWS instance by using the command ssh –i your-key.pem user@aws-public-hostname
- Once the Silver Fabric Broker AWS instance has been successfully started, it can be safely stopped and restarted at a later time. However, before you stop the Silver Fabric Broker AWS instance, make sure all running stacks in the Silver Fabric Administration Tool are completely stopped.
- Do not manually start or stop AWS instances for Silver Fabric Engines.
Copyright © Cloud Software Group, Inc. All rights reserved.