knife-cosmic

DESCRIPTION:

This is the Edmunds Knife plugin for cosmic. This plugin gives Knife the ability to create, bootstrap and manage cosmic instances.

INSTALLATION:

This plugin is distributed as a Ruby Gem. To install it, run:

gem install knife-cosmic

Depending on your system’s configuration, you may need to run this command with root privileges.

CONFIGURATION:

In order to communicate with the cosmic API you will have to tell Knife about your cosmic API Key, your Secret Key and the URL of the API endpoint. The easiest way to accomplish this is to create some entries in your knife.rb file:

knife[:cosmic_url] = "http://yourcosmicosmicerver.com:8080/client/api
knife[:cosmic_api_key]  = "Your cosmic API Key"
knife[:cosmic_secret_key] = "Your cosmic Secret Key"

If your knife.rb file will be checked into a SCM system (ie readable by others) you may want to read the values from environment variables:

knife[:cosmic_url] = "#{ENV['cosmic_URL']}"
knife[:cosmic_api_key] = "#{ENV['cosmic_API_KEY']}"
knife[:cosmic_secret_key] = "#{ENV['cosmic_SECRET_KEY']}"

You also have the option of passing your cosmic URL and API/Secret Keys into the individual knife subcommands using the -U (or --cosmic-url), -A (or --cosmic-api-key ) -K (or --cosmic-secret-key) command options

# provision a new small RHEL 5.6 webserver
knife cosmic server create my-server -r 'role[webserver]' -S "small" -T "RHEL 5.6 Base" -A 'Your cosmic API Key' -K "Your cosmic Secret Key" --distro "rhel5-gems"

Additionally the following options may be set in your knife.rb:

Public Clouds (Tata InstaCompute, Ninefold etc):

To get this plugin to work in public clouds, it is essential that the virtual network (and router) be allocated to the account. cosmic clouds automatically creates a virtual network when the first VM is requested to be created. Hence, it is essential to create the first VM (of a newly created account) manually(which can be terminated immediately if not required) to ensure the virtual network is created.

SUBCOMMANDS:

This plugin provides the following Knife subcommands. Specific command options can be found by invoking the subcommand with a --help flag

knife cosmic <command> list

--filter Filters your output on the fields that you specify. Filter can be a string ‘name’ or regexp ‘/name/i’

Example: knife cosmic server list --filter "instancename:/i-xxx/i, account:accname"

--fields The fields that you want to display in your output. Default outputs will be shown if no fields are defined.

Example: knife cosmic server list --fields "name, instancename, domain"

--fieldlist This will display all information returned by the cosmic API. These fields can be used within the --fields or --filter to create custom output. The data displayed is based on first result returned in your output. You can get other result by using the --filter option.

--noheader this will remove the column header from the output.

knife cosmic server create

Provisions a new server in cosmic and then performs a Chef bootstrap (using the SSH protocol). The goal of the bootstrap is to get Chef installed on the target system so it can run Chef Client with a Chef Server. The main assumption is a baseline OS installation exists (provided by the provisioning). It is primarily intended for Chef Client systems that talk to a Chef server. By default the server is bootstrapped using the ‘chef-full’ template (default bootstrap option for knife, (Ref. docosmic.opscode.com/knife_bootstrap.html). This can be overridden using the -d or --template-file command options. VM provided with --no-bootstrap option have no forwarded ports or forwared ip rules (in case --static-nat is used).

By default, new servers are allocated a public IP address mapping to the cosmic private IP address. If you do not want this behavior, pass the --no-public-ip option.

Forcing cloustack to reserve and use a certain private ipv4 address when deploying an instance inside an advanced zone is possible by passing <option>–private-ip</option> followed by the intended ipv4 address. Ip Address must be contained within the network mask of the isolated network, of course.

Example: knife cosmic server create testvm --private-ip 192.168.1.1 --networks mynetwork

Here mynetwork must have a network mask for example like 192.168.1.0/24.

Port forwarding rules for virtual networks

The --port-rules option takes a comma separated list of port forwarding rules. These rules are created on the virtual public ip address of the server. Note that this option only applies to servers using a virtual network; it is ignored if the server’s public ip address is on a direct attached network.

Port forwarding rules have the syntax PUBLIC_PORT[:PRIVATE_PORT[:PROTOCOL]]. PRIVATE_PORT and PROTOCOL are optional. The default value of PRIVATE_PORT is PUBLIC_PORT and the default PROTOCOL is ‘TCP’. For example, a rule to forward from public port 80 to private port 7000 would look like 80:7000:TCP. Since ‘TCP’ is the default protocol, the rule can be shortened to 80:7000. A rule can even be shortened to a single number when the public and private ports are the same. For example, a rule to forward from public port 25 to private port 25 can be stated as simply 25. A list of such rules for a webserver might look like 80,443.

IP forwarding rules

The --ipfwd-rules option takes a comma separated list of ip forwarding rules. These rules are created on public ip appdress assigned obtained with --static-nat option. (Ref. cosmic.apache.org/docosmic/api/apidocosmic-4.0.0/root_admin/createIpForwardingRule.html)

Ip forwarding rules have the syntax START_PORT[:END_PORT[:PROTOCOL]]. END_PORT and PROTOCOL are optional. The default value of END_PORT is START_PORT and the default PROTOCOL is ‘TCP’. For example, a rule to forward ports range from 1024 to 10000 would look like 1024:10000:TCP. Since ‘TCP’ is the default protocol, the rule can be shortened to 1024:10000. A rule can even be shortened to a single number when the start and end ports are the same. For example, a rule to forward port 22 can be stated as simply 22. A list of such rules for a webserver might look like 80,443.

Create Firewall Rule for given ip address

The -f, --fw-rules option takes a comma separated list of firewall rules which are applied to the public ip address assigned to the current server.

Firewall rules have the syntax PROTOCOL[:CIDR_LIST[:START_PORT[:END_PORT]]]. START_PORT and END_PORT must not be specified when PROTOCOL is ICMP, CIDR_LIST is always optional. The default value of END_PORT is START_PORT, the default CIDR_LIST is ‘0.0.0.0/0’. For example, a rule to open firewall for port 80 to everyone would look like TCP::80 and a rule to open ICMP to internal network would look like ICMP:10.0.0.0/8.

knife cosmic server delete

Deletes an existing server in the currently configured cosmic account. PLEASE NOTE - this does not delete the associated node and client objects from the Chef server.

knife cosmic server list

Displays a list of all servers in the currently configured cosmic account. PLEASE NOTE - this shows all servers associated with the cosmic account including servers that are not registered with a Chef server.

--listall This will list all the servers, depending on the account that you are using.

--keyword Filters your output on the instance name that you specify:

Example: --keyword "i-324", This will display all servers with <tt>'i-324'</tt> inside the instancename.

--name Filters your output on the name that you specify.

Example: --name "webserver", This will display all servers that contains 'webserver' inside the hostname.

--action This action will be executed on the output of the list command. Actions can be start, stop or destroy.

knife cosmic network list

Displays a list of all networks available in the currently configured cosmic account. A network can be specified when creating a new server by passing the network name as an argument to the -W (or –network) option of the knife cosmic server create command.

--listall This will list all the networks, depending on the account that you are using.

--keyword Filters your output on the keyword that you specify.

Example: --keyword "network1", This will display all networks with 'network1' inside the name.

knife cosmic service list

Displays a list of all service offerings available in the currently configured cosmic account. A service offering can be specified when creating a new server by passing the name as an argument to the -S (or –service) option of the knife cosmic server create command.

--keyword Filters your output on the service name, just like --name here below.

--name Filters your output on the name that you specify.

Example: --name "medium" or --keyword "medium", This will display all cpu service offerings with 'medium' inside the name.

knife cosmic template list

Displays a list of all templates in the currently configured cosmic account. Featured templates are displayed by default. Use the -F (or –filter) option to use a different filter. The allowable filter values are:

featured - templates that are featured and are public (this is the default)
self - templates that have been registered/created by the owner
self-executable - templates that have been registered/created by the owner that can be used to deploy a new VM
executable - all templates that can be used to deploy a new VM
community - templates that are public

A template can be specified when creating a new server by passing the template name as an argument to the -T (or –template) option of the knife cosmic server create command.

--listall This will list all the templates, depending on the account that you are using.

knife cosmic template create

Creates a template based on a volumeID

knife cosmic template extract

Returns a link where an extractable template can be downloaded

knife cosmic template register

Creates a template based on a file

knife cosmic volume create

Creates a volume based on a name

knife cosmic zone list

Displays a list of all zones available in the currently configured cosmic account. A zone can be specified when creating a new server by passing the zone name as an argument to the -Z (or –zone) option of the knife cosmic server create command.

--keyword Filters your output on the keyword that you specify.

Example: --keyword "zone1", This will display all zones with 'zone1' inside the name

knife cosmic hosts

Convenience command that displays the public ip address and fqdn for all servers. Matches /etc/hosts file format.

--listall This will list all the servers, depending on the account that you are using.

--keyword Filters your result based on a keyword.

--name Filters your output on the name that you specify.

knife cosmic account list

Displays all accounts that are currently in your cosmic environment.

--listall This will list all the possible results, depending on the account that you are using.

--keyword Filters your result based on a keyword.

--name Filters your output on the name that you specify.

knife cosmic cluster list

Displays all clusters that are currently available in your cosmic environment.

--keyword Filters your result based on a keyword.

--name Filters your output on the name that you specify.

knife cosmic config list

Displays all the configuration settings that are configured inside the cosmic environment.

--keyword Filters your result based on a keyword.

--name Filters your output on the name that you specify.

knife cosmic disk list

Displays all the disks that are available within cosmic.

--keyword Filters your result based on a keyword.

--name Filters your output on the name that you specify.

knife cosmic domain list

Displays all domains within the cosmic environment.

--listall This will list all the domains, depending on the account that you are using.

knife cosmic firewallrule list

Displays all firewall rules that are currently active within your cosmic environment.

--listall This will list all the firewallrules, depending on the account that you are using.

--keyword Filters your result based on a keyword.

knife cosmic host list

Displays all hosts from the cosmic environment.

--name Filters your output on the name that you specify.

knife cosmic iso list

Displays all iso’s that are available within the cosmic environment.

--listall This will list all the iso’s, depending on the account that you are using.

--keyword Filters your result based on a keyword.

--name Filters your output on the name that you specify.

knife cosmic ostype list

Displays all the os types that are available within the cosmic environment.

--keyword Filters your output on the keyword that you specify.

knife cosmic oscategory list

Displays all os categories that are available --keyword Filters your output on the keyword that you specify.

knife cosmic pod list

Displays all the pods that are currently available within the cosmic environment.

--keyword Filters your result based on a keyword.

--name Filters your output on the name that you specify.

knife cosmic project list

Displays all the projects that are within the cosmic environment.

--listall This will list all the projects, depending on the account that you are using.

--keyword Filters your result based on a keyword.

--name Filters your output on the name that you specify.

knife cosmic publicip list

Displays all public ip’s that are used within the cosmic environment.

--listall This will list all the public ip’s, depending on the account that you are using.

knife cosmic router list

Displays all the routers that are within the cosmic environment.

--listall This will list all the routers, depending on the account that you are using.

--keyword Filters your result based on a keyword.

--name Filters your output on the name that you specify.

knife cosmic user list

Displays all the users that are available within your account.

--listall This will list all the users, depending on the account that you are using.

--keyword Filters your output on the keyword that you specify.

knife cosmic volume list:

Displays all the volumes that are currently available within the cosmic environment.

--listall This will list all the volumes, depending on the account that you are using.

--keyword Filters your output on the keyword that you specify.

--name Filters your output on the name that you specify.

knife cosmic stack create

Creates a “stack” of servers based on a JSON definition file. Simple orchestration can be performed by specifying one or more actions to be executed after a server (or group of servers) is created.

--skip-existing Skip erroring on any servers already created in the stack (default is false)

Example Stack Definition File:

{
  "name": "hadoop_cluster_a",
  "description": "A small hadoop cluster with hbase",
  "version": "1.0",
  "environment": "production",
  "servers": [
    {
      "name": "zookeeper-a, zookeeper-b, zookeeper-c",
      "description": "Zookeeper nodes",
      "template": "rhel-5.6-base",
      "service": "small",
      "port_rules": "2181",
      "run_list": "role[cluster_a], role[zookeeper_server]",
      "actions": [
        { "knife_ssh": ["role:zookeeper_server", "sudo chef-client"] }
      ]
    },
    {
      "name": "hadoop-master",
      "description": "Hadoop master node",
      "template": "rhel-5.6-base",
      "service": "large",
      "networks": "app-net, storage-net",
      "port_rules": "50070, 50030, 60010",
      "run_list": "role[cluster_a], role[hadoop_master], role[hbase_master]"
    },
    {
      "name": "hadoop-worker-a hadoop-worker-b hadoop-worker-c",
      "description": "Hadoop worker nodes",
      "template": "rhel-5.6-base",
      "service": "medium",
      "port_rules": "50075, 50060, 60030",
      "run_list": "role[cluster_a], role[hadoop_worker], role[hbase_regionserver]",
      "actions": [
        { "knife_ssh": ["role:hadoop_master", "sudo chef-client"] },
        { "http_request": "http://${hadoop-master}:50070/index.jsp" }
      ]
    }
  ]
}

Stack Attributes

name: Stack name. May be used in the future to e.g. look up a stack in a databag.
version: Stack version. This attribute is not currently used.
environment: Chef environment used for all servers in the stack.
servers: List of servers to create.

Server Attributes

name: The name of the new server. Specify a list of names separated by commas and/or spaces to create multiple servers.
description: Server description.
template: cosmic template name.
service: cosmic service offering name.
port_rules: Port forwarding rules for virtual networks. These are ignored when using direct or other network types.
run_list: Chef run list.
actions: List of actions to perform after the server (or group of servers) is created.

Actions

knife-ssh: Performs a knife-ssh command. Takes the search query and the command to execute as an array of arguments.
http_request: Performs an http request. Supports references to server names in the stack with $server_name.

Upcoming Actions

The following actions are under development:

runlist_add: Adds an entry from the run list.
runlist_remove: Removes an entry from the run list.

knife cosmic stack delete

Deletes all servers in the specified stack definition.

knife cosmic server start

Starts the specified virtual machine(s).

knife cosmic server stop

Stops the specified virtual machine(s). Force stop is optional.

knife cosmic server reboot

Reboots the specified virtual machines(s).

LICENSE:

Original knife-cloudstack author: Ryan Holmes <[email protected]>
Original knife-cloudstack author: KC Braunschweig <[email protected]>
Original knife-cloudstack author: John E. Vincent <[email protected]>
Original knife-cloudstack author: Sander Botman <[email protected]>
Original knife-cloudstack author: Frank Breedijk <[email protected]>
Original knife-cloudstack author: Jeremy Baumont <[email protected]>
License: Apache License, Version 2.0

Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Trademark Attribution

knife is a trademark of Opscode

cosmic is a trademark or registered trademark of The Apache Software Foundation

The use of these marks is nominative in describing the function and integration of this software and does not imply endorsement by Opscode, the Apache Software Foundation, or Apache cosmic.