Deploying Cloud Foundry on OpenStack Juno and XenServer (Part II)

link

http://rabbitstack.github.io/deploying-cloud-foundry-on-openstack-juno-and-xenserver-part-ii/

Let‘s move on. We should have our OpenStack instance prepared for Cloud Foundry. The most usual way of deploying Cloud Foundry is through BOSH. For the who still didn‘t hear about it, BOSH is the platform for automation and lifecycle management of software and distributed services. It is also capable of monitoring and failure recovery of processes and virtual machines. There are already a few IT automation platforms in the market like Chef or Puppet, so, why to learn / use BOSH then?

One notable difference is that BOSH is able to perform the deployment from the sterile environment, i.e. package source code and dependencies, create the virtual machines (jobs in BOSH terminology) from the so calledstemcell template (VM which has BOSH agent installed and is used to generate the jobs), and finally install, start and monitor the required services and VMs. Visit the official page from the link above to learn more about BOSH.

Deploying MicroBOSH

MicroBOSH is a single VM which contains all the necessary components to boot BOSH, including the blobstore, nats, director, health manager etc. Once you have an instance of MicroBOSH running, you can deploy BOSH if you wish. Install BOSH CLI gems (Ruby >= 1.9.3 is required).

$ gem install bosh_cli bosh_cli_plugin_micro

You will need to create a keypair in OpenStack and configure bosh security group with the rules shown in the table below. You can do it by accessing the Horizon dashboard or by using nova CLI.

$ nova keypair-add microbosh > microbosh.pem
$ chmod 600 microbosh.pem

BOSH uses a variety of artifacts in order to complete the deployment life cycle. We can basically distinguish between stemcell, release and deployment. To deploy MicroBOSH we will only need a stemcell which can be downloaded using the bosh CLI. First get a list of available stemcells and download thebosh-stemcell-2839-openstack-kvm-centos-go_agent-raw.tgz.

$ bosh public stemcells
+-----------------------------------------------------------------+
| Name                                                            |
+-----------------------------------------------------------------+
| bosh-stemcell-2427-aws-xen-ubuntu.tgz                           |
| bosh-stemcell-2652-aws-xen-centos.tgz                           |
| bosh-stemcell-2839-aws-xen-centos-go_agent.tgz                  |
| bosh-stemcell-2427-aws-xen-ubuntu-go_agent.tgz                  |
| bosh-stemcell-2710-aws-xen-ubuntu-lucid-go_agent.tgz            |
| bosh-stemcell-2652-aws-xen-ubuntu-lucid.tgz                     |
| bosh-stemcell-2839-aws-xen-ubuntu-trusty-go_agent.tgz           |
| bosh-stemcell-2690.6-aws-xen-ubuntu-trusty-go_agent.tgz         |
| bosh-stemcell-2719.1-aws-xen-centos-go_agent.tgz                |
| bosh-stemcell-2719.1-aws-xen-ubuntu-trusty-go_agent.tgz         |
| bosh-stemcell-2719.2-aws-xen-centos-go_agent.tgz                |
| bosh-stemcell-2719.2-aws-xen-ubuntu-trusty-go_agent.tgz         |
| bosh-stemcell-2719.3-aws-xen-ubuntu-trusty-go_agent.tgz         |
| light-bosh-stemcell-2427-aws-xen-ubuntu.tgz                     |
| light-bosh-stemcell-2652-aws-xen-centos.tgz                     |
| light-bosh-stemcell-2839-aws-xen-centos-go_agent.tgz            |
| light-bosh-stemcell-2427-aws-xen-ubuntu-go_agent.tgz            |
| light-bosh-stemcell-2710-aws-xen-ubuntu-lucid-go_agent.tgz      |
| light-bosh-stemcell-2652-aws-xen-ubuntu-lucid.tgz               |
| light-bosh-stemcell-2839-aws-xen-ubuntu-trusty-go_agent.tgz     |
| light-bosh-stemcell-2690.6-aws-xen-ubuntu-trusty-go_agent.tgz   |
| light-bosh-stemcell-2719.1-aws-xen-centos-go_agent.tgz          |
| light-bosh-stemcell-2719.1-aws-xen-ubuntu-trusty-go_agent.tgz   |
| light-bosh-stemcell-2719.2-aws-xen-centos-go_agent.tgz          |
| light-bosh-stemcell-2719.2-aws-xen-ubuntu-trusty-go_agent.tgz   |
| light-bosh-stemcell-2719.3-aws-xen-ubuntu-trusty-go_agent.tgz   |
| light-bosh-stemcell-2839-aws-xen-hvm-centos-go_agent.tgz        |
| light-bosh-stemcell-2839-aws-xen-hvm-ubuntu-trusty-go_agent.tgz |
| bosh-stemcell-2427-openstack-kvm-ubuntu.tgz                     |
| bosh-stemcell-2624-openstack-kvm-centos.tgz                     |
| bosh-stemcell-2624-openstack-kvm-ubuntu-lucid.tgz               |
| bosh-stemcell-2839-openstack-kvm-centos-go_agent.tgz            |
| bosh-stemcell-2839-openstack-kvm-ubuntu-trusty-go_agent.tgz     |
| bosh-stemcell-2652-openstack-kvm-ubuntu-lucid-go_agent.tgz      |
| bosh-stemcell-2719.1-openstack-kvm-centos-go_agent.tgz          |
| bosh-stemcell-2719.1-openstack-kvm-ubuntu-trusty-go_agent.tgz   |
| bosh-stemcell-2719.2-openstack-kvm-centos-go_agent.tgz          |
| bosh-stemcell-2719.2-openstack-kvm-ubuntu-trusty-go_agent.tgz   |
| bosh-stemcell-2719.3-openstack-kvm-ubuntu-trusty-go_agent.tgz   |
| bosh-stemcell-2839-openstack-kvm-centos-go_agent-raw.tgz        |
| bosh-stemcell-2839-openstack-kvm-ubuntu-trusty-go_agent-raw.tgz |
| bosh-stemcell-2427-vcloud-esxi-ubuntu.tgz                       |
| bosh-stemcell-2652-vcloud-esxi-ubuntu-lucid.tgz                 |
| bosh-stemcell-2839-vcloud-esxi-ubuntu-trusty-go_agent.tgz       |
| bosh-stemcell-2690.5-vcloud-esxi-ubuntu-trusty-go_agent.tgz     |
| bosh-stemcell-2690.6-vcloud-esxi-ubuntu-trusty-go_agent.tgz     |
| bosh-stemcell-2710-vcloud-esxi-ubuntu-lucid-go_agent.tgz        |
| bosh-stemcell-2427-vsphere-esxi-ubuntu.tgz                      |
| bosh-stemcell-2624-vsphere-esxi-centos.tgz                      |
| bosh-stemcell-2839-vsphere-esxi-centos-go_agent.tgz             |
| bosh-stemcell-2427-vsphere-esxi-ubuntu-go_agent.tgz             |
| bosh-stemcell-2710-vsphere-esxi-ubuntu-lucid-go_agent.tgz       |
| bosh-stemcell-2624-vsphere-esxi-ubuntu-lucid.tgz                |
| bosh-stemcell-2839-vsphere-esxi-ubuntu-trusty-go_agent.tgz      |
| bosh-stemcell-2719.1-vsphere-esxi-centos-go_agent.tgz           |
| bosh-stemcell-2719.1-vsphere-esxi-ubuntu-trusty-go_agent.tgz    |
| bosh-stemcell-2719.2-vsphere-esxi-ubuntu-trusty-go_agent.tgz    |
| bosh-stemcell-2719.2-vsphere-esxi-centos-go_agent.tgz           |
| bosh-stemcell-2719.3-vsphere-esxi-ubuntu-trusty-go_agent.tgz    |
| bosh-stemcell-2690.6-vsphere-esxi-ubuntu-trusty-go_agent.tgz    |
| bosh-stemcell-389-warden-boshlite-ubuntu-trusty-go_agent.tgz    |
| bosh-stemcell-53-warden-boshlite-ubuntu.tgz                     |
| bosh-stemcell-389-warden-boshlite-centos-go_agent.tgz           |
| bosh-stemcell-64-warden-boshlite-ubuntu-lucid-go_agent.tgz      |
+-----------------------------------------------------------------+

$ bosh download public stemcell bosh-stemcell-2839-openstack-kvm-centos-go_agent-raw.tgz
bosh-stemcell:   4% |ooo                              |  24.4MB 753.0KB/s ETA:  00:11:43

Now we are ready to create the MicroBOSH deployment manifestmicrobosh-openstack.yml file. You will need to change net_id with your OpenStack instance network identifier, ip with the ip address from the network pool. You can find out that information by executing the following commands.

$ nova network-list
+--------------------------------------+----------+----------------+
| ID                                   | Label    | Cidr           |
+--------------------------------------+----------+----------------+
| 3f36d40e-1097-49a0-a023-4606dbf3a1f5 | yuna-net | 192.168.1.0/24 |
+--------------------------------------+----------+----------------+

$ nova network-show 3f36d40e-1097-49a0-a023-4606dbf3a1f5
+---------------------+--------------------------------------+
| Property            | Value                                |
+---------------------+--------------------------------------+
| bridge              | xenbr0                               |
| bridge_interface    | eth0                                 |
| broadcast           | 192.168.1.255                        |
| cidr                | 192.168.1.0/24                       |
| cidr_v6             | -                                    |
| created_at          | 2014-12-28T17:18:14.000000           |
| deleted             | False                                |
| deleted_at          | -                                    |
| dhcp_server         | 192.168.1.50                         |
| dhcp_start          | 192.168.1.51                         |
| dns1                | 8.8.4.4                              |
| dns2                | -                                    |
| enable_dhcp         | True                                 |
| gateway             | 192.168.1.50                         |
| gateway_v6          | -                                    |
| host                | -                                    |
| id                  | 3f36d40e-1097-49a0-a023-4606dbf3a1f5 |
| injected            | False                                |
| label               | yuna-net                             |
| mtu                 | -                                    |
| multi_host          | True                                 |
| netmask             | 255.255.255.0                        |
| netmask_v6          | -                                    |
| priority            | -                                    |
| project_id          | -                                    |
| rxtx_base           | -                                    |
| share_address       | True                                 |
| updated_at          | -                                    |
| vlan                | -                                    |
| vpn_private_address | -                                    |
| vpn_public_address  | -                                    |
| vpn_public_port     | -                                    |
+---------------------+--------------------------------------+

Under the openstack section change the Identity service endpoint, OpenStack credentials, the private key location, and optionally set the timeout for OpenStack resources.

---
name: microbosh-openstack

logging:
  level: DEBUG

network:
  type: manual
  ip: 192.168.1.55
  cloud_properties:
    net_id: 3f36d40e-1097-49a0-a023-4606dbf3a1f5

resources:
  persistent_disk: 16384
  cloud_properties:
    instance_type: m1.medium

cloud:
  plugin: openstack
  properties:
    openstack:
      auth_url: http://controller:5000/v2.0
      username: admin
      api_key: admin
      tenant: admin
      default_security_groups: ["bosh"]
      default_key_name: microbosh
      private_key: /root/microbosh.pem
      state_timeout: 900

apply_spec:
  properties:
    director:
      max_threads: 3
    hm:
      resurrector_enabled: true
    ntp:
      - 0.europe.pool.ntp.org
      - 1.europe.pool.ntp.org

Finally, set the current deployment manifest file and deploy MicroBOSH.

$ bosh micro deployment microbosh-openstack.yml
$ bosh micro deploy bosh-stemcell-2839-openstack-kvm-centos-go_agent-raw.tgz

If everything goes well you should login into the MicroBOSH instance (use admin, for both username and password).

$ bosh target 192.168.1.55
Target set to ‘microbosh-openstack‘
Your username: admin
Enter password: *****
Logged in as ‘admin‘

Deploying Cloud Foundry

Start by cloning the Cloud Foundry repository. Enter the newly created cf-releasedirectory and execute the update script to update all submodules.

$ git clone https://github.com/cloudfoundry/cf-release.git
$ cd cf-release
$ ./update

Upload the stemcell to the BOSH Director.

$ bosh upload stemcell bosh-stemcell-2839-openstack-kvm-centos-go_agent-raw.tgz

In BOSH terminology, release is a collection of packages and source code, dependencies, configuration properties, and any other components required to perform a deployment. To create a Cloud Foundry release, use this command fromcf-release directory.

$ bosh create release

This will download the required blobs from the S3 storage service and generate a release tarball. You should end up with the similar directory structures.

$ ls blobs
buildpack_cache    git           haproxy         mysql             php-buildpack     rootfs      ruby-buildpack
cli                go-buildpack  java-buildpack  nginx             postgres          ruby        sqlite
debian_nfs_server  golang        libyaml         nodejs-buildpack  python-buildpack  ruby-2.1.4  uaa

$ ls packages
acceptance-tests        buildpack_python     dea_next             golang     loggregator_trafficcontroller  postgres        warden
buildpack_cache         buildpack_ruby       debian_nfs_server    golang1.3  login                          rootfs_lucid64
buildpack_go            cli                  doppler              gorouter   metron_agent                   ruby
buildpack_java          cloud_controller_ng  etcd                 haproxy    mysqlclient                    ruby-2.1.4
buildpack_java_offline  collector            etcd_metrics_server  hm9000     nats                           smoke-tests
buildpack_nodejs        common               git                  libpq      nginx                          sqlite
buildpack_php           dea_logging_agent    gnatsd               libyaml    nginx_newrelic_plugin          uaa

Now you can upload the release to the BOSH Director.

$ bosh upload release

The most complex part of Cloud Foundry BOSH deployment is the manifest file where all components are tied together - computing resource specifications, VMs, software releases, and configuration properties. You can use the deployment which worked great on my environment. Don’t forget to create cf.small and cf.medium flavors in OpenStack.

view rawcf-194.yml hosted with ? by GitHub

Set and initiate the deploy. This process can take a few hours. Relax.

$ bosh deployment cf-deployment.yml
$ bosh deploy

Pushing an application

Download the cf CLI from https://github.com/cloudfoundry/cli/releases. Make sure you can access the API endpoint of the Cloud Foundry instance. If so, use cf loginwith your username, organization and space.

$ curl http://api.192.168.1.249.xip.io/info
$ cf login -a api.192.168.1.249.xip.io -u user -o rabbitstack -s qa

To test our instance we are going to push a very simple node.js app. Create a new directory and place server.js and the application manifest.yml file in it.

var http = require("http");

var server = http.createServer(function (req, res) {
    res.writeHeader(200, {
        "Content-Type":"text/html"
    });
    res.end("Bunnies on Cloud Foundry. Port is " + process.env.VCAP_APP_PORT);
}).listen(process.env.VCAP_APP_PORT);

---
applications:
- name: rabbitstack
  path: .
  memory: 256M
  instances: 1

From within the directory run cf push and accesshttp://rabbitstack.192.168.1.249.xip.io from the browser. Play with cf scale and see how port number changes on every request.

Congratulations! You now have a fully functional private Cloud Foundry.