Adding a Runtime and Framework to the New Cloud Foundry

In my previous post I described how to add a runtime and framework to a Cloud Foundry created with the traditional setup install script found on github. The site also provides an experimental Chef-based installation (which appears to be the preferred method going forward), and I described how to create a private cloud following these instructions. In this post we will walk through the process of adding Perl as a runtime and framework to this new cloud. (Before starting we need a tunnel from the client to the server, we need a shell on the server, a shell on the client, and a logged-in VMC user.)

Using ‘vmc runtimes‘ and ‘vmc frameworks‘ on the client we see that Perl is not in the list of things supported on the server.

On the server, change to the VCAP directory (subsequent server commands are from this directory) and edit the following file:

cd ~/cloudfoundry/vcap/
vim cloud_controller/app/models/app.rb

Add ‘perl5’ as a runtime and ‘perl’ as a framework on lines 26 and 27:

Runtimes = %w[perl5 ruby18 ruby19 java node php erlangR14B02 python26]
Frameworks = %w[perl sinatra rails3 java_web spring grails node php otp_rebar lift wsgi django unknown]

Add Perl to the function normalize_legacy_staging_strings! starting at line 541:

when "perl/1.0"
  self.framework = 'perl'
  self.runtime = 'perl5'

Save this file and then edit the following file:

vim dev_setup/cookbooks/cloud_controller/attributes/default.rb

Add the following at line 10:

default[:cloud_controller][:staging][:perl] = "perl.yml"

Save this file and then edit cloud_controller.yml.erb (command broken up into parts to avoid loss of formatting in blog post):

(cd dev_setup/cookbooks/cloud_controller/templates/; 
 vim default/cloud_controller.yml.erb)

Add the following two lines starting at line 110:

perl5:
  version: 5.10

Save this file and then create the following file:

vim dev_setup/cookbooks/cloud_controller/templates/default/perl.yml.erb

The contents of the file is the following 11 lines:

---
name: "perl"
runtimes:
  - "perl5":
      version: "5.10"
      description: "Perl 5"
      executable: "/usr/bin/perl"
      default: true
      environment:
detection:
  - "main.pl"

Save this file and then edit the following file:

vim dev_setup/cookbooks/dea/attributes/default.rb

Edit line 3 to add ‘perl5’ as a runtime:

default[:dea][:runtimes] = ["perl5", "ruby18", "ruby19", "nodejs", "java", "erlang", "php"]

Save this file and then edit the following file:

vim dev_setup/cookbooks/dea/templates/default/dea.yml.erb

Add the following to the runtimes: list starting at line 46:

<% if node[:dea][:runtimes].include?("perl5") %>
  perl5:
    executable: /usr/bin/perl
    version: 5.10
    version_flag: '-v'
    environment:
<% end %>

Save this file and then create four new directories:

mkdir dev_setup/cookbooks/perl5/
mkdir dev_setup/cookbooks/perl5/attributes/
mkdir dev_setup/cookbooks/perl5/recipes/
mkdir staging/lib/vcap/staging/plugin/perl/

Create the following file:

vim dev_setup/cookbooks/perl5/README.rdoc

Set the contents of the file to the following 8 lines:

= DESCRIPTION:

= REQUIREMENTS:

= ATTRIBUTES:

= USAGE:

Save this file and then create another new file:

vim dev_setup/cookbooks/perl5/attributes/default.rb

Set the contents of the file to the following lines (backslash and new line added to ensure it all shows in the browser):

include_attribute "deployment"
default[:perl5][:version] = "5.10"
default[:perl5][:source] = \
 "http://www.cpan.org/src/5.0/perl-#{perl5[:version]}.tar.gz"
default[:perl5][:path] = \
 File.join(node[:deployment][:home], "deploy", "perl5")

Save this file and then create another new file:

vim dev_setup/cookbooks/perl5/metadata.rb

Set the contents of the file to the following 6 lines:

maintainer       "VMware"
maintainer_email "support@vmware.com"
license          "Apache 2.0"
description      "Installs/Configures Perl 5"
long_description IO.read(File.join(File.dirname(__FILE__), 'README.rdoc'))
version          "0.0.1"

Save this file and then create another new file:

vim dev_setup/cookbooks/perl5/recipes/default.rb

Set the contents of the file to the following 8 lines:

#
# Cookbook Name:: perl
# Recipe:: default
#
# Copyright 2012, VMware
#
#
Chef::Log.info("Running fake perl install...nothing to do!")

Save this file and then edit the following file:

vim setup/vcap_setup

Add the following three lines to the DEA setup code starting at line 211:

    # Perl
    echo "=== TODO: Add real perl support to vcap_setup"

Save this file and then create another new file:

vim staging/lib/vcap/staging/plugin/manifests/perl.yml

Set the contents of the file to the following 12 lines:

---
name: "perl"
runtimes:
  - perl5:
      version: '5.10'
      description: 'Perl 5'
      executable: perl
      default: true
app_servers:
detection:
  - "main.pl": '.'
staged_services:

Save this file and then create another new file:

vim staging/lib/vcap/staging/plugin/perl/plugin.rb

Set the contents of the file to the following lines:

class PerlPlugin < StagingPlugin
  include GemfileSupport
  def framework
    'perl'
  end

  def stage_application
    Dir.chdir(destination_directory) do
      create_app_directories
      copy_source_files
      create_startup_script
      create_stop_script
    end
  end

  def start_command
    # perl_main = detect_main_file
    "#{local_runtime} ./main.pl $@"
  end

  private
  def startup_script
    vars = environment_hash
    # PWD here is after we change to the 'app' directory.
    generate_startup_script(vars) do
      # plugin_specific_startup
      "# Hello from perl plugin"
    end
  end

  def stop_script
    vars = environment_hash
    generate_stop_script(vars)
  end

  # Not used
  def plugin_specific_startup
    cmds = []
    cmds << "mkdir ruby"
    cmds << 'echo "\$stdout.sync = true" >> ./ruby/stdsync.rb'
    cmds.join("\n")
  end

  # TODO - I'm fairly sure this problem of 'no standard startup command' is
  # going to be limited to Perl and Node.js. If not, it probably deserves
  # a place in the perl.yml manifest.
  def detect_main_file
    file = app_files_matching_patterns.first
    # TODO - Currently staging exceptions are not handled well.
    # Convert to using exit status and return value on a case-by-case basis.
    raise "Unable to determine Perl startup command" unless file
    file
  end
end

Save this file and then create another new file:

vim staging/lib/vcap/staging/plugin/perl/stage

Set the contents of the file to the following lines:

#!/usr/bin/env ruby
require File.expand_path('../../common', __FILE__)
plugin_class = StagingPlugin.load_plugin_for('perl')
plugin_class.validate_arguments!
plugin_class.new(*ARGV).stage_application

At this point we can update the cloud:

vcap stop
dev_setup/bin/vcap_dev_setup -d ~/cloudfoundry/

The vcap_dev_setup script updates the cloud deployment found in ~/cloudfoundry/.deployments/devbox with some of the needed files. For reasons I don’t yet understand (watch this space for an update!), not all the needed files are deployed. To fix this, use the following script (note that pip_support.rb is simply a file known to be in the directory of interest):

fromDir=`find ~/cloudfoundry/vcap/staging/ | grep pip_support.rb`
fromDir=`dirname $fromDir`
toDir=`find ~/cloudfoundry/.deployments/devbox/ | grep pip_support.rb`
toDir=`dirname $toDir`
cp -r $fromDir/perl $toDir
cp $fromDir/manifests/perl.yml $toDir/manifests/perl.yml

At this point we can start the cloud:

vcap start

Now from the client, get a list of the runtimes and frameworks:

vmc runtimes
vmc frameworks

We should see perl in both lists. At this point I am able to push my application and see it running in a browser (as we have done before). Note that the Chef scripts are is a state of flux and the steps described above are almost certainly not ideal, but they worked for me today!

The next task will be to see if we can add Smalltalk as a runtime.