In our previous post we used Chef to install GemStone/S 64 Bit into Cloud Foundry. Now we add code so that GemStone is recognized as a service offered by Cloud Foundry. On the server, we start by adding “gemstone” to the cloud_controller by modifying
~/cloudfoundry/vcap/dev_setup/cookbooks/cloud_controller/attributes/default.rb line 22:
default[:cloud_controller][:builtin_services] = ["gemstone", "redis", "mongodb", "mysql", "neo4j"]
Create ~/cloudfoundry/vcap/bin/services/gemstone_gateway with:
#!/usr/bin/env ruby
# Copyright (c) 2012 VMware, Inc.
#
exec(File.expand_path("../../../services/gemstone/bin/gemstone_gateway",
__FILE__), *ARGV)
Create ~/cloudfoundry/vcap/bin/services/gemstone_node with:
#!/usr/bin/env ruby
# Copyright (c) 2012 VMware, Inc.
#
exec(File.expand_path("../../../services/gemstone/bin/gemstone_node",
__FILE__), *ARGV)
Set these two files to be executable:
chmod 0755 ~/cloudfoundry/vcap/bin/services/gemstone_*
The rest of the work is creating following directories and files to manage the service:
- ~/cloudfoundry/vcap/services/gemstone/
- bin/
- gemstone_gateway
- gemstone_node
- config/
- gemstone_gateway.yml
- gemstone_node.yml
- lib/
- gemstone_service/
- common.rb
- error.rb
- node.rb
- provisioner.rb
- resources/
- provision.tpz.erb
- unprovision.tpz.erb
- Gemfile
- Gemfile.lock
- Rakefile
- README
- .gitignore
We create the directory structure:
mkdir ~/cloudfoundry/vcap/services/gemstone
cd ~/cloudfoundry/vcap/services/gemstone
mkdir bin config lib lib/gemstone_service resources
Create Gemfile with:
source "http://rubygems.org"
#
gem "nats"
gem "datamapper", ">= 0.10.2"
gem "do_sqlite3", :require => nil
gem "dm-sqlite-adapter"
gem "eventmachine"
gem "em-http-request"
gem "json"
gem "mysql"
gem "uuidtools"
gem "ruby-hmac", :require => "hmac-sha1"
gem "thin"
gem "sinatra"
#
gem 'vcap_common', '~> 1.0.3', :require => ['vcap/common', 'vcap/component']
gem 'vcap_logging', '>=0.1.3', :require => ['vcap/logging']
Create Gemfile.lock with:
GEM
remote: http://rubygems.org/
specs:
addressable (2.2.4)
bcrypt-ruby (2.1.4)
daemons (1.1.5)
data_objects (0.10.3)
addressable (~> 2.1)
datamapper (1.1.0)
dm-aggregates (= 1.1.0)
dm-constraints (= 1.1.0)
dm-core (= 1.1.0)
dm-migrations (= 1.1.0)
dm-serializer (= 1.1.0)
dm-timestamps (= 1.1.0)
dm-transactions (= 1.1.0)
dm-types (= 1.1.0)
dm-validations (= 1.1.0)
dm-aggregates (1.1.0)
dm-core (~> 1.1.0)
dm-constraints (1.1.0)
dm-core (~> 1.1.0)
dm-core (1.1.0)
addressable (~> 2.2.4)
dm-do-adapter (1.1.0)
data_objects (~> 0.10.2)
dm-core (~> 1.1.0)
dm-migrations (1.1.0)
dm-core (~> 1.1.0)
dm-serializer (1.1.0)
dm-core (~> 1.1.0)
fastercsv (~> 1.5.4)
json (~> 1.4.6)
dm-sqlite-adapter (1.1.0)
dm-do-adapter (~> 1.1.0)
do_sqlite3 (~> 0.10.2)
dm-timestamps (1.1.0)
dm-core (~> 1.1.0)
dm-transactions (1.1.0)
dm-core (~> 1.1.0)
dm-types (1.1.0)
bcrypt-ruby (~> 2.1.4)
dm-core (~> 1.1.0)
fastercsv (~> 1.5.4)
json (~> 1.4.6)
stringex (~> 1.2.0)
uuidtools (~> 2.1.2)
dm-validations (1.1.0)
dm-core (~> 1.1.0)
do_sqlite3 (0.10.3)
data_objects (= 0.10.3)
em-http-request (0.3.0)
addressable (>= 2.0.0)
escape_utils
eventmachine (>= 0.12.9)
escape_utils (0.2.3)
eventmachine (0.12.11.cloudfoundry.3)
fastercsv (1.5.4)
json (1.4.6)
json_pure (1.6.4)
little-plugger (1.1.3)
logging (1.6.1)
little-plugger (>= 1.1.2)
mysql (2.8.1)
nats (0.4.22.beta.4)
daemons (>= 1.1.4)
eventmachine (>= 0.12.10)
json_pure (>= 1.6.1)
thin (>= 1.3.1)
posix-spawn (0.3.6)
rack (1.4.0)
ruby-hmac (0.4.0)
sinatra (1.2.1)
rack (~> 1.1)
tilt (>= 1.2.2, < 2.0)
stringex (1.2.1)
thin (1.3.1)
daemons (>= 1.0.9)
eventmachine (>= 0.12.6)
rack (>= 1.0.0)
tilt (1.2.2)
uuidtools (2.1.2)
vcap_common (1.0.3)
eventmachine (~> 0.12.11.cloudfoundry.3)
logging (>= 1.5.0)
nats (~> 0.4.22.beta.4)
posix-spawn (~> 0.3.6)
thin (~> 1.3.1)
yajl-ruby (~> 0.8.3)
vcap_logging (0.1.3)
yajl-ruby (0.8.3)
#
PLATFORMS
ruby
#
DEPENDENCIES
datamapper (>= 0.10.2)
dm-sqlite-adapter
do_sqlite3
em-http-request
eventmachine
json
mysql
nats
ruby-hmac
sinatra
thin
uuidtools
vcap_common (~> 1.0.3)
vcap_logging (>= 0.1.3)
Create .gitignore with:
*.db
Create Rakefile with:
require 'rake'
#
desc "Run specs"
task "spec" => ["bundler:install:test", "test:spec"]
#
desc "Run specs using RCov"
task "spec:rcov" => ["bundler:install:test", "test:spec:rcov"]
#
namespace "bundler" do
desc "Install gems"
task "install" do
sh("bundle install")
end
#
desc "Install gems for test"
task "install:test" do
sh("bundle install --without development production")
end
#
desc "Install gems for production"
task "install:production" do
sh("bundle install --without development test")
end
#
desc "Install gems for development"
task "install:development" do
sh("bundle install --without test production")
end
end
#
namespace "test" do
task "spec" do |t|
sh("cd spec && ../../base/bin/nats-util start &&"
" rake spec && ../../base/bin/nats-util stop")
end
#
task "spec:rcov" do |t|
sh("cd spec && rake spec:rcov")
end
end
Create README with:
This code provides GemStone/S 64 Bit as a Cloud Foundry service.
Create bin/gemstone_gateway with:
#!/usr/bin/env ruby
# -*- mode: ruby -*-
#
# Copyright (c) 2009-2011 VMware, Inc.
#
ENV["BUNDLE_GEMFILE"] ||= File.expand_path('../../Gemfile', __FILE__)
#
$LOAD_PATH.unshift File.join(File.dirname(__FILE__), '..', '..',
'base', 'lib')
require 'base/gateway'
#
$LOAD_PATH.unshift File.join(File.dirname(__FILE__), '..', 'lib')
require 'gemstone_service/provisioner'
#
class VCAP::Services::Gemstone::Gateway < VCAP::Services::Base::Gateway
#
def provisioner_class
VCAP::Services::Gemstone::Provisioner
end
#
def default_config_file
config_base_dir = ENV["CLOUD_FOUNDRY_CONFIG_PATH"] ||
File.join(File.dirname(__FILE__), '..', 'config')
File.join(config_base_dir, 'gemstone_gateway.yml')
end
#
end
#
VCAP::Services::Gemstone::Gateway.new.start
Create bin/gemstone_node with:
#!/usr/bin/env ruby
# -*- mode: ruby -*-
# Copyright (c) 2009-2011 VMware, Inc.
#
ENV["BUNDLE_GEMFILE"] ||= File.expand_path("../../Gemfile", __FILE__)
#
$LOAD_PATH.unshift File.join(File.dirname(__FILE__), '..', '..',
'base', 'lib')
require 'base/node_bin'
#
$LOAD_PATH.unshift(File.expand_path("../../lib", __FILE__))
require "gemstone_service/node"
#
class VCAP::Services::Gemstone::NodeBin < VCAP::Services::Base::NodeBin
#
def node_class
VCAP::Services::Gemstone::Node
end
#
def default_config_file
config_base_dir = ENV["CLOUD_FOUNDRY_CONFIG_PATH"] ||
File.join(File.dirname(__FILE__), '..', 'config')
File.join(config_base_dir, 'gemstone_node.yml')
end
#
def additional_config(options, config)
options
end
#
end
#
VCAP::Services::Gemstone::NodeBin.new.start
Change these two files to be executable:
chmod 0755 bin/gemstone_*
Create config/gemstone_gateway.yml with:
---
#cloud_controller_uri: api.vcap.me
service:
name: gemstone
version: "3.0"
description: 'GemStone/S 64 Bit database service'
plans: ['free']
tags: ['gemstone', 'gemstone-3.0', 'object']
ip_route: localhost
index: 0
token: "0xdeadbeef"
logging:
level: debug
mbus: nats://localhost:4222
pid: /var/vcap/sys/run/gemstone_service.pid
node_timeout: 2
Create config/gemstone_node.yml with:
---
local_db: sqlite3:/var/vcap/services/gemstone/gemstone_node.db
base_dir: /var/vcap/services/gemstone/
ip_route: 127.0.0.1
mbus: nats://localhost:4222
index: 0
logging:
level: debug
pid: /var/vcap/sys/run/gemstone_node.pid
available_storage: 1024
node_id: gemstone_node_1
migration_nfs: /mnt/migration
gemstone:
host: localhost
port: 50377
user: gemroot
pass: gemroot
Create lib/gemstone_service/common.rb with:
# Copyright (c) 2012 VMware, Inc.
module VCAP
module Services
module Gemstone
module Common
def service_name
"GaaS"
end
end
end
end
end
Create lib/gemstone_service/error.rb with:
# Copyright (c) 2012 VMware, Inc.
$LOAD_PATH.unshift File.join(File.dirname(__FILE__), '..', '..', '..',
'base', 'lib')
#
require "base/service_error"
#
class VCAP::Services::Gemstone::GemstoneError<
VCAP::Services::Base::Error::ServiceError
GSS_LOCAL_DB_ERROR = [1, HTTP_INTERNAL, 'Problem with CF database']
end
Create lib/gemstone_service/provisioner.rb with:
# Copyright (c) 2012 VMware, Inc.
$LOAD_PATH.unshift File.join(File.dirname(__FILE__), '..', '..', '..',
'base', 'lib')
#
require 'base/provisioner'
require 'gemstone_service/common'
#
class VCAP::Services::Gemstone::Provisioner <
VCAP::Services::Base::Provisioner
#
include VCAP::Services::Gemstone::Common
#
def node_score(node)
10 # > 0 for ~/cloudfoundry/vcap/services/base/lib/base/provisioner.rb
end
#
end
Create lib/gemstone_service/node.rb with:
# Copyright (c) 2012 VMware, Inc.
require "erb"
require "fileutils"
require "logger"
require "pp"
#
require "uuidtools"
require "open3"
require "thread"
#
$LOAD_PATH.unshift File.join(File.dirname(__FILE__), '..', '..', '..',
'base', 'lib')
require 'base/node'
require 'base/service_error'
require "datamapper_l"
#
module VCAP
module Services
module Gemstone
class Node < VCAP::Services::Base::Node
end
end
end
end
#
require "gemstone_service/common"
require "gemstone_service/error"
#
class VCAP::Services::Gemstone::Node
#
include VCAP::Services::Gemstone::Common
include VCAP::Services::Gemstone
#
# ProvisionedService is the data stored by CloudFoundry for a provisioned
# instance of the GemstoneService. This info is made available throughout
# the system, e.g., at the time an app is staged / run.
class ProvisionedService
include DataMapper::Resource
#
property :name, String, :key => true
property :plan, Enum[:free], :required => true
property :user, String, :required => true
property :pass, String, :required => true
end
#
# +options+ includes the info in ../../config/gemstone_node.yml
def initialize(options)
super(options) # handles @node_id, @logger, @local_ip, @node_nats
# @logger.debug("initialize( #{options.keys.inspect}")
template_path = File.expand_path('../../resources/provision.tpz.erb',
File.dirname(__FILE__))
@provision_template = ERB.new(File.read(template_path))
template_path = File.expand_path('../../resources/unprovision.tpz.erb',
File.dirname(__FILE__))
@unprovision_template = ERB.new(File.read(template_path))
start_local_db(options[:local_db])
@GEMSTONE = options[:base_dir]
start_gemstone
end
#
def start_local_db(local_db)
DataMapper.setup(:default, local_db)
DataMapper::auto_upgrade!
end
#
def start_gemstone
`#{@GEMSTONE}/bin/startstone`
end
#
def shutdown
@logger.debug("shutdown")
`#{@GEMSTONE}/bin/stopstone`
super
end
def pre_send_announcement
# @logger.debug("pre_send_announcement")
super
end
def announcement
# @logger.debug("announcement")
a = {
:some_random_data => 42,
}
end
#
def provision(plan, credential=nil)
@logger.info("provision: plan #{plan} credential: #{credential.inspect}")
provisioned_service = ProvisionedService.new
provisioned_service.plan = plan
if credential
provisioned_service.name = credential[:name]
provisioned_service.user = credential[:user]
provisioned_service.pass = credential[:password]
else
provisioned_service.name = 'GS-' +
UUIDTools::UUID.random_create.to_s.gsub(/-/, '')
provisioned_service.user = 'CF-' + generate_credential
provisioned_service.pass = generate_credential
end
#
`#{@GEMSTONE}/bin/topaz_dc <<EOF
#{@provision_template.result(binding)}
EOF`
#
if not provisioned_service.save
@logger.error("Could not save entry: " \
"#{provisoned_service.errors.inspect}")
raise GemstoneError.new(GemstoneError::GSS_LOCAL_DB_ERROR)
end
#
response = {
"name" => provisioned_service.name,
"user" => provisioned_service.user,
"pass" => provisioned_service.pass,
"host" => @local_ip,
"port" => 0,
}
end
#
def unprovision(name, credentials)
@logger.info("unprovision: name #{name} " \
"credentials: #{credentials.inspect}")
return if name.nil?
provisioned_service = ProvisionedService.get(name)
raise GemstoneError.new(GemstoneError::GSS_LOCAL_DB_ERROR) \
if provisioned_service.nil?
#
`#{@GEMSTONE}/bin/topaz_dc <<EOF
#{@unprovision_template.result(binding)}
EOF`
#
if not provisioned_service.destroy
@logger.error("Could not delete service: " \
"#{provisioned_service.errors.inspect}")
raise GemstoneError.new(GemstoneError::GSS_LOCAL_DB_ERROR)
end
end
#
def bind(name, bind_opts, credential=nil)
@logger.debug("Bind request: name=#{name}, \
bind_opts=#{bind_opts}, credential=#{credential.inspect}")
provisioned_service = ProvisionedService.get(name)
raise GemstoneError.new(GemstoneError::GSS_LOCAL_DB_ERROR) \
if provisioned_service.nil?
response = {
"user" => provisioned_service.user,
"pass" => provisioned_service.pass,
"host" => @local_ip,
"port" => 0,
}
end
#
def unbind(credentials)
@logger.debug("Unbind request: credentials=#{credentials}")
end
#
CREDENTIAL_CHARACTERS =
("A".."Z").to_a + ("a".."z").to_a + ("0".."9").to_a
def generate_credential(length=12)
Array.new(length) {
CREDENTIAL_CHARACTERS[rand(CREDENTIAL_CHARACTERS.length)] }.join
end
#
end
Create resources/provision.tpz.erb with:
! In GemStone/S 64 Bit we respond to a provisioning request by adding a user
! output pushnew provision.out
!
run
| securityPolicy newUser |
securityPolicy := (GsObjectSecurityPolicy newInRepository: SystemRepository)
ownerAuthorization: #'write';
worldAuthorization: #'none';
yourself.
System commitTransaction.
newUser := AllUsers
userWithId: '<%= provisioned_service.user %>'
ifAbsent: [ nil ].
newUser ~~ nil ifTrue: [
AllUsers removeAndCleanup: newUser.
System commitTransaction.
].
newUser := AllUsers
addNewUserWithId: '<%= provisioned_service.user %>'
password: '<%= provisioned_service.pass %>'
defaultObjectSecurityPolicy: securityPolicy
privileges: #(
#'CodeModification'
#'NoPerformOnServer'
#'NoUserAction'
#'NoGsFileOnServer'
#'NoGsFileOnClient')
inGroups: #().
securityPolicy owner: newUser.
System commitTransaction.
%
logout
exit
Create resources/unprovision.tpz.erb with:
! We respond to an uprovisioning request by removing a user
! output pushnew unprovision.out
!
run
| user |
user := AllUsers
userWithId: '<%= provisioned_service.user %>'
ifAbsent: [^true].
AllUsers removeAndCleanup: user.
System commitTransaction.
%
logout
exit
At this point we can repeat the deployment of VCAP:
~/cloudfoundry/vcap/dev_setup/bin/vcap_dev_setup -d ~/cloudfoundry/
~/cloudfoundry/vcap/dev_setup/bin/vcap_dev start
If the startup process reports that it cannot find the correct version of datamapper, then stop vcap, edit ~/cloudfoundry/vcap/services/gemstone/Gemfile.lock, change (1.2.0) to (1.1.0), and restart vcap.
Trying it out
From the client we can verify that the service exists:
vmc services
This shows GemStone/S as a service along with MongoDB, MySQL, Neo4j, and Redis. We can verify that Cloud Foundry handles the new service by pushing our original Ruby application (created here) and adding a gemstone service as part of the push process. Once the application is running, go to a browser and navigate to http://ruby-env.vcap.me/env. In the environment list we see something like the following:
VCAP_SERVICES: {
"gemstone-3.0":[{
"name":"gs-env",
"label":"gemstone-3.0",
"plan":"free",
"tags":["gemstone","object"],
"credentials":{
"user":"CF-1vztH87mLJhW",
"pass":"J47kFHmC2CvY",
"host":"192.168.9.155",
"port":0
}
}]
}
We are particularly interested in the user and password provided as part of the credentials. Returning to the server, we can verify that GemStone is running and the user/password provided is valid:
/var/vcap/services/gemstone/bin/topaz <<EOF
set user CF-1vztH87mLJhW pass J47kFHmC2CvY
login
run
100 factorial asString size
%
logout
exit
EOF
This demonstrates that we have provisioned a new service, bound it to the new application, and that an application could use the provided credentials to connect to GemStone/S and execute Smalltalk code.
