Skip to content
Kevin Menard edited this page May 19, 2014 · 38 revisions

You can follow these steps to create a demo app on a single instance

Create a simple rails project

rails new rubbertest -d postgresql
cd rubbertest
rails g scaffold post title:string body:text

Make sure the asset pipeline will compile by adding the following to your Gemfile

gem 'therubyracer', :group => :assets

Install rubber

First install the gem:

gem install rubber

Then vulcanize your project:

rubber vulcanize complete_passenger_postgresql

This should work for non-rails projects as well, though you’ll likely need to do some tweaking to recipes to get it to work.

The complete_passenger_postgresql generator gives you a deployment setup which includes passenger, apache, haproxy, postgresql (automatic master/slave replication if you create other instances in the postgresql_slave role) monit, collectd+graphite and graylog.

Note that if you don’t use a complete_* recipe, your app may not respond to http requests. (see this issue for more info)

Configure rubber

Edit config/rubber/rubber.yml. To do a simple demo, you need real values for these keys:

  • cloud_providers → aws → access_key
  • cloud_providers → aws → secret_access_key
  • cloud_providers → aws → account
  • cloud_providers → aws → key_name
  • cloud_providers → aws → key_file
  • web_tools_user
  • web_tools_password

See the FAQ if having connection issues.

For a real app, you should go through all the settings in rubber.yml as well as the settings in rubber-MODULE.yml which contains settings specific to each MODULE (apache, passenger, etc.)

For ease of initial setup, rubber deploys by zipping up the project tree and pushing it to your instance. If you configure capistrano to use the git scm provider, you probably do not want sensitive data in your rubber*.yml files and thus in git. Rubber handles this by allowing you to point it to a rubber-secret.yml file outside of your project, which will then get pushed to your instance during deploy. Edit the rubber_secret parameter in rubber.yml to set this up.

Create, bootstrap, then deploy to instance

Use the convenience task which creates a single instance with all needed roles. This is also very useful for environments other than production so that each developer can create their own complete staging instance to test on:

cap rubber:create_staging
# Hit enter at prompts to accept the defaults

Or do the entire process manually:

cap rubber:create
cap rubber:bootstrap
cap deploy

Then you should be able to browse to http://production.foo.com – rubber maps aliases to instances in the /etc/hosts file by default, so you don’t need your own domain for the demo, however with your own domain you can map aliases to IPs using a dns provider like nettica/zerigo/route53. Note that rubber:bootstrap should be idempotent, so feel free to run it multiple times in the event of a failure like a timeout getting gems from rubygems.

Note that cap rubber:create_staging will use production as the environment for your Rails app. If you want to use staging as the Rails environment you should have a config/environments/staging.rb file and then use:

cap RUBBER_ENV=staging rubber:create_staging

Create a separate staging environment

Want each developer to have their own custom staging instance on demand? Easy, just:

cp config/environments/production.rb config/environments/myenv.rb
RUBBER_ENV=myenv cap rubber:create_staging

If you accepted the defaults, you should now be able to browse to http://myenv.foo.com By default, these instances are isolated from each other with AWS security groups, so even if you have some hardcoded addresses in your production config, your production servers will be safe.

Create an admin server

By default, rubber:create_staging also sets up the web_tools role on your staging instance. This lets you check on server statistics (graphite), logs (graylog) or hit the monit/haproxy web admin UIs. Since these are sensitive tools, they are password protected with a random password – to make the password non-random, edit rubber(-secret).yml and set web_tools_user/password, then deploy. If you setup your instances without that role, you can dedicate another instance to be your admin server, e.g.:

ALIAS=tools ROLES=web_tools cap rubber:create
cap rubber:bootstrap
cap deploy

Now you can hit https://admin:sekret@tools.foo.com:8443/ to see stats on all the servers in your cluster. Note that clusters are isolated by RUBBER_ENV/RAILS_ENV, so don’t expect to see your staging server’s data in your production server’s web admin. If you only want a single production instance, you can add the web_tools role when creating that production instance. As an additional benefit, monit and munin are configured to automatically send email to admin_email (rubber.yml) whenever anything interesting happens (restart of services, too much machine load, not enough disk space, etc).

Note that the full stack with all admin tools is pretty memory intensive (I blame java :), you you may need a m1.medium to be comfortable. You can edit rubber.yml and set the staging_roles key to exclude the web_tools related roles if you don’t want that overhead. You could also edit the recipes to force the java daemons (graylog, elasticsearch, etc) to use less memory.

To add another app server to an existing deployment

The roles in the default templates can scale horizontally by default, so scaling up to handle increased app server load is as simple as:

ALIAS=app02 ROLES=app cap rubber:create
cap rubber:bootstrap
cap deploy

For a more complex production setup

Since the roles in the default templates are horizontally scalable by default, you can break those roles out onto whatever instances makes sense for your deployment, and scale them all independently:

ALIAS=web01 ROLES=web cap rubber:create # haproxy load balancer
ALIAS=app01 ROLES=app cap rubber:create # app server (passenger+apache)
ALIAS=app02 ROLES=app cap rubber:create # app server (passenger+apache)
ALIAS=db01 ROLES=postgresql_master cap rubber:create # this is the postgresql master
ALIAS=tools ROLES=web_tools cap rubber:create
cap rubber:bootstrap
cap deploy
# add a postgresql slave too
ALIAS=db02 ROLES=postgresql_slave cap rubber:create
cap rubber:bootstrap
cap deploy

Note the separate instances for postgresql masters and slaves – rubber automatically sets up replication for any new slave that gets created, though I leave it up to your application code to determine how to load balance.

Changing cloud configurations per host

You can specify specific configuration options within the hosts declaration if you need something other than the global configuration. For example, if the master AWS configuration specified image types of c1.medium and you wanted to launch the “stag-01” host as an m1.small you would do this inside rubber.yml:

hosts:
  stag-01:
    cloud_providers:
      aws:
        image_type: m1.small

Making things quicker

If you are building staging instances a lot, or even just building many instances of a similar type (app servers, resque workers), you can bundle an existing instance as a starting point for subsequent creates. Simply create your instance with the roles that you are happy with, do a rubber:bundle, record the AMI it reports at the end, and use that AMI for the image_id in rubber.yml

ALIAS=app01 ROLES=app cap rubber:create
FILTER=app01 cap rubber:bundle
# add AMI to rubber.yml: cloud_providers -> aws -> image_id
# If you want this to be specific to the app role, you can use a role configuration override
ALIAS=app02~app10 ROLES=app cap rubber:create # create 9 more app servers
FILTER=app02~app10 ROLES=app cap rubber:bootstrap # bootstrap the 9 new app servers, way faster!