vm_provisioner

A Vagrantfile helper that does very basic virtual machine provisioning.

Overview

vm_provisioner is a small provisioning helper for use with Vagrant. It is a simplistic version of tools such as Puppet, Chef, Ansible, etc. It was created as an exercise to learn Vagrant and to aid the (re)creation of OSX-based development environments.

Here is a fragment of a Vagrantfile showing how vm_provisioner is typically used:

...

Vagrant.configure(API_VERSION) do |vagrant_config|

  vagrant_config.vm.box = BOX

  vagrant_config.vm.provider(PROVIDER_NAME) do |vb|
    vb.name = VM_NAME
    vb.gui = true
  end

  with vagrant_config do
    install_atom
    install_iterm2
    install_gpg
    install_git
    install_github_for_mac
    install_homebrew
    install_ruby '2.1.2'
    install_bundler
    git_clone PROJECT_GITHUB_URL, PROJECT_VM_DIR
    cd PROJECT_VM_DIR do
      bundle_install
    end
    reboot_vm
  end

end

...

Installation

vm_provisioner installs itself each time Vagrant runs a Vagrantifile that references it. However, these are the prerequisits for using it:

• Vagrant must be installed on the host machine.
• OS X must be the operating system running on the virtual machine.
• The OS X Command Line Tools should be installed on the virtual machine.
• The host machine must have access to github.com when Vagrant runs.

Quickstart

1. Create and cd into a directory for a new project:

   $ mkdir -p ~/work/demo
   $ cd ~/work/demo
   

2. Create a Vagrantfile containing the following:

   #
   # Vagrantfile to create demo development environment.
   #
   
   # TODO: annotate
   API_VERSION     = "2"
   BOX             = "OSX109"
   PROVIDER_NAME   = "vmware_fusion"
   PROJECT_NAME    = "demo"
   VM_NAME         = PROJECT_NAME
   PROJECT_VM_DIR  = "/Users/vagrant/Documents/#{PROJECT_NAME}"
   PROVISIONER_URL = "https://raw.githubusercontent.com/milewdev/vm-provisioner/v2/Provisioner.rb"
   
   
   Vagrant.configure(API_VERSION) do |vagrant_config|
   
     vagrant_config.vm.box = BOX
   
     vagrant_config.vm.provider(PROVIDER_NAME) do |vb|
       vb.name = VM_NAME
       vb.gui = true
     end
   
     with vagrant_config do
       install_atom
       install_iterm2
       install_git
       install_github_for_mac
       install_homebrew          # needed to install ruby
       install_ruby '2.1.2'
       install_bundler
       reboot_vm
     end
   
   end
   
   
   def with(vagrant_config, &block)
     require "open-uri"
     File.write "Provisioner.rb", open(PROVISIONER_URL).read
     require_relative "Provisioner"
     Provisioner.provision(vagrant_config, &block)
     File.delete "Provisioner.rb"
   end
   

3. Run vagrant up:

   $ vagrant up --provider=vmware_fusion

4. See the Reference for a list of currently supported installers. See Adding a Product for instructions on how to add an installer.

Reference

add_to_path path

Prepend path to the PATH environment variable by doing:

echo 'export PATH=path:$PATH' >> ~/.bash_profile

For example, say you have installed nodejs, installed your project, and you have run npm install to install any dependencies. Any nodejs binaries, such as grunt, are now located in node_modules/.bin. So, you can add node_modules/.bin to your path:

PROJECT_GITHUB_URL = 'https://https://github.com/...'
PROJECT_VM_DIR = '/Users/vagrant/Documents/...'
...
with vagrant_config do
  install_git
  install_node
  git_clone PROJECT_GITHUB_URL, PROJECT_VM_DIR
  cd PROJECT_VM_DIR do
    npm_install
  end
  add_to_path "#{PROJECT_VM_DIR}/node_modules/.bin"
end

Now, rather than having to do:

$ node_modules/.bin/grunt ...

you can just do:

$ grunt ...

bundle_install

Run bundle install. bundler must be installed beforehand.

You would typically install the project sources and then run bundle install to install any project dependencies. Note that bundle install is run in the current directory so you will likely want to change to the project directory directory first:

PROJECT_GITHUB_URL = 'https://https://github.com/...'
PROJECT_VM_DIR = '/Users/vagrant/Documents/...'
...
with vagrant_config do
  install_git
  install_homebrew
  install_ruby '2.1.2'
  install_bundler
  git_clone PROJECT_GITHUB_URL, PROJECT_VM_DIR
  cd PROJECT_VM_DIR do
    bundle_install
  end
end

cd path do ... end

Temporarily change to directory path. For example, you might do this before running bundle_install or npm_install:

PROJECT_GITHUB_URL = 'https://https://github.com/...'
PROJECT_VM_DIR = '/Users/vagrant/Documents/...'
...
with vagrant_config do
  install_git
  install_homebrew
  install_ruby '2.1.2'
  install_bundler
  install_node
  git_clone PROJECT_GITHUB_URL, PROJECT_VM_DIR
  cd PROJECT_VM_DIR do
    bundle_install
    npm_install
  end
end

Warning: cd cannot be nested:

with vagrant_config do
  cd 'somewhere' do
    cd 'somewhere/else' do
      ...
    end
    # Whoops! back at ~, not 'somewhere'
  end
end

git_clone git_repository, path

Clone git_repository to path. Use this command to install project source code from a git repository onto the virtual machine:

PROJECT_GITHUB_URL = 'https://https://github.com/...'
PROJECT_VM_DIR = '/Users/vagrant/Documents/...'
...
with vagrant_config do
  install_git
  git_clone PROJECT_GITHUB_URL, PROJECT_VM_DIR
end

install_atom

Install the atom editor.

with vagrant_config do
  install_atom
end

install_bundler

Install the bundler Ruby gem package manager. Ruby must be installed beforehand.

with vagrant_config do
  install_homebrew
  install_ruby '2.1.2'
  install_bundler
end

install_chrome

Install the Google Chrome web browser.

with vagrant_config do
  install_chrome
end

install_firefox

Install the Mozilla Firefox web browser.

with vagrant_config do
  install_firefox
end

install_git [version]

Install git. Also copy the file ~/.gitconfig on the host machine to /Users/vagrant/.gitconfig on the virtual machine. If the optional version parameter is not provided, version '2.0.1' is installed.

with vagrant_config do
  install_git '2.2.0'
end

install_github_for_mac

Install the GitHub for Mac git client.

with vagrant_config do
  install_github_for_mac
end

install_gpg

Install the gpg encryption tools. Also copy the following files from ~/.gnupg on the host machine to /Users/vagrant/.gnupg on the virtual machine: pubring.gpg, secring.gpg, trustdb.gpg, pubring.gpg~, and random_seed.

with vagrant_config do
  install_gpg
end

install_heroku_toolbelt

Install heroku toolbelt, the Heroku command line utility.

with vagrant_config do
  install_heroku_toolbelt
end

homebrew

Install the Homebrew OS X package manager. The OS X command line tools must be installed beforehand on the base Vagrant box. During installation, homebrew/versions and homebrew/dupes are tapped.

Homebrew is a prerequisit for other vm_provisioner tasks, such as install_ruby.

with vagrant_config do
  install_homebrew
end

install_iterm2

Install the iterm2 OS X terminal replacement.

with vagrant_config do
  install_iterm2
end

install_node [version]

Install Node.js. If the optional version parameter is not provided, version 'v0.10.32' is installed.

with vagrant_config do
  install_node 'v0.10.33'
end

install_phantomjs

Install the PhantomJS headless browser. Homebrew must be installed beforehand.

with vagrant_config do
  install_homebrew
  install_phantomjs
end

install_postgresql [version]

Install the PostgreSQL database management system. If the optional version parameter is not supplied, the latest version is installed. Homebrew must be installed beforehand. During installation initdb is called, and the database is set up to start when user vagrant logs in.

with vagrant_config do
  install_homebrew
  install_postgresql '92'
end

install_python3

Install python (v3.4.1).

with vagrant_config do
  install_python3
end

install_qt

Install qt. Homebrew must be installed beforehand.

with vagrant_config do
  install_homebrew
  install_qt
end

install_ruby [version]

Install Ruby. If the optional version parameter is not supplied, version '2.1.2' is installed. Homebrew must be installed beforehand. During installation, GCC 4.2, rbenv, and ruby-build are also installed.

with vagrant_config do
  install_homebrew
  install_ruby '2.1.5'
end

install_textmate

Install the TextMate editor.

with vagrant_config do
  install_textmate
end

install_virtualenv

Install virtualenv (v1.11.6), the Python environment manager.

PROJECT_VM_DIR = '/Users/vagrant/Documents/...'
PROJECT_GITHUB_URL = 'https://https://github.com/...'
...
with vagrant_config do
  install_git
  install_python3
  install_virtualenv
  git_clone PROJECT_GITHUB_URL, PROJECT_VM_DIR
  cd PROJECT_VM_DIR do
    virtualenv_create
    pip_install
  end
end

npm_install

Run npm install. Node.js must be installed beforehand.

You would typically install the project sources and then run npm install to install any project dependencies. Note that npm install is run in the current directory so you will likely want to change to the project directory directory first:

PROJECT_GITHUB_URL = 'https://https://github.com/...'
PROJECT_VM_DIR = '/Users/vagrant/Documents/...'
...
with vagrant_config do
  install_git
  install_node
  git_clone PROJECT_GITHUB_URL, PROJECT_VM_DIR
  cd PROJECT_VM_DIR do
    npm_install
  end
end

pip_install

Runs pip install --requirements.txt. Python must be installed beforehand.

You would typically install the project sources and then run pip install to install any project dependencies. Note that pip install is run in the current directory so you will likely want to change to the project directory directory first:

PROJECT_VM_DIR = '/Users/vagrant/Documents/...'
PROJECT_GITHUB_URL = 'https://https://github.com/...'
...
with vagrant_config do
  install_git
  install_python3
  install_virtualenv
  git_clone PROJECT_GITHUB_URL, PROJECT_VM_DIR
  cd PROJECT_VM_DIR do
    virtualenv_create
    pip_install
  end
end

reboot_vm

Reboots the virtual machine. This may be useful on some operating systems after installing software.

with vagrant_config do
  install_git
  install_homebrew
  install_postgresql
  # ...
  reboot_vm
end

virtualenv_create

Runs virtualenv --no-site-packages --python=which python3 env. virtualenv must be installed beforehand.

You would typically install the project sources and then run virtualenv to create a new python environment in the project directory.

PROJECT_VM_DIR = '/Users/vagrant/Documents/...'
PROJECT_GITHUB_URL = 'https://https://github.com/...'
...
with vagrant_config do
  install_git
  install_python3
  install_virtualenv
  git_clone PROJECT_GITHUB_URL, PROJECT_VM_DIR
  cd PROJECT_VM_DIR do
    virtualenv_create
    pip_install
  end
end

Source Installation

$ git clone https://github.com/milewdev/vm-provisioner.git ~/work/vm-provisioner

Source Overview

The source consists of one class, Provisioner, defined in the file Provisioner.rb. There are a large number of methods but they are very repetitive and there is not yet the incentive to break them out into smaller classes.

A Vagrantfile bootstraps vm-provisioner by downloading Provisioner.rb from github and then invoking the Provisioner.provision class method, i.e. the code entry point:

PROVISIONER_URL = "https://raw.githubusercontent.com/milewdev/vm-provisioner/v2/Provisioner.rb"
...
Vagrant.configure(API_VERSION) do |vagrant_config|
  ...
  with vagrant_config do
    ...
  end
end


def with(vagrant_config, &block)
  require "open-uri"
  File.write "Provisioner.rb", open(PROVISIONER_URL).read
  require_relative "Provisioner"
  Provisioner.provision(vagrant_config, &block)
  File.delete "Provisioner.rb"
end

Note that the URL references branch v2 rather than master:

PROVISIONER_URL = "https:// ... /vm-provisioner/v2/Provisioner.rb"

Interface-breaking changes to vm-provisioner should be stored in github in their own branch so as not to break existing Vagrantfiles.

Although there is only the one Provisioner class, it has been divided into three parts, the first for initialization and launch (where Provisioner.provision is defined), the second containing the provisioning utilities called from a Vagrantfile, for example install_git and reboot_vm, and the last containing API methods used in the implementations of the utilities, for instance dmg_install and copy_host_file_to_vm.

Adding a Product

Adding a product will usually involve adding an install_something method to the second part of the Provisioner class:

class Provisioner
  ...
  def install_atom
    say "Installing atom editor"
    zip_install 'https://atom.io/download/mac'
  end
  ...
end

Sometimes a little more work is required:

class Provisioner
  ...
  def install_postgresql(version = nil)
    version ||= ''                        # blank version defaults to latest version
    say "Installing PostgreSQL"
    run_script <<-"EOF"
       brew install postgresql#{version}
       rm -rf /usr/local/var/postgres/
       initdb /usr/local/var/postgres -E UTF8
       mkdir -p ~/Library/LaunchAgents
       ln -sfv /usr/local/opt/postgresql92/*.plist ~/Library/LaunchAgents
    EOF
  end
  ...
end

The say method prints a header line in the terminal window where Vagrant is run from. This simply helps to delimit one provisioning step from another which can be helpful when debugging errors, slow downloads, etc.

...
--------------- Installing atom editor ---------------
[default] Running provisioner: shell...
[default] Running: inline script
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   389  100   389    0     0    396      0 --:--:-- --:--:-- --:--:--   396
100 65.7M  100 65.7M    0     0  1122k      0  0:00:59  0:00:59 --:--:-- 1058k
[default] Running provisioner: shell...
[default] Running: inline script
--------------- Installing iTerm2 ---------------
[default] Running provisioner: shell...
[default] Running: inline script
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 3111k  100 3111k    0     0   880k      0  0:00:03  0:00:03 --:--:--  880k
[default] Running provisioner: shell...
[default] Running: inline script
...

The API Reference contains a list of methods that can help to write an installer, such as zip_install, run_script, etc. Add additional helper methods, if required, or enhance the existing ones. Note: if you introduce a change that would break existing Vagrantfiles then create a new branch in github, e.g. v3, to contain the enhancements.

API Reference

copy_host_file_to_vm host_path, vm_path

Copy the file host_path from the host machine to vm_path on the virtual machine.

def install_git(version = '2.0.1')
  say "Installing git and copying .gitconfig from vm host"
  dmg_install "http://sourceforge.net/projects/git-osx-installer/files/git-#{version}-intel-universal-snow-leopard.dmg/download?use_mirror=autoselect"
  copy_host_file_to_vm "~/.gitconfig", ".gitconfig"
end

copy_host_file_to_vm is shorthand for Vagrant's config.vm.provision 'file' command. The example above could be written as:

def install_git(version = '2.0.1')
  say "Installing git and copying .gitconfig from vm host"
  dmg_install "http://sourceforge.net/projects/git-osx-installer/files/git-#{version}-intel-universal-snow-leopard.dmg/download?use_mirror=autoselect"
  @vagrant_config.vm.provision 'file', source: '~/.gitconfig', destination: '.gitconfig'
end

dmg_install url_of_dmg_file

Install a dmg file located at url_of_dmg_file.

def install_chrome
  say 'Installing google chrome browser'
  dmg_install 'https://dl.google.com/chrome/mac/stable/GGRO/googlechrome.dmg'
end

If the dmg file contains one file with the extension .pkg or .mpkg then the file is installed using installer. If the dmg file contains one directory with the extension .app then the directory is copied to /Applications.

pkg_install url_of_pkg_file

Install a pkg file located at url_of_pkg_file.

def install_heroku_toolbelt
  say 'Installing Heroku Toolbelt'
  pkg_install 'https://toolbelt.heroku.com/download/osx'
end

The pkg file is installed using installer.

@run_in_directory

Contains the name of the directory where run_script is run. This is the value of 'path' passed to cd when inside of the cd block, or /Users/vagrant otherwise.

with vagrant_config do
  # @run_in_directory = '/Users/vagrant'
  cd '/tmp' do
    # @run_in_directory = '/tmp'
  end
  # @run_in_directory = '/Users/vagrant'
end

run_script script_code

Run script_code on the virtual machine in the current directory.

def bundle_install
  say "Running 'bundle install'"
  run_script "bundle install"
end

run_script is shorthand for Vagrant's config.vm.provision 'shell' command. The example above could be written as:

def bundle_install
  say "Running 'bundle install'"
  @vagrant_config.vm.provision :shell, privileged: false, inline: <<-"EOF"
    pushd #{@run_in_directory} > /dev/null
    #{script_code}
    popd > /dev/null
  EOF
end

Note that the current directory (@run_in_directory) is normally /User/vagrant, but it can be changed temporarily using the cd command. For example, suppose we have the following provisioning method:

def pwd
  puts @run_in_directory
end

This Vagrantfile snippit shows the effect of the cd command on @run_in_directory:

with vagrant_config do
  pwd                   # /Users/vagrant
  cd '/tmp' do
    pwd                 # /tmp
  end
  pwd                   # /Users/vagrant
end

say

Prints a header line in the terminal window where Vagrant is run from. This simply helps to delimit one provisioning step from another and is intended as an aid for debugging errors, identifying slow downloads, etc.

For example, suppose we have defined the following utility methods:

def got_here
  say 'got here'
  puts 'got here too'
end

def got_there
  say 'got there'
  puts 'got there too'
end

And we include the following in our Vagrantfile:

with vagrant_config do
  got_here
  got_there
end

Then running vagrant up results in:

$ vagrant up --provider=vmware_fusion
...
--------------- got here ---------------
got here too
--------------- got there ---------------
got there too
...
$

tar_install url_of_tar_file

Install a tar file located at url_of_tar_file.

def install_textmate
  say "Installing TextMate"
  tar_install 'https://api.textmate.org/downloads/release'
end

The contents of the tar file are simply extracted to /Applications.

@vagrant_config

The vagrant config instance that was passed to Provisioner.provision. Here is the entire trail of the vagrant configuration, starting with the Vagrantfile:

Vagrant.configure(API_VERSION) do |vagrant_config|
  with vagrant_config do
    ls
  end
end

def with(vagrant_config, &block)
  require "open-uri"
  File.write "Provisioner.rb", open(PROVISIONER_URL).read
  require_relative "Provisioner"
  Provisioner.provision(vagrant_config, &block)
  File.delete "Provisioner.rb"
end

And inside Provisioner.rb we may have something like:

class Provisioner

    def self.provision(vagrant_config, &block)
      Provisioner.new(vagrant_config).send(:run, &block)
    end

  private

    def initialize(vagrant_config)
      @vagrant_config = vagrant_config
      @run_in_directory = @run_in_directory_default = '.'
    end

    ...
end

...

class Provisioner
  def ls
    run_script 'ls'
  end
end

...

class Provisioner
  def run_script(script_code)
    @vagrant_config.vm.provision :shell, privileged: false, inline: <<-"EOF"
      pushd #{@run_in_directory} > /dev/null
      #{script_code}
      popd > /dev/null
    EOF
  end
end

zip_install url_of_zip_file

Install a zip file located at url_of_zip_file.

def install_atom
  say "Installing atom editor"
  zip_install 'https://atom.io/download/mac'
end

The contents of the zip file are simply extracted to /Applications.