J'aime pas les piles

My lost place

Creating Vagrant Base Boxes

| Comments

Create a VM inside VirtualBox

  • No usb
  • No audio
  • One dynamic disk (10GiB)
  • 512 MiB of RAM
  • One vcpu
  • One network card in NAT mode

Install the base system, as minimal as possible

Scientific Linux 5

  • Retrieve ISOs CD 1 and 2
1
wget http://ftp1.scientificlinux.org/linux/scientific/5x/iso/x86_64/cd/SL.510.110513.CD.x86_64.disc{1,2}.iso
  • Boot CD
  • Use basic video driver installation
  • Use default configuration
    • except:
      • DHCP for ipv6
      • Deselct every packages sets
  • Use vagrant as root password

Scientific Linux 6

  • Retrieve netinstall ISO CD
1
wget http://ftp.scientificlinux.org/linux/scientific/6x/x86_64/iso/SL-64-x86_64-2013-03-18-boot.iso
  • Boot CD
  • Use network install URL:
  • Use default configuration
  • Use vagrant as root password

Debian 7: retrieve netinstall ISO CD

1
wget http://cdimage.debian.org/debian-cd/7.3.0/amd64/iso-cd/debian-7.3.0-amd64-netinst.iso

System configuration

Scientific Linux 5/6

  • Update system and reboot if kernel was updated
1
2
yum clean all && yum update -y
reboot
  • Add a vagrant user with vagrant as password
1
2
adduser vagrant
passd vagrant
  • Configure password-less sudo for vagrant user
1
2
3
visudo
Default:vagrant !requiretty
vagrant ALL=(ALL) NOPASSWD: ALL
  • Configure ssh server
1
sed -i 's/^#UseDNS yes/UseDNS no/' /etc/ssh/sshd_config
  • Configure ssh authorized_keys for vagrant user
1
2
3
4
5
6
mkdir ~vagrant/.ssh
curl -o ~/vagrant/.ssh/authorized_keys \
  https://raw.github.com/mitchellh/vagrant/master/keys/vagrant.pub
chmod 0700 ~vagrant/.ssh
chmod 0600 ~vagrant/.ssh/authorized_keys
chown -R vagrant:vagrant ~vagrant/.ssh

Scientific Linux 6

Minimal SL6 install does not install acpid nor perl

1
2
yum install -y acpid perl
service acpid start

VirtualBox Additions installation

Scientific Linux 5/6

  • Insert Guest additions CD image using VirtualBox device menu
  • Install required software for build the VirtualBox additions
1
yum install -y gcc make kernel-devel
  • Build and install VirtualBox additions

Error about OpenGL or Window System drivers are “normal”.

1
2
3
mount /dev/cdrom /mnt
sh /mnt/VBoxLinuxAdditions.run
umount /mnt

Cleaning image

Scientific Linux 5/6

1
2
3
4
5
yum clean all
: > /var/log/messages
: > /var/log/secure
: > ~/.bash_history
kill -9 $$

ACPI shutdown VM using VirtualBox Machine menu.

Packing the boxes

Scientific Linux 5

1
2
vagrant package --output sl5-64-VB436-nocm.box --base scientificlinux5
vagrant package --output sl6-64-VB436-nocm.box --base scientificlinux6

Testing the boxes

1
2
vagrant box add sl5-64-nocm sl5-64-VB436-nocm.box
vagrant box add sl6-64-nocm sl6-64-VB436-nocm.box
1
2
3
4
5
6
7
8
9
10
11
mkdir ~/sl5-64-nocm-tests && cd $_
vagrant init sl5-64-nocm
vagrant up
vagrant ssh
ping -c 3 gnu.org
sudo -s
exit
exit
vagrant destroy -f
cd ..
rm -rf ~/sl5-64-nocm-tests
1
2
3
4
5
6
7
8
9
10
11
mkdir ~/sl6-64-nocm-tests && cd $_
vagrant init sl6-64-nocm
vagrant up
vagrant ssh
ping -c 3 gnu.org
sudo -s
exit
exit
vagrant destroy -f
cd ..
rm -rf ~/sl6-64-nocm-tests

Node-less Puppet Setup Using Hiera

| Comments

Why?

Following a big puppet 2.7 => 3.3 space jump (it took quite some times to test/setup/adapt everything) I am trying to get a cleaner/saner puppet usage to avoid errors, duplication (allowing to easily override some conf for a specific deployment site at some specific location) and to avoid slapping my lazyness with a truit.

So I crawled a bit the web, and read a lot of different posts/bugs/idas/rants, and did not find the golden-wonderfull-definitive set-up guide, so here are the things that are on the way: * Use hiera for storing the nodes configuration * Assign classes using hiera (node-less setup?) * Create roles and profiles modules to allow to encapsulate contents not configurable using hiera

How?

Hiera base setup

Nothing to fancy here as shown in the hiera.yaml file, just an environment-dependent datadir and a first draft of the hierarchy that will be used.

hiera.yaml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
---
:backends:
  - json
:json:
  :datadir: /etc/puppet/environments/%{::environment}/hieradata/
:hierarchy:
  - "%{::fqdn}"
  - "%{::company_role}"
  - "%{::company_location}"
  - "%{::virtual}"
  - "%{::operatingsystem}-${lsbdistrelease}"
  - "%{::operatingsystem}-${lsbmajdistrelease}"
  - "%{::operatingsystem}"
  - "%{::osfamily}"
  - common

# vim: set ft=yaml et smarttab sw=2 ts=2 sts=2:

Custom facts for hiera

Here two hiera data sources are meant to be able to easily configure a node according to its location or role. (location meaning more or less a more or less physical location with some specific network configuration or other specific rules/requirements)

In order to be able to assign the role and location, custom facts were added (company_role and company_location), based on the content of a file (/etc/company.conf) that have to be available on the server. (see XXX for more)

/etc/company.conf
1
2
role=puppet
location=ki
dist/site/lib/facter/gnbila-facts.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
require 'facter'

if File.exist?('/etc/company.conf')
  File.readlines('/etc/company.conf').each do |line|
    if line =~ /^(.+)=(.+)$/
      varname = "company" + $1.strip
      value = $2.strip

      Facter.add(varname) do
        setcode { value }
      end
    end
  end
end

# vim: set expandtab smarttab shiftwidth=2 tabstop=2 softtabstop=2 nocindent noautoindent:

Assigning class to nodes using hiera

hieradata/common.json
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
{
  "classes" : [
    "unix",
    "skel",
    "requiredsoftware",
    "pamldap",
    "git",
    "liquidprompt",
    "ruby",
    "ruby::dev",
    "postfix",
    "ntp",
    "sudo"
  ],

  "ntp::server" : [
      "0.fr.pool.ntp.org",
      "1.fr.pool.ntp.org",
      "2.fr.pool.ntp.org"
  ],
}
manifests/site.pp
1
2
# Load classes from hiera conf merging all classes for inclusion
hiera_include('classes')

Assigning defines to nodes using hiera

Defines parameters are stored in a hash, the key is the resource title and the value is a hash of define parameters.

hieradata/common.json
1
2
3
4
5
6
7
8
9
10
  "rsyslog_configs" : {
    "iptables.conf" : {
      "ensure" : "present",
      "source" : "puppet:///modules/site/rsyslog/rsyslog.d/iptables.conf"
    },
    "puppet-agent.conf" : {
      "ensure" : "present",
      "source" : "puppet:///modules/site/rsyslog/rsyslog.d/puppet-agent.conf"
      },
  },

Defines have to be instanciated calling create_resource with the retrieved define configuration hash.

manifests.y/site.pp
1
2
3
4
5
6
7
8
node default {
  # Load classes from hiera conf merging all classes for inclusion
  hiera_include('classes')

  # Retrieve rsyslog configurations if any
  $rsyslog_configs = hiera_hash('rsyslog_configs', {})
  create_resources('rsyslog::config', $rsyslog_configs)
}

Puppet Tests Using Vagrant

| Comments

XXX Work in progress.

Intro

Read first

Prerequisites

Yaourting VirtualBox, git and Vagrant on Archlinux
1
2
3
yaourt -S git
yaourt -S virtualbox virtualbox-guest-iso
yaourt -S vagrant

VMs list

  • One puppet master running Debian 7 64bits
  • One puppet client running Debian 7 64bits
  • One puppet client running Scientific Linux 5.x 64 bits
  • One puppet client running Scientific Linux 6.x 64 bits
  • One puppet client running CentOS 6.x 64 bits

Planned workflow

Boxes URLs

Boxes repositories

Boxes list

Box installation

Box installation will take some time as the boxes have to be downloaded locally.

Yaourting VirtualBox, git and Vagrant on Archlinux
1
2
3
vagrant box add debian7-64 http://puppet-vagrant-boxes.puppetlabs.com/debian-70rc1-x64-vbox4210-nocm.box
vagrant box add sl6-64 http://lyte.id.au/vagrant/sl6-64-lyte.box
vagrant box add centos6-64 http://puppet-vagrant-boxes.puppetlabs.com/centos-64-x64-vbox4210-nocm.box

Vagrant configuration

Puppet will be boostraped using a small shell script

shell/base.sh
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
#!/bin/sh

if [ $(id -u) -ne 0 ]; then
  echo 'This script must be run as root.' >&2
  exit 1
fi

if which puppet > /dev/null 2>&1; then
  echo 'Puppet is already installed'
  exit 0
fi

# Add puppetlabs repo definitions
echo 'deb http://apt.puppetlabs.com wheezy main' > /etc/apt/sources.list.d/puppetlabs.list
echo 'deb http://apt.puppetlabs.com wheezy dependencies' > /etc/apt/sources.list.d/puppetlabs-dependencies.list

# Add puppetlabs repo key
apt-key adv --keyserver keyserver.ubuntu.com --recv 4BD6EC30

# Update packages list
aptitude update

# Upgrade system
# Not working yet due to debconf wanting input
#aptitude -V -y upgrade
#aptitude -V -y dist-upgrade

# Install puppet
aptitude -y install puppet
echo 'Puppet successfully installed'

Then nitial role will be set using another shell script

shell/role.sh
1
2
3
4
5
6
7
8
#!/bin/sh

if [ $(hostname) = 'puppet' ]; then
  echo 'role=puppetmaster' >> /etc/company.conf
fi
if [ $(hostname) = 'client' ]; then
  echo 'role=puppet' >> /etc/company.conf
fi

And the puppetmaster is boostraped using the puppet apply provider. The client will get its configuration from the puppetmaster using the puppet agent provider.

Two directories from the host are made available to the guest, they contain the puppet modules that will be used for the puppetmaster bootstrap: * Local puppet modules are available in the relatvie ../../dist directory * A local copy of the remote puppet modules managed using the Puppetfile is made using r10k (using a symbolicaly linked Puppetfile)

1
2
gem install r10k
r10k -v INFO puppetfile install
Vagrantfile
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
# -*- mode: ruby -*-
# vi: set ft=ruby :

# Vagrantfile API/syntax version. Don't touch unless you know what you're doing!
VAGRANTFILE_API_VERSION = "2"

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|

  # Every Vagrant virtual environment requires a box to build off of.
  config.vm.box = "debian7-64-nocm"

  # The url from where the 'config.vm.box' box will be fetched if it
  # doesn't already exist on the user's system.
  config.vm.box_url = "http://puppet-vagrant-boxes.puppetlabs.com/debian-70rc1-x64-vbox4210-nocm.box"

  # Setup the Puppet master
  config.vm.define :master do |master|
    # Configure memory
    master.vm.provider :virtualbox do |vb|
      vb.customize ["modifyvm", :id, "--memory", "1024"]
    end
    # Set hostname - role will be set based onto it
    master.vm.hostname = "puppet.local"
    # Shell provisionner for bootstrapping puppet agent
    master.vm.provision "shell", path: "shell/base.sh"
    # Shell provisionner for bootstrapping gnubila conf
    master.vm.provision "shell", path: "shell/role.sh"

    # Share puppet develop branch as puppet production folder
    master.vm.synced_folder  "../../../puppet", "/puppet"
    master.vm.synced_folder  "../../hieradata", "/vagrant/hieradata"

    # Create a puppetmaster using puppet apply
    master.vm.provision :puppet do |puppet|
      # Path on host to puppet manifests
      puppet.manifests_path = "../../manifests"
      # Relative path to the default manifest
      # Path on host to puppet modules
      puppet.module_path = ["../../dist", "modules"]
      puppet.manifest_file  = "site.pp"
      # Path on host to hiera.yaml
      puppet.hiera_config_path = "puppet/hiera.yaml"
      # Working directory on the guest
      puppet.working_directory = '/vagrant'
    end
  end
  config.vm.define :client do |client|
    # Configure memory
    client.vm.provider :virtualbox do |vb|
      vb.customize ["modifyvm", :id, "--memory", "1024"]
    end
    # Set hostname - role will be set based onto it
    client.vm.hostname = "client.local"
    # Shell provisionner for bootstrapping puppet agent
    client.vm.provision "shell", path: "base.sh"
    # Shell provisionner for bootstrapping gnubila conf
    client.vm.provision "shell", path: "role.sh"
    # TODO Configure server using puppet agent against the master vm
  end
end

A copy of the hiera.yaml has been made with a custom datadir configuration to allow puppet apply to find the conf exposed into the vm by Vagrant.

Go play!

The boxes are started using

1
vagrant up
  • Connect using ssh
1
2
vagrant ssh master
vagrant ssh client

Later

Creating custom boxes using veewee

Blog Up Your Hands!

| Comments

Let’s start again with a new blog… Not coded by myself so perhaps posts will come to life :)