Testing Network Automation with Cumulus VX

Userlevel 1
This article shows you how to combine Cumulus VX, Vagrant and Serverspec with your configuration management tool as an effective way to test changes prior to deploying them.
Most configuration management tools encourage you to break things up into individual modules. For example, you may have a module that configures local users and SSH keys, and another module that configures NTP.
This article shows you how to:
  • Create a simple example module
  • Use Cumulus VX and Vagrant to deploy that module within a virtual environment
  • Use Serverspec to test the effect of that module
RequirementsExample ModuleIn this step, you will create a simple Ansible role that creates two users, rocket and turtle, and installs an SSH key for the rocket user.
. ├── example.yml └── roles     └── example         └── tasks             └── main.yml
Where main.yml is:
- name: "Create Rocket user"   user:     name=rocket     shell=/bin/bash     password=$1$uqRkM4YB$DPm1kPONNIOVCCUNJY/En.     groups=sudo  - name: "Install SSH key for the Rocket user"   authorized_key:     user=cumulus     key="{{ lookup('file', 'files/id_rsa.pub') }}"  - name: "Create Turtle user"   user:     name=tutrle     shell=/bin/bash     password=$1$5sR7Y1cc$kensm93iDBQg9p76x8Ol60
And example.yml is:
---   - hosts: all     roles:       - example 
Configuring VagrantVagrant requires a single configuration file (Vagrantfile). Normally this file is created at the top directory level of the module, alongside theexample.yml top-level Ansible task file. The Vagrantfile creates a Cumulus VX virtual machine and then uses the Ansible provisioner to run the Ansible task.
Since Vagrant takes care of most of the complexities of creating a new virtual machine, the Vagrantfile is very simple:
Vagrant.configure(2) do |config|    # Create a new Cumulus VX virtual machine   config.vm.box = "cumulus-vx-2.5.3"    # Use Ansible to run our tasks   config.vm.provision :ansible do |ansible|     ansible.playbook = "example.yml"     ansible.sudo = true   end  end
Next, run vagrant up. Vagrant:
  • Creates a single Cumulus VX virtual machine
  • Uses Ansible to run the top-level Ansible task (example.yml)
First RunThe first time you run the new Ansible module, run Vagrant to see what happens:
$ vagrant up Bringing machine 'default' up with 'virtualbox' provider... ==> default: Importing base box 'cumulus-vx-2.5.3'... ==> default: Matching MAC address for NAT networking... ==> default: Setting the name of the VM: vx-example_default_1437732585709_35793 ==> default: Clearing any previously set network interfaces... ==> default: Preparing network interfaces based on configuration...     default: Adapter 1: nat ==> default: Forwarding ports...     default: 22 => 2222 (adapter 1) ==> default: Booting VM... ==> default: Waiting for machine to boot. This may take a few minutes...     default: SSH address:     default: SSH username: vagrant     default: SSH auth method: private key     default: Warning: Connection timeout. Retrying...     default:     default: Vagrant insecure key detected. Vagrant will automatically replace     default: this with a newly generated keypair for better security.     default:     default: Inserting generated public key within guest...     default: Removing insecure key from the guest if it's present...     default: Key inserted! Disconnecting and reconnecting using new SSH key... ==> default: Machine booted and ready! ==> default: Checking for guest additions in VM... ==> default: Mounting shared folders...     default: /vagrant => /Users/cumulus/vx-example ==> default: Running provisioner: ansible...  PLAY [all] ********************************************************************  GATHERING FACTS *************************************************************** ok: [default]  TASK: [example | Create Rocket user] ****************************************** changed: [default]  TASK: [example | Install SSH key for the Rocket user] ************************* fatal: [default] => could not locate file in lookup: files/id_rsa.pub  FATAL: all hosts have already failed -- aborting  PLAY RECAP ********************************************************************            to retry, use: --limit @/Users/cumulus/example.retry  default                    : ok=2    changed=1    unreachable=1    failed=0  Ansible failed to complete successfully. Any error output should be visible above. Please fix these errors and try again.
The Ansible module has failed to run. When you created the module, you forgot to create the SSH public key that is needed to install for the rocket user!
Once you've by added the missing file to fix the error, verify that it works. Note that the virtual machine is still running; instead of creating a new virtual machine, you can just tell Vagrant to run the Ansible provisioner again with the vagrant provision command.
$ vagrant provision ==> default: Running provisioner: ansible...  PLAY [all] ********************************************************************  GATHERING FACTS *************************************************************** ok: [default]  TASK: [example | Create Rocket user] ****************************************** ok: [default]  TASK: [example | Install SSH key for the Rocket user] ************************* changed: [default]  TASK: [example | Create Turtle user] ****************************************** changed: [default]  PLAY RECAP ******************************************************************** default                    : ok=4    changed=2    unreachable=0    failed=0
This time the Ansible tasks were completed.
Verifying the Changes with ServerspecEven though the Ansible playbook runs successfully, how do you know that it did what you intended it to do? You could log in to the virtual machine with the vagrant ssh command and manually check that the two users were created and that the SSH key was installed for the rocket user, but that's likely to become tedious and error prone as the module becomes more complex, or if you're making rapid changes.
Instead, use Serverspec to test the changes. Serverspec can test a variety of system resources and integrates with Vagrant to run its tests inside of the virtual machine.
First, have Serverspec create a default configuration:
$ serverspec-init Select OS type:    1) UN*X   2) Windows  Select number: 1  Select a backend type:    1) SSH   2) Exec (local)  Select number: 1  Vagrant instance y/n: y Auto-configure Vagrant from Vagrantfile? y/n: y  + spec/  + spec/default/  + spec/default/sample_spec.rb  + spec/spec_helper.rb  + Rakefile  + .rspec
Serverspec-init asks some questions, including if you want to integrate your tests with Vagrant. It then creates a configuration file (spec_helper.rb), a Rakefile (for running the tests) and a sample test file (sample_spec.rb).
The sample file is just an example, so you should change it to something more appropriate for your playbook:
require 'spec_helper'  describe user('rocket') do   it { should exist } end  describe file('/home/rocket/.ssh/authorized_keys') do   it { should exist } end  describe user('turtle') do   it { should exist } end
Now run Serverspec to test the configuration inside the virtual machine:
$ rake spec:_default  User "rocket"   should exist  File "/home/rocket/.ssh/authorized_keys"   should exist  User "turtle"   should exist (FAILED - 1)  Failures:    1) User "turtle" should exist      Failure/Error: it { should exist }        expected User "turtle" to exist      # ./spec/default/sample_spec.rb:12:in 'block (2 levels) in '  Finished in 0.16075 seconds (files took 3.32 seconds to load) 3 examples, 1 failure  Failed examples:  rspec ./spec/default/sample_spec.rb:12 # User "turtle" should exist
Although Ansible ran and claimed to create the turtle user, something went wrong. If you look closely at the task, you can see the problem: a typo means Ansible has created a user called "tutrle". Luckily you ran the playbook inside a virtual machine and tested it before deploying it to production and creating the erroneous user everywhere!
Now that we have Vagrant and Serverspec set up, it's easy to fix the playbook, apply the changes and test that the fix worked:
$ vim roles/example/tasks/main.yml $ vagrant provision ==> default: Running provisioner: ansible...  PLAY [all] ********************************************************************  GATHERING FACTS *************************************************************** ok: [default]  TASK: [example | Create Rocket user] ****************************************** ok: [default]  TASK: [example | Install SSH key for the Rocket user] ************************* ok: [default]  TASK: [example | Create Turtle user] ****************************************** changed: [default]  PLAY RECAP ******************************************************************** default                    : ok=4    changed=1    unreachable=0    failed=0  $ rake spec:_default  User "rocket"   should exist  File "/home/rocket/.ssh/authorized_keys"   should exist  User "turtle"   should exist  Finished in 0.15233 seconds (files took 3.38 seconds to load) 3 examples, 0 failures
This time the Ansible playbook was successful. You can go ahead and deploy it to real systems confident that it works as intended.
Cumulus Networks Vagrant ExamplesCumulus Networks has created some examples of using Cumulus VX with Vagrant. These examples include Serverspec tests that show how to test more complex configurations and how to test network resources such as interfaces, bridges, bonds and routes.
You can find the examples under the vagrant/demos/ directory of the cumulus-vx-vagrant Github repository.
Related Links

4 replies

There isn't much here in between this and a Docker image. Have you done any of that and will it be released?
Userlevel 1
Josh Saul wrote:

There isn't much here in between this and a Docker image. Have you done any of that and will it ...

There are currently no plans to target Docker, and I'm not sure we ever would; Cumulus Linux naturally has its own kernel, so running everything under the host kernel inside a container wouldn't be the same.
Userlevel 2
In the "related links" section, the Cumulus VX user guide link needs updated to the following -> Cumulus VX Getting Started Guide
Userlevel 5
Fixed broken link.