Sparring - a test robot suite for OpenStack platform

Sparring is a test robot suite for OpenStack platform.

There are 3 robots for now.

  • funcbot: a functional test robot for openstack
  • perfbot: a performance test robot for openstack
  • cdbot: a continuous deployment test robot for openstack API services

funcbot

  • Use robotframework and gabbi library for functional tests

  • There are 5 test suites

    • identity: Keystone API test
    • network: Neutron API test
    • image: Glance API test
    • volume: Cinder API test
    • compute: Nova API test
  • There are 3 areas - Keyword, Code, Manifest

    • Keyword: This area is for tester. Just put keywords from Code area.
    • Code: This area is for programmer. Program logic of keywords are here.
    • Manifest: This area is for programmer. OpenStack API calls are here.

Keyword example: Given-When-Then example:

Start the server
  [Tags]    compute     critical
  Given Compute service is available
  When Start the server
  Then Check if the server is active

Code example: Use robotframework libraries:

Check if the server is active
  Wait Until Keyword Succeeds   2m   3s
  ...   check server is active     url=${COMPUTE_SERVICE}

Manifest example: API manifests are defined in yaml format using gabbi:

- name: check server is active
  GET: /servers/$ENVIRON['TEST_SERVER_ID']
  status: 200
  response_json_paths:
    $.server.status: ACTIVE

How to use funcbot

Get the sample openstack_settings.robot file to /tmp/ and Edit settings above “Do not touch below!!!” line.:

docker run --rm --name sparring jijisa/sparring \
     --show-os-settings > /tmp/settings.robot

To run all test suites in funcbot (This is default.):

docker run --rm --tty --network=host --name sparring \
  -v /etc/hosts:/etc/hosts:ro -v /tmp/output:/tmp/output \
  -v /tmp/settings.robot:/sparring/resources/openstack_settings.robot:ro \
  jijisa/sparring

To run specific test suites(e.g. network and image) in funcbot:

docker run --rm --tty --network=host --name sparring \
  -v /etc/hosts:/etc/hosts:ro -v /tmp/output:/tmp/output \
  -v /tmp/settings.robot:/sparring/resources/openstack_settings.robot:ro \
  jijisa/sparring --run-funcbot network image

To run all test cases except evacuation test in funcbot (funcbot forces compute service down to evacuate test VM instance so the option(-e) excluding evacuation test is useful when you run Sparring funcbot in production environment.):

docker run --rm --tty --network=host --name sparring \
  -v /etc/hosts:/etc/hosts:ro -v /tmp/output:/tmp/output \
  -v /tmp/settings.robot:/sparring/resources/openstack_settings.robot:ro \
  jijisa/sparring --run-funcbot -e

Deliverables

Demo

screencast