Practical Glusto Example with Jonathan Holloway at Gluster Developer Summit

1. Practical Glusto
Example
Jonathan Holloway
Principal Quality Engineer - Red Hat Storage

2. Practical Glusto Example
Very Brief Intro
to Glusto

3. Glusto is...
● Glusto is a “framework” or collection of commonly used tools
for developing scripts and tests for distributed systems.
● Primary components
− A class combining a collection of tools in a single, easy-to-use interface.
− Command-line wrapper to make dynamic config files available to test frameworks
● Built on open source and standard tools.
− (e.g., Python modules, RPyC, SSH, REST, PyUnit, PyTest, Nose, etc.)
● Provides flexibility minus complexity during development.
− Simple import into existing scripts
− Access to functionality via Python Interactive Interpreter
− Code/Test/Code with IDE (e.g., Eclipse via PyDev)

4. Key Features
● Remote calls via SSH
● Remote calls via RPyC
● Read and store config files in yaml, json, and ini formats
● Logging
● Templates
● Provides a wrapper for unit test discovery and configuration
● Works with multiple unit test frameworks (PyUnit, PyTest, Nose)
● Simple REST client
● Accessible via Python module, Python Interactive Interpreter, and a CLI
client.

5. Practical Glusto Example
Install Glusto

6. Installing Glusto
● Installs via setuptools
− Directly from github with pip command
− # pip install –upgrade
git+git://github.com/loadtheaccumulator/glusto.git
− Via setuptools with python setup.py
− # git clone https://github.com/loadtheaccumulator/glusto.git
# cd glusto
# python setup.py install
● Docker container
− docker.io/loadtheaccumulator/glusto

7. Installing glustolibs-gluster
● Installs via setuptools
− Directly from github with pip command
− # pip install –upgrade
git+git://github.com/glusto-tests/glustolibs/glusto.git
− Via setuptools with python setup.py
− # git clone https://github.com/gluster/glusto-tests.git
# cd glusto-tests/glustolibs-gluster
# python setup.py install
● Docker container
− Contains both Glusto and glustolibs-gluster libraries
− May be available by the time you see this

8. Practical Glusto Example
Write Tests

9. Test Scripts with Glusto
● Standard PyUnit, PyTest, or Nose format
● Extendable via standard Python subclassing.
● Glusto just plugs in via import without dependency on
Glusto to run basic tests.
− from glusto.core import Glusto as g
● YAML config file containing the hosts
● clients: [192.168.1.225]
servers: [192.168.1.221, 192.168.1.222, 192.168.1.223, 192.168.1.224]

10. Basic PyUnit Script Format
● import unittest
class MyTestClass(unittest.TestCase):
@classmethod
def setUpClass(cls):
# runs before all test_ methods in the class
def setUp(self):
# runs before each test_ method in the class
def test_example1(self):
# test method representing a test case
def test_example2(self):
# test method representing a test case
def tearDown(self):
# runs after each test_ method in the class
@classmethod
def tearDownClass(cls):
# runs after all test_ methods in the class

11. Basic PyUnit Script Format
● import unittest
class MyTestClass(unittest.TestCase):
@classmethod
def setUpClass(cls):
# runs before all test_ methods in the class
def setUp(self):
# runs before each test_ method in the class
def test_example1(self):
# test method representing a test case
def test_example2(self):
# test method representing a test case
def tearDown(self):
# runs after each test_ method in the class
@classmethod
def tearDownClass(cls):
# runs after all test_ methods in the class

12. Basic PyUnit Script Format
● import unittest
class MyTestClass(unittest.TestCase):
@classmethod
def setUpClass(cls):
# runs before all test_ methods in the class
def setUp(self):
# runs before each test_ method in the class
def test_example1(self):
# test method representing a test case
def test_example2(self):
# test method representing a test case
def tearDown(self):
# runs after each test_ method in the class
@classmethod
def tearDownClass(cls):
# runs after all test_ methods in the class

13. Basic PyUnit Script Format
● import unittest
class MyTestClass(unittest.TestCase):
@classmethod
def setUpClass(cls):
# runs before all test_ methods in the class
def setUp(self):
# runs before each test_ method in the class
def test_example1(self):
# test method representing a test case
def test_example2(self):
# test method representing a test case
def tearDown(self):
# runs after each test_ method in the class
@classmethod
def tearDownClass(cls):
# runs after all test_ methods in the class

14. Basic PyUnit Script Format
● import unittest
class MyTestClass(unittest.TestCase):
@classmethod
def setUpClass(cls):
# runs before all test_ methods in the class
def setUp(self):
# runs before each test_ method in the class
def test_example1(self):
# test method representing a test case
def test_example2(self):
# test method representing a test case
def tearDown(self):
# runs after each test_ method in the class
@classmethod
def tearDownClass(cls):
# runs after all test_ methods in the class

15. import and class definition
● import unittest
import pytest
from glusto.core import Glusto as g
from gluster_base_class import runs_on
@runs_on([['distributed', 'replicated'], ['glusterfs', 'nfs']])
class GlusterDemoTest(unittest.TestCase):
@classmethod
def setUpClass(cls):
# runs before all test_ methods in the class
def setUp(self):
# runs before each test_ method in the class
...
def tearDown(self):
# runs after each test_ method in the class
@classmethod
def tearDownClass(cls):
# runs after all test_ methods in the class

16. setUpClass/tearDownClass
● ...
@runs_on([['distributed', 'replicated'], ['glusterfs', 'nfs']])
class GlusterDemoTest(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.masternode = g.config['servers'][0]
cls.client = g.config['clients'][0]
cls.mountpoint = '/mnt/gluster-mount'
g.run(cls.masternode, 'service glusterd start')
g.run(cls.masternode, 'gluster volume start glustervol1')
g.run(cls.client, 'mount /mnt/gluster-mount')
...
@classmethod
def tearDownClass(cls):
g.run(cls.client, 'umount %s' % cls.mountpoint)
g.run(cls.masternode, 'gluster volume stop glustervol1')
g.run(cls.masternode, 'service glusterd stop')

17. setUp/tearDown
● ...
@runs_on([['distributed', 'replicated'], ['glusterfs', 'nfs']])
class GlusterDemoTest(unittest.TestCase):
@classmethod
def setUpClass(cls):
...
def setUp(self):
self.filename = '%s/%s-%s' %
(self.mountpoint, self.volume, self.mount)
print('nCreating file %s' % self.filename)
g.run(self.client, 'touch %s' % self.filename)
...
def tearDown(self):
print('Removing file %s' % self.filename)
g.run(self.client, 'rm -f %s' % self.filename)
@classmethod
def tearDownClass(cls):
...

18. test_ method(s)
● ...
def setUp(self):
...
def test_mount_type(self):
rcode, _, _ = g.run(self.client,
'mount | grep %s' % self.mount)
self.assertEqual(rcode, 0,
'Mounted volume is not type %s' % self.mount)
@pytest.mark.bvt
def test_create_file_with_touch(self):
print('nTesting file %s' % self.filename)
rcode, rout, rerr = g.run(self.client, 'ls %s' % self.filename)
self.assertEqual(rcode, 0, 'File does not exist')
def tearDown(self):
...
...

19. Practical Glusto Example
Run Tests

20. Run via /usr/bin/glusto
● Glusto CLI wrapper provides config file(s) to tests
− $ glusto -c myconfig.yml --pytest='test_demo1.py'
● PyTest provides xUnit style output
− $ glusto -c myconfig.yml
--pytest='test_demo1.py --junitxml=results.xml'
● Accepts PyTest marker (-m) and filter (-k) parameters
− $ glusto -c myconfig.yml
--pytest='test_demo1.py -m bvt'
$ glusto -c myconfig.yml
--pytest='test_demo1.py -k mount'

21. Practical Glusto Example
Demo
https://asciinema.org/a/7sb4vox7ucasmgbwyeii4rheh

22. Info
● Contact
− Email: gluster-devel@gluster.org
− IRC: FreeNode #gluster-dev (loadtheacc)
● Libraries and Tests Repo
− git clone http://review.gluster.org/glusto-tests
− Mirrored at http://github.com/gluster/glusto-tests
● Glusto Repo and Docs
− http://github.com/loadtheaccumulator/glusto
− http://glusto.readthedocs.io/