Tests are central to our development process and hence some understanding of how testing is done will also help you to understand Reahl – and the rest of this tutorial.
Here’s how we like to think of tests: if you had to explain a system to a newcomer from scratch, you’d explain a number of facts – some facts also build upon on other facts that have already been explained. We try to identify these explanatory facts, and write a test per fact. The code of the test itself provides an example and/or more detail about that fact. Tests are thus a specification of the requirements of the system, and can be used to explain the system to someone.
In order to facilitate these ideas, some testing tools grew with Reahl. This section gives a brief introduction.
In order to test something, you usually need to create a number of related objects first for use in the test itself. Creating these objects can be tedious – especially if they depend on each other.
Such a collection of objects used together by a test is called a test
Fixture. Reahl allows one to write a
Fixture as a separate
class. Defining a
Fixture separately from the test that is using it
allows for the
Fixture to be re-used by different tests, and even by
Fixture classes. Separating the
Fixture also helps keep the test
itself to read more like a requirements specification and less like
Fixtures work: For each object that will form part of the
Fixture, a programmer writes a method on the
Fixture class which will
create the object when called. The name of the method is “new_”
prefixed to the name of the object. (Assuming an object named
my_object, you would add a method to the fixture named
new_my_object.) In order to use a
Fixture with a specific test,
decorate the test using the @test() decorator, passing the
class to @test(). The test method or function should also
expect a single argument in its signature: the
The first time a program references .my_object on the
instance, the corresponding new_ method will be called behind the
scenes to create the object. Subsequent accesses of .my_object will
always bring back the same instance which was created on the first
Here is a simple test which illustrates how
from __future__ import print_function, unicode_literals, absolute_import, division from reahl.tofu import test, Fixture class SimpleFixture(Fixture): def new_name(self): return 'John' @test(SimpleFixture) def fixture_singletons(fixture): """Accessing an attribute on the Fixture always brings back the same object, as created by a similarly named new_ method on the fixture.""" assert fixture.name == 'John' assert fixture.name is fixture.name # ie, the same object is always returned
Fixtures are all about objects that depend on one another. The
following example shows a
Fixture with two objects: a .name, and a
.user. The User object is initialised using the .name which
accompanies it on the
Fixture. For simplicity, the actual code of User
is just included in the test. Note also how SimpleFixture is reused
here, by deriving from it:
class User(object): def __init__(self, name): self.name = name class InterestingFixture(SimpleFixture): def new_user(self): return User(self.name) @test(InterestingFixture) def dependent_setup_objects(fixture): """Different attributes on a Fixture can reference one another.""" assert fixture.user.name is fixture.name
Things can get more interesting though. A useful convention is to
create new_* methods with keyword arguments. This way one can use
the new_* method to create slightly modified versions of the default
object available on the
class MoreInterestingFixture(SimpleFixture): def new_user(self, name=None): return User(name or self.name) @test(MoreInterestingFixture) def bypassing_the_singleton(fixture): """new_ methods can be supplied with kwargs in order to create test objects that differ from the default.""" jane = fixture.new_user(name='Jane') assert jane.name == 'Jane' assert fixture.user is not jane other_jane = fixture.new_user(name='Jane') assert jane is not other_jane
Running the tests:
Set_up, tear_down, and run fixtures¶
Fixtures mostly consist of a bunch of new_ methods, they
usually do not need traditional methods for settting up a fixture, or
tearing the fixture down afterwards. In some rare cases this is still
needed though. One may want to start a web server before a test, and
stop it afterwards, for example. Any method on a
Fixture can be run
before or after the test it is used with. Just annotate the method
with @set_up or @tear_down respectively.
Sometimes such setup can take a long time and would slow down tests if
it happens for each and every test. When testing web applications, for
example, you may want to fire up a browser before the test – something
that takes quite a long time. For this reason Reahl provides an
extension to nosetests to which you can specify a “run
Fixture”. A run
Fixture is used for all tests that are run together. It is set up
before all the tests are run, and torn down at the end of all test
Fixtures that are attached to individual tests also have
access to the current run
To specify which run
Fixture should be used for a test run, use
--with-run-fixture (or -F) argument to nosetests.
Testing without a real browser¶
Enough talk. Its time for a first test.
In the previous section an application was
developed which lets one add, view and edit Addresses on different
Views. The second version of it used
Buttons to navigate to the edit
Views of individual Addresses. This application is starting to get
cumbersome to test by hand each time a change is made to it. It needs
a test. A Test for it will also make it possible to set breakpoints
and investigate its internals while running.
Here is a first stab at a test for it. Sometimes it is useful to write a test per explained fact; other times it is useful to write a test for a little scenario illustrating a “user story”. This test is an example of the latter:
from __future__ import print_function, unicode_literals, absolute_import, division from reahl.tofu import test from reahl.web_dev.fixtures import WebFixture from reahl.webdev.tools import Browser, XPath from reahl.doc.examples.tutorial.parameterised2.parameterised2 import AddressBookUI @test(WebFixture) def adding_an_address(fixture): """To add a new address, a user clicks on "Add Address" link on the menu, then supplies the information for the new address and clicks the Save button. Upon successful addition of the address, the user is returned to the home page where the new address is now listed.""" browser = Browser(fixture.new_wsgi_app(site_root=AddressBookUI)) browser.open('/') browser.click(XPath.link_with_text('Add an address')) actual_title = browser.title assert actual_title == 'Add an address', 'Expected to be on the Add an address page' browser.type(XPath.input_labelled('Name'), 'John Doe') browser.type(XPath.input_labelled('Email'), '[email protected]') browser.click(XPath.button_labelled('Save')) actual_title = browser.title assert actual_title == 'Addresses', 'Expected to be back on the home page after editing' assert browser.is_element_present(XPath.paragraph_containing('John Doe: [email protected]')), \ 'Expected the newly added address to be listed on the home page'
The test should be run with nosetests,
There are a number of
Fixtures available as part of Reahl to help you
test applications. Explaining all of them is outside of the scope of
this introductory section. For this test, it is useful to know some of
the things that
BrowserSetup does: Before the tests start to run, it
creates an empty database with the complete schema of your
application. The Elixir/Sqlalchemy classes used by your application
are also initialised properly (no need to create_all/setup_all, etc).
The only obvious feature of
WebFixture used in this little test is the
creation of a WSGI application. WebFixture.new_wsgi_app() can create a
WSGI application in various different ways. This example shows how to
create a ReahlWSGIApplication from a supplied
is not turned on (although, as you will see later, this can be
specified as well).
Behind the scenes,
WebFixture also has the job of starting a new
database transaction before each test, and rolling it back after each
test. Hence no real need to tear down anything...
Two other important tools introduced in this test are:
Browser is not a real browser – it is our thin wrapper on top
of what is available from WebTest.
Browser has the interface of a
Browser though. It has methods like .open() (to open an url),
.click() (to click on anything), and .type() (to type something
into an element of the web page).
We like this interface, because it makes the tests more readable to a non-programmer.
When the browser is instructed to click on something or type into
XPath expression is used to specify how to find that
element on the web page.
XPath has handy methods for constructing
XPath expressions while keeping the code readable. Compare, for
example the following two lines to appreciate what
XPath does. Isn’t
the one using
XPath much more explicit and readable?
Often in tests, there are small bits of code which would be more
readable if it was extracted into properly named methods (as done with
XPath above). If you create a
Fixture specially for testing the
AddressBookUI, such a fixture is an ideal home for such methods. In
the expanded version of our test below, AddressAppFixture was
added. AddressAppFixture now contains the nasty implementation of a
couple of things we’d like to assert about the application, as well as
some objects used by the tests. (Compare this implementation of
.adding_an_address() to the previous implementation.)
from __future__ import print_function, unicode_literals, absolute_import, division from reahl.tofu import test from reahl.web_dev.fixtures import WebFixture from reahl.webdev.tools import Browser, XPath from reahl.doc.examples.tutorial.parameterised2.parameterised2 import AddressBookUI, Address class AddressAppFixture(WebFixture): def new_browser(self): return Browser(self.new_wsgi_app(site_root=AddressBookUI)) def new_existing_address(self): address = Address(name='John Doe', email_address='[email protected]') address.save() return address def is_on_home_page(self): return self.browser.title == 'Addresses' def is_on_add_page(self): return self.browser.title == 'Add an address' def is_on_edit_page_for(self, address): return self.browser.title == 'Edit %s' % address.name def address_is_listed_as(self, name, email_address): return self.browser.is_element_present(XPath.paragraph_containing('%s: %s' % (name, email_address))) @test(AddressAppFixture) def adding_an_address(fixture): """To add a new address, a user clicks on "Add Address" link on the menu, then supplies the information for the new address and clicks the Save button. Upon success addition of the address, the user is returned to the home page where the new address is now listed.""" browser = fixture.browser browser.open('/') browser.click(XPath.link_with_text('Add an address')) assert fixture.is_on_add_page() browser.type(XPath.input_labelled('Name'), 'John Doe') browser.type(XPath.input_labelled('Email'), '[email protected]') browser.click(XPath.button_labelled('Save')) assert fixture.is_on_home_page() assert fixture.address_is_listed_as('John Doe', '[email protected]') @test(AddressAppFixture) def editing_an_address(fixture): """To edit an existing address, a user clicks on the "Edit" button next to the chosen Address on the "Addresses" page. The user is then taken to an "Edit" View for the chosen Address and can change the name or email address. Upon clicking the "Update" Button, the user is sent back to the "Addresses" page where the changes are visible.""" browser = fixture.browser existing_address = fixture.existing_address browser.open('/') browser.click(XPath.button_labelled('Edit')) assert fixture.is_on_edit_page_for(existing_address) browser.type(XPath.input_labelled('Name'), 'John Doe-changed') browser.type(XPath.input_labelled('Email'), '[email protected]') browser.click(XPath.button_labelled('Update')) assert fixture.is_on_home_page() assert fixture.address_is_listed_as('John Doe-changed', '[email protected]')
Browser class used above cannot be used for all tests, since it
in something like an actual browser. Doing this requires Selenium, and
the use of the web server started by
BrowserSetup for exactly this
DriverBrowser is a class which provides a thin layer over Selenium’s
WebDriver interface. It gives a programmer a similar set of methods
to those provided by the
Browser, as used above. An instance of it is
available on the
WebFixture via its .driver_browser attribute.
Fixtures that form part of Reahl use Chromium by default
in order to run tests, and they expect the executable for chromium to
be in ‘/usr/lib/chromium-browser/chromium-browser’.
You can change which browser is used by creating a new run
Fixture that inherits
BrowserSetup, and overriding its .web_driver() method. Some
details regarding how the browser is configured (such as the binary
location of the browser) can similarly be changed by overriding the
relevant .new_ method of that class.
When the following is executed, a browser will be fired up, and
Selenium used to test that the validation provided by
web application upon creation:
from __future__ import print_function, unicode_literals, absolute_import, division from reahl.tofu import test from reahl.web_dev.fixtures import WebFixture from reahl.webdev.tools import XPath from reahl.doc.examples.tutorial.parameterised2.parameterised2 import AddressBookUI, Address class AddressAppErrorFixture(WebFixture): def new_wsgi_app(self): return super(AddressAppErrorFixture, self).new_wsgi_app(site_root=AddressBookUI, enable_js=True) def new_existing_address(self): address = Address(name='John Doe', email_address='[email protected]') address.save() return address def error_is_displayed(self, text): return self.driver_browser.is_element_present(XPath.error_label_containing(text)) @test(AddressAppErrorFixture) def edit_errors(fixture): """Email addresses on the Edit an address page have to be valid email addresses.""" fixture.reahl_server.set_app(fixture.wsgi_app) browser = fixture.driver_browser existing_address = fixture.existing_address # Doing this just creates the Address in the database browser.open('/') browser.click(XPath.button_labelled('Edit')) browser.type(XPath.input_labelled('Email'), 'invalid email address') assert fixture.error_is_displayed('Email should be a valid email address')