Posts Tagged ‘testing software’

h1

Jumping on the Rails Pt.4 – Spec’n our form

July 11, 2009

Well carrying on from the previous post, we’ll focus on some actual functionality in this post & start on creating the functionality for users to create new projects. As always we’ll start off with our feature:


	Feature: As a registered user I must be able to create projects
	  In order to create a new project I must be logged in
	  As a registered user
	  I want I want to be able to create new projects

		Scenario: A registered user must be able to create projects
		  Given an activated user logged in as 'reggie'
		  When I click new project
			And fill in the new project form
		  Then I should get confirmation of the successful creation

As we’re using restful-authentication’s stories we can save our selves a step ;), as well as having knowing that the user will be logged in.

This leads us to our next step, we need to make sure that we can click the new project link, so that we can actually create our new project, so we take our snippet & modify it to look like the example below.


	When /^I click new project$/ do
	  visit "/projects/new"
	end

Now that we have our first failing test

No route matches "/projects/new" with {:method=>:get} (ActionController::RoutingError)

We need to set up a route for a projects controller.

map.resource :projects

Now we’ll get another expected error

uninitialized constant ProjectsController (NameError)

So we should create projects_controller.rb,


class ProjectsController < ActionController::Base
  def new

  end
end

We’ll finally need to create our new template.

Missing template projects/new.erb in view path app/views (ActionView::MissingTemplate)

Once you’ve created the template, it’s finally time to jump down into our view specs, we’ll start with the view first. So we will use the welcome indexes view spec to start with:

/PROJECT/spec/view/welcome/index.html.erb_spec.rb

Now there are a few things here that we not specified in the our story:

  • a guest user should not see the new projects link
  • only registered users can see the new projects link

now for those of you that have never created a spec before, we’ll start off slowly.

All specs have the following template


	require File.expand_path(File.dirname(__FILE__) + '/../../spec_helper')

	describe "something" do
	end

Now the first line brings in our specs helper which basically sets up rspec for us, the next significant line is:


	describe "welcome/index.html.erb" do
	end

Within this we describe the things we are testing, in this case the index view of our welcome page.

With the basics out of the way, we’ll start by setting up the first test.


   it "should have a sign up link" do
     response.should have_selector("a", :content => "Sign up")
   end

This test should pass straight away seeing as we haven’t added the link yet. Next we’ll do a little refactoring as our specs are now defining different types of user actions, our code now look like this:


	require File.expand_path(File.dirname(__FILE__) + '/../../spec_helper')

	describe "/welcome/index" do
	  context "viewing as a unregistered user" do
	    before(:each) do
	      render
	    end
	    it "should have a sign up link" do
	      response.should have_selector("a", :content => "Sign up")
	    end

	    it "should not have a new project link" do
	      response.should_not have_selector("a", :content => "New Project")
	    end
	  end

	  context "viewing as a registered user" do
	  end
	end

Now we’re wrapped our specs in context blocks, i’ve done this basically to help visually and mentally tell the difference between the two blocks of test, the first context block focusses on users that are not registered, whilst the second deals with specs concentrating on users viewing the index page as a registered user.

Now that the refactoring is out of the way lets deal with our next spec.


	context "viewing as a registered user" do
    it "should be able to see the new project link" do
      response.should have_selector("a", :content => "New Project")
    end
  end

Our tests will now fail so now its time to get our test to pass.


	<a href="/project">New Project</a>

I know, we can make this a lot more Railsy but let concerntratet on the matter at hand, we still have a failing test.


	'/welcome/index should not have a new project link' FAILED
	expected following output to omit a New Project:

Our initially passing test is now failing, lets get that to pass and move along.


	context "viewing as a registered user" do
    before(:each) do
      template.stub!(:current_user).and_return mock_model(User)
      render
    end
    it "should be able to see the new project link" do
      response.should have_selector("a", :content => "New Project")
    end
  end

Our spec is passing but out definition is still failing, thought we are making some progress, we are greated by our next friendly error.


	When I click new project                                  # features/step_definitions/registered_users_step.rb:2
    No route matches "/projects/new" with {:method=>:get} (ActionController::RoutingError)

Well we don’t have a route for our project, so lets get that setup, we add the following to

PROJECT/config/routes.rb

:


	map.resource :projects

With our route now setup, we have another error:


	When I click new project                                  # features/step_definitions/registered_users_step.rb:2
	      uninitialized constant ProjectsController (NameError)

That seems simple enough, we just need to create a controller for our projects, creating

PROJECT/app/controllers/projects_controller.rb

:


	class ProjectsController < ApplicationController

	end

Our tests run again but wait? yet another error:


	When I click new project                                  # features/step_definitions/registered_users_step.rb:2
	      No action responded to new. Actions: authorized?, current_user, and logged_in? (ActionController::UnknownAction)

Oh yeah, we need to create the new method block:


	class ProjectsController < ApplicationController
	  def new

	  end
	end

Now we run the tests again, next our tests tell us to create a view for our project.


	When I click new project                                  # features/step_definitions/registered_users_step.rb:2
	      Missing template projects/new.erb in view path app/views (ActionView::MissingTemplate)

So lets create the template

PROJECT/views/projects/new.html.erb

. Our tests are now passing and we can move on to the next step.

Before we jump in, we take a step back and think about what type of information we want to store in our project.

Well we certainly want a project title & we could do with a small description of what the project is about. It would also be nice for the project to have aims.

Ok, now that we have a rough idea of what data we want to store in our project, it’s time to setup our definition.


	When /^fill in the new project form$/ do
	  fill_in 'project_title', :with => 'new project'
	  fill_in 'project_description', :with => 'This project will help us manage stuff'
	  fill_in 'project_aim', :with => 'To provide a service'
	  submit_form 'new_project'
	end

So now we have our first error:


	Could not find field: "projects_title" (Webrat::NotFoundError)

We need to create a field with the id projects_title. So lets dip down into our view specs to step out this functionality.

We create

PROJECT/specs/views/projects/new.html.erb_spec.rb

first and the add the following code:


	require File.expand_path(File.dirname(__FILE__) + '/../../spec_helper')

	describe "/projects/new.html.erb" do

	  it "should have a form" do
			@project = mock_model(Project)
	    assigns[:project] = @project
	    render
	    response.should have_tag("form[action=/project][method=post]") do
	      with_tag('input#project_title[name=?]', "project[title]")
	      with_tag('textarea#project_description[name=?]', "project[description]")
	      with_tag('textarea#project_aim[name=?]', "project[aim]")
	    end
	  end
	end

Now this is a little involved but we’re basically looking for a form which contains a text field and two textarea’s, which will store our title, description & project aim. I also mocked out the project model from the jump, though it’s a bit naughty to do this all the time, I knew I was going to need a Project model here, so I just mocked it out.

We now get the following error:


	'/projects/new.html.erb should have a form' FAILED
	Expected at least 1 element matching "form[action='/project'][method='post']", found 0.
	<false> is not true.

This is basically telling us need to now create our form. So we create our new view

PROJECT/app/views/projects/new.html.rb

and add the following code:


	<% form_for @project do |f| -%>

	<% end -%>

Next we get the following error:


	NameError in '/projects/new.html.erb should have a form'
	uninitialized constant Project

So this error tells us we need a new project to pass the to the form, so we’ll add it to make our controller look like below


	class ProjectsController < ApplicationController
	  def new
	    @project = Project.new
	  end
	end

We’ll get a similar error if we don’t create our actual model, so lets quickly do that know:


	class Project < ActiveRecord::Base
	end

And we’ll need to create our migration table also.


	class CreateProject < ActiveRecord::Migration
	  def self.up
	    create_table "users", :force => true do |t|
	      t.column :title,                    :string
	      t.column :description,              :text
	      t.column :aim,                      :text
	      t.column :created_at,               :datetime
	      t.column :updated_at,               :datetime
	    end
	  end

	  def self.down
	    drop_table "users"
	  end
	end

And run rake to migrate our changes:


	rake db:migrate RAILS_ENV=test

Now we make our title spec form spec pass it time to deal with the projects data, so on to our next test.


	'/projects/new.html.erb should have a form' FAILED
	Expected at least 1 element matching "field#projects_title[name='projects[title]']", found 0.
	<false> is not true.

So here we are being told that the title does not exist so lets create it, our view file now looks like the code below:


	<% form_for @project, :url => project_path do |f| -%>
	  <%= f.text_field :title %>
	<% end -%>

Now we are getting an error relating to our stub previously created mock.


	ActionView::TemplateError in '/projects/new.html.erb should have a form'
	undefined method `title' for #<Proc:0x25664e8>

To fix this we modify our mock to look like the snippet below:


	    @project = mock_model(Project,
	                          :title=>'A cool project')

All we have done here is set the title for our project within our mock, fixing our previously broken test.

Now that we have resolved that issue we should be on to our next failing spec.


	'/projects/new.html.erb should have a form' FAILED
	Expected at least 1 element matching "textarea#project_description[name='project[description]']", found 0.
	<false> is not true.

Again this is pretty straight forward, we simply add a text area field to our view, making our view look like below:


	<% form_for @project, :url => project_path do |f| -%>
	  <%= f.text_field :title %>
		<%= f.text_area :description %>
	<% end -%>

Now we stumble upon another issue with our mock, which recieved an unexpected message:


	ActionView::TemplateError in '/projects/new.html.erb should have a form'
	Mock 'Project_1001' received unexpected message :description with (no args)

we could stub out each of our properties as we go along but to be honest that gets very boring very quick.

Lets get this solved once & for all, we’ll modify our mock_model as follows:


	@project = mock_model(Project,
                        :null_object=>true)

This gets us back on track & strolling on to the next expected failure, our missing aims field:


	'/projects/new.html.erb should have a form' FAILED
	Expected at least 1 element matching "textarea#project_aim[name='project[aim]']", found 0.
	 is not true.

Now with our view looking like the example below:


	<% form_for @project, :url => project_path do |f| -%>
	  <%= f.text_field :title %>
		<%= f.text_area :description %>
		<%= f.text_area :aim %>
	<% end -%>

Our specs are now passing but we still have one step missing from our definitions, we left the submit button out of our spec’s but we specified them here luckly enough, let’s get this test passing.


	<% form_for @project, :url => project_path do |f| -%>
	  <%= f.text_field :title %>
		<%= f.text_area :description %>
		<%= f.text_area :aim %>
		<%= f.submit "Create", :disable_with => 'Creating...' %>
	<% end -%>

We have another failing test


	No action responded to create. Actions: authorized?, current_user, logged_in?, and new (ActionController::UnknownAction)

So we basically need to setup a create method in our project controller, we stub out the response for a later scenario.


	def create
    redirect_to projects_path @project
  end

Doing this will create another error, we need to create a project show view, so create that & we are ready to move on to the next step.

Thees last two are nice & easy.


       Then /^the form will not be re-rendered$/ do
	  response.should_not have_selector :form
	end

Then /^I should get confirmation of the successful creation$/ do flash.should contain "You have successfully created your new project" end
Our projects controller looks now looks like the following:


	class ProjectsController & ApplicationController
	  def new
	    @project = Project.new
	  end

	  def create
            @project = Project.new params[:project]
            @project.save
            flash[:notice] = "You have successfully created your new project"
	    redirect_to projects_path @project
	  end
	end

We’ll need to do one more thing to get all our tests passing, we’ll need to create a show action in our controller, we’ll stub it out for now


    def show
        respond_to do |format|
            format.html
        end
    end

All our steps are now passing, lets quickly clean up our view whilst we have our code in a stable passing state, lets sort out our welcome page.

	<%= link_to "Sign up", signup_path %>
	<% if false == self.current_user.nil? %>
	<%= link_to "New Project", new_project_path %>
	<% end -%>

Now along with our specs & our step definitions all passing, we can safely move on to the next story.

We'll leave that for the next post, what I will leave for now is an outline of what we need to cover on the next few iterations.

  • When submitting the form the user must be alerted if the title is not present
  • When submitting the form the user must be alerted if the description is not present
  • When submitting the form the user must be alerted if the aim is not present
  • When submitting the form the user must be alerted if the project already exists
  • When submitting the form the user successfully, the project will be save
h1

SimpleTest Vs PHPUnit

January 28, 2009

Ideology

We want to be creating tests for every piece of functionality being developed. This will help us to keep our project scalable as well as alerting us to any state or behavioural errors/smells that may arise over the projects life time.

Tests are typically used as a way of making a project as stable as possible with the view of spending as little time as possible on debugging and error finding.

As you may know from personal experience, it can be more than a pain to figure out where a particular error is coming from, not to mention, what fired it off in the first place. Test cases are there to relieve this, firstly by way of testing each expected and unexpected response to the test case. The other way is allowing us to create test situations that would rarely take place in untested situations, allowing us to deal with those issues before they come up as opposed to waiting for them to appear later on in development or worst yet, when in production after a compromise.

Test suites are not just used as a safety net, they can be used to give the developer a better understanding of the implementation of the system, as well as documentation for other developers, describing what the developer does and doesn’t expect from the case.

The idea is to incrementally create test cases & then accompany them with the actual implementation of the functionality in questions. The PHPUnit tutorials explain this procedure pretty well so I will not reiterate (see http://www.phpunit.de/pocket_guide/3.2/en/test-first-programming.html for more info). Developing this way not only helps find bugs as soon as they appear, it also helps to find them later down the line it. TDD also helps to realise over sights in the design and implementation of the system allowing us to deal with them as soon as they appear.

Ideals

  • Test incrementally, creating test first, then the implementation.
  • Have a testing suite that allows us to run test via a web browser and command line (needs to be possible with no change to code).
  • Tests are integrated into phing, so tests are run before system is deployed or updated.
  • Tests are able to run separately, a group & as a whole.
  • Able to customise results front end so we can view pass & fail results ( useful to ascertain that we actually have the data we expect.
  • Use Reflection API to test a classes structure (properties, access type, etc.).
  • Test for the unexpected as well as expected results & errors.
  • Test of exceptions & exception handling.

Findings

I’ve been looking into both PHPUnit3 & SimpleTest to determine the best test suite for us to use. Both are pretty good suites at a glance but there a few fundamental differences to be noted.

PHPUnit3

Iis the most widely used and the most popular to date though it does present a few problems. Since version 3 mock objects have been introduced but still lacks the power that SimpleTest possesses. It can also only be run via a command prompt so view-ability can be an issue, especially when the suite grows. This can be eleviated with the use of reports which can be generated once a test is run, allowing for testers to view the results without needing to know the actual command to run the suite by itself. As of ZFW 1.6 Zend_Test_PHPUnit is now integrated allowing us to test our zend application explicitly with PHPUnit. This is an obvious attraction as Zend_Test_PHPUnit will have functionality specific to the framework, allowing us to spend time on the actual tests and not creating the functionality for them.

pros

  • Widely used, part of ZFW.
  • Loads of example on-line.
  • Extended by Zend_Test_PHPUnit as of 1.6RC1
  • Able to test controllers with no further extending.
  • Can create various type of reports.
  • Customisable tests results.

cons

  • Mock objects not as fluent as SimpleTest
  • Can not run directly via a web browser.
  • Less functional than SimpleTest.

SimpleTest is the not as widely used as the above but has some fundamental differences. It allows us to not just test an objects validity but also test our application in varying ways (check its state, behaviour). With SimpleTest we are able to not just test the back end integrity but we can also test that the front end also deals with situations as we expect it to.

pros

  • Can be used with PHPUnit.
  • Can custom output.
  • Can be run via both command line and browser.
  • Can test both states & behaviour.
  • Customisable tests results.
  • Can test both state, behaviour & front end functionality.

cons

  • Not as well documented as PHPUnit
  • Will need to extend to use with ZFW
  • Not naturally a part of ZFW.

Over the past few years I’ve used both suites quite extensively and found that SimpleTest is by far the most flexible. First off we’ll be able to customise the display of our results so we can properly determine whether a test is correctly passed or not, I’ve found that sometimes, though a test passes, it can sometimes be a false positive. SimpleTest allows us to not just display the test result, but also display the actual result data. Mock objects are also exceptionally powerful in SimpleTest, as mentioned before mock objects allow us to create instances of an object and set its return values. Once this is done, we can then test to make sure a method is only run ‘x’ amount of times, as well as being able to test for results, behaviour & states as well as property types. On top of all that it lessen the dependency issues that can arise from having to use real objects to test other objects (see http://simpletest.sourceforge.net/en/mock_objects_documentation.html for more info).

Conclusion

Both suites can be used with Zend framework (SimpleTest needing some extending), as well both having an Eclipse plugin (PHPUnit with ZFE out of the box) which has a feature allowing developers to run unit tests within the IDE. Both need to be downloaded and placed somewhere PHP can see it (include_path/webroot). As well as both frameworks will allow us to test a systems state plus its behaviours.

After initially going for SimpleTest, ZFW released 1.6RC1 (19/07/08), which now includes testing framework that allows us to test our controllers easier. This is a large factor in the decision making as it now means that by using SimpleTest, we will have to create a simular wrapper to which is already implemented within ZFW already using PHPUnit3. For this reason I prefer to work with PHPUnit, along with ZFW 1.7 giving me functionality needed to test ZF based applications.

I’ve added a couple of links to better explain the concept of stub and mock objects.

Resource Links

http://martinfowler.com/articles/mocksArentStubs.html – An excellent article explaining the difference between stubs & mocks

http://simpletest.org/api/– SimpleTest

http://phpunit.de/pocket_guide/3.3/en/ – PHPUnit3

h1

Why test anyway?

October 27, 2008

Before I start I’d like to point out that this is purely from my perspective and pretty much a subjective point of view. There are so many articles out there from an objective stand point, that I thought I do this from my own, seeing as personal accounts are pretty much one sided & usually only highlight the good points. Here, though I’m obviously a TDD advocate I will try to address the issues I have come across whilst working with the methodology.

I know I’m not the only one who hates spending hours trying to track a bug & have phrases like: “where did that bug come from?” and “i didn’t change that” swish around my brain.

Over the years I’ve learnt that there is no such thing as a perfect solution but have adopted TDD purely because I have always felt that software projects lacked the flexibility of car designers (who test our concepts before making a final decision). Being security conscious and taking pride in my work I thought it would be of benefit to explore the practice first hand.

I’ve always hated pros & cons, especially as some peoples pros are others cons, so I’ll just bullet each methodologies attributes that I am aware of, make of it what you will.

No tests

  • Quick to initially code.
  • No need for test suite.
  • Cab be hard to trace errors & debug.
  • Can leads to misconceptions in design.
  • Changes could break existing code, without knowing and/or introducing new bugs.
  • Developers feel uneasy making changes to the system.
  • Debugging gets left to last or even out entirely due to time restraints.
  • Functionality can get left out or reproduced.

Tests

  • Need a test suite at hand.
  • Able to test each module as its built.
  • Easy to created dependencies within test cases.
  • Ideas can be played with without actually implement production code.
  • Need to adopt a different way of approaching problems.
  • Easy to track when the application is deliverable.
  • Errors/bugs can be picked up quicker.
  • There is always something to test.
  • Easy to recreate rare scenario’s/exception.
  • Can take time to learn how to create useful tests.

With those points out of the way, I’ll explain my reasons for taking the testing route.

Though software development is maturing at a pace we still have an urgent need to track change & check our assumptions. The quicker we can check our assumptions are correct the quicker we’ll find bugs in our software, I’m the first to admit I get bored quick, when my attention wains so does the software I’m coding, whilst testing I find that I become more productive & my focus is more on problem at hand, rather than tracking errors or debugging as most of the errors are expected and handled quickly.

Below is a list of my loves & hates in regards to the testing process, like I mentioned earlier I don’t believe in perfect solutions so I think it prudent not to gloss over the fact that just because something is tested doesn’t mean it does what it is supposed to or that a project will end up better.

Loves

  • Sped up my understanding of the problem at hand.
  • Alerted me to error I’ve over seen.
  • Easy to replicate hard to detect errors & exceptions.
  • Allows me to test ideas before implementing them.
  • Very handy when doing exploratory development (not sure how something works).

Hates

  • Creating useful tests can take time to learn.
  • Just because a test passes, doesn’t mean it is bug free.
  • Is easy to create useless tests.
  • If not careful planned, can find yourself testing more than creating production code.

From personal experience I found that once I grasped the concepts of testing (more so the TDD process, which I’ll go over in later blogs), the quicker I was able to implement functional code, whilst allowing me to test out assumptions I may have had. This along with a few other techniques (mainly design patterns & refactoring) has evolved my programming allowing me to create flexible, reusable code which I end up using in a range of projects.