Step 4. Plug your tests into Continuous Integration

Any automated solution would be incomplete without plugging it into a Continuous Integration server. Hudson (recently forked as Jenkins) is one of the most popular CI servers in the Java world. Hudson/Jenkins is very easy to deploy and it could be run under non-administrative Windows account. Because you are going to drive real browsers, you would need a Windows machine to drive IE. However, if your main server is running on a different OS, you can deploy a secondary server, even on a Windows desktop, using Hudson/Jenkins master-slave architecture.

Cucumber is easy to plug into Hudson/Jenkins because it is a command-line tool and it can produce execution reports in JUnit format.

My choice: Hudson/Jenkins

Other choices: CruiseControl, CruiseControl.NET, Apache Continuum

What to do:

• Add Cucumber to your automated build. Make sure to fail the build if the exit code is not 0.
  Here is an example for Ant:

  <macrodef name="cuke" xmlns:c="antlib:net.sf.antcontrib">
      <attribute name="env"/>
      <sequential>
          <c:var name="-cuke.parameters" value="--tags ~@wip --format pretty --format junit --out build/testreport/xml" />
          <c:if><not><equals arg1="@{env}" arg2="LOCAL"/></not><then>
            <c:var name="-cuke.parameters" value="${-cuke.parameters} USER=${user} 'PWD=${pwd}'" />
          </then></c:if>
  
          <c:if><os family="windows"/><then>
              <property name="-cuke.runner" value="cucumber.bat" />
              <echo>
  Testing in IE (Windows only)
              </echo>
              <exec executable="${-cuke.runner}" osfamily="windows" failonerror="true">
                  <!-- <arg value="..."> is parsed incorrectly -->
                  <arg line="--profile @{env}" />
                  <arg line="--profile ie" />
                  <arg line="${-cuke.parameters}" />
              </exec>
          </then><else>
              <property name="-cuke.runner" value="cucumber" />
          </else></c:if>
  
          <echo>
  Testing in Firefox
          </echo>
          <exec executable="${-cuke.runner}" failonerror="true">
              <!-- <arg value="..."> is parsed incorrectly -->
              <arg line="--profile @{env}" />
              <arg line="--profile firefox" />
              <arg line="${-cuke.parameters}" />
          </exec>
  
      </sequential>
  </macrodef>

• You might have noticed that this macro passes profile parameters and USER and PWD variables to Cucumber. Profiles in Cucumber are simply handy combinations of variables and parameters. USER, PWD and other VARIABLE=value parameters are passed to Cucumber glue code as environment variables and are accessible via ENV["VARIABLE"]Ruby construct. Create couple of profiles suitable for your environment:

  # Cucumber YAML configuration
  ---
  # == Default profile ==
  default: --profile DEV --profile firefox
  
  # == Environments ==
  # In order to use DEV environment, you must pass USER='{app user}' PWD='{password}'
  DEV: APP_ROOT_URL=http://devhost/app
  LOCAL: APP_ROOT_URL=http://localhost:8500/app
  
  # == Browsers ==
  firefox: BROWSER=Fx
  ie: BROWSER=IE
  keep_browser_open: STAY_OPEN=Y

  Save it into config/cucumber.yml.

• Now, all you have to do in Hudson/Jenkins is to configure a job launching your build as usually. Pass correct parameters and publish Cucumber test results along with other automated test results:

Enjoy!

Update (Feb 4, 2012): Updated some obsolete information. Added hyperlinks.