JUnit and HP ALM Integration Guide¶

Description¶

Bumblebee provides a set of Java annotations which allows you to send your test results into HP ALM from your favorite IDE or CI system.

Technically it consists of three parts:

@Bumblebee annotation
JUnit test run listener: com.agiletestware.bumblebee.annotations.BumblebeeJUnitListener
XML configuration file which contains ALM connection and other details needed for functioning of bumblebee-annotations

Prerequisites¶

Bumblebee Server must be installed and be accessible via HTTP from user machine
Maven project with JUnit tests

Installation and configuration¶

Setting up pom.xml¶

Modify your pom.xml file and add agiletestware repository and bumblebee-annotations dependency:

<dependencies>
  <dependency>
      <groupId>com.agiletestware</groupId>
      <artifactId>bumblebee-annotations</artifactId>
      <version>0.1.6</version>
      <scope>test</scope>
   </dependency>
</dependencies>     
<repositories>
    <repository>
        <id>nexus.agiletestware.com</id>
        <url>https://nexus.agiletestware.com/repository/maven-public</url>
    </repository>
</repositories>

Configuring Maven surefire plugin:¶

<build>
    <plugins>   
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.19</version>
            <configuration>
                <properties>
                    <property>
                        <name>listener</name>
                        <value>com.agiletestware.bumblebee.annotations.BumblebeeJUnitListener</value>
                    </property>
                </properties>
            </configuration>
        </plugin>
    </plugins>
</build>

Creating bumblebee_config.xml¶

Create a bumblebee_config.xml file and put it into your project root folder. Here is a description of configuration file:

<?xml version="1.0"?>
<bumblebee>
  <!-- URL of the Bumblebee Server -->
  <bumblebee_url>http://server_name:port/bumblebee</bumblebee_url>
  <!-- URL of HP ALM Server -->
  <alm_url>http://server_name:port/qcbin</alm_url>
  <!-- Name of HP ALM User -->
  <alm_user>qcuser</alm_user>
  <!-- Encrypted password: please use http://server_name:port/bumblebee/password/encrypt to encrypt your plain text password  -->
  <alm_encrypted_pass>fd4OMOXLJjkMR6e64RJh3Q==</alm_encrypted_pass>
  <!-- HP ALM Domain -->
  <alm_domain>DEFAULT</alm_domain>
  <!-- HP ALM Project -->
  <alm_project>annotations_demo</alm_project>
  <!-- Asynchronous (offline) update -->
  <async_update>false</async_update>
</bumblebee>

Overriding configuration options with Java Properties (since 0.1.3)¶

Due to security or other reasons, you might want to override or replace XML configuration with Java Properties.

The following Java configuration properties are available:

Name	Description
bumblebee.config	Path to a configuration file. If not present, then ${project.basedir}/bumblebee_config.xml is used
bumblebee.config.bumblebee_url	URL of the Bumblebee Server
bumblebee.config.alm_url	URL of ALM Server
bumblebee.config.alm_user	Name of ALM User
bumblebee.config.alm_password	Encrypted user password
bumblebee.config.alm_domain	Name of ALM Domain
bumblebee.config.alm_project	Name of ALM Project
bumblebee.config.async_update	Asynchronous (offline) update
bumblebee.config.take_screenshot	Boolean flag indicating whether a screenshot of a failure should be automatically done (for WebDriver tests only)
bumblebee.config.enabled	Boolean flag indicating whether Bumblebee functionality is enabled or not
bumblebee.config.update_testplan	Boolean flag indicating whether Bumblebee should update existing tests in ALM TestPlan module

To provide values to Maven via command line, please use -D, e.g.

mvn clean test -Dbumblebee.config.bumblebee_url=http://localhost:8080/bumblebee

Note

Values provided with Java Properties override the ones provided in XML configuration.

Add Bumblebee annotations to your JUnit test classes/methods¶

@Bumblebee Java annotation can be added on class or method level When class or method is marked with Bumblebee annotation, its result will be exported to HP ALM.

Bumblebee annotation parameters¶

Name	Description	Applicable	Required
testplan	Path to test in a Test Plan in HP ALM. e.g. Subject\Test1	Class, method	Yes
testlab	Path to test set in a test lab in HP ALM. e.g. Root\Test1	Class, method	Yes
testset	Name of test set in HP ALM	Class, method	Yes
testname	If specified, then the value of testname parameter will be set as the,name of test in HP ALM. If not set, test name will be set to fully,qualified method name, e.g. com.annotations.Demo.method1	Method	No
almid	Defines id of a test in a HP ALM test plan which needs to be updated. If specified then testplan and testname are ignored	Method	No
description	Description for test in HP ALM	Method	No
overwriteAlmSteps	If set to BooleanValue.TRUE, all existing test steps will be deleted	Class, method	No
parameters	An array of custom parameters which are passed to the bumblebee server and then mapped to HP ALM custom fields. Please refer to documentation documentation on how to setup mappings on Bumblebee server	Class, method	No

Bumblebee annotation on method level overrides values set by Bumblebee annotation on class level.

Defining dynamic values for annotation parameters (since 0.1.0)¶

The following parameters can contain dynamic values:

testplan
testlab
testset
testname
description

To define a dynamic value, just add a placeholder in format: ${property_name}, where property_name is the name of property containing value for the parameter.
Bumblebee resolves dynamic values in run time using Java System Properties or XML configuration file.

Providing values with bumblebee_config.xml file¶

To set a value for dynamic parameter in bumblebee_config.xml file, just add new property element. E.g. to provide a value for ${"example"} placeholer

@Bumblebee(testset="Test set: ${example}")

<property name="example">some value</property>

Providing values with Java System Properties¶

To provide a value with Java System Property, just pass then into your Maven build, e.g.

mvn clean test -Dexample=Something

If both XML configuration and Java System Property are provided, value in Java System Property will be used.

Run Maven build¶

To run JUnit tests and send results to HP ALM, just run Maven test phase on your project: e.g.: mvn test

Examples¶

Bumblebee annotation on class level:¶

@Bumblebee(testlab = "Root\\junit_simple", testset = "class_annotations", testplan = "Subject\\junit_simple")
public class SomePassSomeFailTest {

  @Test
  public void test1() {
    Assert.fail("test1 failed");
  }

  @Test
  public void test2() {
    System.out.println("test2 passed");
  }

  @Ignore
  @Test
  public void test3() {
    System.out.println("test3 passed");
  }

  @Test
  public void test4() {

    throw new RuntimeException("test4 failed with exception");
  }

  @Test
  public void test5() {
    System.out.println("test5 passed");
  }
}

After export to HP ALM: Result in HP ALM

Bumblebee annotation on method level:¶

public class SomePassSomeFailTest {

    @Test
    public void test1() {
        Assert.fail("test1 failed");
    }

    @Bumblebee(testname="JUnit test",testlab = "Root\\junit_simple", testset = "method_annotation", testplan = "Subject\\junit_simple")
    @Test
    public void test2() {
        System.out.println("test2 passed");
    }
}

Results in HP ALM:

Result in HP ALM

Overriding annotation values on methods:¶

@Bumblebee(testlab = "Root\\junit_simple", testset = "override_annotations", testplan = "Subject\\junit_simple")
public class SomePassSomeFailTest {

    @Test
    public void test1() {
        Assert.fail("test1 failed");
    }

    @Bumblebee(testname="JUnit test", description="Test description")
    @Test
    public void test2() {
        System.out.println("test2 passed");
    }

    @Ignore
    @Test
    public void test3() {
        System.out.println("test3 passed");
    }

    @Test
    public void test4() {

        throw new RuntimeException("test4 failed with exception");
    }

    @Test
    public void test5() {
        System.out.println("test5 passed");
    }
}

Results:

Result in HP ALM

Map JUnit test method to existing test in HP ALM:¶

Here is the existing Manual test in HP ALM Test Plan with id = 50:

Map to existing test

It contains two design steps:

Map to existing test

To map test method in java class to the test in HP ALM, almid property can be used. Also it is necessary to define testlab and testset properties :

public class UpdateExistingTest {
    @Bumblebee(almid=50, description="Updated description", testlab = "Root\\junit existing test", testset = "map existing test")
    @Test
    public void test1() {
        Assert.fail("test1 failed");
    }
}

Results in HP ALM: Test description in Test Plan is updated with the value of Bumblebee description field:

Result in HP ALM

There is also a new test step with name Test Results:

Result in HP ALM

And results are exported to the HP ALM Test Lab. In such a case, all steps get the same status:

Result in HP ALM

Mapping single test to multiple tests in ALM

Bumblebee almid field can accept an array of test IDs, e.g.: almid={42,24}. In such a case test result will be mapped into two tests in ALM.

Setting custom fields in HP ALM from JUnit class¶

Bumblebee annotation allows users to pass custom parameters to Bumblebee Server which then are mapped to custom user fields in HP ALM. Please refer to documentation on how to setup mappings on Bumblebee server. Parameter name shall match to the "label" attribute of mapping in mapping XML on the server:

<mapping alm_field="TS_USER_01" label="user field"/>

To map some value to TS_USER_01 field in HP ALM, define a parameter with name="user field":

public class UpdateExistingTest {
    @Bumblebee(testplan = "Subject\\custom fields", description = "Test description",
            testlab = "Root\\custom fields", testset = "map custom fields",
            parameters = {@Parameter(name = "user field", value = "value from JUnit") })
    @Test
    public void test1() {
        Assert.fail("test1 failed");
    }
}

After export TS_USER_01 field of the test in HP ALM test plan gets the value from Java class:

Result in HP ALM

Currently Bumblebee supports setting of custom fields for HP ALM Test, Test Set, Run, Design Step, Run Step, Test Instance

Overriding Bumblebee custom fields configuration¶

It is also possible to override server side configuration with the bumblebee XML configuration. To do that, just add a new set of mapping elements into configuration:

<bumblebee>
    <bumblebee_url>http://bbe:8888/bumblebee</bumblebee_url>
    <alm_url>http://alm:8080/qcbin</alm_url>
    <alm_user>user</alm_user>
    <alm_encrypted_pass>fd4OMOXLJjkMR6e64RJh3Q==</alm_encrypted_pass>
    <alm_domain>DEFAULT</alm_domain>
    <alm_project>demo</alm_project>

    <mapping field_type="test" field_label="user field" bumblebee_label="test user field" default_value="something"/>   
</bumblebee>

mapping element has the following attributes:

field_type - a type of entity in ALM for which mapping is provided. The following are supported:
- test - test in TestPlan
- test-set - test set in TestLab
- test-instance - test instance in TestLab
- run-step - step of a test in TestPlan
- step - step of a test in a run in TestLab
- run - run in TestLab
field_label - label of a field in ALM (not a system name as in mapping configuration on server), e.g. 'Status' or 'User field', etc...
bumblebee_label - this is a label which should be used as @Parameter.name to pass a different value
default_value - default value

E.g. if you wish to provide value for test's "User field", the following element should be used:

<mapping field_type="test" field_label="user field" bumblebee_label="test user field" default_value="something"/>

When running Maven build, the following notification is displayed in the console output:

Bumblebee: Passing mappings configuration to the server: [Mapping [fieldLabel=user field, fieldType=TEST, defaultValue=something, bumblebeeLabel=test user field]]
Bumblebee: Mapping configuration on Bumblebee Server will be ignored

And results in ALM:

Adding arbitrary attachments¶

To add an arbitrary attachment to a run in ALM, Bumblebee provides a static utility class com.agiletestware.bumblebee.annotations.util.CurrentTest with a set of methods:

addAttachment(File) - add a single file
addAttachment(String, InputStream) - add a single attachment from a stream
addAttachments(List) - add list of files

Here is an example of adding attachments:

@Bumblebee(testlab = "Root\\junit_simple", testset = "class_annotations", testplan = "Subject\\junit_simple")
public class SomePassSomeFailTest {

    @Test
    public void test1() {
        CurrentTest.addAttachment(new File("test.log"));
    }
}

Results in ALM:

Adding arbitrary test steps (since 0.1.1)¶

To add additional test steps to test in ALM, Bumblebee provides com.agiletestware.bumblebee.annotations.util.CurrentTest.addStep(String) method which returns a builder object which provides convenient way to set attributes of step.

@Bumblebee(testlab = "Root\\junit_simple", testset = "class_annotations", testplan = "Subject\\junit_simple")
public class SomePassSomeFailTest {

    @Test
    public void test_additional_step() {
        final Step step = CurrentTest.addStep("Additional step")
                .description("description")
                .expected("expected")
                .actual("actual")
                .status(Status.FAILED)
                .build();
        step.addAttachment(new File("test.log"));
    }
}

Results in ALM:

Adding log output into test step's actual (since 0.1.3)¶

To add some log messages into test step's actual field, use com.agiletestware.bumblebee.annotations.util.CurrentTest.log(String) method:

import org.junit.Test;

import com.agiletestware.bumblebee.annotations.Bumblebee;
import com.agiletestware.bumblebee.annotations.util.CurrentTest;

@Bumblebee(testlab = "Root\\junit_simple", testset = "class_annotations", testplan = "Subject\\junit_simple")
public class SomePassSomeFailTest {

    @Test
    public void test1() {
        CurrentTest.log("Log line one");
        CurrentTest.log("Log line two");
    }
}

Results in ALM:

Disabling update of existing tests in TestPlan (since 0.1.3)¶

Sometimes it is not desired to update your existing test in TestPlan, but just export results into TestLab. For this purpose, Bumblebee offers an update_testplan configuration switch. By default, it is set to true, so Bumblebee updates test in TestPlan for each run. To disable update, just add true element into bumblebee XML configuration:

<bumblebee>
    <bumblebee_url>http://bbe:8888/bumblebee</bumblebee_url>
    <alm_url>http://alm:8080/qcbin</alm_url>
    <alm_user>user</alm_user>
    <alm_encrypted_pass>fd4OMOXLJjkMR6e64RJh3Q==</alm_encrypted_pass>
    <alm_domain>DEFAULT</alm_domain>
    <alm_project>demo</alm_project>
    <update_testplan>false</update_testplan>
</bumblebee>

In such a case, Bumblebee will not update existing tests in TestPlan and add a new Comments sections into a run in TestLab:

while existing test steps will get No Run status:

Disabling Bumblebee functionality¶

There are two ways to disable Bumblebee functionality:

Disable exporting of all test results into HP ALM¶

To disable exporting all test results into HP ALM, just add <enabled>false</enabled> into bumblebee_config.xml file:

<?xml version="1.0"?>
    <bumblebee>
      <!-- URL of the Bumblebee Server -->
      <bumblebee_url>http://server_name:port/bumblebee</bumblebee_url>
      <!-- URL of HP ALM Server -->
      <alm_url>http://server_name:port/qcbin</alm_url>
      <!-- Name of HP ALM User -->
      <alm_user>qcuser</alm_user>
      <!-- Encrypted password: please use http://server_name:port/bumblebee/password/encrypt to encrypt your plain text password  -->
      <alm_encrypted_pass>fd4OMOXLJjkMR6e64RJh3Q==</alm_encrypted_pass>
      <!-- HP ALM Domain -->
      <alm_domain>DEFAULT</alm_domain>
      <!-- HP ALM Project -->
      <alm_project>annotations_demo</alm_project>
      <enabled>false</enabled>
    </bumblebee>

During the execution the following message will be shown in console output: Bumblebee: Bumblebee has been disabled in configuration → Results will not be exported to HP ALM

Disable exporting results for a particular test class or method¶

To disable exporting results for a particular test the one just need to add enabled = BooleanValue.FALSE attribute to Bumblebee annotation:

@Bumblebee(testlab = "Root\\junit_simple", testset = "class_annotations", testplan = "Subject\\junit_simple", enabled = BooleanValue.FALSE)
    public class SomePassSomeFailTest {

      @Test
      public void test1() {
        Assert.fail("test1 failed");
      }
    }