NUnit and HP ALM Integration Guide

Description

Bumblebee provides a set of C# attributes which allows you to send your test reports directly from Visual Studio to HP ALM

Technically it consists of two parts:

• BumblebeeTestFixture and BumblebeeTest attributes
• XML configuration which is a part of your application configuration file.

Prerequisites

• Agiletestware Bumblebee Server must be installed and be accessible via HTTP from user machine
• Visual Studio 2012 or later with NUnit3TestAdapter installed
• NuGet
• NUnit 3

For running examples with Selenium WebDriver the following NuGet packages are needed:

• Selenium.WebDriver
• Selenium.WebDriver.ChromeDriver

Installation and configuration

1) Add "AgileTestware.Bumblebee.NUnit", "NUnit", "NUnit3TestAdapter" NuGet packages to your solution:

2) Add a new section and Bumblebee elements into your App.config file:

<configSections>
    <section name="Bumblebee"
             type="AgileTestware.Bumblebee.Core.Configurations.BumblebeeSection, AgileTestware.Bumblebee.Core"
             allowDefinition="Everywhere"
             allowLocation="true"/>  
</configSections>

<Bumblebee          
    bumblebee_url="http://bumblebee_server:port/bumblebee"          
    alm_url="http://alm_server:port/qcbin"
    alm_user="user"
    alm_encrypted_pass="fd4OMOXLJjkMR6e64RJh3Q=="
    alm_domain="DEFAULT"
    alm_project="project">
</Bumblebee>

Where

• bumblebee_url - URL of Bumblebee Server
• alm_url - URL of HP ALM server
• alm_user - name of a user in HP ALM
• alm_encrypted_pass - encrypted password for HP ALM user, please use http://bumblebee_server:port/bumblebee/password/encrypt to encrypt your plain text password
• alm_domain - domain name in HP ALM
• alm_project - project name in HP ALM

Add BumblebeeTestFixture and BumblebeeTest attributes to your NUnit test classes

In order to tell Bumblebee to export results of test class execution, that class needs to be marked with BumblebeeTestFixture attribute and all test methods shall be marked with BumblebeeTest attributes. If test method is not marked with BumblebeeTest attribute, its result will not be uploaded to HP ALM.

using AgileTestware.Bumblebee.NUnit.Attributes;
using NUnit.Framework;
using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
using OpenQA.Selenium.Support.UI;
using System;
using System.Collections.Generic;
using System.Collections.ObjectModel;

namespace Agiletstware.Demo.NUnit
{
    [BumblebeeTestFixture(TestPlan = "Subject\\Folder1\\Folder2", TestLab = "Root\\Folder1\\Folder2", TestSet = "Class attributes")]
    [TestFixture]
    public class ClassAttributesDemo
    {
        private IWebDriver webDriver;

        [SetUp]
        public void SetUp()
        {
            webDriver = new ChromeDriver();
        }

          [BumblebeeTest(Description ="Test that google returns Agiletestware.com as a first result for HP ALM and JUnit integration query")]
        [Test]
        public void SearchForTheBestIntegrationTool()
        {
            webDriver.Navigate().GoToUrl("http://google.com");
            IWebElement element = webDriver.FindElement(By.Name("q"));
            element.SendKeys("HP ALM and JUnit integration");
            element.Submit();
            IWebElement myDynamicElement = (new WebDriverWait(webDriver, TimeSpan.FromSeconds(1))
           .Until((d) =>
           {
               IWebElement el = d.FindElement(By.Id("resultStats"));
               if (el.Displayed && el.Enabled)
               {
                   return el;
               }
               return null;
           }));
            ReadOnlyCollection<IWebElement> findElements = webDriver.FindElements(By.XPath("//*[@id='rso']//h3/a"));
            Assert.True(findElements.Count > 0);
            Assert.True(findElements[0].GetAttribute("href").Contains("agiletestware"), "Ooops, we are not on the first place");
        }

        [BumblebeeTest]
        [Test]
        public void SomeFailingTest()
        {
            webDriver.Navigate().GoToUrl("http://google.com");
            webDriver.FindElement(By.LinkText("this is indeed a bad text for a link"));            
        }

        [TearDown]
        public void TearDown()
        {
            if (webDriver != null)
            {
                webDriver.Close();
                webDriver.Quit();
            }
        }
    }
}

BumblebeeTestFixture and BumblebeeTest attributes

BumblebeeTestFixture and BumblebeeTest attributes have the following properties:

Name	Description	Applicable	Required
TestPlan	Path to a test folder in HP ALM TestPlan where tests shall be created, e.g. Subject\Folder1\Folder2	Class, method	Yes
TestLab	Path to a test lab folder in HP ALM TestLab where test set shall be created, e.g. Root\Folder1\Folder2	Class, method	Yes
TestSet	Name of a test set in HP ALM TestLab to be created/updated	Class, method	Yes
TestName	If specified, then the value of TestName parameter will be set as the,name of test in HP ALM. If not set, test name will be set to method name	Method	No
AlmId	Defines id of a test in a HP ALM test plan which needs to be updated. If specified then TestPlan and TestName are ignored	Method	No
Description	Description for test in HP ALM	Method	No
OverwriteAlmSteps	If set to True, all existing test steps will be deleted	Class, method	No
Parameters	JSON array of objects containing name and value for custom parameters which are passed to the bumblebee server and then mapped to HP ALM custom fields, e.g. [{'Name':'parameter 1','Value':'value 1'}, {'Name':'parameter 2','Value':'value 2'}]. Please refer to documentation documentation on how to setup mappings on Bumblebee server	Class, method	No
AlmReqIds	An array of HP ALM requirements IDs to which this test should be linked	Class, method	No
AlmReqRecursive	If set to True, test is linked to all child requirements of requirements defined y AlmReqIds parameter	Class, method	No

If the same property is defined on both class and method attribute, the method one takes precedence.

Exporting test results into HP ALM

Once your tests are marked with BumblebeeTestFixture and BumblebeeTest attributes, each time they are executed with NUnit adapter, results are automatically exported into HP ALM.

Examples

Export results of all test methods

using AgileTestware.Bumblebee.NUnit.Attributes;
using NUnit.Framework;
using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
using OpenQA.Selenium.Support.UI;
using System;
using System.Collections.Generic;
using System.Collections.ObjectModel;

namespace Agiletstware.Demo.NUnit
{
    [BumblebeeTestFixture(TestPlan = "Subject\\Folder1\\Folder2", TestLab = "Root\\Folder1\\Folder2", TestSet = "Class attributes")]
    [TestFixture]
    public class NunitDemoTest
    {
        private IWebDriver webDriver;

        [SetUp]
        public void SetUp()
        {
            webDriver = new ChromeDriver();
        }

        [BumblebeeTest]
        [Test]
        public void SearchForTheBestIntegrationTool()
        {
            webDriver.Navigate().GoToUrl("http://google.com");
            IWebElement element = webDriver.FindElement(By.Name("q"));
            element.SendKeys("HP ALM and JUnit integration");
            element.Submit();
            IWebElement myDynamicElement = (new WebDriverWait(webDriver, TimeSpan.FromSeconds(1))
           .Until((d) =>
           {
               IWebElement el = d.FindElement(By.Id("resultStats"));
               if (el.Displayed && el.Enabled)
               {
                   return el;
               }
               return null;
           }));
            ReadOnlyCollection<IWebElement> findElements = webDriver.FindElements(By.XPath("//*[@id='rso']//h3/a"));
            Assert.True(findElements.Count > 0);
            Assert.True(findElements[0].GetAttribute("href").Contains("agiletestware"), "Ooops, we are not on the first place");
        }

        [BumblebeeTest]
        [Test]
        public void SomeFailingTest()
        {
            webDriver.Navigate().GoToUrl("http://google.com");
            webDriver.FindElement(By.LinkText("this is indeed a bad text for a link"));            
        }

        [TearDown]
        public void TearDown()
        {
            if (webDriver != null)
            {
                webDriver.Close();
                webDriver.Quit();
            }
        }
    }
}

Results in HP ALM TestPlan:

TestLab:

Overriding class attributes on a particular method

Bumblebee allows you to override values defined in BumblebeeTestFixture for a single test method by setting values to BumblebeeTest attribute properties:

using AgileTestware.Bumblebee.NUnit.Attributes;
using NUnit.Framework;
using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
using OpenQA.Selenium.Support.UI;
using System;
using System.Collections.Generic;
using System.Collections.ObjectModel;

namespace Agiletstware.Demo.NUnit
{
    [BumblebeeTestFixture(TestPlan = "Subject\\Folder1\\Folder2", TestLab = "Root\\Folder1\\Folder2", TestSet = "Class attributes")]
    [TestFixture]
    public class NunitDemoTest
    {
        private IWebDriver webDriver;

        [SetUp]
        public void SetUp()
        {
            webDriver = new ChromeDriver();
        }

        [BumblebeeTest(TestName = "Overridden test name", Description = "Test that google returns Agiletestware.com as a first result for HP ALM and JUnit integration query")]
        [Test]
        public void SearchForTheBestIntegrationTool()
        {
            webDriver.Navigate().GoToUrl("http://google.com");
            IWebElement element = webDriver.FindElement(By.Name("q"));
            element.SendKeys("HP ALM and JUnit integration");
            element.Submit();
            IWebElement myDynamicElement = (new WebDriverWait(webDriver, TimeSpan.FromSeconds(1))
           .Until((d) =>
           {
               IWebElement el = d.FindElement(By.Id("resultStats"));
               if (el.Displayed && el.Enabled)
               {
                   return el;
               }
               return null;
           }));
            ReadOnlyCollection<IWebElement> findElements = webDriver.FindElements(By.XPath("//*[@id='rso']//h3/a"));
            Assert.True(findElements.Count > 0);
            Assert.True(findElements[0].GetAttribute("href").Contains("agiletestware"), "Ooops, we are not on the first place");
        }

        [BumblebeeTest]
        [Test]
        public void SomeFailingTest()
        {
            webDriver.Navigate().GoToUrl("http://google.com");
            webDriver.FindElement(By.LinkText("this is indeed a bad text for a link"));            
        }

        [TearDown]
        public void TearDown()
        {
            if (webDriver != null)
            {
                webDriver.Close();
                webDriver.Quit();
            }
        }
    }
}

Results in HP ALM TestPlan:

TestLab:

Setting custom fields in HP ALM from NUnit class

BumblebeeTestFixture and BumblebeeTest attributes allow users to pass custom parameters to Bumblebee Server which then are mapped to custom user fields in HP ALM. Please refer to documentation on how to setup mappings on Bumblebee server. Parameter name shall match to the "label" attribute of mapping in mapping XML on the server:

<mapping alm_field="TS_USER_01" label="user field"/>

To map some value to TS_USER_01 field in HP ALM, define a parameter with name="user field":

using AgileTestware.Bumblebee.NUnit.Attributes;
using NUnit.Framework;
using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
using OpenQA.Selenium.Support.UI;
using System;
using System.Collections.Generic;
using System.Collections.ObjectModel;

namespace Agiletstware.Demo.NUnit
{
    [BumblebeeTestFixture(TestPlan = "Subject\\Folder1\\Folder2", TestLab = "Root\\Folder1\\Folder2", TestSet = "Class attributes", Parameters = "[{'Name':'user field','Value':'some value'}]")]
    [TestFixture]
    public class NunitDemoTest
    {
        private IWebDriver webDriver;

        [SetUp]
        public void SetUp()
        {
            webDriver = new ChromeDriver();
        }

        [BumblebeeTest]
        [Test]
        public void SearchForTheBestIntegrationTool()
        {
            webDriver.Navigate().GoToUrl("http://google.com");
            IWebElement element = webDriver.FindElement(By.Name("q"));
            element.SendKeys("HP ALM and JUnit integration");
            element.Submit();
            IWebElement myDynamicElement = (new WebDriverWait(webDriver, TimeSpan.FromSeconds(1))
           .Until((d) =>
           {
               IWebElement el = d.FindElement(By.Id("resultStats"));
               if (el.Displayed && el.Enabled)
               {
                   return el;
               }
               return null;
           }));
            ReadOnlyCollection<IWebElement> findElements = webDriver.FindElements(By.XPath("//*[@id='rso']//h3/a"));
            Assert.True(findElements.Count > 0);
            Assert.True(findElements[0].GetAttribute("href").Contains("agiletestware"), "Ooops, we are not on the first place");
        }

        [TearDown]
        public void TearDown()
        {
            if (webDriver != null)
            {
                webDriver.Close();
                webDriver.Quit();
            }
        }
    }
}

After export TS_USER_01 field of the test in HP ALM test plan gets the value from your class:

Currently Bumblebee supports setting of custom fields for HP ALM Test, Test Set, Run, Design Step, Run Step, Test Instance

You can also override Parameters property for individual test method by setting its value in corresponding BumblebeeTest attribute.

Mapping tests to HP ALM requirements

Bumblebee annotations allows users to map their test classes/methods to HP ALM requirements with a help of two parameters: * AlmReqIds - an array of requirements IDs in HP ALM * AlmReqRecursive - defines if test should be also linked to child requirements

e.g.:

using AgileTestware.Bumblebee.NUnit.Attributes;
using NUnit.Framework;
using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
using OpenQA.Selenium.Support.UI;
using System;
using System.Collections.Generic;
using System.Collections.ObjectModel;

namespace Agiletstware.Demo.NUnit
{
    [BumblebeeTestFixture(TestPlan = "Subject\\Folder1\\Folder2", TestLab = "Root\\Folder1\\Folder2", TestSet = "Class attributes", AlmReqIds = new int[] {1, 3}, AlmReqRecursive = true)]
    [TestFixture]
    public class NunitDemoTest
    {
        private IWebDriver webDriver;

        [SetUp]
        public void SetUp()
        {
            webDriver = new ChromeDriver();
        }

        [BumblebeeTest ]
        [Test]
        public void SearchForTheBestIntegrationTool()
        {
            webDriver.Navigate().GoToUrl("http://google.com");
            IWebElement element = webDriver.FindElement(By.Name("q"));
            element.SendKeys("HP ALM and JUnit integration");
            element.Submit();
            IWebElement myDynamicElement = (new WebDriverWait(webDriver, TimeSpan.FromSeconds(1))
           .Until((d) =>
           {
               IWebElement el = d.FindElement(By.Id("resultStats"));
               if (el.Displayed && el.Enabled)
               {
                   return el;
               }
               return null;
           }));
            ReadOnlyCollection<IWebElement> findElements = webDriver.FindElements(By.XPath("//*[@id='rso']//h3/a"));
            Assert.True(findElements.Count > 0);
            Assert.True(findElements[0].GetAttribute("href").Contains("agiletestware"), "Ooops, we are not on the first place");
        }

        [TearDown]
        public void TearDown()
        {
            if (webDriver != null)
            {
                webDriver.Close();
                webDriver.Quit();
            }
        }
    }
}

After running tests, Bumblebee creates test in HP ALM and maps it to the requirements with ID 1 and 3, and also to their children:

If requirement mappings are specified on a class-level annotation, they will be applied to all test methods of the class.

Map NUnit test method to existing test in HP ALM:

Here is the existing Manual test in HP ALM Test Plan with id = 16:

It contains two design steps:

To map test method in C# class to the test in HP ALM, AlmId property can be used. Also it is necessary to define TestLab and TestSet properties :

using AgileTestware.Bumblebee.NUnit.Attributes;
using NUnit.Framework;
using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
using OpenQA.Selenium.Support.UI;
using System;
using System.Collections.Generic;
using System.Collections.ObjectModel;

namespace Agiletstware.Demo.NUnit
{
    [BumblebeeTestFixture(TestPlan = "Subject\\Folder1\\Folder2", TestLab = "Root\\Folder1\\Folder2", TestSet = "Demo set")]
    [TestFixture]
    public class ClassAttributesDemo
    {
        private IWebDriver webDriver;

        [SetUp]
        public void SetUp()
        {
            webDriver = new ChromeDriver();
        }

        [BumblebeeTest(AlmId = 16, Description = "Test that google returns Agiletestware.com as a first result for HP ALM and JUnit integration query")]
        [Test]
        public void SearchForTheBestIntegrationTool()
        {
            webDriver.Navigate().GoToUrl("http://google.com");
            IWebElement element = webDriver.FindElement(By.Name("q"));
            element.SendKeys("HP ALM and JUnit integration");
            element.Submit();
            IWebElement myDynamicElement = (new WebDriverWait(webDriver, TimeSpan.FromSeconds(1))
           .Until((d) =>
           {
               IWebElement el = d.FindElement(By.Id("resultStats"));
               if (el.Displayed && el.Enabled)
               {
                   return el;
               }
               return null;
           }));
            ReadOnlyCollection<IWebElement> findElements = webDriver.FindElements(By.XPath("//*[@id='rso']//h3/a"));
            Assert.True(findElements.Count > 0);
            Assert.True(findElements[0].GetAttribute("href").Contains("agiletestware"), "Ooops, we are not on the first place");
        }

        [TearDown]
        public void TearDown()
        {
            if (webDriver != null)
            {
                webDriver.Close();
                webDriver.Quit();
            }
        }
    }
}

Results in HP ALM: Test description in Test Plan is updated with the value of Bumblebee description field:

There is also a new test step with name Test Results:

And results are exported to the HP ALM Test Lab:

Disabling export of results to HP ALM

Sometimes exporting of test results into HP ALM is not desired (e.g. during debugging tests by developers), so it can be easily switched off by modifying Bumblebee section in App.config file.
Just add isEnabled="false" attribute to Bumblebee section and save App.config file.
e.g.:

<Bumblebee          
    bumblebee_url="http://bumblebee_server:port/bumblebee"          
    alm_url="http://alm_server:port/qcbin"
    alm_user="user"
    alm_encrypted_pass="fd4OMOXLJjkMR6e64RJh3Q=="
    alm_domain="DEFAULT"
    alm_project="project"
    isEnabled="false">
</Bumblebee>

During test execution you will get the following record in test output: Skipping HP ALM update. Bumblebee is disabled.

Adding attachments to test run in HP ALM (since 1.1.3)

In some cases it could be beneficial to have some test data like screenshots or logs accessible in HP ALM.

To add an arbitrary attachment to your test results in HP ALM you can use AgileTestware.Bumblebee.NUnit.CurrentTest class which provides AddAttachment method.
AddAttachment method takes an instance of AgileTestware.Bumblebee.Core.Attachment as a parameter and Attachment class provides several convenient static methods for creating attachments:

• FromFile - creates an attachments from some local file
• FromStream - creates attachments from Stream
• FromByteArray - creates attachments from byte array

Here is a code snippet for adding some attachment to HP ALM:

using AgileTestware.Bumblebee.Core;
using AgileTestware.Bumblebee.NUnit;
using AgileTestware.Bumblebee.NUnit.Attributes;
using NUnit.Framework;
using System.Text;

namespace Agiletestware.Demo.NUnit
{
    [TestFixture]
    [BumblebeeTestFixture(TestPlan = "Subject\\NUnit\\Attachments", TestLab = "Root\\NUnit", TestSet = "Attachments Demo")]
    class SimpleAttachmentsDemo
    {
        [BumblebeeTest]
        [Test]
        public void AddSomeAttachmentTest()
        {
            //do something
            CurrentTest.AddAttachment(Attachment.FromByteArray(Encoding.ASCII.GetBytes("Some content"), "attachment.txt", "text/plain"));
            //do something
        }
    }
}

Results in HP ALM:

!

Adding test steps to HP ALM test (since 1.1.4)

Bumblebee provides functionality for adding test steps to HP ALM programmatically from test code.
By default, Bumblebee creates one synthetic test step with "Test Result" name which reflects execution status of NUnit test method. To add more steps, just call AddTestStep method of AgileTestware.Bumblebee.NUnit.CurrentTest class:

using AgileTestware.Bumblebee.Core;
using AgileTestware.Bumblebee.NUnit;
using AgileTestware.Bumblebee.NUnit.Attributes;
using NUnit.Framework;
using System.Text;

namespace Agiletestware.Demo.NUnit
{
    [TestFixture]
    [BumblebeeTestFixture(TestPlan = "Subject\\NUnit\\Steps", TestLab = "Root\\NUnit", TestSet = "Steps Demo")]
    class AddStepsDemo
    {


        [BumblebeeTest(TestName = "Test steps", Description = "Steps demo", OverwriteAlmSteps =true)]
        [Test]
        public void TestFirstGoogleResultIsReallyAboutPizza()
        {                        
            TestStep step = CurrentTest.AddTestStep("Step One", "Description for the first step", AgileTestware.Bumblebee.Core.Enums.TestCaseStatus.Failed);
            step.Expected = "Expected";
            step.Actual = "Actual";
            step.Error = "Some error";
            //Add attachment                
            step.Attachments.Add(Attachment.FromByteArray(Encoding.ASCII.GetBytes("Some content"), "attachment.txt", "text/plain"));

            CurrentTest.AddTestStep("Step Two", "Description for the second step", AgileTestware.Bumblebee.Core.Enums.TestCaseStatus.Passed);                                    
        }

    }
}

Results in HP ALM:

---