Tests and issue trackers – how to manage the integration

In highly disciplined teams when a bug is discovered the following happens:

  • test (or most likely a set of tests) is written that fails exposing the bug
  • a ticket in issue tracking system is created
  • a developer fixes the bug, runs all the tests including the new ones and if everything is green, the ticket is resolved

My question is what if some time later (say several weeks after) a developer wants to find which issue relates to the tests he’s looking at, or which tests were documenting the bug he’s looking at. How do you manage the link between the tests and issues in your tracker?

Solution one – file per ticket

Quite common solution to this problem is to have a special folder in your test project and for each ticket have a file named after the ticket’s ID like this:

Tests solution

That works and is quite easy to discover. However the downside of this solution is that it introduces fragmentation. If a bug was found in a shipping module does it really make sense to keep some tests for shipping module in ShippingModuleTests class and some other in CRM_4 class merely because the latter ones were discovered by a tester and not original developer?

Solution two – ticket id in method name

To alleviate that another solution is often used. The tests end up in ShippingModuleTests bug the id of the issue is encoded in the name of the test method. like this:

[Test]
public void CRM_4_Gold_preferred_customer_should_have_his_bonus_applied_to_net_price()
{
   //some logic here
}

[Test]
public void CRM_4_Silver_preferred_customer_should_have_his_bonus_applied_to_net_price()
{
   //some logic here
}

That’s a step in the right direction. It makes the link explicit and you can quickly navigate the relation either direction. However I don’t like it very much, because most of the time I couldn’t care less about the fact that this test documents a long fixed bug. Yet I am constantly reminded about it every time I run my tests.

tests

Solution three – description

The solution I found myself using the most recently is to leverage the description most testing frameworks let you associate with your tests.

[Test(Description = "CRM_4")]
public void Gold_preferred_customer_should_have_his_bonus_applied_to_net_price()
{
   //some logic here
}

[Test(Description = "CRM_4")]
public void Silver_preferred_customer_should_have_his_bonus_applied_to_net_price()
{
   //some logic here
}

This still makes the association explicit and searchable, but doesn’t remind me of it constantly where I don’t care.

tests

What about you? What approach do you employ to manage that?

  • I think quite common approach is to use ticket reference/id in VCS commit message – something like “Fixed #123 NullReferenceException when logging-in with empty username”. In this way you can easily search in VCS commit history and also create some commit-hooks etc which can associate your commits with related task/bug in your issue tracker so as you can see commits related to an issue directly in your tracker. I like this approach with TargetProcess or Trac we used few years ago…

    • Yes, I do use that approach too, and YouTrack and Teamcity integrate with it nicely, but that’s a different problem from the one discussed in the post.

      • I don’t think it’s different problem – it’s just about the tool you use to locate the corresponding issue to the test you’re looking at or vice versa. If you use some VCS integration tool in VS you can easily use it to find tickets related to a test class file in commit history or perform the opposite search directly from VS similarly as you’d search over test descriptions as you suggest in the third option in your post. I didn’t mean opening issue tracker and finding related source code from there 🙂 I use this approach for quite long time and quite like it so far – I don’t need to manage the code vs issue reference on two places…