Sometimes for loops containing Catch sections segfault/internal assert fail

[jamespharvey20]

Description

Sometimes for loops containing Catch sections execute fine, sometimes they don't. I've been writing some like this without running into this before, so something's unique about this layout. Granted, this sample code could easily be "unrolled" so there's no for loop , or be implemented using a function or fixture. But, I've found it useful to be able to do this especially when deep in a BDD structure. And, the part in the for loop can be quite large, and the cases can be more, so unrolling can be painful.

Steps to reproduce

This compiles but fails to execute. Without debug, it segfaults. With debug, it fails an internal catch assertion: catch.hpp:5983: void Catch::TestCaseTracking::TrackerBase::moveToParent(): Assertion `m_parent' failed.

#include <catch.hpp>

TEST_CASE("") {
   for(int a = 0; a < 2; ++a) {
      SECTION("1") {
         SECTION("A") {
         }
         SECTION("B") {
         }
      }
   }
}

Moving the for loop into the first section allows it to compile and successfully execute.

#include <catch.hpp>

TEST_CASE("") {
   SECTION("1") {
   for(int a = 0; a < 2; ++a) {
         SECTION("A") {
         }
         SECTION("B") {
         }
      }
   }
}

But, moving the failing code within a section still fails to execute.

#include <catch.hpp>

TEST_CASE("") {
   SECTION("Added") {
   for(int a = 0; a < 2; ++a) {
         SECTION("1") {
            SECTION("A") {
            }
            SECTION("B") {
            }
         }
      }
   }
}

Extra information

• Catch version: Sort of v1.7.1 (current git master: 4d0cd60) -- I tried a really old commit from 12 months ago, and it still failed.
• Operating System: Arch Linux
• Compiler+version: clang++ 3.9.1 & g++ 6.3.1

[jamespharvey20]

Actually, moving this into a function, class, or fixture doesn't work either, which is another thing I've done a few times. These segfault/fail the internal assertion as well:

#include <catch.hpp>

void func() {
   SECTION("1") {
      SECTION("A") {
      }
      SECTION("B") {
      }
   }
}

TEST_CASE("") {
   func();
   func();
}

#include <catch.hpp>

class foo{
public:
   foo() {
      SECTION("1") {
         SECTION("A") {
         }
         SECTION("B") {
         }
      }
   }
};

TEST_CASE("") {
   foo A{};
   foo B{};
}

#include <catch.hpp>

class fixture{
public:
   fixture() {
      SECTION("1") {
         SECTION("A") {
         }
         SECTION("B") {
         }
      }
   }
};

TEST_CASE_METHOD(fixture, "A") {
   fixture A{};
}

TEST_CASE_METHOD(fixture, "B") {
   fixture B{};
}

[philsquared]

The short answer is: this is not supported (I really need to make this clear in the docs).
The way forward will be generators - but they've been deferred to Catch2 now.

But you can workaround this: you just need to make the section names unique. What I have done, the couple of times I've needed to do similar, is to form the section name string to include the loop counter - e.g.:

TEST_CASE("") {
   for(int a = 0; a < 2; ++a) {
      SECTION("1. + std::to_string( a )) {
         SECTION("A." + std::to_string( a )) {
         }
         SECTION("B." + std::to_string( a )) {
         }
      }
   }
}

[philsquared]

@jamespharvey20 does this resolve your issue?

[stervo4ka]

Isn't it possible to allow a section execute multiple times during one test case run? As I understand, catch runs the same test case multiple times disabling sections one by one, but it should be ok to let a section execute more than once during one such run, shouldn't it?

[stervo4ka]

Another helpful feature would be some kind of a scoped context automatically appending a string to sections names.

[philsquared]

@stervo4ka although it's probably technically possible to execute the same section multiple times during a single run I think this goes against the spirit of what sections are for - which is to say that each leaf section should be executed exactly once.

The scoped context idea is interesting. I might consider that. Do bear in mind though that this is really all just workarounds for missing generators.