AndroidTestingBox

Android project to experiment various testing tools. It targets Java and Kotlin languages. Priority is given to fluency and ease of use. The idea is to provide a toolbox to write elegant and intelligible tests, with modern techniques like behavior-driven testing frameworks or fluent assertions.

• AndroidTestingBox in the news
• System under test (SUT)
  • Simple Java class
  • Android Activity
• JUnit
  • Fluent assertions: truth
    • Alternative: AssertJ
  • Frutilla
  • Fluent test method names
  • Specifications framework: Spectrum
    • Alternative: Oleaster
  • Hierarchies in JUnit: junit-hierarchicalcontextrunner
    • Novelty to consider: JUnit 5 Nested Tests
  • BDD tools
    • Cucumber
    • JGiven
  • Mutation testing: Zester plugin
  • Alternative to JUnit: TestNG
• Kotlin
  • Fluent assertions: Kluent
    • Alternative: Expekt
  • Specifications framework: Spek
• Android
  • Fluent assertions: AssertJ Android
  • Robotium
  • Espresso
  • Robolectric
  • Cucumber support
  • JGiven support
• IDE configuration
• Nota Bene
• Bibliography
• Interesting repositories
• Interesting articles
• Resources
• Logo credits

AndroidTestingBox in the news

• Android Weekly #242

System under test (SUT)

Simple Java class

publicclassSum {
publicfinalinta;
publicfinalintb;
privatefinalLazyInitializer<Integer> mSum;
publicSum(inta, intb) {
this.a = a;
this.b = b;
mSum = newLazyInitializer<Integer>() {
@OverrideprotectedIntegerinitialize() throwsConcurrentException {
returnSum.this.a + Sum.this.b;
}
};
}
publicintgetSum() throwsConcurrentException {
returnmSum.get();
}
}

Android Activity

Here stands the layout file:

<?xml version="1.0" encoding="utf-8"?>
<RelativeLayoutandroid:id="@+id/activity_main"xmlns:android="http://schemas.android.com/apk/res/android"android:layout_width="match_parent"android:layout_height="match_parent">
<TextViewandroid:id="@+id/ActivityMain_TextView"android:layout_width="wrap_content"android:layout_height="wrap_content"android:layout_centerInParent="true"android:text="@string/app_name"/>
<Buttonandroid:id="@+id/ActivityMain_Button"android:layout_width="wrap_content"android:layout_height="wrap_content"android:layout_below="@id/ActivityMain_TextView"android:layout_centerHorizontal="true"android:text="@string/click_me"/>
</RelativeLayout>

and here stands the corresponding Activity:

classMainActivity : AppCompatActivity() {
overridefunonCreate(savedInstanceState:Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_main)
val textView:TextView= findViewById(R.id.ActivityMain_TextView) asTextViewval button = findViewById(R.id.ActivityMain_Button)
button.setOnClickListener({ view:View-> textView.setText(R.string.text_changed_after_button_click) })
}
}

JUnit

Fluent assertions: truth

• https://google.github.io/truth/

Alternative: AssertJ

• http://joel-costigliola.github.io/assertj/

Frutilla

• https://github.com/ignaciotcrespo/frutilla

@RunWith(value = org.frutilla.FrutillaTestRunner.class)
publicclassFrutillaSumTest {
@Frutilla(
Given = "two numbers a = 1 and b = 3",
When = "computing the sum of these 2 numbers",
Then = "should compute sum = 4"
)
@Testpublicvoidtest_addition_isCorrect() throwsException {
given("two numbers", () -> {
finalinta = 1;
finalintb = 3;
when("computing the sum of these 2 numbers", () -> {
finalSumsum = newSum(a, b);
then("should compute sum = 4", () -> assertThat(sum.getSum()).isEqualTo(4));
});
});
}
}

Fluent test method names

• https://jijeshmohan.wordpress.com/2011/12/17/junit-readable-test-names/

Specifications framework: Spectrum

• https://github.com/greghaskins/spectrum
• http://www.greghaskins.com/archive/introducing-spectrum-bdd-style-test-runner-for-java-junit

importstaticcom.google.common.truth.Truth.assertThat;
importstaticcom.greghaskins.spectrum.Spectrum.describe;
importstaticcom.greghaskins.spectrum.Spectrum.it;
@RunWith(Spectrum.class)
publicclassSpectrumSumTest {
{
describe("Given two numbers a = 1 and b = 3", () -> {
finalinta = 1;
finalintb = 3;
it("computing the sum of these 2 numbers, should compute sum = 4", () -> {
finalSumsum = newSum(a, b);
assertThat(sum.getSum()).isEqualTo(4);
});
});
}
}

Alternative: Oleaster

• https://github.com/mscharhag/oleaster

Hierarchies in JUnit: junit-hierarchicalcontextrunner

• https://github.com/bechte/junit-hierarchicalcontextrunner

@RunWith(HierarchicalContextRunner.class)
publicclassHCRSumTest {
publicclassGivenTwoNumbers1And3 {
privateinta = 1;
privateintb = 3;
@BeforepublicvoidsetUp() {
a = 1;
b = 3;
}
publicclassWhenComputingSum {
privateSumsum;
@BeforepublicvoidsetUp() {
sum = newSum(a, b);
}
@TestpublicvoidthenShouldBeEqualTo4() throwsConcurrentException {
assertThat(sum.getSum()).isEqualTo(4);
}
}
publicclassWhenMultiplying {
privateintmultiply;
@BeforepublicvoidsetUp() {
multiply = a * b;
}
@TestpublicvoidthenShouldBeEqualTo3() throwsConcurrentException {
assertThat(multiply).isEqualTo(3);
}
}
}
}

Novelty to consider: JUnit 5 Nested Tests

• http://junit.org/junit5/docs/current/user-guide/#writing-tests-nested
• The @Nested and @DisplayName annotations allow developers to reach an elegant "given/when/then" canvas

BDD tools

Cucumber

• https://cucumber.io/

• Define the .feature file:

Feature: Sum computationScenario Outline: Sum 2 integersGiven two int <a> and <b> to sum
When computing sum
Then it should be <sum>Examples:
 | a | b | sum | | 1 | 3 | 4 | | -1 | -3 | -4 | | -1 | 3 | 2 |

• Define the corresponding steps:

publicclassSumSteps {
SummoSum;
intmiSum;
@Given("^two int (-?\\d+) and (-?\\d+) to sum$")
publicvoidtwoIntToSum(finalinta, finalintb) {
moSum = newSum(a, b);
}
@When("^computing sum$")
publicvoidcomputingSum() throwsConcurrentException {
miSum = moSum.getSum();
}
@Then("^it should be (-?\\d+)$")
publicvoiditShouldBe(finalintexpected) {
Assert.assertEquals(expected, miSum);
}
}

• Define the specific runner:

@RunWith(Cucumber.class)
@CucumberOptions(
features = "src/test/resources/"
)
publicclassSumTestRunner {
}

• Relevant tools:
  • to write Gherkin features: Tidy Gherkin
  • to display Gherkin features in Chrome a way pretty way: Pretty Gherkin
  • to generating specifications from Gherkin source files: featurebook

JGiven

• http://jgiven.org/

publicclassJGivenSumTestextendsSimpleScenarioTest<JGivenSumTest.TestSteps> {
@Testpublicvoidaddition_isCorrect() throwsConcurrentException {
given().first_number_$(1).and().second_number_$(3);
when().computing_sum();
then().it_should_be_$(4);
}
publicstaticclassTestStepsextendsStage<TestSteps> {
privateintmA;
privateintmB;
privateSummSum;
publicTestStepsfirst_number_$(finalintpiA) {
mA = piA;
returnthis;
}
publicvoidsecond_number_$(finalintpiB) {
mB = piB;
}
publicvoidcomputing_sum() {
mSum = newSum(mA, mB);
}
publicvoidit_should_be_$(finalintpiExpected) throwsConcurrentException {
assertThat(mSum.getSum()).isEqualTo(piExpected);
}
}
}

Mutation testing: Zester plugin

• https://plugins.jetbrains.com/plugin/8281
• https://tech.zalando.com/blog/zester-mutation-testing/

For this sample project, define a new "Run configuration" with Zester such as:

Target classes: com.guddy.android_testing_box.zester.*
Test class: com.guddy.android_testing_box.zester.ZesterExampleTest

It generates an HTML report in the build/reports/zester/ directory, showing that 2 "mutants" survived to unit tests (so potential bugs, and in this case, yes it is).

Alternative to JUnit: TestNG

• http://testng.org/doc/index.html

Kotlin

Fluent assertions: Kluent

• https://github.com/MarkusAmshove/Kluent

Alternative: Expekt

• https://github.com/winterbe/expekt

Specifications framework: Spek

• https://github.com/JetBrains/spek
• http://jetbrains.github.io/spek/docs/latest/#setting-up

@RunWith(JUnitPlatform::class)
classSpekSumTest : Spek({
given("two numbers a = 1 and b = 3") {
vala:Int = 1valb:Int = 3
on("computing the sum of these 2 numbers") {
valsum:Sum = Sum(a, b)
it("should compute sum = 4") {
sum.sum shouldBe 4
}
}
}
})

Android

Fluent assertions: AssertJ Android

• http://square.github.io/assertj-android/

Robotium

• https://github.com/RobotiumTech/robotium

@RunWith(AndroidJUnit4.class)
publicclassMainActivityTest {
//region Rule@RulepublicfinalActivityTestRule<MainActivity> mActivityTestRule = newActivityTestRule<>(MainActivity.class, true, false);
//endregion//region FieldsprivateSolomSolo;
privateMainActivitymActivity;
privateContextmContextTarget;
//endregion//region Test lifecycle@BeforepublicvoidsetUp() throwsException {
mActivity = mActivityTestRule.getActivity();
mSolo = newSolo(InstrumentationRegistry.getInstrumentation(), mActivity);
mContextTarget = InstrumentationRegistry.getTargetContext();
}
@AfterpublicvoidtearDown() throwsException {
mSolo.finishOpenedActivities();
}
//endregion//region Test methods@TestpublicvoidtestTextDisplayed() throwsException {
given("the main activity", () -> {
when("launching activity", () -> {
mActivity = mActivityTestRule.launchActivity(null);
then("should display 'app_name'", () -> {
finalbooleanlbFoundAppName = mSolo.waitForText(mContextTarget.getString(R.string.app_name), 1, 5000L, true);
assertThat(lbFoundAppName);
});
});
});
}
//endregion
}

Espresso

• https://google.github.io/android-testing-support-library/docs/espresso/

Robolectric

• http://robolectric.org/
• http://robolectric.org/using-add-on-modules/

 testCompile 'org.robolectric:robolectric:3.2.2'
testCompile 'org.robolectric:shadows-multidex:3.2.2'
testCompile 'org.robolectric:shadows-support-v4:3.2.2'
testCompile 'org.khronos:opengl-api:gl1.1-android-2.1_r1'

@RunWith(RobolectricTestRunner.class)
@Config(constants = BuildConfig.class)
publicclassRobolectricMainActivityTest {
@Testpublicvoidtest_clickingButton_shouldChangeText() throwsException {
given("The MainActivity", () -> {
finalMainActivityloActivity = Robolectric.setupActivity(MainActivity.class);
finalButtonloButton = (Button) loActivity.findViewById(R.id.ActivityMain_Button);
finalTextViewloTextView = (TextView) loActivity.findViewById(R.id.ActivityMain_TextView);
when("clicking on the button", () -> {
loButton.performClick();
then("text should have changed", () -> assertThat(loTextView.getText().toString()).isEqualTo("Text changed after button click"));
});
});
}
}

Cucumber support

• https://github.com/cucumber/cucumber-jvm/tree/master/examples/android

• Configure the build.gradle file:

android {
defaultConfig {
testApplicationId "com.guddy.android_testing_box.ui"
testInstrumentationRunner "com.guddy.android_testing_box.ui.CucumberInstrumentationRunner"
}
sourceSets {
androidTest {
assets.srcDirs = ['src/androidTest/assets']
}
}
}

• Write features in the src/androidTest/assets directory, for example this main.feature file:

Feature: Main activityScenario: Click on the buttonGiven the initial state is shown
When clicking on the button
Then the text changed to "Text changed after button click"

• Define the corresponding steps:

@CucumberOptions(features = "features")
publicclassCucumberMainActivityStepsextendsActivityInstrumentationTestCase2<MainActivity> {
publicCucumberMainActivitySteps() {
super(MainActivity.class);
}
@Given("^the initial state is shown$")
publicvoidthe_initial_main_activity_is_shown() {
// Call the activity before each test.getActivity();
}
@When("^clicking on the button$")
publicvoidclicking_the_Click_Me_button() {
onView(withId(R.id.ActivityMain_Button)).perform(click());
}
@Then("^the text changed to \"([^\"]*)\"$")
publicvoidtext_$_is_shown(finalStrings) {
onView(withId(R.id.ActivityMain_TextView)).check(matches(withText(s)));
}
}

• Define the specific runner:

publicclassCucumberInstrumentationRunnerextendsMonitoringInstrumentation {
privatefinalCucumberInstrumentationCoremInstrumentationCore = newCucumberInstrumentationCore(this);
@OverridepublicvoidonCreate(Bundlearguments) {
super.onCreate(arguments);
mInstrumentationCore.create(arguments);
start();
}
@OverridepublicvoidonStart() {
super.onStart();
waitForIdleSync();
mInstrumentationCore.start();
}
}

JGiven support

• http://jgiven.org/userguide/#_android
• https://github.com/TNG/JGiven/tree/master/example-projects/android

@RunWith(AndroidJUnit4.class)
publicclassEspressoJGivenMainActivityTestextendsSimpleScenarioTest<EspressoJGivenMainActivityTest.Steps> {
@Rule@ScenarioStatepublicActivityTestRule<MainActivity> activityTestRule = newActivityTestRule<>(MainActivity.class);
@RulepublicAndroidJGivenTestRuleandroidJGivenTestRule = newAndroidJGivenTestRule(this.getScenario());
@Testpublicvoidclicking_ClickMe_changes_the_text() {
given().the_initial_main_activity_is_shown()
.with().text("AndroidTestingBox");
when().clicking_the_Click_Me_button();
then().text_$_is_shown("Text changed after button click");
}
publicstaticclassStepsextendsStage<Steps> {
@ScenarioStateCurrentStepcurrentStep;
@ScenarioStateActivityTestRule<MainActivity> activityTestRule;
publicStepsthe_initial_main_activity_is_shown() {
// nothing to do, just for reportingreturnthis;
}
publicStepsclicking_the_Click_Me_button() {
onView(withId(R.id.ActivityMain_Button)).perform(click());
returnthis;
}
publicStepstext(@QuotedStrings) {
returntext_$_is_shown(s);
}
publicStepstext_$_is_shown(@QuotedStrings) {
onView(withId(R.id.ActivityMain_TextView)).check(matches(withText(s)));
takeScreenshot();
returnthis;
}
privatevoidtakeScreenshot() {
currentStep.addAttachment(
Attachment.fromBinaryBytes(ScreenshotUtils.takeScreenshot(activityTestRule.getActivity()), MediaType.PNG)
.showDirectly());
}
}
}

IDE configuration

• MoreUnit plugin: https://plugins.jetbrains.com/plugin/7105

Nota Bene

A relevant combination of Dagger2 and mockito is already described in a previous post I wrote: http://roroche.github.io/AndroidStarter/

Bibliography

• https://blog.codecentric.de/en/2016/01/writing-better-tests-junit/
• https://www.petrikainulainen.net/programming/unit-testing/3-reasons-why-we-should-not-use-inheritance-in-our-tests/
• http://blog.xebia.com/mutation-testing-how-good-are-your-unit-tests/

Interesting repositories

• https://github.com/googlesamples/android-testing
• https://github.com/TNG/JGiven/tree/master/jgiven-examples
• https://github.com/ahus1/bdd-examples
• https://github.com/chiuki/android-test-demo

Interesting articles

• https://www.philosophicalhacker.com/post/some-resources-for-learning-how-to-test-android-apps/
• https://www.sitepoint.com/property-based-testing-with-javaslang/
• https://medium.com/@fabioCollini/android-testing-using-dagger-2-mockito-and-a-custom-junit-rule-c8487ed01b56
• https://offbeattesting.com/2017/04/13/unit-test/

Resources

• https://www.petrikainulainen.net/writing-clean-tests/
• https://www.petrikainulainen.net/category/weekly/

Logo credits

Science graphic by Pixel perfect from Flaticon is licensed under CC BY 3.0. Made with Logo Maker