ExUnit is fine for small, simple applications, but when you want to do more complex test cases, it has limitations. ShouldI provides nested contexts to eliminate duplication in tests, and has better support for naming tests based on behavior. This API is based on the shoulda framework for Ruby on Rails.
Just add the hex dependency to your mix file:
defpdepsdo[{:shouldi,only: :test}]endand add
...useShouldI...to your test script in place of
...useExUnit.Case...When you're testing behavior, you can get better names with a more descriptive macro. The test code...
test"should return ok on parse"doassert:ok==Parser.parseend...can become more descriptive and shorter with...
should"return :ok on parse"doassert:ok==Parser.parseendSay you have a test case that needs some setup. ExUnit has support for a context that can be set once, and passed to all clients. You can use the setup method to pass a map to each of your test cases, like this:
defmoduleMyFlatTestdosetupcontextdo{:ok,Dict.putcontext,:necessary_key,:necessary_value}endtest("this test needs :necessary_key",context)doassertcontext.necessary_key==:necessary_valueendendThis approach breaks down when several, but not all, tests need the same set of values. ShouldI solves this problem with nested contexts, which you can provide with the having keyword, like this:
defmoduleMyFatTestdohaving"necessary_key"dosetupcontextdoDict.putcontext,:necessary_key,:necessary_valueendshould("have necessary key",context)doassertcontext.necessary_key==:necessary_valueendendhaving"sometimes_necessary_key"dosetupcontextdoDict.putcontext,:sometimes_necessary_key,:sometimes_necessary_valueendshould("have necessary key",context)doassertcontext.sometimes_necessary_key==:sometimes_necessary_valueendendendThis approach is much nicer than the alternatives when you're testing something like a controller with dramatically different requirements across tests:
having"a logged in user"dosetupcontextdologincontext,userend...endhaving"a logged out user"do...endhaving"a logged in admin"dosetupcontextdologincontext,adminend...endassign is a macro that is syntactic sugar for updating the context.
setupcontextdoDict.putcontext,:necessary_key,:necessary_valueendbecomes
setupcontextdoassigncontext,necessary_key: :necessary_valueendYou can package macros that write your own tests. Matchers encode common assertion patterns. For example, our plug matchers
having"a logged in admin"dosetupcontextdologincontext,adminendhaving"a get to :index"dosetupcontextdo# process getendshould_respond_with:successshould_match_body_to"<html>"endendThe two matchers, should_respond_with and should_match_body_to, will run in a single test, against the context created in the setup function (or setup functions, if you've used multiple contexts). Even if both of these tests fail, you'll see two failures in your output.
We come prepackaged with a set of matchers, but you can code your own as well. The following is the matcher to check for existence of a dictionary key in the context:
defmatchershould_assign_key([{key,value}])doquotedoassertvar!(context)[unquote(key)]==unquote(value)endendThis macro allows you to build a matcher macro.
We'll have more information about creating matchers later. In the mean time, you can read through the matchers we've created in the project. Matchers should be stateless, as all matchers within a having clause will run to completion, unless there is an error, even if a test fails.
- Context
should_assign_key key, value: assert that the value forkeyin the context isvalueshould_match_key key, expected: assert that the value forkeyin the context satisfies the pattern matchexpectedshould_have_key key: assert thatkeyexists in the contextshould_not_have_key key: assert thatkeydoes not exist in the context
- Plug
should_respond_with expected: Assert that the value forcontext.connection.statusin the context matches a reasonable value for:success,:redirect,:bad_request,:unauthorized,:missingor:errorshould_match_body_to expected: Assert that the value forcontext.resp_bodycontains the textexpected.
When running tests asynchronously it can be useful to have a way to generate IDs or names that will not conflict with other tests that run concurrently. uid() will generate an ID unique for the current test and setup. If it is called again during the same test it will return the same ID. An additional string can be given uid("some string") so multiple IDs can be generated during the same test.
The philosophy is that experiments go in setup and measurements go into matchers. shouldi will make sure that the context is passed between them cleanly so that things compose correctly.
When you run a shouldi test, for each context:
- one
shouldtest is created, collecting all of the matchers in ahavingclause. - one exunit
testis created for eachshouldblock - for each test
- all of the ancestor
setupfunctions will fire, from outermost to innermost.
- all of the ancestor
- the test will fire
- if a test is a matcher test, all of the matchers will run to completion, even if there is a failure, stopping only on errors.
- if a test is a
shouldblock, the first failure will halt the test, as inExUnit.
- if a test is a
Happy testing. Open an issue if there are any matchers you'd like to see. Feedback and pull requests are welcome. Send a pull request if you'd like to contribute.
Special thanks to ThoughtBot's shoulda, which formed the foundation for this approach.