[Yanel-dev] running Canoo tests independently

Guillaume Déflache guillaume.deflache at wyona.com
Fri Mar 26 19:55:18 CET 2010


Hi!

While fixing a lot of tests I suffered again the pain of having to run 
the full test suite of a project just to check the outcome of a single 
test because they are all interdependent.
(Fortunately on that project at least thanks to memo we have an 
automatic script to reset the application data to the initial state.)


One small inexpensive improvement could be to distinguish independent 
tests from the non-independent ones so that one would know at least when 
running the full suite is not needed.
This had been done on this project and kick-started for the 'use-cases' 
realms (which AFAICU is supposed to contain the Canoo tests that are not 
related to a particular realm or may involve several realms), cf. 
http://svn.wyona.com/repos/public/yanel/trunk/src/realms/use-cases/yanel/src/test/canoo/tests/allTests.xml


One further more expensive improvement I recently discussed offline with 
memo would be to try leverage Ant target dependency tracking (e.g. the 
'depends' attribute of targets) to make dependencies between Canoo tests 
explicit.
That would allow running a non-independent test alone by automatically 
executing its dependencies first. This may probably need new cleaning 
targets at the beginning of the dependency chain, probably more focused 
but that could still be shared by several tests as a kind of common test 
data subset.

I really the explanation is pretty cryptic, I will try to come with more 
concrete examples if there is some interest.


The underpinning idea here is that making every test fully 
self-contained is a lot of work (either lots of copy-paste or a very 
sophisticated organization of test data) and may slow down the whole 
test suite so much it will not be practically anyway.
So IMHO we have to find a middle ground if we want to keep adding more 
and more end-to-end tests.

HTH,
    Guillaume


More information about the Yanel-development mailing list