[Yanel-dev] running Canoo tests independently
Guillaume Déflache
guillaume.deflache at wyona.com
Fri Mar 26 19:55:18 CET 2010
Hi!
While fixing a lot of tests I suffered again the pain of having to run
the full test suite of a project just to check the outcome of a single
test because they are all interdependent.
(Fortunately on that project at least thanks to memo we have an
automatic script to reset the application data to the initial state.)
One small inexpensive improvement could be to distinguish independent
tests from the non-independent ones so that one would know at least when
running the full suite is not needed.
This had been done on this project and kick-started for the 'use-cases'
realms (which AFAICU is supposed to contain the Canoo tests that are not
related to a particular realm or may involve several realms), cf.
http://svn.wyona.com/repos/public/yanel/trunk/src/realms/use-cases/yanel/src/test/canoo/tests/allTests.xml
One further more expensive improvement I recently discussed offline with
memo would be to try leverage Ant target dependency tracking (e.g. the
'depends' attribute of targets) to make dependencies between Canoo tests
explicit.
That would allow running a non-independent test alone by automatically
executing its dependencies first. This may probably need new cleaning
targets at the beginning of the dependency chain, probably more focused
but that could still be shared by several tests as a kind of common test
data subset.
I really the explanation is pretty cryptic, I will try to come with more
concrete examples if there is some interest.
The underpinning idea here is that making every test fully
self-contained is a lot of work (either lots of copy-paste or a very
sophisticated organization of test data) and may slow down the whole
test suite so much it will not be practically anyway.
So IMHO we have to find a middle ground if we want to keep adding more
and more end-to-end tests.
HTH,
Guillaume
More information about the Yanel-development
mailing list