Initial nose experience

I've been using unittest for many years, but have steadfastly (perhaps
stubbornly) avoided newfangled improvements like nose. I finally
decided to take a serious look at nose. There were a few pain points I
had to work through to get our existing collection of tests to run under
nose. I figured I'd share them for the benefit of others who may be
going through the same process.

First nose won't import executable files, at least not by default.

All of our test files are executable, with a "#!/usr/bin/env python"
line at the top, and a "if __name__ == '__main__'" block, which does
some setup and invokes unittest.main(), at the bottom. Going to the top
of our source tree and typing "nosetests" was an uninspiring experience:

> $ nosetests
>
> ----------------------------------------------------------------------
> Ran 0 tests in 0.001s
>
> OK

The fix is either make them non-executable, or do "nosetests --exe".

Next up was the the setup in the "if __name__ == '__main__'" block
wasn't running. The solution here is to move all the setup to
setUpModule(), where it belongs. SetUpModule() is new in Python 2.7 but
it turns out it's trivial to drop that version into older systems
(http://pypi.python.org/pypi/unittest2).

We found a bunch of tests which require some specific setup before they
could run (most blatantly, some selenium tests which depend on X11).
When we were running tests semi-manually, that was not a big deal. With
nose's "find them all and run them" strategy, this fails. The obvious
fix is that every test needs to either set up the right environment, or
be protected with the appropriate @skip decorator so it doesn't run if
the environment isn't good.

Lastly, nose, by default, doesn't say much. When things go wrong and
you have no clue what's happening, --verbose and --debug are your
friends.

Hi Roy,

> I've been using unittest for many years, but have steadfastly
(perhaps stubbornly) avoided newfangled improvements like nose.
I finally decided to take a serious look at nose.

Thanks for sharing your nose experience.

What motivated you to migrate from unittest to nose?

After years of using unittest, what would you say are the pros and
cons of nose?

Thank you,
Malcolm

In article <mailman.2149.1342375358.4697.python-list@python.org>,
python@bdurham.com wrote:

> Hi Roy,
>
> > I've been using unittest for many years, but have steadfastly
> (perhaps stubbornly) avoided newfangled improvements like nose.
> I finally decided to take a serious look at nose.
>
> Thanks for sharing your nose experience.
>
> What motivated you to migrate from unittest to nose?

Mostly I was just looking for a better way to run our existing tests.
We've got a bunch of tests written in standard unittest, but no good way
to start at the top of the tree and run them all with a single command.

On 07/15/2012 08:58 PM, Roy Smith wrote:
>> What motivated you to migrate from unittest to nose?
> Mostly I was just looking for a better way to run our existing tests.
> We've got a bunch of tests written in standard unittest, but no good way
> to start at the top of the tree and run them all with a single command.

Currently, $ python -m unittest does nothing useful (afaik). Would it
break anything to look in . , ./test, ./tests for any files matching
test_* , and execute those?

- Philipp


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEAREKAAYFAlAD8tEACgkQ9eq1gvr7CFwZkACeI5ItZTw3co gMyF0R88p8aTPM
DbgAn0KAcoqy5/0UqQPPXF5VZWDrcf0L
=r1wp
-----END PGP SIGNATURE-----

In article <mailman.2149.1342375358.4697.python-list@python.org>,
python@bdurham.com wrote:

> After years of using unittest, what would you say are the pros and
> cons of nose?

BTW, although I'm currently using nose just as a unittest aggregator, I
can see some nice advantages to native nose functionality. The most
obvious is that tests can be plain-old static functions at the top level
of a module.

In unittest, you have to subclass TestCase, then write methods for that
(showing its JUnit/SUnit roots). In 99% of the tests I write, I don't
do anything special in my TestCase subclasses, so that's all just
boilerplate busywork. I like the idea that I can skip that all now.

Philipp Hagemeister wrote:

> Currently, $ python -m unittest does nothing useful (afaik). Would it
> break anything to look in . , ./test, ./tests for any files matching
> test_* , and execute those?

http://docs.python.org/library/unittest#test-discovery

On 07/16/2012 01:47 PM, Peter Otten wrote:
> http://docs.python.org/library/unittest#test-discovery

That's precisely it. Can we improve the discoverability of the discover
option, for example by making it the default action, or including a
message "use discover to find test files automatically" if there are no
arguments?

- Philipp


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEAREKAAYFAlAECw8ACgkQ9eq1gvr7CFzGfgCgsAbj1OfYWM EzBX5acuJcSELV
ejAAoLsCs7nPQ9eQhD9bSqUq43JzwUAx
=062U
-----END PGP SIGNATURE-----

Roy Smith <roy@panix.com> writes:

> In article <mailman.2149.1342375358.4697.python-list@python.org>,
> python@bdurham.com wrote:
>
> > After years of using unittest, what would you say are the pros and
> > cons of nose?
>
> BTW, although I'm currently using nose just as a unittest aggregator

Be aware that Python 2.7 and higher has unit test discovery in the
standard library:

$ python -m unittest discover

will look through all packages within the current directory and collect
unit tests it finds, then run them and report
<URL:http://docs.python.org/library/unittest.html#test-discovery>.

You can get the same in earlier versions of Python with ‘unittest2’
<URL:http://pypi.python.org/pypi/unittest2>.

--
\ “I prayed for twenty years but received no answer until I |
`\ prayed with my legs.� —Frederick Douglass, escaped slave |
_o__) |
Ben Finney

Philipp Hagemeister <phihag@phihag.de> writes:

> On 07/16/2012 01:47 PM, Peter Otten wrote:
> > http://docs.python.org/library/unittest#test-discovery
>
> That's precisely it. Can we improve the discoverability of the discover
> option

Having it prominently in the documentation for the ‘unittest’ module is
already pretty discoverable, IMO.

> for example by making it the default action, or including a message
> "use discover to find test files automatically" if there are no
> arguments?

Well, “we� includes you; feel free to make a patch to that effect and
submit a bug report requesting it, if you think it's needed
<URL:http://bugs.python.org/>.

--
\ “Do not enter the lift backwards, and only when lit up.� |
`\ —elevator, Leipzig |
_o__) |
Ben Finney

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iQIcBAEBCAAGBQJQBAz1AAoJELiyTAasEoQFGqcQAMWRp5pqRN vTLHT1AUaJVEf4
NtcxEcD1H/HAge7+Z8VD7eTbNLaJlaGFW7UVX4ZN0eVL3++ex50ORh8zi6Bm cJDJ
1UE1g+D8a8mbkk/GQUe0rOvJEqT70pnyILisqfh9Lp81rBg84Sqgip4VocKUeC/L
lWkRZn8oIj2MY6ch/1l+O2BPuaANMFL4G/DDa9JMTtoGtKdC3sE+vEReypB1XYfG
INVRyk1wsIjOydjVJ4YTkN5jxYZJR12PV70yrhwB7jYVLr42lH Rg1YS701ooQYxl
ezdn3YzvqWbgBupjJ4FQsfiZ1NQHBjFBPRBHF2Fetq74eBtsLR 0YJH5U2ZVdyIqY
Tyz+R7GAjWPrZuVrVolIqDGj+WpHlSJ+sXZxz14HF5VrOEXBST S16Pq1WGIvulkp
8A+PGEx1MjJv19Dy8t285fgL2iCxK00S/+TURsaaeetIAHm2WKZU0c4WLObqJlrN
jvIYCcRYpLxDENQZwG7LsvzRctIT7rOeEl5bm4/ax5zU9lRWWkMXcbunPeXob98Z
dcYhN965N4tLFcRUDI1yi67e9caOfu4ltfqsVK293NSAqxtJmj lytnsjiyrLxWJm
0RncfoG8WsF7MR6IU9XWY5PCUOWPbcjarl2WsfSvA+oGa69wLH ahA+CNqdvWBLrj
8nrbAeDVAB5Dt68JzR5J
=tooM
-----END PGP SIGNATURE-----

On 07/16/2012 02:37 PM, Philipp Hagemeister wrote:
> Can we improve the discoverability of the discover
> option, for example by making it the default action, or including a
> message "use discover to find test files automatically" if there are no
> arguments?
Oops, already implemented as of Python 3.2. Sorry, should've checked before.

- Philipp


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEAREKAAYFAlAEDcQACgkQ9eq1gvr7CFzxKACgu/ynpNNx9TxZB5HXjNWenqIX
BCcAnRAOgkAaaPmxRjQGO+WOAaIGV62Q
=wfvU
-----END PGP SIGNATURE-----

Does nose run all of its collected tests in a single process?

I've got a test which monkey-patches an imported module. Will all of the other tests collected in the same run of nosetests see the patch?

In article <roy-F2685A.08422113072012@news.panix.com>,
Roy Smith <roy@panix.com> wrote:

> Lastly, nose, by default, doesn't say much. When things go wrong and
> you have no clue what's happening, --verbose and --debug are your
> friends.

I found another example of nose not saying much, and this one is kind of
annoying. Unittest has much richer assertions, and better reporting
when they fail. If a nose assertion fails, you just get:

------------------------------------------------------------------
Traceback (most recent call last):
File
"/home/roy/production/python/lib/python2.6/site-packages/nose/case.py",
line 197, in runTest
self.test(*self.arg)
File "/home/roy/songza/pyza/djapi/test_middleware.py", line 48, in
test_update_non_json_cookie
assert user_list == [9990]
AssertionError
------------------------------------------------------------------

Under unittest, it would have printed the value of user_list. Yeah, I
know, I can stick a "print user_list" statement into the test, and the
output will get suppressed if the test fails. But that means when a
test fails, I need to go back and edit the test code, which is a pain.

On the other hand, there *is* an incremental efficiency gain of writing:

assert x == y

instead of

assertEqual(x, y)

many times. So maybe overall it's a wash.