It looks like there are two issues at hand here.
- Should there be one test module per real module? My opinion is that the
answer is generally yes, but there are cases where this would be awkward,
so we shouldn't make it a hard rule. For example, `test_pickle.py` tests
the picklability of various objects, from simple ones to entire figures.
Of course, each of the tests could be moved to the test module
corresponding to the module where the object is defined, but I doubt this
would make things clearer.
- Should there be a `matplotlib.tests` package containing all the tests
(possibly replicating the matplotlib hierarchy) or one `.tests` subpackage
per package? Let's realize that matplotlib has a very flat hierarchy
compared to the other projects Nelle mentioned. Basically, the change
would entail moving
- test_axes to axes/tests
- test_backend_* to backends/tests
- test_cbook to cbook/tests
- test_triangulation to tri/tests
- test_style to style/tests
The other subpackages are projections and sphinxext, for which there does
not appear to be a dedicated test module. Frankly, which such a small
number of exceptions, I feel it would be more confusing to move the
corresponding tests out to their own subpackage. The situation is not the
same as in numpy, scipy, skimage, pandas, sklearn and sympy, which (I just
checked) all have more subpackages (i.e. folders) than submodules (i.e.
files) directly under the main package. (IPython has more direct
submodules, but many of them seem to be just shims left over from the big
split). For these projects, subpackages provide a clear partitioning
between parts of the code, and it is thus natural to split the tests in the
same manner.
Antony
2017-01-28 17:40 GMT-08:00 Matthias Bussonnier <bussonniermatthias at gmail.com
:
> numpy, scipy, skimage, pandas, sklearn, sympy, ipython all follow this
standard.
We do our best in IPython to try to "catch up" on standard, and right
now we are not the best example.
Like matplotlib our test harness is relatively complex (for different
reasons than matplotlib). But we are doing
our best to push things forward. Migrating to PyTest is one of the
things we hope to one day achieve (or already start).
"The bit split" helped, but in 2014 out tests layout was kind of horrible.
I am not involved enough in matplotlib to tell you whether you should
(or not) decide to change (that's your choice, and only core dev can
likely correctly weight on cost/benefits balance), but IMHO, and the
experience I have:
- With IPython : non traditional package layout and test harness
make it difficult for new/occasional contributors.
- E.g in IPython a couple of time per year, we have people
running `nosetest` from the root of the directory.
- we have a specific guard against running iptest from source
directory.
- contributing to our test suite is harder
That does not seem much, but integrating something small through time
can lead to quite a lot.
- You may not realise it but a lot of people when wondering how to
do things will look up to "core" packages.
- They either will not understand, potentially giving up.
- Or will replicate a complex setup
That add to the myth that "python packaging is hard", and I think that
weighting on the overall community impact is important.
I definitely feel a "layout context switching" when working on
matplotlib. Test layout is not the only one, but it definitively
interfere with my editor which smart search on nearby files.
Catching up on "Standard" does not mean either changing all at once.
AFAIU you do that for pep8 where the number of violation can only go down,
so you can perfectly start a policy:
- new test need to be next to thing they test.
Sidenote: I don't see why a testing submodule should not be importable.
Don't you have utility function that would be useful for others ?
Wouldn't it bother you that only a test submodule can be imported but
only by testing tools ?
I'm going to guess "just post 2.0" might not be a good time to
actually shuffle things around, but looking at "what would it break"
(beyond people habits) can be a way to see a couple of assumptions
that are not supposed to be here*, and lay out the foundation to
moving things around if one day you decide to.
Thanks !
--
Matthias
[*] source: PyCon 2014, I deleted the test suite. IPython was refusing
to start.
>>> Actually, it is really a pain that Matplotlib doesn't follow python
>>> standards for any contributors that contributes to any other python
>>> packages.
>>
>>
>> very true, but...
>>
>> 1) If we were starting from scratcth that would be one thing, but a lot
of
>> thrash jsut to meet a "standard" is probably not worth it.
>
>
> The benefits of following standards for maintainability has been
> demonstrated over and over.
>
>>
>> 2) I don't know how standardized this concept is, and personally, I
don''t
>> like it (if I understand it correctly, anyway)
>
>
> It is extremely standard. Matplotlib is not the only project not
following
> standards, but it's probably in the 1% of projects not following
standards.
> More importantly, it is how we (the community) teach how to organize
> packages and tests.
>
> numpy, scipy, skimage, pandas, sklearn, sympy, ipython all follow this
> standard.
> The only other mainstream scientific python package that may not entirely
> follow this convention is cython.
>
>>
>> I prefer that ALL tests be in one single directory tree, ideally with
that
>> tree mtaching the package / subpacakge/module tree of the project, ie:
>
>
> The whole point of convention is to avoid this.
>
>>
>>
>> project
>> __init__.py
>> proj_mod1.py
>> sub_package1
>> __init__.py
>> sub_mod1.py
>> ...
>>
>> test_project
>> test_proj_mod1.py
>> test_sub_package1
>> test_sub_mod1.py
>>
>> i.e. one test file for each module file.
>>
>> With this method, it would not be hard to find the tests that correspond
>> to a particular python module.
>>
>> I'm ambivalent about whether the test files should be importable
packages,
>> and I think there is a lot to be said for keeping the test code outside
the
>> installable package tree altogether, but I definitely don't like them
all
>> intermingled.
>>
>> though I still don't advocate a lot of churn for this...
>>
>> -CHB
>>
>> --
>>
>> Christopher Barker, Ph.D.
>> Oceanographer
>>
>> Emergency Response Division
>> NOAA/NOS/OR&R (206) 526-6959 voice
>> 7600 Sand Point Way NE (206) 526-6329 fax
>> Seattle, WA 98115 (206) 526-6317 main reception
>>
>> Chris.Barker at noaa.gov
>
>
>
> _______________________________________________
> Matplotlib-devel mailing list
> Matplotlib-devel at python.org
> Matplotlib-devel Info Page
>
_______________________________________________
Matplotlib-devel mailing list
Matplotlib-devel at python.org
Matplotlib-devel Info Page
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/matplotlib-devel/attachments/20170129/7a17f3fb/attachment-0001.html>
···
On Sat, Jan 28, 2017 at 10:27 AM, Nelle Varoquaux > <nelle.varoquaux at gmail.com> wrote: