[Repronim-trd3] Odd mongodb package error in niceman travis-ci (temporary?)

Yaroslav Halchenko yoh at onerussian.com
Wed Jan 10 19:58:30 PST 2018 

On Thu, 11 Jan 2018, Robert Buccigrossi wrote:

> That worked!

> Also, I found some weird thing with test_utils.py.  I moved my tests above
> the "@line_profile" line and suddenly they are detected.  It looks like
> "@line_profile" is confusing codecov and causing it to not detect anything
> below that call to f(3).  So suddenly my pull request claims 20.46% more
> coverage than the trunk...

hm...  what a "neat" finding!  I can confirm it somewhat:

before I installed line-profiler module:

$> python -m pytest -s -v --cov=niceman.tests.test_utils niceman/tests/test_utils.py 
============================================= test session starts ==============================================
platform linux2 -- Python 2.7.14, pytest-3.2.1, py-1.4.34, pluggy-0.4.0 -- /home/yoh/proj/repronim/niceman-master/venvs/dev/bin/python
cachedir: .cache
rootdir: /home/yoh/proj/repronim/niceman-master, inifile:
plugins: xdist-1.18.2, localserver-0.3.7, cov-2.5.1, hypothesis-3.12.0
collected 27 items                                                                                              

niceman/tests/test_utils.py::test_swallow_outputs PASSED
niceman/tests/test_utils.py::test_swallow_logs PASSED
niceman/tests/test_utils.py::test_setup_exceptionhook::[0] PASSED
niceman/tests/test_utils.py::test_setup_exceptionhook::[1] PASSED
niceman/tests/test_utils.py::test_md5sum PASSED
niceman/tests/test_utils.py::test_updated PASSED
niceman/tests/test_utils.py::test_get_local_file_url_windows SKIPPED
niceman/tests/test_utils.py::test_auto_repr PASSED
niceman/tests/test_utils.py::test_assure_list_from_str PASSED
niceman/tests/test_utils.py::test_assure_dict_from_str PASSED
niceman/tests/test_utils.py::test_any_re_search PASSED
niceman/tests/test_utils.py::test_find_files PASSED
niceman/tests/test_utils.py::test_not_supported_on_windows PASSED
niceman/tests/test_utils.py::test_file_basename PASSED
niceman/tests/test_utils.py::test_expandpath PASSED
niceman/tests/test_utils.py::test_is_explicit_path PASSED
niceman/tests/test_utils.py::test_make_tempfile PASSED
niceman/tests/test_utils.py::test_unique PASSED
niceman/tests/test_utils.py::test_path_ PASSED
niceman/tests/test_utils.py::test_unicode_and_binary_conversion PASSED
niceman/tests/test_utils.py::test_generate_unique_set PASSED
niceman/tests/test_utils.py::test_hashable_dict PASSED
niceman/tests/test_utils.py::test_execute_command_batch PASSED
niceman/tests/test_utils.py::test_line_profile SKIPPED
niceman/tests/test_utils.py::test_find_files_exclude_vcs <- niceman/tests/utils.py PASSED
niceman/tests/test_utils.py::test_getpwd_symlink <- niceman/tests/utils.py PASSED
niceman/tests/test_utils.py::test_getpwd_basic <- niceman/tests/utils.py PASSED

---------- coverage: platform linux2, python 2.7.14-final-0 ----------
Name                          Stmts   Miss  Cover
-------------------------------------------------
niceman/tests/test_utils.py     313     27    91%


so  test_line_profile  is skipped. and then after I install it:

$> python -m pytest -s -v --cov=niceman.tests.test_utils niceman/tests/test_utils.py
============================================= test session starts ==============================================
platform linux2 -- Python 2.7.14, pytest-3.2.1, py-1.4.34, pluggy-0.4.0 -- /home/yoh/proj/repronim/niceman-master/venvs/dev/bin/python
cachedir: .cache
rootdir: /home/yoh/proj/repronim/niceman-master, inifile:
plugins: xdist-1.18.2, localserver-0.3.7, cov-2.5.1, hypothesis-3.12.0
collected 27 items                                                                                              

niceman/tests/test_utils.py::test_swallow_outputs PASSED
niceman/tests/test_utils.py::test_swallow_logs PASSED
niceman/tests/test_utils.py::test_setup_exceptionhook::[0] PASSED
niceman/tests/test_utils.py::test_setup_exceptionhook::[1] PASSED
niceman/tests/test_utils.py::test_md5sum PASSED
niceman/tests/test_utils.py::test_updated PASSED
niceman/tests/test_utils.py::test_get_local_file_url_windows SKIPPED
niceman/tests/test_utils.py::test_auto_repr PASSED
niceman/tests/test_utils.py::test_assure_list_from_str PASSED
niceman/tests/test_utils.py::test_assure_dict_from_str PASSED
niceman/tests/test_utils.py::test_any_re_search PASSED
niceman/tests/test_utils.py::test_find_files PASSED
niceman/tests/test_utils.py::test_not_supported_on_windows PASSED
niceman/tests/test_utils.py::test_file_basename PASSED
niceman/tests/test_utils.py::test_expandpath PASSED
niceman/tests/test_utils.py::test_is_explicit_path PASSED
niceman/tests/test_utils.py::test_make_tempfile PASSED
niceman/tests/test_utils.py::test_unique PASSED
niceman/tests/test_utils.py::test_path_ PASSED
niceman/tests/test_utils.py::test_unicode_and_binary_conversion PASSED
niceman/tests/test_utils.py::test_generate_unique_set PASSED
niceman/tests/test_utils.py::test_hashable_dict PASSED
niceman/tests/test_utils.py::test_execute_command_batch PASSED
niceman/tests/test_utils.py::test_line_profile PASSED
niceman/tests/test_utils.py::test_find_files_exclude_vcs <- niceman/tests/utils.py PASSED
niceman/tests/test_utils.py::test_getpwd_symlink <- niceman/tests/utils.py PASSED
niceman/tests/test_utils.py::test_getpwd_basic <- niceman/tests/utils.py PASSED

---------- coverage: platform linux2, python 2.7.14-final-0 ----------
Name                          Stmts   Miss  Cover
-------------------------------------------------
niceman/tests/test_utils.py     313     64    80%


it is not skipped now BUT coverage drops to 80% from 91%.

and that is with your change where line_profile test is the last one. On
master, it would go from 91% to 59% . 

similarish seems to be using nose on master:

with line_profiler:

rm .coverage; python -m nose -s -v --with-coverage niceman/tests/test_utils.py; python-coverage report niceman/tests/test_utils.py
...
OK (SKIP=1)
Name                          Stmts   Miss  Cover
-------------------------------------------------
niceman/tests/test_utils.py     302    117    61%

and then if removed:

Name                          Stmts   Miss  Cover
-------------------------------------------------
niceman/tests/test_utils.py     302     26    91%


an it is specifically if line_profile'd function is called


my wild guess is that line_profiler rewrites the code (to profile), making it
longer but not really executing all the statements. that leads to this drop of
coverage.

adding  "# pragma: no cover" to even the entire test doesn't have an effect
though which suggests that I am wrong in above assumption... although your
observation that moving the test alters the coverage already says that my
assumption was wrong

IMHO it is worth distilling and reporting to coverage folks.  Interested? ;)

-- 
Yaroslav O. Halchenko
Center for Open Neuroscience     http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik

More information about the Repronim-trd3 mailing list