Rewriting the integration tests to make them reliable and easier to understand #198
Conversation
…y is registered and wait with the Tests for it
there might still be content in the Buffer
This is much more reliable
| # can either give multiple identifier separated by comma (,) or put this option | ||
| # multiple time (only on the command line, not in the configuration file where | ||
| # it should appear only once). | ||
| disable=E0611,E0202,R0801,E1101,E1002,R0201,F0401,W0141,R0904,too-few-public-methods |
There was a problem hiding this comment.
Do you need all these disables? If so can you please describe why. It would also be good to use the "short-names" rather than the numbers.
There was a problem hiding this comment.
no, i just copied the .pylint_integrationtests to .pylint_integrationtests2, I did not take a look at pylint compatibility for the new tests yet.
| @@ -182,85 +206,86 @@ class MockConnection(object): | |||
|
|
|||
| """A class which mocks the Connection class""" | |||
|
|
|||
| def __init__(self, mode): | |||
| self.mode = mode | |||
| def __init__(self, return_variant, should_fail=False): | |||
There was a problem hiding this comment.
What is return_variant used for? Docstring would be good :)
|
|
|
Generally a very awesome pull request! Thank you so much for you work on this. |
|
|
|
|
Rewriting the integration tests to make them reliable and easier to understand
These changes introduce a cleaner, reliable and more complete set of integration tests increasing unit test coverage to >99%.
It creates a new common testing base system, which does not rely on sleeping n-seconds (to give the server time to start) but instead closely monitors the server's stdout to know when the server is ready for a specific test to begin. This way the tests run much faster and don't rely on the speed of the running machine. This should greatly improve overall test reliability, especially on highly variable environments like Travis-CI.
Tests which continue to have problems after this change are marked as @xfail and reference GitHub-Issues about why the tests fail.
This pull request closes the following open issues;
test_get_preview_portssporadically fails.test_set_composite_mode_preview.make testNUMandFACTORstuff in test_controller is incomprehensible