jlane9 / pytest-needle Goto Github PK

For some reason landscape.io's code health check sits in a continuously waiting phase. Figure out how to get this to build. https://landscape.io/github/jlane9/pytest-needle/2

Documentation is looking a little bare. http://pytest-needle.readthedocs.io/en/latest/

Expected Behavior

pytest-needle creates diff images and a report
I am using:
pytest --driver Firefox --needle-engine imagemagick --html=report.html --needle-viewport-size "1200 x 800" --needle-cleanup-on-success

Actual Behavior

Test fails with
`E AttributeError: 'AssertionError' object has no attribute 'message'

/MYVIRTENVPATH/lib/python3.6/site-packages/pytest_needle/driver.py:287: AttributeError
`and does not generate a new report.

Steps to Reproduce the Problem

1. Use the following command in a directory with a test that fails the image comparision
   pytest --driver Firefox --needle-engine imagemagick --html=report.html --needle-viewport-size "1200 x 800" --needle-cleanup-on-success
2. Watch output and check for a report.html

Specifications

• Version:
  needle==0.5.0
  pytest-needle==0.3.4
• Platform: Linux

Expected Behavior

When maximize_window() command is used, Browser is expected to stay in maximized state.

Actual Behavior

Browser collapses into smaller window at the beginning of every pytest script. As a workaround, I must use maximize_window() at the beginning of every script. Also when headless chrome is under the control of NeedleDriver, any arguments to change the browser size does not work.

Steps to Reproduce the Problem

1. Initialize Browser,
2. give maximize_window() command using NeedleDriver

• Browser size will collapse at the beginning of every test.

Specifications

• Version: pytest-needle 3.11, selenium 3.141
• Platform: Windows 7

Hi Team,
Subject: pytest-needle 0.3.11
In report, unable to get screenshot with colour difference like Heat map difference. Its showing actual screen shot, expected screenshot and difference screenshot. but unable to find what is the difference between two screenshot with color change. Is there any workaround for this.
I tried in different engine like PIL,Perceptualdiff etc..
Thank in advance.

Regards,
Rafeek

Expected Behavior

Run Pytest-needle on docker selenium-hub

Actual Behavior

I’m not able to figure out what to put in the command line to make selenium on selenium-hubs remote webdriver.

Thanks for making this!!

Need to get code coverage working. https://coveralls.io/github/jlane9/pytest-needle

Expected Behavior

When a test runs, It creates a fresh screenshot before checking if the baseline image is available.

Actual Behavior

It does not create a screenshot if there is no corresponding baseline screenshot.

Steps to Reproduce the Problem

1. insert the following code into a new file test_fresh_screenshot.py:

def test_screenshot(needle):
    needle.driver.get('https://www.example.com')
    needle.assert_screenshot('test_fresh_screenshot')

2. run pytest test_fresh_screenshot
3. No test_fresh_screenshot.png is found in the screnshots folder

Reasoning

If fresh screenshots are generated without requiring a baseline screenshot, a user can check them manually and copy them to the baseline folder. Thus she/he does not have to call --needle-save-baseline which would overwrite every already existing screenshot while reviewing every new screenshot.

Specifications

• Version: 0.3.6

Original issue

run pytest my/test/file.py::mytest to ensure the other screenshots do not contain any error
run pytest my/test/file.py::mytest --needle-save-baseline to create new baseline images
check all baseline screenshots for mytest if the contain errors
Since I overwrite all new baseline screenshots. Thus there is still a slight propability of a randomly failing test to screw up my base line images. Thus the manual check.

Implement the needle-capture parameter to automatically create missing baseline images
Create fresh images even if a baseline image is missing (and report an IOError for the missing baseline image)

Feature Suggestion

Originally this was a feature suggestion which is continued in issue #10 (The original content of this issue can be found there)

Now this issue covers proposals by @jlane9
a. Save baselines for different screen sizes
b. Save baselines for different browsers
c. Save incremental baselines for each successful run, compare against most recent baseline set (Sort of like your 2nd suggestion)

Utilize JrGoodle's setup of auto-deployments in clowder https://github.com/JrGoodle/clowder/

Expected Behavior

All works fine.

Actual Behavior

Traceback (most recent call last):
  File "C:\Python36\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Python36\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\dev\test\env\Scripts\pytest.exe\__main__.py", line 9, in <module>
  File "c:\dev\test\env\lib\site-packages\_pytest\config\__init__.py", line 56, in main
    config = _prepareconfig(args, plugins)
  File "c:\dev\test\env\lib\site-packages\_pytest\config\__init__.py", line 181, in _prepareconfig
    pluginmanager=pluginmanager, args=args
  File "c:\dev\test\env\lib\site-packages\pluggy\__init__.py", line 617, in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
  File "c:\dev\test\env\lib\site-packages\pluggy\__init__.py", line 222, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "c:\dev\test\env\lib\site-packages\pluggy\__init__.py", line 216, in <lambda>
    firstresult=hook.spec_opts.get('firstresult'),
  File "c:\dev\test\env\lib\site-packages\pluggy\callers.py", line 196, in _multicall
    gen.send(outcome)
  File "c:\dev\test\env\lib\site-packages\_pytest\helpconfig.py", line 89, in pytest_cmdline_parse
    config = outcome.get_result()
  File "c:\dev\test\env\lib\site-packages\pluggy\callers.py", line 76, in get_result
    raise ex[1].with_traceback(ex[2])
  File "c:\dev\test\env\lib\site-packages\pluggy\callers.py", line 180, in _multicall
    res = hook_impl.function(*args)
  File "c:\dev\test\env\lib\site-packages\_pytest\config\__init__.py", line 607, in pytest_cmdline_parse
    self.parse(args)
  File "c:\dev\test\env\lib\site-packages\_pytest\config\__init__.py", line 772, in parse
    self._preparse(args, addopts=addopts)
  File "c:\dev\test\env\lib\site-packages\_pytest\config\__init__.py", line 724, in _preparse
    self.pluginmanager.load_setuptools_entrypoints("pytest11")
  File "c:\dev\test\env\lib\site-packages\pluggy\__init__.py", line 397, in load_setuptools_entrypoints
    plugin = ep.load()
  File "c:\dev\test\env\lib\site-packages\pkg_resources\__init__.py", line 2324, in load
    return self.resolve()
  File "c:\dev\test\env\lib\site-packages\pkg_resources\__init__.py", line 2330, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "c:\dev\test\env\lib\site-packages\_pytest\assertion\rewrite.py", line 216, in load_module
    py.builtin.exec_(co, mod.__dict__)
  File "c:\dev\test\env\lib\site-packages\pytest_needle\plugin.py", line 11, in <module>
    from pytest_needle.driver import DEFAULT_BASELINE_DIR, DEFAULT_OUTPUT_DIR, DEFAULT_ENGINE, \
  File "c:\dev\test\env\lib\site-packages\_pytest\assertion\rewrite.py", line 216, in load_module
    py.builtin.exec_(co, mod.__dict__)
  File "c:\dev\test\env\lib\site-packages\pytest_needle\driver.py", line 14, in <module>
    from test_needle.cases import import_from_string
ModuleNotFoundError: No module named 'test_needle'

Steps to Reproduce the Problem

1. Follow the tutorial example steps

Specifications

• Version: python 3.6
• Platform: win10