Hi,

I do this in my app

def mqtt_message_recieved_event(self, event_name, data, kwargs):  
        print(data)      
        payload = json.loads(data["payload"])
        message = ""
        for msg in payload["Messages"]:
            message += self.message_builder(msg)

        if message != "":
            self.fire_event("MQTT_PUBLISH", topic = "appdaemon/notification/tts", message = json.dumps({"message": message})) 
        else:
            self.log("Nothing to publish")

def initialize(self):
        self.log("NotitfyApp Init")
        self.set_namespace("mqtt")
        self.listen_event(self.mqtt_message_recieved_event, "MQTT_MESSAGE", topic = "homeassistant/notification/tts")
        self.set_namespace("hass")

How do I raise an event to test my mqtt_message_recieved_event method and assert that an event was fired?

Thanks

get_state supports an attribute kwarg (https://appdaemon.readthedocs.io/en/latest/AD_API_REFERENCE.html#state-operations) but when running automations that use this under the test framework, I get an error:

TypeError: <lambda>() got an unexpected keyword argument 'attribute'

Deprecation of hass_functions

Hi everyone,

To prepare for the addition of absolutely amazing features related to time-travel, worked on by @lifeisafractal, we are deprecating the direct use of hass_functions in favor of accessing them via HassMocks.

If you have any issues with the deprecation, please comment on this thread 🙂

Quick migration guide:

• hass_functions['listen_event']
  Replace with XXXX
• hass_functions['listen_state']
  Replace with XXXX
• hass_functions['now_is_between']
  Replace with XXXX
• hass_functions['notify']
  Replace with XXXX
  Etc . . . . .

I can see you are using semantic versioning, which is awesome! Can you please tag your releases in Git such that they show up in Githubs "Releases" tab?

One of my apps has the following code:

`class Sonos(hass.Hass):

def initialize(self):
 self.run_hourly(self.run_hourly_callback, time)`

That last line is breaking one of my tests with the following error:

`self = <sonos.sonos.Sonos object at 0x7f5d82296400>

def get_now(self):
   return self.AD.get_now()

   AttributeError: 'Sonos' object has no attribute 'AD'`

My script only has one very simple test which runs fine if I comment out the self.run_hourly line in the app. My test script doesn't reference this in any way, I presume it's being run because it's within the initialize method in the app.

I am running version 2.5 and AppDaemon 3.0.5. Any help would be appreciated.

I have a test that calls the notify api on appdaemon apps such as:

class ExampleApp(hass.Hass):
    def initialize(self):
        self.notify(message="test", name="html5")

This does not work in the tests because the AD attribute never get instantiated and raises an AttributeError here:

def hass_check(func):
    def func_wrapper(*args, **kwargs):
        self = args[0]
        if not self.AD.get_plugin(self._get_namespace(**kwargs)).reading_messages:
            self.AD.log("WARNING", "Attempt to call Home Assistant while disconnected: {}".format(func))
            return lambda *args: None
        else:
            return func(*args, **kwargs)

    return (func_wrapper)

I could certainly overwrite the notify method to call self.call_service directly, but it seems like it would be better to test against the current appdaemon API. I was able to monkey patch the AD attribute, but I then ran into another issue with the app's namespace never being set. I had to call app.set_namespace("default") to get past that, but it seems these should be handled in the test initialization.

Hi,
I'm a developer and am having some problems getting the time travel to work,

import pytest

from apps.demo_class import MotionLight

# Important:
# For this example to work, do not forget to copy the `conftest.py` file.
# See README.md for more info
TEST_LIGHT = 'light.test_light';
TEST_SENSOR = 'binary_sensor.test_sensor';
DELAY = 120;
@pytest.fixture
def motion_light(given_that):
    motion_light = MotionLight(None, None, None, None, None, None, None, None)
    given_that.time_is(0)
    motion_light.initialize()
    given_that.mock_functions_are_cleared()
    return motion_light


def test_demo(given_that, motion_light, assert_that, time_travel):
    given_that.state_of('light.test_light').is_set_to('on') 
    time_travel.assert_current_time(0).seconds()
    motion_light.motion(DELAY)
    time_travel.fast_forward(DELAY).seconds()
    time_travel.assert_current_time(DELAY).seconds()
    assert_that('light.test_light').was.turned_off()

Here is the demo class i am trying to test:

import appdaemon.plugins.hass.hassapi as hass

class MotionLight(hass.Hass):
    def initialize(self):
        self.timer = None

    def motion(self, delay):
        """
            Sensor callback: Called when the supplied sensor/s change state.
        """

        self.start_timer(delay);
    
    def start_timer(self, delay):
        if self.timer:
            self.cancel_timer(self.timer) # cancel previous timer
            self.timer = self.run_in(self.light_off, delay)

    def light_off(self):
        self.log("fds");
        self.turn_off('light.test_light')

For some reason, even though run_in time and time_travel time match, the turn off service call is not found.

=================================== FAILURES ===================================
__________________________________ test_demo ___________________________________

given_that = <appdaemontestframework.given_that.GivenThatWrapper object at 0x7fde397b00f0>
motion_light = <apps.demo_class.MotionLight object at 0x7fde39755a90>
assert_that = <appdaemontestframework.assert_that.AssertThatWrapper object at 0x7fde39755ac8>
time_travel = <appdaemontestframework.time_travel.TimeTravelWrapper object at 0x7fde39755b38>

    def test_demo(given_that, motion_light, assert_that, time_travel):
        given_that.state_of('light.test_light').is_set_to('on')
        time_travel.assert_current_time(0).seconds()
        motion_light.motion(DELAY)
        time_travel.fast_forward(DELAY).seconds()
        time_travel.assert_current_time(DELAY).seconds()
>       assert_that('light.test_light').was.turned_off()

tests/test_motion_lights.py:26:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <appdaemontestframework.assert_that.WasWrapper object at 0x7fde39755ba8>

    def turned_off(self):
        """ Assert that a given entity_id has been turned off """
        entity_id = self.thing_to_check

        service_not_called = _capture_assert_failure_exception(
            lambda: self.hass_functions['call_service'].assert_any_call(
                ServiceOnAnyDomain('turn_off'),
                entity_id=entity_id))

        turn_off_helper_not_called = _capture_assert_failure_exception(
            lambda: self.hass_functions['turn_off'].assert_any_call(
                entity_id))

        if service_not_called and turn_off_helper_not_called:
            raise EitherOrAssertionError(
>               service_not_called, turn_off_helper_not_called)
E           appdaemontestframework.assert_that.EitherOrAssertionError:
E
E           At least ONE of the following exceptions should not have been raised
E
E           The problem is EITHER:
E           call_service('ANY_DOMAIN/turn_off', entity_id='light.test_light') call not found
E
E           OR
E           turn_off('light.test_light') call not found

/usr/local/lib/python3.6/site-packages/appdaemontestframework/assert_that.py:106: EitherOrAssertionError
=========================== 1 failed in 0.08 seconds ===========================

Hi!

Thank you for this amazing piece of software. I am not a python guru but I used a lot appdaemon in the past.
I am starting with a different approach about how to use appdaemon and home assistant.

Basically I am splitting my "big classes" in small modules and everything is going to be saved in home assistant entities. This is now really easy because the helpers entities can be added directly in the front end.

Based on this, I have small classes that sets different partial config states. This is really helpful in order to debug the current state and adding some test to the class is straightforward.

I am facing with a doubt that I cannot solve alone.
I am rewriting the noone, someone, everyone status in appdaemon because the way appdaemon uses it, or how home assistant define the home/away does not suit my needs.
In order to re-implement it I am using an input_select with defined states.

My class is changing this status based on device_trackers.

and this is the relevant part of the class:

   def evaluate_state(self):
        noone = True
        everyone = True
        for person in self._persons:
            state = self.get_state(person)
            self.log("{} is {}".format(person, state))
            if state == "home":
                noone = False
            elif state == "not_home":
                everyone = False
        if noone:
            self.log("Evaluated noone at home")
            self.select_option(self._selector, "noone")
        elif everyone:
            self.log("Evaluated everyone at home")
            self.select_option(self._selector, "everyone")
        else:
            self.log("Evaluated someone at home")
            self.select_option(self._selector, "someone")

And now the question:

I don't find any way to check if select_option sets the right status. It seems that at the moment the select_option call is not wrapped. Running the test I see in the error message:

AttributeError: 'PersonAtHome' object has no attribute '_namespace'

Am I missing something obvious here?
Thank you in advance

Andrea

Apparently I was wrong and the wildcard was needed. 🤦‍♂️

AppDaemon has a concept of module dependencies via a separate directive: https://appdaemon.readthedocs.io/en/latest/APPGUIDE.html#global-module-dependencies in addition to normal module dependencies (which cause the same issue).

AppDaemon modifies the PYTHONPATH such that it appears these modules are actual packages. Not only makes this development more difficult (since the imports are in fact invalid during development time) but it also makes it impossible to use this nice test framework.

Is there a way to solve this?

AppDaemon 3.x allows for the creation of custom constraints:

self.register_constraint('constrain_cloudy')

However, Appdaemon-Test-Framework doesn't seem to understand them:

self = <settings.apps.hass.TestApp object at 0x10d11c780>, name = 'constrain_cloudy'

    def register_constraint(self, name):
>       self.constraints.append(name)
E       AttributeError: 'AutoVacationMode' object has no attribute 'constraints'

I assume that this needs to be patched somehow, but I'm clueless on this one. 😆 Any thoughts?

EDIT: Actually, I think it might be pretty straightforward; we'll see. Submitted a PR; appreciate your review!

First, great work! Thank you very much!

If I am not wrong, currently it is not possible to write tests for an app that uses the AppDaemon run_at_sunrise() and run_at_sunset() calls.

May it possible to add this feature?

Context

A bug found by @chbndrhnns was present when using appdaemon version 4.0.0 or later. A fix was quickly introduced. But @lifeisafractal raised the question:
Should we be supporting Appdaemon v3 and v4? or only the latest (v4)?

Let's use this issue to discuss the pros and cons and decide how to move forward.

Hi Florian,

Is there any way to capture Appdaemons logs (the self.log calls) and show them in pytests logging output?

Pytest is showing all logs created with

import logging

logger = logging.getLogger(__name__)

in init():

self.logger.info("Hello")

But it is not showing any Appdaemon specific logs using the appdaemon supplied logging mechanism (the only one that will show in Appdaemons logs)
self.log("Updating graph in state: " + self.state)

I tried using Appdaemons log calback as well with the following error:

    def custom_log(self, **kwargs):
        self.logger.error("Callback")
        self.logger.info(kwargs);

Error:

___________________________ test_basic_duration_sad ____________________________
 
given_that = <appdaemontestframework.given_that.GivenThatWrapper object at 0x7f9dad43c2e8>
ml = <apps.LightingSM.LightingSM object at 0x7f9dad424da0>
assert_that = <appdaemontestframework.assert_that.AssertThatWrapper object at 0x7f9dad424dd8>
time_travel = <appdaemontestframework.time_travel.TimeTravelWrapper object at 0x7f9dad424e48>
 
    def test_basic_duration_sad(given_that, ml, assert_that, time_travel):
        given_that.passed_arg('entity').is_set_to(CONTROL_ENTITY)
        given_that.passed_arg('sensor').is_set_to(SENSOR_ENTITY)
        given_that.passed_arg('sensor_type_duration').is_set_to(True)
        given_that.state_of(CONTROL_ENTITY).is_set_to('off')
 
>       ml.initialize()
 
tests/test_lighting_sm.py:90:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apps/LightingSM.py:52: in initialize
    self.listen_log(self.custom_log)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
 
self = <apps.LightingSM.LightingSM object at 0x7f9dad424da0>
cb = <bound method LightingSM.custom_log of <apps.LightingSM.LightingSM object at 0x7f9dad424da0>>
 
    def listen_log(self, cb):
>       return self.AD.add_log_callback(self.name, cb)
E       AttributeError: 'LightingSM' object has no attribute 'AD'

Any advise?

I have pytest and this framework installed in a python 3.8 buster docker container. My Hass and AD instances are running in their own containers. When running pytest I get this error, any ideas?

============================================================== test session starts ===============================================================
platform linux -- Python 3.8.1, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /appdaemon/test
collected 0 items                                                                                                                                
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 197, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 246, in _main
INTERNALERROR>     config.hook.pytest_collection(session=session)
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/pluggy/manager.py", line 84, in <lambda>
INTERNALERROR>     self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 256, in pytest_collection
INTERNALERROR>     return session.perform_collect()
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 458, in perform_collect
INTERNALERROR>     items = self._perform_collect(args, genitems)
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 496, in _perform_collect
INTERNALERROR>     self.items.extend(self.genitems(node))
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 724, in genitems
INTERNALERROR>     yield from self.genitems(subnode)
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 721, in genitems
INTERNALERROR>     rep = collect_one_node(node)
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/_pytest/runner.py", line 379, in collect_one_node
INTERNALERROR>     rep = ihook.pytest_make_collect_report(collector=collector)
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/pluggy/manager.py", line 84, in <lambda>
INTERNALERROR>     self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/pluggy/callers.py", line 203, in _multicall
INTERNALERROR>     gen.send(outcome)
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/_pytest/capture.py", line 204, in pytest_make_collect_report
INTERNALERROR>     rep = outcome.get_result()
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/_pytest/runner.py", line 276, in pytest_make_collect_report
INTERNALERROR>     errorinfo = collector.repr_failure(call.excinfo)
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/_pytest/nodes.py", line 379, in repr_failure
INTERNALERROR>     return self._repr_failure_py(excinfo, style=tbstyle)
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/_pytest/nodes.py", line 320, in _repr_failure_py
INTERNALERROR>     return excinfo.getrepr(
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/_pytest/_code/code.py", line 631, in getrepr
INTERNALERROR>     return fmt.repr_excinfo(self)
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/_pytest/_code/code.py", line 877, in repr_excinfo
INTERNALERROR>     reprtraceback = self.repr_traceback(excinfo_)
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/_pytest/_code/code.py", line 821, in repr_traceback
INTERNALERROR>     reprentry = self.repr_traceback_entry(entry, einfo)
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/_pytest/_code/code.py", line 781, in repr_traceback_entry
INTERNALERROR>     s = self.get_source(source, line_index, excinfo, short=short)
INTERNALERROR>   File "/usr/local/lib/python3.8/site-packages/_pytest/_code/code.py", line 711, in get_source
INTERNALERROR>     lines.append(space_prefix + source.lines[line_index].strip())
INTERNALERROR> IndexError: list index out of range

============================================================= no tests ran in 0.17s ==============================================================

Currently, only run_in is support by the test framework. It would be great for testing if all scheduler calls were supported as well as dispatching these callbacks. Additional, a more expressive API for moving through time in tests would help cover more test cases I'm interned in when TDD'ing my tests.

I'm currently working on a branch with some ideas of how to approach this if you want ot have a look: master...lifeisafractal:expand_time_travel

I'd like to discuss design decisions here before I get too far down the wrong path, but I'm also happy puttering along on this (hey, it's fun!).

General design thoughts:

• Make the time_travel fixture the central place for handling all time and callback scheduling.
• Rather than mocking each call (run_hourly, run_at, etc.), mock lower level AD APIs so we can reuse the appapi implementations and logic. (i.e. appapi.insert_schedule)
  • Pros:
    • Less code to write
    • Behavior during test is exactly as it will be in the wild
  • Cons:
    • Depends on the internal API structure of AppDaemon. The upcoming 4.0beta already would break this, but is fixable
• Proposed breaking change: remove given_that.time_is() and replace with time_travel.reset_time(). If the time_travel fixture takes over the whole notion of time in the system, using the given_that fixture to set time could produce inconstant tests or require much more coupling between given_that and time_travel which are currently unaware of each other. I'd love to see a solution that could keep this API cleanly, but right now I'm leaning to the best thing being to bite the bullet and make the breaking change.

I'll keep putting along on my branch, but I'd love to talk about why this is needed and how to implement it as I'm going along on it.

Also, the intent would be to get this up for PR and merged, but then follow on with an extension that support sun related callbacks and calls.

The "appdaemon_mock" folder isn't included in the distro, which leads to a bunch of missing file errors.
The "release" version throws problems with "AD" when interacting with get_utc() from AppDaemon....

Soo I just found a bug in an automation I wrote. I wrote notify.mobile_app_fairphone instead of notify/mobile_app_fairphone

The reason it took me a while to find, is that I made the same mistake in the test I wrote for it. So that's why I thought: what if appdaemontestframework would validate service names are in the correct format, in the assertion?

I will probably implement this myself, as I think it is a good first issue, but just putting this here as a reminder for myself and documenting the reason to want it.

The documentation seems to imply when configuring a fixture I can provide arguments that will be accessible during initialize:

# Single Class w/ params:
@automation_fixture((upstairs.Bedroom, {'motion': 'binary_sensor.bedroom_motion'}))

However, my test failed when trying to access this.args within initialize:

self = <apps.alarm_auto_arm.MyAutomation object at 0x108216820>

    def initialize(self):
>       print(self.args['foo'])
E       KeyError: 'foo'

apps/alarm_auto_arm.py:14: KeyError

Example Code:

class MyAutomation(hass.Hass):
    def initialize(self):
        print(self.args['foo'])


def test_my_automation(given_that, my_automation, assert_that):
    pass


@automation_fixture((MyAutomation, {
   'foo': 'bar'
}))
def my_automation():
    pass

My workaround is currently to do the following, but wondering if there was a more "blessed" way to achieve this:

@pytest.fixture
def my_automation(given_that):
    my_automation = MyAutomation(None, None, None, None, None, None, None)
    my_automation.args = {
        'foo': 'bar',
    }
    my_automation.initialize()
    given_that.mock_functions_are_cleared()
    return my_automation

Following on the discussion on the Patch 'now_in_between' pull-request.

While now_in_between is now patched, it would be nice to have a seamless integration with the framework.

Probably an interesting way would be to integrate it with the time_travel module, so that run_in, now_in_between and time_travel.assert_current_time end up all coherent.

I have created a fully isolated test case so you can easily reproduce and check if there is nothing wrong with config.
requirements.txt for all version
stack_trace.txt to show cmd: what I run and what happens.

May well be a duplicate of #51 .
AppDaemon_test_case.zip

Hi, thanks for all the awesome work on this library!

I'm relatively new to pytest in general and had a question on general setup. Where does pytest need to be installed/run (my local machine, the HASS docker container, the Appdaemon docker container)?

So far whenever, I call assert_that('my_entity_id'), and try to run a test, I receive an appdaemontestframework.assert_that.EitherOrAssertionError error on the pytest console.

Here's an example of a simple test I've tried:

from apps.front_porch_lights import FrontPorchLights
import pytest
from appdaemontestframework import automation_fixture


@automation_fixture(FrontPorchLights)
def front_porch():
    pass


def test_light_turns_on_at_sunset(given_that, front_porch, assert_that):
    given_that.passed_arg('entities').is_set_to('light.front_porch_sconces')
    given_that.state_of('light.front_porch_sconces').is_set_to("off")
    front_porch.lights_on(None, None)
    assert_that('light.front_porch_sconces').was.turned_on()

Thanks in advance for any help in getting up and running!

It appears that inherited AppDaemon apps cannot be tested. For example, consider a case where a subclass calls listen_event:

apps/base.py

from appdaemon.plugins.hass.hassapi import Hass  # type: ignore


class Base(Hass):
    """Define a base automation object."""

    def initialize(self):
        """Initialize."""
        # Do some stuff...

apps/subclass.py

from .base import Base


class Subclass(Base):
    """Define automated alterations to vacation mode."""

    def initialize(self) -> None:
        """Initialize."""
        super().initialize()

       self.listen_event(self.callback, 'EVENT_NAME')  

    def callback(self, event_name, data, kwargs):
        """Run the callback."""
        # Do some stuff...

If I were to write a simple test of Subclass:

tests/test_subclass.py

import pytest

from ..apps.hass import AutoVacationMode


@pytest.fixture
def subclass(given_that):
    """Define a fixture for Subclass."""
    app = Subclass(None, None, None, None, None, None, None, None)
    app.initialize()
    given_that.mock_functions_are_cleared()
    return app


def test_subclass(hass_functions, subclass):
    """Test."""
    listen_event = hass_functions['listen_event']
    listen_event.assert_any_call(subclass.callback, 'EVENT_NAME')

...I am greeted with a stacktrace that looks like this:

>   ???
E   AssertionError: listen_event(<bound method Subclass.callback of <settings.apps.hass.Subclass object at 0x108abb908>>, 'EVENT_NAME') call not found

Could totally be due to my incompetence with pytest, but would appreciate thoughts.

I'm trying to get this testing framework to work for my own Hass setup, but the examples in the docs folder use old and deprecated code which no longer works when I set it up. I'm having a really hard time getting it to work and understand which interfaces to use. Is here anyway the examples could be updated? Happy to help with the writing/coding if there is somebody that can help me get up to speed in the project because I'm kind of at a loss at the moment.

In version 5.4.* of pytest, an error is thrown when capturing the logging

TerminalReporter.writer attribute is deprecated, use TerminalReporter._tw instead at your own risk.
  See https://docs.pytest.org/en/latest/deprecations.html#terminalreporter-writer for more information.

I have no time to investigate it now, so I pinned down the version 5.3.* so that travis build stop failing but this is obviously an issue we will want to tackle 🙂
Quick fix commit: 5873630f0979381bdec751729cf648037fbb681c

I'll look at it in the following weeks 🙂

This is a great little utility I'm using very much.

However, for the sake of having more expressive tests, it would be great to have a when helper that has the ability to trigger one of the HA "events" as opposed to having to invoke the callback directly.

E.g. a test like this:

def test_entering_the_kitchen(given_that, kitchen_activity, assert_that):
    given_that.state_of(devices.KITCHEN_MOTION).is_set_to(states.OFF)

    when.state_of(devices.KITCHEN_MOTION).changes_to(states.ON)

    assert_that(services.HELPER_SELECT_SET).was.set_to_activity(helpers.KITCHEN_ACTIVITY, activities.PRESENT)

as opposed to a test like this:

def test_entering_the_kitchen(given_that, kitchen_activity, assert_that):
    given_that.state_of(devices.KITCHEN_MOTION).is_set_to(states.ON)

    kitchen_activity.kitchen_activity_controller(None, None, None, None, None)

    assert_that(services.HELPER_SELECT_SET).was.set_to_activity(helpers.KITCHEN_ACTIVITY, activities.PRESENT)

If I want to create abstractions or unit tests I wouldn't really be using this lib, and I think this lib is actually great for testing the HA boundary, at which point accessing implementation details of an app-daemon class seems like a breach of boundaries.

As an added benefits, it renders the callback tests documented in the "bonus" area redundant, and those too are a bit brittle, as they rely on the thing being called, and the exact parameters set on them.

The demo file I created from your instructions contains just these lines:

"""Demo"""
import appdaemon.plugins.hass.hassapi as hass
from appdaemontestframework import automation_fixture


class LivingRoom(hass.Hass):
    def initialize(self):
        ...


@automation_fixture(LivingRoom)
def living_room():
    pass


def test_during_night_light_turn_on(given_that, living_room, assert_that):
    given_that.state_of('sensor.living_room_illumination').is_set_to(
        200)  # 200lm == night
    living_room._new_motion(None, None, None)
    assert_that('light.living_room').was.turned_on()

Running this test with pytest gives me the error message TypeError: too many positional arguments.

Would you have any idea what causes this?

 ✘  (ad-lights-hI87TkkR) ~/dev/ad-lights  pytest
============================================================================ test session starts ============================================================================
platform darwin -- Python 3.7.6, pytest-5.3.4, py-1.8.1, pluggy-0.13.1
rootdir: /Users/jo/dev/ad-lights
collected 1 item

test_all.py E                                                                                                                                                         [100%]

================================================================================== ERRORS ===================================================================================
_______________________________________________________ ERROR at setup of test_during_night_light_turn_on[LivingRoom] _______________________________________________________

request = <SubRequest 'living_room' for <Function test_during_night_light_turn_on[LivingRoom]>>
given_that = <appdaemontestframework.given_that.GivenThatWrapper object at 0x10ce2f090>
hass_functions = {'__init__': <function __init__ at 0x10ce287a0>, 'args': <MagicMock name='args' id='4510310032'>, 'call_service': <MagicMock name='call_service' id='4510503760'>, 'cancel_timer': <MagicMock name='cancel_timer' id='4511695376'>, ...}

    @pytest.fixture(params=self.automation_classes, ids=self._generate_id)
    def automation_fixture_with_initialisation(request, given_that, hass_functions):
        automation_class = request.param
>       return _instantiate_and_initialize_automation(function, automation_class, given_that, hass_functions)

../../.virtualenvs/ad-lights-hI87TkkR/lib/python3.7/site-packages/appdaemontestframework/automation_fixture.py:78:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../.virtualenvs/ad-lights-hI87TkkR/lib/python3.7/site-packages/appdaemontestframework/automation_fixture.py:16: in _instantiate_and_initialize_automation
    None, automation_class.__name__, None, None, None, None, None, None)
../../.virtualenvs/ad-lights-hI87TkkR/lib/python3.7/site-packages/mock/mock.py:270: in checksig
    sig.bind(*args, **kwargs)
/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/inspect.py:3015: in bind
    return args[0]._bind(args[1:], kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <Signature (self, ad: appdaemon.appdaemon.AppDaemon, name, logging, args, config, app_config, global_vars)>
args = (<test_all.LivingRoom object at 0x10ce2fcd0>, None, 'LivingRoom', None, None, None, ...), kwargs = {}

    def _bind(self, args, kwargs, *, partial=False):
        """Private method. Don't use directly."""

        arguments = OrderedDict()

        parameters = iter(self.parameters.values())
        parameters_ex = ()
        arg_vals = iter(args)

        while True:
            # Let's iterate through the positional arguments and corresponding
            # parameters
            try:
                arg_val = next(arg_vals)
            except StopIteration:
                # No more positional arguments
                try:
                    param = next(parameters)
                except StopIteration:
                    # No more parameters. That's it. Just need to check that
                    # we have no `kwargs` after this while loop
                    break
                else:
                    if param.kind == _VAR_POSITIONAL:
                        # That's OK, just empty *args.  Let's start parsing
                        # kwargs
                        break
                    elif param.name in kwargs:
                        if param.kind == _POSITIONAL_ONLY:
                            msg = '{arg!r} parameter is positional only, ' \
                                  'but was passed as a keyword'
                            msg = msg.format(arg=param.name)
                            raise TypeError(msg) from None
                        parameters_ex = (param,)
                        break
                    elif (param.kind == _VAR_KEYWORD or
                                                param.default is not _empty):
                        # That's fine too - we have a default value for this
                        # parameter.  So, lets start parsing `kwargs`, starting
                        # with the current parameter
                        parameters_ex = (param,)
                        break
                    else:
                        # No default, not VAR_KEYWORD, not VAR_POSITIONAL,
                        # not in `kwargs`
                        if partial:
                            parameters_ex = (param,)
                            break
                        else:
                            msg = 'missing a required argument: {arg!r}'
                            msg = msg.format(arg=param.name)
                            raise TypeError(msg) from None
            else:
                # We have a positional argument to process
                try:
                    param = next(parameters)
                except StopIteration:
>                   raise TypeError('too many positional arguments') from None
E                   TypeError: too many positional arguments

/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/inspect.py:2936: TypeError
============================================================================= 1 error in 0.19s ==============================================================================

AppDaemon has example of using Hass and MQTT from same automation, using ADBase rather than Hass from plugin

https://appdaemon.readthedocs.io/en/latest/APPGUIDE.html?highlight=adbase#using-multiple-apis-from-one-app

The test framework will reject this, since expects base class to be Hass

Currently the Tox tests are only run against Python 3.6. #37 and #36 both show Python 3.8 issues that could be cause with automated test coverage.

Proposal:

• We create a policy of what Python versions are supported. I suggest we follow upstream HomeAssistant policy of supporting the 2 most recent minor stable releases.
• Extend Tox coverage to test with both these supported versions both for local testing and for the automated testing

Immediate consequences:

• Python 3.6 would be deprecated and heading for removal in the next couple releases.

Following a conversation with @lifeisafractal on one of his PR, he recommended we keep a changelog and use proper semantic versioning. I agree it would be a great idea. So here we are 🙂

This issue is here to track the progress made on the Changelog and to host any discussions related to changelog / semantic versioning.

Resources

• Changelog: https://keepachangelog.com/en/1.0.0/
• Semantic Versioning: https://semver.org/

we would like to

I think it would be natural if state listener callbacks were automatically called when a mocked state changes. In the example below, I had to call it myself from the test, which increases the amount of code needed in tests, and the probability of errors.

given_that.state_of("binary_sensor.normal_activity").is_set_to("on")
area.entity_state_changed_handler("binary_sensor.normal_activity", None, "off", "on", None)

If this isn't the intended behavior, please give me a hint on what I'm doing wrong. Otherwise, do you think it would be desirable to make the framework automatically call the listeners on state change?