UDM Pro issue when using main function about aiounifi HOT 51 CLOSED

Its working!

from aiounifi.

I've got access to a UDMP now but unfortunately I've been svamped with stuff and on top of it all got sick today, I hope to have some energy to progress here during the weekend.

from aiounifi.

@Kane610 get better soon!

from aiounifi.

Hey, the script is updated and "nearly" works with UDMP now, current issue is I get a 404 when trying to login, and I can't understand why, because exactly the same is sent using firefox

from aiounifi.

You will know when I close this issue :)

from aiounifi.

The base UDM is not yet officially on 1.6.x which is when the new OS is implemented
however under 1.6.x it does function the same as the UDM-Pro
I can confirm this as my UDM is now on 1.6.5 with the new os

under the current official 1.5.5 firmware its still using the old method and works but fails after a few hours for unknown reasons at this point

that being said i'd vote for
is_embedded

from aiounifi.

I just realized that I was on the master branch & not the udm_support branch, so I am deleting this comment & retesting.

Sorry for the white noise.

Will post back findings...

from aiounifi.

@Kane610 and the rest, I've got a working solution here: #17

I'm not very familiar with aiohttp.CookieJar, but I think the problem is there. My PR implements a workaround to demonstrate how this should be working by including the missing x-csrf-token header that should be accompanying the cookie header.

from aiounifi.

I have a couple of other issues to investigate before I create a PR to hass but it is not far off. It seems to work fine. I really want to thank you all for your support ❤️

from aiounifi.

Hmm I didn't see this problem when I tried it initially. I will look at it tomorrow

from aiounifi.

Looks like you are running some kind of alpha version of aiohttp

I use 3.6.1

❯ pip show aiohttp
Name: aiohttp
Version: 3.6.1

from aiounifi.

Yes, you're right. It seems like an alpha was installed. Weird.

pip3 show aiohttp
Name: aiohttp
Version: 4.0.0a1

Changed to your version and this is what I got:

Using selector: KqueueSelector
Starting aioUniFi
Error connecting to the UniFi controller at 127.0.0.1
Couldn't connect to UniFi controller.
Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x10ac73410>

from aiounifi.

Here is a hacky diff that shows the differences needed to work with UDMPro using this cmd line:

python  -m aiounifi --port 443 <hostname> <user> <password>

diff --git a/aiounifi/controller.py b/aiounifi/controller.py
index a2b3b9d..1e9f676 100644
--- a/aiounifi/controller.py
+++ b/aiounifi/controller.py
@@ -59,7 +59,7 @@ class Controller:
         self.wlans = None
 
     async def login(self):
-        url = "login"
+        url = "auth/login"
         auth = {
             "username": self.username,
             "password": self.password,
@@ -141,9 +141,14 @@ class Controller:
 
     async def request(self, method, path, json=None):
         """Make a request to the API."""
-        url = f"https://{self.host}:{self.port}/api/"
-        url += path.format(site=self.site)
+        print (f"path: {path}")
+        if path == "auth/login":
+            url = f"https://{self.host}:{self.port}/api/"
+        else:
+            url = f"https://{self.host}:{self.port}/proxy/network/api/"
 
+        url += path.format(site=self.site)
+        print(f'URL: {url}')
         try:
             async with self.session.request(
                 method, url, json=json, ssl=self.sslcontext
@@ -154,9 +159,11 @@ class Controller:
 
                 response = await res.json()
 
-                _raise_on_error(response)
-
-                return response["data"]
+                if path == "auth/login":
+                    return response
+                else:
+                    _raise_on_error(response)
+                    return response["data"]
 
         except client_exceptions.ClientError as err:
             raise RequestError(
@@ -166,5 +173,7 @@ class Controller:
 
 def _raise_on_error(data):
     """Check response for error message."""
+    from pprint import pprint
+    pprint(data)
     if isinstance(data, dict) and data["meta"]["rc"] == "error":
         raise_error(data["meta"]["msg"])

I'm guessing a method like is_unifios() would be useful to support both seamlessly.

from aiounifi.

Great! I will put something together within a couple of days. I guess there is no anonymous way to identify if it is unifi os or standalone controller?

from aiounifi.

@analogue isn't it possible to use the same url when logging in as with the rest of the commands?

+        if path == "auth/login":
+            url = f"https://{self.host}:{self.port}/api/"
+        else:
+            url = f"https://{self.host}:{self.port}/proxy/network/api/"

from aiounifi.

@Kane610 I tried once more with the path for login set to just login and it failed.

Using selector: EpollSelector
Starting aioUniFi
path: login
URL: https://<redacted>:443/api/login
Unknown UniFi communication error occurred
Traceback (most recent call last):
  File "/home/analogue/git/aiounifi/aiounifi/__main__.py", line 34, in unifi_controller
    await controller.login()
  File "/home/analogue/git/aiounifi/aiounifi/controller.py", line 71, in login
    await self.request("post", url, json=auth)
  File "/home/analogue/git/aiounifi/aiounifi/controller.py", line 161, in request
    raise ResponseError(f"Invalid content type: {res.content_type}")
aiounifi.errors.ResponseError: Invalid content type: text/html
Couldn't connect to UniFi controller.
Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x7f664d193d90>
Unclosed connector
connections: ['[(<aiohttp.client_proto.ResponseHandler object at 0x7f664d1544b0>, 3843944.194195639)]']
connector: <aiohttp.connector.TCPConnector object at 0x7f664d193d10>

Looks like auth/login is required.

Also, the other route of using auth/login with the /proxy/network/api/ prefix failed:

sing selector: EpollSelector
Starting aioUniFi
path: auth/login
URL: https://<redacted>:443/proxy/network/api/auth/login
path: s/{site}/stat/sta
URL: https://<redacted>:443/proxy/network/api/s/default/stat/sta
{'data': [], 'meta': {'msg': 'api.err.LoginRequired', 'rc': 'error'}}

As far as being to to detect the embedded controller vs a hosted controller, you could probably mimic what is happening here: https://github.com/Art-of-WiFi/UniFi-API-client/blob/master/src/Client.php#L135

FYI, is_unifios is probably a bad name since Unifi have announced UMGPro which runs UnifiOS but does not have an embedded controller. is_embedded?

from aiounifi.

Does the regular UDM work with the integration? I assume it seems the same challenges as the UDM pro? If that's the case, I'd vote for either is_embedded or is_udm.

from aiounifi.

Could you try if this works?

diff --git a/aiounifi/controller.py b/aiounifi/controller.py
index a2b3b9d..15ea2f9 100644
--- a/aiounifi/controller.py
+++ b/aiounifi/controller.py
@@ -8,7 +8,7 @@ from aiohttp import client_exceptions
 
 from .clients import Clients, URL as client_url, ClientsAll, URL_ALL as all_client_url
 from .devices import Devices, URL as device_url
-from .errors import raise_error, ResponseError, RequestError
+from .errors import raise_error, LoginRequired, ResponseError, RequestError
 from .events import event
 from .websocket import WSClient, SIGNAL_CONNECTION_STATE, SIGNAL_DATA, STATE_RUNNING
 from .wlan import Wlans, URL as wlan_url
@@ -45,6 +45,7 @@ class Controller:
         self.username = username
         self.password = password
         self.site = site
+        self.use_proxy_path = False
         self.sslcontext = sslcontext
 
         self.callback = callback
@@ -65,7 +66,12 @@ class Controller:
             "password": self.password,
             "remember": True,
         }
-        await self.request("post", url, json=auth)
+        try:
+            await self.request("post", url, json=auth)
+        except LoginRequired:
+            url = "auth/login"
+            await self.request("post", url, json=auth)
+            self.use_proxy_path = True
 
     async def sites(self):
         url = "self/sites"
@@ -141,14 +147,18 @@ class Controller:
 
     async def request(self, method, path, json=None):
         """Make a request to the API."""
-        url = f"https://{self.host}:{self.port}/api/"
+        base_path = "api/"
+        if self.use_proxy_path:
+            base_path = "proxy/network/api/"
+
+        url = f"https://{self.host}:{self.port}/{base_path}"
         url += path.format(site=self.site)
 
         try:
             async with self.session.request(
                 method, url, json=json, ssl=self.sslcontext
             ) as res:
-
+                print(res)
                 if res.content_type != "application/json":
                     raise ResponseError(f"Invalid content type: {res.content_type}")

from aiounifi.

Created a PR to track the work

from aiounifi.

using controller.py from udm branch which has changes from above
still getting original error

"Bad user credentials"

with below in the log file

Unknown UniFi communication error occurred
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/components/unifi/controller.py", line 368, in get_controller
await controller.login()
File "/usr/local/lib/python3.7/site-packages/aiounifi/controller.py", line 70, in login
await self.request("post", url, json=auth)
File "/usr/local/lib/python3.7/site-packages/aiounifi/controller.py", line 163, in request
raise ResponseError(f"Invalid content type: {res.content_type}")
aiounifi.errors.ResponseError: Invalid content type: text/html

from aiounifi.

Anything we can do to help on this issue?

from aiounifi.

Hey, the script is updated and "nearly" works with UDMP now, current issue is I get a 404 when trying to login, and I can't understand why, because exactly the same is sent using firefox

Thank you for your hard work on this!

from aiounifi.

@SeraphimSerapis @plastikk80 if you can try it out since my install is not a regular one I just want to make sure that it is not the environment playing me tricks

from aiounifi.

@SeraphimSerapis @plastikk80 if you can try it out since my install is not a regular one I just want to make sure that it is not the environment playing me tricks

Sure can did you upload a new version to test?

from aiounifi.

Also happy to test first thing tomorrow morning -- should we just go with the latest commit from the udm_support branch?

from aiounifi.

Yeah latest is good! And thanks

from aiounifi.

I have a UDM Pro and would like to help test this. How can I install and test this in HA? Should I see it under available integrations after adding the aiounifi folder to my custom components directory?

from aiounifi.

Download the code from this repository and run python3 -m aiounifi host username password -p port

from aiounifi.

@Kane610 Downloaded and copied to the custom components. Where do I run the python command. The hassio console doesn't seem to have python available.

from aiounifi.

@Kane610 unfortunately I'm seeing 404 from the login post:

https://redacted:443/api/auth/login
404
<ClientResponse(https://redacted:443/api/auth/login) [404 Not Found]>

But I can confirm that it's reachable via a curl request. Seems like it's still limited to just the app.

from aiounifi.

@plastikk80 you run it from your normal computer, you don't need hass to use the library

Also! It might be a cookie_jar issue, Im trying out disabling different things and it seems to be working. So some progress. Continuing tomorrow.

from aiounifi.

@Kane610
Going to preface my comment with Not a developer
So I was getting 404 as well. I made a change to your self.base_path in your controller file. My UDMP adds "login?redirect=%2F" to the login URL. I added the same thing there and I got back 200 OK. The following is the exact message:

Starting aioUniFi
https://192.168.3.1/login?redirect=%2F
200 <ClientResponse(https://192.168.3.1/login?redirect=/) [200 OK]>
<CIMultiDictProxy('Vary': 'Origin', 'X-DNS-Prefetch-Control': 'off', 'X-Frame-Options': 'SAMEORIGIN', 'Strict-Transport-Security': 'max-age=15552000; includeSubDomains', 'X-Download-Options': 'noopen', 'X-Content-Type-Options': 'nosniff', 'X-XSS-Protection': '1; mode=block', 'X-CSRF-Token': '700e8e9e-34fh-2f63788c513f', 'Content-Type': 'text/html; charset=utf-8', 'Content-Length': '357', 'X-Response-Time': '4ms', 'Set-Cookie': 'TOKEN=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjc3JmVG9rZW4iOiI3MDBlOGU5ZS1iMzQyLTRlM; path=/; secure; httponly', 'Date': 'Tue, 31 Mar 2020 22:19:22 GMT', 'Connection': 'keep-alive')>

If I attempt via the "https://192.168.3.1/api/auth/login" it gives me a 404 Not found. If I attempt that via a browser it tells me method not allowed.

I also get 200 if I drop the "auth" from the url. ex. https://192.168.3.1/api/login works.

Hoping this helps, if not just disregard.

from aiounifi.

OK, here is what I have figured out...

I get the 404 with the udm_support branch, so I started playing around in Postman. During this process, I had my json wrong, so I got a 401, so I fixed my payload & send again. This time I got a 404 even though I had not changed anything with the uri.

I closed Postman & tried again--It worked. I got back the response, so I sent the request again & I got back a 404. This was really strange. I saved my request, closed postman, reopened it, and it ran correctly! So I ran it again, and got a 404.

This got me to looking at the cookie, which has a TOKEN in it. If I delete the TOKEN from the cookie & request it work exactly 1 time. I have to clear the cookie in order to log in.

I would guess that they have some guard/bug in their login that says that if you have a TOKEN, that you should not login, so return a 404?

Anyhow, I don't know my way around Python to find it, but I would think that your check_unifi_os is getting a TOKEN in your cookie, then your login is getting the 404?

I am sorry that I cannot dig deeper, but hopefully this will help!

Thanks so much for all that you do!

from aiounifi.

Thanks for your awesome help guys! I will follow up on your comments tonight

from aiounifi.

Seems like @JohnLahr and @jimmypuckett have been quicker than I was. Need another sample set of data or are we good for now?

from aiounifi.

So what is the next step in getting this in master? Is there anything that I can do to help?

from aiounifi.

I need to have some time to write tests now to make sure that it works as intended on both UDMP and ordinary controller

from aiounifi.

OK, sounds great. Let me know if I can help in any way. Thanks again for all that you have done!

from aiounifi.

I think next step would be for you guys to verify it works when everything has been merged to hass dev

from aiounifi.

Is there a way to know that it has been merged so that I know to test?

from aiounifi.

Wait, I was meaning how is the best way to know that it is hass so that I can test. I did not know if there was something to subscribe/watch.

from aiounifi.

@jimmypuckett I will notify in this thread when it is merged to hass

from aiounifi.

Just merged support to HASS dev. Feel free to try it out!

from aiounifi.

Did this get rolled into .108? I seem to have issues trying to add my controller.

from aiounifi.

No its ok hass dev. It will be a part of 0.109. Please try out the dev branch to verify it works as expected

from aiounifi.

Ah, thanks much, I will give it a shot when I get a chance this weekend. Thanks for all the work.

from aiounifi.

FYI, I just tried this with the UDM Pro and it connects and lists devices as expected. So great work with the fix.
Unfortunately all devices are listed in state 'home' and never goes to 'away'. I don't think this has anything to do with this integration. This is most likely due to some rather big bugs in the current UDM Pro software (UDM firmware 1.6.6).

from aiounifi.

If you can enable debug in hass pet unifi integration instructions and share the logs we can see what is happening

from aiounifi.

Hi all, and sorry for brining up an old issue. I was able to log in and see all the devices, and adding ones I want to block access for. Problem is when I try to block access I get the following error, is this related to the new UDMPro UnifiOS and changed paths?
aiounifi.errors.ResponseError: Call https://192.168.1.1:443/proxy/network/api/s/default/cmd/stamgr received 404 Not Found

from aiounifi.

Hi @tnowakow,

I'm also running on a udm pro. My own python script is working and I can block and unblock clients. Basically the path is correct but depends if you're sending the right header (cookie and x-csrf-token) and the json payload (cmd & mac).

from aiounifi.

@phen0 Seems to be working correctly for me as well now so not sure what changed, but good for now :)

from aiounifi.