crashing when retrieving feed today about fr24 HOT 9 CLOSED

Thanks. I just tried to add the line into my code but i get an error when i try it. Its bound to be me doing it wrong - i dont fully understand all the async stuff as yet. Here is a simplified bit of working code for me.

from fr24.core import FR24
import asyncio
import httpx

async def my_feed() -> None:
    async with FR24() as fr24:
        lf = fr24.livefeed()
        response = await lf.api.fetch()
        print(response)
        lf.data.add_api_response(response)
        print(lf.data.df)
        lf.data.save_parquet()
        print(lf.data.fp)

a=asyncio.run(my_feed())

If I add the suggested lines for http2 into it i get errors

Any ideas what I'm doing wrong?

EDIT : Seems like a platform related issue. Doesn't work on Linux (python 3.11.5) , but does work on Windows (python 3.12.2) . Versions of httpx are both 0.27.0

from fr24.core import FR24
import asyncio
import httpx

async def my_feed() -> None:
    client = httpx.AsyncClient(http2=True)
    async with FR24(client) as fr24:
        lf = fr24.livefeed()
        response = await lf.api.fetch()
        print(response)
        lf.data.add_api_response(response)
        print(lf.data.df)
        lf.data.save_parquet()
        print(lf.data.fp)

a=asyncio.run(my_feed())

(grpc) [root@smj-centos-6 grpc]# python w2.py
Traceback (most recent call last):
File "/root/e1radar/grpc/w2.py", line 16, in
a=asyncio.run(my_feed())
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/asyncio/runners.py", line 190, in run
return runner.run(main)
^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/asyncio/base_events.py", line 653, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File "/root/e1radar/grpc/w2.py", line 8, in my_feed
lf = fr24.livefeed()
^^^^^^^^^^^^^^^
File "/root/e1radar/grpc/lib64/python3.11/site-packages/fr24/core.py", line 117, in livefeed
return LiveFeedService(
^^^^^^^^^^^^^^^^
File "/root/e1radar/grpc/lib64/python3.11/site-packages/fr24/core.py", line 414, in init
fs = LiveFeedArrow(base_dir, ctx)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/e1radar/grpc/lib64/python3.11/site-packages/fr24/base.py", line 62, in init
self.base_dir = Path(base_dir)
^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/pathlib.py", line 871, in new
self = cls._from_parts(args)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/pathlib.py", line 509, in _from_parts
drv, root, parts = self._parse_args(args)
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/pathlib.py", line 493, in _parse_args
a = os.fspath(a)
^^^^^^^^^^^^
TypeError: expected str, bytes or os.PathLike object, not AsyncClient

from fr24.

livefeed return empty data too

from fr24.

yes, the main CLI and the whole world feed uses the livefeed component I think which is what is broken. I think maybe FR24 have changed the protocol so it needs updating. Hopefully the author can fix it soon because this is a great library.

from fr24.

I am unable to reproduce the empty response on my end: the manually triggered test here also indicate all passes.

I had a quick check and there seems to be no protocol changes.

In my experience, empty responses usually occurs when you are being ratelimited. Are you authenticated with your subscription key and/or username+password?

from fr24.

@sejo21 Upon closer inspection, the response code was 464, indicating that there was a mismatch in HTTP protocol. HTTPX uses HTTP/1.1 by default: can you modify this line to httpx.AsyncClient(http2=True)

and check that pytest tests/test_misc.py::test_simple passes? If so, I think we should move towards enabling HTTP/2 by default.

from fr24.

Hi, Thankyou for looking at it.

I figured I should check the conditions again before testing the change, and interestingly it is working again without me doing anything (and had been for a while since I came across your library). I think maybe something going on at FR24 (load balancers perhaps?) or something between me and them.

Anyway I did make the modification and run the test with HTTP/2 enabled and this also completes successfully (once I had installed h2 package). Perhaps something to include for the future as you say.

In the meantime, if it happens again for me on HTTP1.1 I'll try the test using HTTP/2 and report.

Thanks again

D:\opensky data\e1radar\fr24>pytest tests/test_misc.py::test_simple
================================================= test session starts =================================================
platform win32 -- Python 3.12.2, pytest-8.2.1, pluggy-1.5.0
rootdir: D:\opensky data\e1radar\fr24
configfile: pyproject.toml
plugins: anyio-4.3.0, asyncio-0.23.7
asyncio: mode=Mode.STRICT
collected 1 item

tests\test_misc.py . [100%]

================================================== warnings summary ===================================================
:488
:488: DeprecationWarning: Type google._upb._message.MessageMapContainer uses PyType_Spec with a metaclass that has custom tp_new. This is deprecated and will no longer be allowed in Python 3.14.

:488
:488: DeprecationWarning: Type google._upb._message.ScalarMapContainer uses PyType_Spec with a metaclass that has custom tp_new. This is deprecated and will no longer be allowed in Python 3.14.

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
============================================ 1 passed, 2 warnings in 1.80s ============================================

from fr24.

Great to know it is resolved. It is likely an issue on fr24's side as I have also been encountering intermittent (though very temporary) connection errors.

I'm currently working on a new core and will be enabling HTTP/2 by default. The workaround for now is to pass in a custom AsyncClient to the FR24 constructor:

from fr24.core import FR24
import httpx

async def my_feed():
    client = httpx.AsyncClient(http2=True)  # adding transport=httpx.AsyncHTTPTransport(retries=5) also helps
    async with FR24(client) as fr24:
        ...

from fr24.

The one on Linux seems to be installed via pip, which does not accept custom AsyncClients. Try reinstalling with

pip3 install git+https://github.com/cathaypacific8747/fr24.git

to grab the latest version

from fr24.

Ah, OK. Yes that's fixed it. Just created a new venv and used the git version.

Thanks again.

from fr24.