cathaypacific8747 / fr24 Goto Github PK

View Code? Open in Web Editor NEW

15.0 3.0 3.0 2.57 MB

Download and parse data from flightradar24.com with gRPC/JSON

Home Page: https://cathaypacific8747.github.io/fr24/

License: MIT License

Python 100.00%

ads-b flightradar24 python grpc

fr24's Introduction

Hello 👋

I am a second-year undergraduate majoring in Aerospace Engineering with an extended major in Artificial Intelligence.

My research interests lie in the intersection of aerospace and machine learning, with a particular focus on air traffic management and trajectory prediction.

I think I can finish this in one day. ™

Technologies

Familiar with:

Backend - Python: FastAPI/scrapy/bs4/PyTorch, C++
Frontend - HTML/CSS/TypeScript: React, Google Apps Script
Databases - postgres/mysql/duckdb
Other - QGIS

Current learning:

JAX, Rust
rusty - Cython, Processing, Arduino, Small Basic

Some of my projects

it should work 😁

Name	Links	Technologies
AM4 Discord bot & am4help.com backend - Various calculators and utilities to achieve maximum profit in the game Airline Manager 4	bot discord, bot code, website, backend code	C++, Python, ExpressJS, MySQL
MTR Billboard - Platform for accelerating billboard inspections at MTRC	website (restricted)	ReactJS, FastAPI, pg
EI/MM - Tool to digitise reports into XLSX files	restricted	ReactJS, Flask, pg, tesseract
Swappy - Online bookselling platform, product of CKY Hackathon 2021.	code	HTML/CSS/JS, flask, postgres
IB Geography Extended Essay - "An investigation into the traffic congestion in Kwun Tong, Hong Kong."	code, paper	Python, QGIS
IB Geography Internal Assessment - "To what extent does Hong Kong fit the pattern of decreasing land values with increasing distance from the Peak Land Value Intersection?"	code, paper	Python, QGIS
IB Physics Internal Assessment - "How does the length of the filars in a bifilar pendulum affect its period?"	code, paper	Python
IB Mathematics Internal Assessment - "Modelling the volume and surface area of the cabin of an A350-900"	paper	Python
IB Chemistry Internal Assessment - "An investigation into the effect of pH of wastewater on the adsorption of copper (II) ions."	paper	Python
IB Physics Practice Internal Assessment - "What is the relationship between the wrap angle of a capstan and the loading force?"	paper	Python
IB Selection form - Custom course selection form for incoming IB students at CKY	restricted	ReactJS, Google Apps Script
BTEHK 3D Map - 3D map/visualisation of Hong Kong for builders of the BuildTheEarth HK Community.	website code	HTML/CSS/JS
MTR ETA Explorer - Website to query real-time second-level MTR ETAs.	website code	HTML/CSS/JS
KMB ETA Explorer - Website to query real-time ETAs for 81/72 buses at CKY.	website code	HTML/CSS/JS
AM4 Stopover Finder - Early Website for finding stopovers of routes in AM4.	Website code	HTML/CSS/JS

Why cathaypacific8747?

On 8 October 2016, Cathay Pacific bid farewell to its last Boeing B747-400 passenger aircraft, marking the end of an era. The 747, renowned for its pioneering role in modern-day air travel since the 1960s, became an iconic symbol in the history of aviation. To commemorate the retirement, Cathay Pacific organised a special flight that included a breathtaking low-altitude flyby over Victoria Harbour, at 2,000 feet.

The elegance of the flyby was truly astonishing and the scene has stuck with me ever since.

⬆️ flight CX8747, taken by me at the West Kowloon Cultural District

fr24's People

Contributors

Stargazers

Watchers

Forkers

glostis sejo21

fr24's Issues

crashing when retrieving feed today

Seems to have stopped working today

(grpc) [root@smj-centos-6 grpc]# fr24 feed
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /root/e1radar/grpc/lib64/python3.11/site-packages/fr24/cli.py:98 in feed │
│ │
│ 95 │ │ │ lf.data.save_parquet() │
│ 96 │ │ │ rich.print(get_success_message(lf)) │
│ 97 │ │
│ ❱ 98 │ asyncio.run(feed_()) │
│ 99 │
│ 100 │
│ 101 if name == "main": │
│ │
│ ╭─────────────────────────── locals ───────────────────────────╮ │
│ │ feed_ = <function feed..feed_ at 0x7fddb33f3b00> │ │
│ │ time = None │ │
│ │ timestamp = None │ │
│ ╰──────────────────────────────────────────────────────────────╯ │
│ │
│ /usr/lib64/python3.11/asyncio/runners.py:190 in run │
│ │
│ 187 │ │ │ "asyncio.run() cannot be called from a running event loop") │
│ 188 │ │
│ 189 │ with Runner(debug=debug) as runner: │
│ ❱ 190 │ │ return runner.run(main) │
│ 191 │
│ 192 │
│ 193 def cancel_all_tasks(loop): │
│ │
│ ╭───────────────────────────── locals ──────────────────────────────╮ │
│ │ debug = None │ │
│ │ main = <coroutine object feed..feed at 0x7fddb3400c70> │ │
│ │ runner = <asyncio.runners.Runner object at 0x7fddb321fd50> │ │
│ ╰───────────────────────────────────────────────────────────────────╯ │
│ │
│ /usr/lib64/python3.11/asyncio/runners.py:118 in run │
│ │
│ 115 │ │ │
│ 116 │ │ self.interrupt_count = 0 │
│ 117 │ │ try: │
│ ❱ 118 │ │ │ return self.loop.run_until_complete(task) │
│ 119 │ │ except exceptions.CancelledError: │
│ 120 │ │ │ if self.interrupt_count > 0: │
│ 121 │ │ │ │ uncancel = getattr(task, "uncancel", None) │
│ │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │ context = <contextvars.Context object at 0x7fddb3a2dec0> │ │
│ │ coro = <coroutine object feed..feed at 0x7fddb3400c70> │ │
│ │ self = <asyncio.runners.Runner object at 0x7fddb321fd50> │ │
│ │ sigint_handler = functools.partial(<bound method Runner.on_sigint of │ │
│ │ <asyncio.runners.Runner object at 0x7fddb321fd50>>, main_task=<Task │ │
│ │ finished name='Task-1' coro=<feed..feed() done, defined at │ │
│ │ /root/e1radar/grpc/lib64/python3.11/site-packages/fr24/cli.py:91> │ │
│ │ exception=AssertionError()>) │ │
│ │ task = <Task finished name='Task-1' coro=<feed..feed() done, defined at │ │
│ │ /root/e1radar/grpc/lib64/python3.11/site-packages/fr24/cli.py:91> │ │
│ │ exception=AssertionError()> │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /usr/lib64/python3.11/asyncio/base_events.py:653 in run_until_complete │
│ │
│ 650 │ │ if not future.done(): │
│ 651 │ │ │ raise RuntimeError('Event loop stopped before Future completed.') │
│ 652 │ │ │
│ ❱ 653 │ │ return future.result() │
│ 654 │ │
│ 655 │ def stop(self): │
│ 656 │ │ """Stop running the event loop. │
│ │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │ future = <Task finished name='Task-1' coro=<feed..feed() done, defined at │ │
│ │ /root/e1radar/grpc/lib64/python3.11/site-packages/fr24/cli.py:91> │ │
│ │ exception=AssertionError()> │ │
│ │ new_task = False │ │
│ │ self = <UnixSelectorEventLoop running=False closed=True debug=False> │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /root/e1radar/grpc/lib64/python3.11/site-packages/fr24/cli.py:94 in feed │
│ │
│ 91 │ async def feed() -> None: │
│ 92 │ │ async with FR24() as fr24: │
│ 93 │ │ │ lf = fr24.livefeed(timestamp) │
│ ❱ 94 │ │ │ lf.data.add_api_response(await lf.api.fetch()) │
│ 95 │ │ │ lf.data.save_parquet() │
│ 96 │ │ │ rich.print(get_success_message(lf)) │
│ 97 │
│ │
│ ╭───────────────────────────── locals ─────────────────────────────╮ │
│ │ fr24 = <fr24.core.FR24 object at 0x7fddb3228710> │ │
│ │ lf = <fr24.core.LiveFeedService object at 0x7fddb3228dd0> │ │
│ │ timestamp = None │ │
│ ╰──────────────────────────────────────────────────────────────────╯ │
│ │
│ /root/e1radar/grpc/lib64/python3.11/site-packages/fr24/core.py:366 in fetch │
│ │
│ 363 │ │ │ │ **kw, # type: ignore[arg-type] │
│ 364 │ │ │ │ auth=self.http.auth, │
│ 365 │ │ │ ) │
│ ❱ 366 │ │ resp = await livefeed_world_data(self.http.client, self.http.auth) │
│ 367 │ │ self.ctx["timestamp"] = int(time.time()) │
│ 368 │ │ return resp │
│ 369 │
│ │
│ ╭──────────────────────── locals ─────────────────────────╮ │
│ │ self = <fr24.core.LiveFeedAPI object at 0x7fddb3228e50> │ │
│ │ ts = None │ │
│ ╰─────────────────────────────────────────────────────────╯ │
│ │
│ /root/e1radar/grpc/lib64/python3.11/site-packages/fr24/livefeed.py:224 in livefeed_world_data │
│ │
│ 221 │ client: httpx.AsyncClient, auth: None | Authentication = None │
│ 222 ) -> list[LiveFeedRecord]: │
│ 223 │ """Retrieve live feed data for the entire world, in chunks.""" │
│ ❱ 224 │ results = await asyncio.gather( │
│ 225 │ │ *[ │
│ 226 │ │ │ livefeed_post( │
│ 227 │ │ │ │ client, │
│ │
│ ╭─────────────────────── locals ────────────────────────╮ │
│ │ auth = None │ │
│ │ client = <httpx.AsyncClient object at 0x7fddb3228850> │ │
│ ╰───────────────────────────────────────────────────────╯ │
│ │
│ /root/e1radar/grpc/lib64/python3.11/site-packages/fr24/livefeed.py:176 in livefeed_post │
│ │
│ 173 │ """Send the request and extract the raw protobuf message.""" │
│ 174 │ response = await client.send(request) │
│ 175 │ data = response.content │
│ ❱ 176 │ assert len(data) and data[0] == 0 │
│ 177 │ data_len = int.from_bytes(data[1:5], byteorder="big") │
│ 178 │ return data[5 : 5 + data_len] │
│ 179 │
│ │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │ client = <httpx.AsyncClient object at 0x7fddb3228850> │ │
│ │ data = b'' │ │
│ │ request = <Request('POST', │ │
│ │ 'https://data-feed.flightradar24.com/fr24.feed.api.v1.Feed/LiveFeed')> │ │
│ │ response = <Response [464 ]> │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯

How to pull back squawk code from live feed

Assuming I am using the example world feed code and I am authenticated...

from fr24.core import FR24

async def my_feed() -> None:
    async with FR24() as fr24:
        lf = fr24.livefeed()
        response = await lf.api.fetch()
        print(response)
        lf.data.add_api_response(response)
        print(lf.data.df)
        lf.data.save_parquet()
        print(lf.data.fp)

await my_feed()

I can see that there is reference in one of the lower level functions ( livefeed_message_create) to return optional extra fields if I am authenticated

When authenticated, squawk, vspeed, airspace, logo_id and age can be included

Is there a way to do this from the higher level core calls? I want to pull back the squawk code for all aircraft in the livefeed.

Thanks ( I am sure there is - I'm just not sure how i would do it from the top level)

Authentication issues

I don't have time to dig much more into this right now, so let me document it here first before I forget. I currently have some authentication issues (context: trying to port my code from the older version to the newer API, and struggling to find where I could just get the JSON before the pyarrow...)

More generally, it would be nice to have an extra option as I could prefer to run anonymously by default, and login only when necessary:

async with FR24(login: bool | {'username': user, 'password': pass}) as fr24:
    ...

Currently, on my side, authentication in the asynccontextmanager fails with

ConnectError: All connection attempts failed

Surprisingly, in the TUI part of the code, I just use the following (without the FR24 client) and it still works:

self.auth = await login(self.client)

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.