burnysc2 / monorepo Goto Github PK
View Code? Open in Web Editor NEWPersonal projects all in one location
License: MIT License
Personal projects all in one location
License: MIT License
Traceback (most recent call last):
File "/app/src/telegram_downloader.py", line 583, in <module>
asyncio.run(main(), debug=False)
File "/usr/local/lib/python3.10/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/usr/local/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
return future.result()
File "/app/src/telegram_downloader.py", line 556, in main
await parse_channel_messages()
File "/app/src/telegram_downloader.py", line 428, in parse_channel_messages
async for message in messages_iter:
File "/root/.cache/pypoetry/virtualenvs/transcriber-backend-9TtSrW0h-py3.10/lib/python3.10/site-packages/pyrogram/methods/messages/get_chat_history.py", line 101, in get_chat_history
messages = await get_chunk(
File "/root/.cache/pypoetry/virtualenvs/transcriber-backend-9TtSrW0h-py3.10/lib/python3.10/site-packages/pyrogram/methods/messages/get_chat_history.py", line 49, in get_chunk
return await utils.parse_messages(client, messages, replies=0)
File "/root/.cache/pypoetry/virtualenvs/transcriber-backend-9TtSrW0h-py3.10/lib/python3.10/site-packages/pyrogram/utils.py", line 101, in parse_messages
parsed_messages.append(await types.Message._parse(client, message, users, chats, replies=0))
File "/root/.cache/pypoetry/virtualenvs/transcriber-backend-9TtSrW0h-py3.10/lib/python3.10/site-packages/pyrogram/types/messages_and_media/message.py", line 590, in _parse
parsed_message.pinned_message = await client.get_messages(
File "/root/.cache/pypoetry/virtualenvs/transcriber-backend-9TtSrW0h-py3.10/lib/python3.10/site-packages/pyrogram/methods/messages/get_messages.py", line 115, in get_messages
r = await self.invoke(rpc, sleep_threshold=-1)
...
File "/root/.cache/pypoetry/virtualenvs/transcriber-backend-9TtSrW0h-py3.10/lib/python3.10/site-packages/pyrogram/connection/transport/tcp/tcp_abridged_o.py", line 61, in send
payload = await self.loop.run_in_executor(pyrogram.crypto_executor, aes.ctr256_encrypt, data, *self.encrypt)
TypeError: Value after * must be an iterable, not NoneType
Except / ignore the TypeError in
I decided to switch to htmx to make my life easier as a backend dev. This means giving up on frontend frameworks and doing server side rendering instead.
Add queries as described in https://zaidesanton.substack.com/p/the-most-underrated-skill-sql-for
Affected folders: https://github.com/BurnySc2/monorepo/tree/develop/transcribe_website/transcriber_backend
Idea:
https://docs.python.org/3/library/concurrent.futures.html#threadpoolexecutor
https://github.com/BurnySc2/monorepo/blob/daab0f49399bd98bb8ea9b1a15877dcf4b7703fb/python_examples/examples/other/multithreading_example_02.py
Why?
Easier to read and use than the asyncio syntax.
Instead of async for ...
we just use for loops
Instead of async with semaphore
we just use with semaphore
Instead of await function()
we just call the function
Instead of asyncio.create_task(func)
we just submit a new function call to the executor, which will only leave the with
statement when all tasks are completed.
Should be easier to test/mock functions
Merge https://github.com/BurnySc2/SC2-Replay-Renamer into fastapi-server to create a .zip with renamed replays.
Save settings in frontend (as cookie?)
From a discussion I had:
army lost
vs active army
Add a README.md file for each project and list the following things:
This repo contains match info and build order overlays
Rewrite them to use htmx to connect to the SC2 API
It was attempted to rewrite them in svelte here https://github.com/BurnySc2/monorepo/blob/7ba9ea45b71944ac97bba559e874854ec98ce93c/supabase_stream_scripts/src/pages/BuildOrder.svelte
Make them available for:
Add multiple modules:
Add rust example code and how to ship it to pypi as a package.
Use pyo3 and maturin according to this video
Idea: keep a rust and python source code file which are identically to be able to maintain the rust code when you only know python
Edit:
Use nim instead - seems much easier
https://github.com/Pebaz/nimporter
https://github.com/yglukhov/nimpy
Use this library https://pypi.org/project/fuzzywuzzy/#description
https://nim-lang.org/
How to set up a nim backend webserver?
How to add nim to frontend?
Also call nim code from Python https://github.com/yglukhov/nimpy
Also need to dockerize the service, and need to pull data from supabase db
https://github.com/BurnySc2/twitch-stream-announcer-webhook
Also need to dockerize the service, and need to pull data from supabase db
https://github.com/BurnySc2/twitch-sc2-irc-bot
Use https://github.com/coqui-ai/TTS to generate .mp3 files from texts, being able to convert a pdf into one or multiple .mp3 files
Will probably require frontend+backend
Change the following (and other lines):
monorepo/fastapi_server/routes/tts/websocket_handler.py
Lines 28 to 35 in daab0f4
Join irc chat anonymously instead with justinfanXXX
where XXX is a number
https://discuss.dev.twitch.com/t/anonymous-connection-using-justinfan/2046
https://gist.github.com/skntxyz/833dd97620d44c345520
https://discuss.dev.twitch.com/t/bot-cant-send-messages-to-other-channel/30021/6
In bored_gems
project I plan on using supabase. However I need to check if my application works but implementing some kind of integration tests, which means:
This way I can assure that at least basics of the application still works. However this poses a few issues:
Tasks:
Use Pony asyncpg (raw SQL queries) to access the supabase db directly instead of using the supabase library.
Alternatively if async/await syntax is prefered, use one the support postgresql and async/await from https://github.com/BurnySc2/monorepo/tree/develop/python_examples/examples/databases
Add Tortoise ORM example to https://github.com/BurnySc2/monorepo/tree/develop/python_examples/examples/databases to check out the syntax
Add to Earthfile on pre-commit a SQL query validation and formatting tool:
https://docs.sqlfluff.com/en/stable/
or
https://pypi.org/project/sqlvalidator/#description
E.g.:
http://peter-hoffmann.com/2010/python-sqlite-namedtuple-factory.html
https://stackoverflow.com/a/3300514
Try if I can use psycopg's namedtuple_rowmaker
See also
https://docs.python.org/3/library/sqlite3.html
def namedtuple_factory(cursor, row):
Because I use supabase as a globally hosted database, and discord bot to access the database, I may as well use it to frequently run checks on the database data?
Alternatively: Find a cheap domain host to host the backend again. This requires me to update the certs every year though
Where to find models for it?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.