muxable / rtchat Goto Github PK

The link preview widget seems to refresh every time there's a new message. We should either:

1. Make the inputs stable so it doesn't cause a refresh when the contents change.
2. Cache the results so it renders immediately.

We should probably start writing some tests...

The models are probably the easiest thing to test first. Getting some of the TTS logic tested would be nice.

While message delivery is pretty robust right now, it's hard to tell if messages are being delayed due to lack of connection. A message that shows up when you're offline would be nice.

Twitch bolds the @muxfd tags. Additionally, tagging the active user gets a red highlight and fancy style.

Thanks @juaoose for finding these links, and the links cannot be previewed. Should be a easy fix.

https://clips.twitch.tv/ObliviousTangibleMarjoramPastaThat-1mJRvHUbilZyszko
https://www.twitch.tv/videos/1088842867

It would be nice if we indexed the message contents with Algolia or something.

I want to be able to tell when someone mentions muxfd in any chat room. 😈

Right now links are opened with url_launcher. Opening them with ChromeSafariBrowser would probably be faster and smoother.

This should be reasonably simple to do, so it would be a good first issue if you're interested in contributing.

It would be cool to receive sharing intents from other apps and populate the text field automatically. This plugin scaffolds the required steps: https://pub.dev/packages/receive_sharing_intent

It seems easier to not use the plugin though and instead write our own native code. We only need to handle text intents.

Right now we don't display channel point redemption or subscription notifications. We will have to go through Twitch PubSub in the agent: https://d-fischer.github.io/twitch-pubsub-client/

Allow the app to be closed from the Android notification. We will also need to re-invoke the background service when the app is opened.

It would be nice to get an option to add timestamps to the chat history.

I like Twitch's approach where the timestamps are actually shown as a message. Maybe one idea is we can add a clock or something that shows the timestamp of the bottom-most message?

In other words, the key idea is that we don't need the timestamp for every message. For a specific message, we can probably show the timestamp in the long press UI and the timestamp in the chat history is just a rough guideline for when that message was sent.

This is technically quite easy to do, but doing it in a way that's elegant and doesn't impose on the UI will be tricky.

Resizing the layout is sort of slow because the webview makes everything slow. We can optimize this by clipping the webview (like what happens when chat history is scrolled) and then when the drag is released, actually do the resize.

Right now, tts gets killed if the app is switched out of because it doesn't run in the foreground.

We can resolve this by using something like https://pub.dev/packages/audio_service to keep a foreground notification and have it play. There's an example of it interacting with flutter_tts in the instructions, however keeping the Firestore connection alive could be a little trickier since the background task runs in an isolate.

It seems that the new v0.18 removes the need to run in an isolate which seems a bit nicer: https://github.com/ryanheise/audio_service/wiki/Migration-Guide#0180-preview

It would be nice to be able to use the app without signing in. The agent supports it, so once we have channel switching this should be possible.

I did a 3 hour stream playing with rtchat and noticed it crashing 5-7 times randomly. Not sure if its because I am running a youtube preview tab or/and with TTS enabled. I am on android Samsung Galaxy S20 Ultra on android 11, on latest OS patch and latest version of rtchat.

If two messages occur from the same sender, there's no point in repeating the author. eg:

muxfd: wat
muxfd: lol

currently produces:

muxfd said: wat. muxfd said: lol

but it would be better to announce

muxfd said: wat. lol.

There are perhaps more clever solutions as well such as selectively reordering messages:

muxfd1: moo
muxfd2: asdf
muxfd1: cows
muxfd2: oink

can probably be announced as

muxfd1 said: moo. cows. muxfd2 said: asdf. oink.

Likely, this coalescing should only occur if the messages are enqueued. That is, if the first message is already spoken by the time the second one comes in, messages should not coalesce.

We can get around some rate limits by using the user's oauth token to listen to their own channel. The big caveat here is that we might not necessarily have the user's oauth token, for example if I join someone else's channel and they haven't signed up for RealtimeChat yet.

So we have to be a little clever about choosing the identity for a channel. Likely we should use the user's oauth if available and if not, fall back to either anonymous or using the RealtimeChat bot.

It would be nice to get link previews. That way I don't need to guess what clip people are posting in chat :)

Right now when the agent swaps there's a small period of downtime. We can resolve this by adding a readiness check: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/

This will probably need to trigger after all the joins are done.

A toggle to mute bots in tts. It should be pretty simple tbh.

The "Send a message..." input doesn't defocus for some reason after the keyboard is closed. This causes the keyboard to pop up whenever the main page is navigated to, which is mildly annoying.

Right now the agent randomly crashes with no information in the logs. I suspect it's a bug with tmi.js. We might be better off rolling a direct IRC connection and managing it ourselves, but it requires some investigation.

If you're interested in this issue, let me know and I can give you access to the GKE container.

Support joining many channels and merging them. We should guard this behind a setting because it's rather complex, but it can be powerful for those who multiplatform stream.

The twitch menu when clicking on a username is a lot useful than the four-button long press menu that shows up right now, not to mention the buttons don't actually work. It would be nice to have a similarly useful popup come up when long pressing a message.

We can probably start off with a subset of functionality but making it look nicer with some more superficial information would be a great first step.

The default twitch chat has sub messages. These are also sent in the IRC API. We should support these as well.

If you're interested in this one, please let me (muxfd) know on Discord. There's some code you can model after pretty easily.

It would be ideal to use a platform that allows the community to provide and evaluate multiple translations for the app.

A candidate is: localazy

Long press menu is shown on every message pressed. We like to limit the menu press to mods and channel owner only.

An easier way to disable TTS would be nice. The difficulty with this feature is that there seems to be a bug in audio_service where the notification doesn't get dismissed when playing is set to false.

This issue is therefore probably quite difficult but would be a great way to learn about how flutter plugins are written.

When chat moves too fast it can be difficult to read. We can improve this by rendering the messages in batches, eg every 1s render the newest messages instead of instantaneously.

It's kind of annoying to get a million catJAM messages. Would be nice to aggregate emote-only or similar messages together into a more compact ui. ie, instead of

muxfd1: catJAM
muxfd2: catJAM
muxfd3: catJAM

We could do something like

muxfd1, muxfd2, +2 others: catJAM

This would need to be a little more intelligent around handling multiple emotes, but it would be really cool.

Permit choosing another channel to join. Also would be nice to save a shortlist of channels.

When the drag handle is dragged too far, it causes a layout error which essentially crashes the app.

It would be cool to have long time followers highlighted or otherwise emphasized so they stand out among the messages.

We can reduce contrast of likely spam or new messages, although the exact behavior we'll need to play around with.

It's basically BTTV/FFZ, but we should support 7tv's emotes.

TTS is a bit aggressive right now. It would be great to filter out emotes, restrict usernames, etc.

Right now there's no way to pick emotes. Would be good to have an emote picker sort of similar to the twitch one.

The scrubber for dragging shouldn't just be the handle. Most people seem to try dragging the username.

They're not supported. Let's just support them by default.

Perhaps we can detect when the user types a command like !b and add it to the list and keep track of the most used commands in an LRU cache.

We can use Google's online TTS to deliver higher quality TTS voices: https://cloud.google.com/text-to-speech

This would be good to gate behind an in-app purchase. It can generate a small amount of revenue which will help cover the API + server costs.

• Mute BTTV/FFZ emotes
• Mute 7TV emotes

Custom added bots are not muted at the moment.

• form added to setting page
• mute all bots including the custom ones when muted bot is enabled

Most of the code is written to one day support a non-twitch provider. YouTube would be a good next step. A single rtchat account can be linked to at most one twitch and at most one YouTube account.