Make aw-qt save logs instead of each module about aw-qt HOT 5 CLOSED

I think the proper way to do it would be to let aw-qt capture stdout and stderr instead of doing it in aw-core and then also make aw-qt responsible for saving the logs to disk.

Sounds like you're just passing the problem along to aw-qt? What if aw-qt crashes? What if I want to run without aw-qt?

If aw-qt captures stdout and stderr, then you won't see output from the modules when you run aw-qt (which makes development a lot less convenient, at least for me). There might be ways to both capture and still pass through to stdout/stderr, but I don't know how that works, not from the SO answer you linked anyway (logger writes to stderr -> stderr writes to logger -> logger writes to stderr -> and so on...).

I think the easy thing is to just make sure that any uncaught exceptions get logged with traceback before crashing (shouldn't be too hard). Surely this must be a common problem, with solutions for both Python and Rust? (I assume...)

There are some errors that I guess you can't redirect to logs (like panics in Rust, I assume). But most of the time the issue is that we just don't log uncaught exceptions, which should be fixable.

from aw-qt.

Sounds like you're just passing the problem along to aw-qt?

When you are the process starting the subprocess you have the direct access to decide how to handle stdout/stderr and can for example pipe the output both to a file and to stdout/stderr. When you are an application which have already started it is not possible to capture stdout/stderr as easily because the filedescriptor is already created and you have to kind of wrap around an already existing filedescriptor.

What if aw-qt crashes?

If aw-qt crashes we still have the same issue as today, lack of logging stderr.
But from most reports I've seen (if not all) in github issues it's been the modules which are crashing and not aw-qt.

What if I want to run without aw-qt?

Then you will have to handle the logging yourself. Most people who do this I expect to have some kind of experience with looking at the stdout/stderr from the application though. I actually think this makes more sense, the watchers get less bloated and don't unknowinly write logs to a file if you don't expect it to. It works as most other processes do. It also allows you to for example skip aw-qt logging and instead for example use the systemd logging system without having to log to two places at the same time.

If aw-qt captures stdout and stderr, then you won't see output from the modules when you run aw-qt (which makes development a lot less convenient, at least for me). There might be ways to both capture and still pass through to stdout/stderr, but I don't know how that works, not from the SO answer you linked anyway (logger writes to stderr -> stderr writes to logger -> logger writes to stderr -> and so on...).

You can tee stdout/stderr to both a file and to stdout/stderr, this should not be an issue. The SO answer I linked is how to solve it with our current solution of each module handling logging itself, that is what I find to be complicated as well.

I think the easy thing is to just make sure that any uncaught exceptions get logged with traceback before crashing (shouldn't be too hard). Surely this must be a common problem, with solutions for both Python and Rust? (I assume...)

There are some errors that I guess you can't redirect to logs (like panics in Rust, I assume). But most of the time the issue is that we just don't log uncaught exceptions, which should be fixable.

Then we would have to do this individually for all of our modules and if there ever comes new third party modules they would have to do the same. I'm sure it's possible with python and it also seems to be possible with rust. A panic in rust is a controlled action, the only case where it's not controlled would be in case of some type of process error like segmentation fault, buserror etc. but that also goes for python.

In general I think it makes more sense for aw-qt to be a "process manager" and let the watchers be specialized at doing what they do and do it well, watching and reporting it to aw-server. Having a shared logging system and forcing every module to use it also works but if feels like the wrong way to go about things.

from aw-qt.

What if I want to run without aw-qt?

Then you will have to handle the logging yourself.

That's exactly my point. I feel like aw-qt shouldn't be a requirement to have working logging for the modules, and since I then need to implement logging for the modules anyway, then what's the point of doing the work twice in aw-qt?

What if aw-qt crashes?

If aw-qt crashes we still have the same issue as today, lack of logging stderr.

My point was that then the logging to file would stop, even if the processes themselves survive. stderr shouldn't contain anything that isn't also in the logs if we just fix the small issue of getting stacktraces to show up in them (which is easy).

I actually think this makes more sense, the watchers get less bloated and don't unknowinly write logs to a file if you don't expect it to.

Logging doesn't add any significant "bloat".

"Unknowingly write logs to a file if you don't expect it to" seems like a very weird argument. I wouldn't expect aw-qt to write the logs when it's just a UI to start the modules.

Then we would have to do this individually for all of our modules and if there ever comes new third party modules they would have to do the same.

But this makes sense to me. Modules should handle logging themselves because they know best what to do with them, not some UI that happens to manage them on some platforms (that GNOME users can't even use).

It also allows you to for example skip aw-qt logging and instead for example use the systemd logging system without having to log to two places at the same time.

If you really want to use systemd (which I assume very few do), then logging to two places at the same time isn't a big deal. I don't find this argument very convincing.

Having a shared logging system and forcing every module to use it also works but if feels like the wrong way to go about things.

But that's basically what you're proposing by moving it all into aw-qt? Modules that want to handle logging themselves would be forced to log to two places at the same time.

Right now we're not forcing every module to use a shared logging system. It's just in aw-core for convenience/code-reuse. Modules can handle logging however they like (which is much simpler).

In general I think it makes more sense for aw-qt to be a "process manager"

In general, I don't think it makes sense to think about aw-qt as a "process manager". It's just a UI that happens to manage processes.

I'm really not convinced by moving all logging to aw-qt just so we can get stderr output into them. It just doesn't feel like it should be the responsibility of aw-qt to have something basic like logging to work. It also breaks functionality in the Python logging module (where you can, for example, filter some messages/loglevels to not appear in stderr but appear in the file).

You kinda make it seem like logging is some big issue to do in the modules themselves, but it's actually rather trivial and it works just fine as it is (apart from stacktraces not being in there, which is easy to fix). Moving logging to aw-qt would make it more complex, and would make modules essentially depend on it for logging. I just don't see the value that would add.

I'm closing this since I really don't like this idea, I'll work on making sure stacktraces show up in the logs instead.

from aw-qt.

@johan-bjareholt The fix was trivial, see: ActivityWatch/aw-core#93

from aw-qt.

In general, I don't think it makes sense to think about aw-qt as a "process manager". It's just a UI that happens to manage processes.

I think this is the root cause of why we disagree about this.

In my opinion it matters little whether it's "a UI that happens to manage processes" or "a process manager which happens to have a UI", the point is that it does both and becomes useless if it only does one of these things.

I also believe that now when aw-qt also supports third-party watchers it becomes more important to be a good process manager and a common way to manage logs makes sense to me no matter what programming language or libraries it is written with.

from aw-qt.