[BUG](Metric) Plots in the UI only show the latest data point about mlflow HOT 7 CLOSED

We're using Bitnami's helm chart and they didn't upgrade yet (server-side on 2.11.3). Will report back once they do.

from mlflow.

Could you share the code used for logging the metrics?

from mlflow.

We're using NVIDIA's NeMo, specifically the MTEncDec model. The logging call is this:

https://github.com/NVIDIA/NeMo/blob/main/nemo/collections/nlp/models/machine_translation/mt_enc_dec_model.py#L435

from mlflow.

@mlflow/mlflow-team Please assign a maintainer and start triaging this issue.

from mlflow.

Hi, I am having the same issue with mlflow tracking server and client 2.11.3 (using pytorch lightning MLFlowLogger logger.log(..., on_step=True, on_epoch=True)). The issue appeared after upgrading mlflow. More precisely, the values logged at each gradient step are shown properly but the values logged at each epoch (average train loss and validation loss) show as a single point.
It seems the logging call refered to in the link above is a pytorch lightning logger call, so the root cause is likely the same.

from mlflow.

Does the issue still persist if you upgrade the tracking server to 2.12.1? There was a bug fix to sampling logic that should be shipped in the latest version.

from mlflow.

This appears to be fixed with 2.12.1 which is now available. Thanks!

from mlflow.