Hi Would it be possible to add custom labels to pushed logs, somethi

you can do it by creating a custom logger class extending the <code class="notranslate

Add custom labels to pushed logs about airflow-provider-grafana-loki HOT 2 CLOSED

snjypl commented on July 18, 2024 1

Add custom labels to pushed logs

from airflow-provider-grafana-loki.

Comments (2)

snjypl commented on July 18, 2024 1

you can do it by creating a custom logger class extending the LokiTaskHandler and overriding the get_labels method. here is an example:

~/airflow/config/lokilogger.py

from grafana_loki_provider.log.loki_task_handler import LokiTaskHandler
from typing import Dict, TYPE_CHECKING

if TYPE_CHECKING:
    from airflow.models.taskinstance import TaskInstance

class CustomLokiHandler(LokiTaskHandler):

    def get_labels(self, ti: "TaskInstance")-> Dict[str, str]:
        labels = super().get_labels(ti)
        labels['run_id'] =str(ti.run_id)
        labels['my_custom_label']='xyz'
        return labels

update the airflow log_config to use the custom class

       elif REMOTE_BASE_LOG_FOLDER.startswith('loki'):
        LOKI_HANDLER: Dict[str, Dict[str, Union[str, bool]]] = {
            'task': {
                'class': 'lokilogger.CustomLokiHandler',
                'formatter': 'airflow',
                'name':"airflow_task",
                'base_log_folder': str(os.path.expanduser(BASE_LOG_FOLDER)),
                'filename_template': FILENAME_TEMPLATE
            },
        }

        DEFAULT_LOGGING_CONFIG['handlers'].update(LOKI_HANDLER)

from airflow-provider-grafana-loki.