Let users know data is sampled about highlight HOT 2 CLOSED

HIG-4646 Let users know data is sampled

from highlight.

To address the issue of data sampling discrepancies, follow these steps:

1. Backend (logs.go): Modify the ReadLogs and ReadLogsHistogram functions to check the main logs table if the sampling table returns no results. Implement a fallback mechanism within these functions.

   Example change for ReadLogs:

   if len(results) == 0 {
       // Fallback to main logs table
       // Re-run the query against LogsTableConfig instead of logsSamplingTableConfig
   }

2. Frontend (TraceLogs.tsx): Add a UI element (e.g., a banner or tooltip) to inform users about potential count discrepancies due to data sampling. This could be conditionally rendered based on a flag set by the backend indicating whether sampling data was used.

   Example UI update:

   {isSampled && <Alert message="Data is sampled and counts may not accurately reflect total records." type="warning" />}

3. Documentation/Comments: Update code comments and documentation to clearly explain the logic for handling sampled data and the potential for discrepancies in counts. This will help maintain clarity for future development and debugging.

References

/backend/clickhouse/logs.go
/frontend/src/pages/Traces/TraceLogs.tsx
/backend/clickhouse/migrations/000024_create_logs_sampling_table.up.sql
/backend/clickhouse/migrations/000043_create_traces_sampling_table.up.sql
/docs-content/general/6_product-features/1_session-replay/filtering-sessions.md
/docs-content/general/6_product-features/2_error-monitoring/filtering-errors.md

Ask Greptile

from highlight.