varadaio / presto-workload-analyzer Goto Github PK

Following https://workload-analyzer.slack.com/archives/C01H9LTDZFX/p1621536293010900.

The analyzer is not able to deal with DynamicFilterSourceOperator to generate the graphs related to joins:

ERROR: analyze: missing keys ('LookupJoinOperator', 'HashBuilderOperator') in {'DynamicFilterSourceOperator': {'node_id': '398', 'type': 'DynamicFilterSourceOperator', 'input_size': 17950.0, 'output_size': 17950.0, 'network_size': 0.0, 'input_rows': 359, 'output_rows': 359, 'network_rows': 0, 'peak_mem': 0.0, 'input_cpu': 0.0007441399999999999, 'output_cpu': 0.00037216, 'finish_cpu': 0.0, 'input_wall': 0.0007452899999999999, 'output_wall': 0.00036805, 'finish_wall': 0.0, 'blocked_wall': 0.0}, 'HashBuilderOperator': {'node_id': '398', 'type': 'HashBuilderOperator', 'input_size': 17950.0, 'output_size': 17950.0, 'network_size': 0.0, 'input_rows': 359, 'output_rows': 359, 'network_rows': 0, 'peak_mem': 196860.0, 'input_cpu': 0.0008561799999999999, 'output_cpu': 0.0, 'finish_cpu': 0.0, 'input_wall': 0.0008708499999999999, 'output_wall': 0.0, 'finish_wall': 0.0, 'blocked_wall': 0.0}}

I am using trino 406 and faced below errors while doing extract and I am using python 3.8.
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 11342.53files/s]
[2023-07-08 12:47:15.354243] INFO: analyze: 28 queries loaded
0%| | 0/28 [00:00<?, ?graphs/s][2023-07-08 12:47:15.685055] ERROR: analyze: failed to generate scheduled_by_date
Traceback (most recent call last):
File "./analyze.py", line 1254, in main
item["doc"]["roots"]["references"].sort(key=lambda r: (r["type"], r["id"]))
TypeError: list indices must be integers or slices, not str
4%|████▉ | 1/28 [00:00<00:08, 3.04graphs/s][2023-07-08 12:47:15.791569] ERROR: analyze: failed to generate scheduled_by_hour
Traceback (most recent call last):
File "./analyze.py", line 1254, in main
item["doc"]["roots"]["references"].sort(key=lambda r: (r["type"], r["id"]))
TypeError: list indices must be integers or slices, not str
7%|█████████▊ | 2/28 [00:00<00:05, 5.05graphs/s][2023-07-08 12:47:15.887112] ERROR: analyze: failed to generate input_by_date
Traceback (most recent call last):
File "./analyze.py", line 1254, in main
item["doc"]["roots"]["references"].sort(key=lambda r: (r["type"], r["id"]))
TypeError: list indices must be integers or slices, not str
[2023-07-08 12:47:15.976779] ERROR: analyze: failed to generate input_by_hour
Traceback (most recent call last):
File "./analyze.py", line 1254, in main
item["doc"]["roots"]["references"].sort(key=lambda r: (r["type"], r["id"]))
TypeError: list indices must be integers or slices, not str
14%|███████████████████▋ | 4/28 [00:00<00:03, 7.58graphs/s][2023-07-08 12:47:16.111376] ERROR: analyze: failed to generate queries_by_date
Traceback (most recent call last):
File "./analyze.py", line 1254, in main
item["doc"]["roots"]["references"].sort(key=lambda r: (r["type"], r["id"]))
TypeError: list indices must be integers or slices, not str
18%|████████████████████████▋ | 5/28 [00:00<00:03, 7.53graphs/s][2023-07-08 12:47:16.222095] ERROR: analyze: failed to generate queries_by_hour
Traceback (most recent call last):
File "./analyze.py", line 1254, in main
item["doc"]["roots"]["references"].sort(key=lambda r: (r["type"], r["id"]))
TypeError: list indices must be integers or slices, not str
21%|█████████████████████████████▌ | 6/28 [00:00<00:02, 7.94graphs/s][2023-07-08 12:47:16.341786] ERROR: analyze: failed to generate peak_mem_by_query