starburstdata / facebook-presto Goto Github PK

After upgrading to 0.203-e.0.2, we observed that when there is a query in RUNNING state, all subsequent queries get QUEUED until the first query finishes. We were previously using 0.195-e.0.6 and did not have this problem. The symptom is the same as described in prestodb#10383. Is the change incorporated into Starburst 0.203-e.0.2?

We do have a very simple resource group configuration file like this:

{
    "rootGroups": [
      {
        "name": "global",
        "softMemoryLimit": "100%",
        "hardConcurrencyLimit": 100,
        "maxQueued": 1000,
        "schedulingPolicy": "weighted",
        "jmxExport": true
      }
    ],
    "selectors": [
      {
        "group": "global"
      }
    ]
  }

A managed internal table with Bucketing (5 buckets) contains the following files:
000000_0
000001_0
000003_0
000005_0

On inserting new records, the files look like this:
000000_0_copy_1
000000_0_copy_2
000000_0_copy_3
000001_0
000003_0
000003_0_copy_1
000003_0_copy_2
000003_0_copy_3
000004_0

This causes Presto to fail, however the Hive Connector changes fixed this issue via implementing hive.empty-bucketed-partitions.enabled and hive.multi-file-bucketing.enabled.

I have run into an issue when loading the above table into another via a CREATE TABLE as SELECT... statement. Hive renames the files to look like:
000000_0
000001_0
000002_0
000003_0
000004_0
000005_0
000006_0
000007_0
000008_0

This causes the following error The number of files in the directory (9) does not match the declared bucket count (5) . I assume for a table with 5 buckets, Presto expects the prefix to range from 000000_ to 000004_ and it's the 000005_ to 000008_ prefixes that cause the PrestoException(HIVE_INVALID_BUCKET_FILES,...) exception. The table is accessible from Hive and Spark but not via Presto.

As a workaround, if a LIMIT clauses is added to the CREATE TABLE as SELECT... statement, it forces hive to create files equal to the number of buckets defined.

I've read an article about work on CBO in tech blog. Awesome results folks! I'm wondering if there's a design doc to read about current work on CBO?

The KILL button in 0.203-e.0.7 UI does not seem to be working. When the button is pushed on a query details page (e.g. https://myserver:8443/ui/query.html?20180828_153406_00009_2vwx5), the following error can be seen from JavaScript console:

Failed to load resource: the server responded with a status of 404 (Not Found)
PUT https://myserver:8443/ui/v1/query/20180828_153406_00009_2vwx5/killed 404 (Not Found)

Inspect the PUT request with Google Chrome Devtools, it reveals:

Request URL: https://myserver:8443/ui/v1/query/20180828_153406_00009_2vwx5/killed
Request Method: PUT
Status Code: 404 Not Found

I think the URL path should have been (without /ui/) https://myserver:8443/v1/query/20180828_153406_00009_2vwx5/killed.