Hi, I'm a student. I'm trying to download the Trace data: clusterData2011_10 
using gsutil with the command "gsutil ls gs://clusterdata-2011-10", the 
following error occurs:

GSResponseError:: status=404, code=NoSuchBucket, reason=Not Found.

What's the matter?

I needs your help. thanks!

A common pattern is to run masters (controllers) and workers in separate jobs (e.g., this is used in MapReduce and similar systems). A worker job that has a master job as a parent​ will automatically be terminated when the ​parent​ exits, even if its workers are still running. A job can have multiple child jobs but only one ​parent​.

Another pattern is to run jobs in a pipeline (this doesn’t apply to alloc sets). If job A says that it should ​run after​ job B, then job A will only be scheduled (made READY) after job B successfully finishes. A job can list multiple jobs that it should ​run after,​ and will only be scheduled after all those jobs finish successfully. 

From my initial analysis of the data set, 51.61% of the tasks have a CPU
column value of 0 -- what does this mean?  Obviously each task must use at
least some fraction of the CPU.  The same goes for memory usage.  0.94% of
all tasks have a memory column value of 0.

A clarification of this would be much appreciated.

I assume that these traces contain many MapReduce jobs. Since my research topic 
is the performance modeling of MapReduce jobs, I am very interested in 
identifying the MapReduce jobs in these traces, and in being able to 
distinguish the map and reduce tasks.

Would you (Google) be able and willing to provide a mapping for all MapReduce 
tasks to task type (eg., (job ID, task index) -> (map|reduce|...)), or if that 
is not feasible maybe just a list of logical jobnames (or job IDs) of MapReduce 
jobs?

bash-5.0# bq shell --project_id=google.com:google-cluster-data
Welcome to BigQuery! (Type help for more information.)
google.com:google-cluster-data> ls
      datasetId       
 -------------------- 
  clusterdata_2011_1  
  clusterdata_2019_a  
  clusterdata_2019_b  
  clusterdata_2019_c  
  clusterdata_2019_d  
  clusterdata_2019_e  
  clusterdata_2019_f  
  clusterdata_2019_g  
  clusterdata_2019_h  
google.com:google-cluster-data> query 'select count(*) from clusterdata_2011_1'
BigQuery error in query operation: Access Denied: Project google.com:google-cluster-data: User does not have bigquery.jobs.create permission in project google.com:google-cluster-data.

What steps will reproduce the problem?
1.
2.
3.

What is the expected output? What do you see instead?


What version of the product are you using? On what operating system?


Please provide any additional information below.

External links
research blog 

is linking to

http://www.blogger.com/post-create.g?blogID=1901025479979892432

The data I downloaded has colums

Time ParentID TaskID JobType NrmlTaskCores NrmlTaskMem

But description says there should be

Time (int)
JobID (int)
TaskID (int)
Job Type (0, 1, 2, 3)
Normalized Task Cores (float)
Normalized Task Memory (float)

JobID is named ParentID?

Are there any restrictions on publishing results derived from this dataset?

It would be quite useful if task durations were included in the data set. 
This way, the data could be used as input to test, say, a job placement
strategy.

The data I downloaded has colums

Time ParentID TaskID JobType NrmlTaskCores NrmlTaskMem

But description says there should be

Time (int)
JobID (int)
TaskID (int)
Job Type (0, 1, 2, 3)
Normalized Task Cores (float)
Normalized Task Memory (float)

JobID is named ParentID?