Comments (2)
Were you able to get 2.3 running using bdutil ?
I tried going Ambari 2.1 - HDP 2.3 route and it worked with some manual work at the end.
Changed the following in ambari.conf
AMBARI_VERSION='2.1.0-1409'
AMBARI_REPO="http://s3.amazonaws.com/dev.hortonworks.com/ambari/centos6/2.x/BUILDS/${AMBARI_VERSION}/ambaribn.repo"
AMBARI_STACK='HDP'
AMBARI_STACK_VERSION='2.3'
The install failed
Sun Jul 19 14:57:08 EDT 2015: Exited 1 : gcloud --project=iron-potion-771 --quiet --verbosity=info compute ssh hadoop-m --command=sudo su -l -c "cd ${PWD} && ./update-ambari-config.sh" 2>>update-ambari-config_deploy.stderr 1>>update-ambari-config_deploy.stdout --ssh-flag=-tt --ssh-flag=-oServerAliveInterval=60 --ssh-flag=-oServerAliveCountMax=3 --ssh-flag=-oConnectTimeout=30 --zone=us-central1-a
Sun Jul 19 14:57:08 EDT 2015: Fetching on-VM logs from hadoop-m
Warning: Permanently added '104.197.109.200' (RSA) to the list of known hosts.
Sun Jul 19 14:57:11 EDT 2015: Command failed: wait ${SUBPROC} on line 326.
Sun Jul 19 14:57:11 EDT 2015: Exit code of failed command: 1
Sun Jul 19 14:57:11 EDT 2015: Detailed debug info available in file: /tmp/bdutil-20150719-143917-aAi/debuginfo.txt
Sun Jul 19 14:57:11 EDT 2015: Check console output for error messages and/or retry your command.
Error file ended with this ..
hadoop-m:tambari_wait status: INPROGRESS
hadoop-m:tambari_wait status: INPROGRESS
hadoop-m:tambari_wait status: INPROGRESS
hadoop-m:tambari_wait status: INPROGRESS
hadoop-m:t-test: Fatal internal error
hadoop-m:tjava.lang.RuntimeException: java.lang.ClassNotFoundException: Class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem not found
hadoop-m:t at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2195)
hadoop-m:t at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2638)
hadoop-m:t at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2651)
hadoop-m:t at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:92)
hadoop-m:t at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2687)
hadoop-m:t at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2669)
hadoop-m:t at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371)
hadoop-m:t at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
hadoop-m:t at org.apache.hadoop.fs.shell.PathData.expandAsGlob(PathData.java:325)
hadoop-m:t at org.apache.hadoop.fs.shell.Command.expandArgument(Command.java:235)
hadoop-m:t at org.apache.hadoop.fs.shell.Command.expandArguments(Command.java:218)
hadoop-m:t at org.apache.hadoop.fs.shell.Command.processRawArguments(Command.java:201)
hadoop-m:t at org.apache.hadoop.fs.shell.Command.run(Command.java:165)
hadoop-m:t at org.apache.hadoop.fs.FsShell.run(FsShell.java:287)
hadoop-m:t at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
hadoop-m:t at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
hadoop-m:t at org.apache.hadoop.fs.FsShell.main(FsShell.java:340)
hadoop-m:tCaused by: java.lang.ClassNotFoundException: Class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem not found
hadoop-m:t at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2101)
hadoop-m:t at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2193)
hadoop-m:t ... 16 more
hadoop-m:tError: There was a problem accessing your configuration bucket using the GCS
hadoop-m:tconnector. Check configuration files. Also make sure have the GCS JSON API
hadoop-m:tenabled as described at https://developers.google.com/storage/docs/json_api/.
hadoop-m:t.
However the JSON API was fine .. but the hadoop-env.sh did not have the class path update to add location of the gcs connector jar.
Added the blow to hadoop env template in ambari config guy
export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/usr/local/lib/hadoop/lib/gcs-connector-1.4.1-hadoop2.jar
Seems like everything is working but I guess I might need some more storage soon :)
hadoop fs -df -h gs://hadoop1213
15/07/19 19:48:22 INFO gcs.GoogleHadoopFileSystemBase: GHFS version: 1.4.1-hadoop2
Filesystem Size Used Available Use%
gs://hadoop1213/ 8.0 E 0 8.0 E 0%
from bdutil.
Were you guys able to run hdp 2.3 with the gdutil utility ??
can you please share the ambari configuration to do so ??
from bdutil.
Related Issues (20)
- gcloud compute ssh tunneling refusing connection HOT 1
- Can't install with Ambari 2.5+ HOT 2
- hortonworks on mac os 10.12.6
- ERROR: (gcloud.compute.ssh) could not fetch resource: The resource 'projects/recommendationshashiproject/zones/us-central1-a/instances/hadoop-m' was not found HOT 5
- subnetwork error HOT 1
- Error Deploy bdutil : Invalid value for field resource.networkInterfaces HOT 1
- Unable to start Ambari NameNode Security Exception
- Command failed: wait ${SUBPROC} on line 326 HOT 2
- unable to deploy ambari using bdutil
- Nonissue..delete
- Command failed line 326, and SSH Key HOT 2
- Cluster creation can fail if apt-get update has warnings
- How to create HDP cluster with High Availability? HOT 1
- Cluster creation is failing with "Failed to find image for alias [debian-7-backports]..."
- Upgrade bdutil to Debian 8
- Does not seem to working with Custom network HOT 2
- bdutil deploy fails with ssh error
- Add a "version" command|flag
- bdutil confusingly reports "using project ${GCE_ZONE}"
- unable to deploy ambari using bdutil HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bdutil.