Giter Club home page Giter Club logo

Comments (3)

harrisiirak avatar harrisiirak commented on September 28, 2024

@gq17 hi! Can you give me any example code that produces such behaviour?

from webhdfs.

gq17 avatar gq17 commented on September 28, 2024

@harrisiirak considering Hadoop(HDFS) is install in a host(192.168.1.90), I run the script below on this host, it works fine.

const WebHDFS = require('webhdfs')

var hdfs = WebHDFS.createClient({
  user:'guoqiang',
  //host:'127.0.0.1',
  //port:'50070',
  //path:''
}, {
  headers: {
    Authorization: "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6MS"
  }
});

const localFilePath = "/home/guoqiang/Downloads/udisk";
const remoteFilePath = "/home/guoqiang/test4"

var upload_hdfs = function(){
  hdfs.exists(remoteFilePath, function (err, exist) {
    function callback(err) {
      if (err){
        console.log("task failed " + err)
      }
      else{
        console.log("ok")
      }
    }
    if(err){console.log('error '+ err)}
    if (exist) {
      console.log("append")
      hdfs.appendFile(remoteFilePath, localFilePath, callback);
    }else {
      console.log("write")
      hdfs.writeFile(remoteFilePath, localFilePath, callback);
    }
  });
}

upload_hdfs();  

then I run the script on another host, I change the webhdfsclient configuration

var hdfs = WebHDFS.createClient({
  user:'guoqiang',
  host:'192.168.1.90',
  //port:'50070',
  //path:''
}, {
  headers: {
    Authorization: "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6MS"
  }
});

then I got the error

error true
write
task failed Error: connect ECONNREFUSED 127.0.0.1:50075

seems it connect the local Hadoop

from webhdfs.

gq17 avatar gq17 commented on September 28, 2024

@harrisiirak the problem is due to network DNS, the host installed with Hadoop needs to be configured(the configuration of Hadoop ) to send back the right hostname and the hostname needs to be recognized by the client host.
ref: http://hadoop.apache.org/docs/r1.0.4/webhdfs.html#CREATE

from webhdfs.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.