Comments (3)
@gq17 hi! Can you give me any example code that produces such behaviour?
from webhdfs.
@harrisiirak considering Hadoop(HDFS) is install in a host(192.168.1.90), I run the script below on this host, it works fine.
const WebHDFS = require('webhdfs')
var hdfs = WebHDFS.createClient({
user:'guoqiang',
//host:'127.0.0.1',
//port:'50070',
//path:''
}, {
headers: {
Authorization: "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6MS"
}
});
const localFilePath = "/home/guoqiang/Downloads/udisk";
const remoteFilePath = "/home/guoqiang/test4"
var upload_hdfs = function(){
hdfs.exists(remoteFilePath, function (err, exist) {
function callback(err) {
if (err){
console.log("task failed " + err)
}
else{
console.log("ok")
}
}
if(err){console.log('error '+ err)}
if (exist) {
console.log("append")
hdfs.appendFile(remoteFilePath, localFilePath, callback);
}else {
console.log("write")
hdfs.writeFile(remoteFilePath, localFilePath, callback);
}
});
}
upload_hdfs();
then I run the script on another host, I change the webhdfsclient configuration
var hdfs = WebHDFS.createClient({
user:'guoqiang',
host:'192.168.1.90',
//port:'50070',
//path:''
}, {
headers: {
Authorization: "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6MS"
}
});
then I got the error
error true
write
task failed Error: connect ECONNREFUSED 127.0.0.1:50075
seems it connect the local Hadoop
from webhdfs.
@harrisiirak the problem is due to network DNS, the host installed with Hadoop needs to be configured(the configuration of Hadoop ) to send back the right hostname and the hostname needs to be recognized by the client host.
ref: http://hadoop.apache.org/docs/r1.0.4/webhdfs.html#CREATE
from webhdfs.
Related Issues (20)
- readFile calls callback twice HOT 5
- On Finish callback calls each time HOT 3
- Calling writeFile not working twice. HOT 2
- Missing features HOT 1
- Piping from tar packing to webhdfs error HOT 3
- Problem when piping an expressjs request into a writable stream HOT 3
- Module throws exception on `write` HOT 4
- Does it support multiple namenodes?
- unable to delete a folder which is not empty HOT 1
- support typescript
- Support webhdfs 2.3.0
- can't upload large size file HOT 1
- Read/Write with Kerberos
- Is this project still upate ?
- faced "java.lang.UnsupportedOperationException","message":"Operation APPEND is not supported."}} (error 400) HOT 1
- exists not returning errors HOT 1
- Using Kerberos for the authentication HOT 3
- WritableStream gets reset after pipe() HOT 2
- Getting SSL connection options through to request HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from webhdfs.