augxhub / libpyhdfs Goto Github PK
View Code? Open in Web Editor NEWAutomatically exported from code.google.com/p/libpyhdfs
Automatically exported from code.google.com/p/libpyhdfs
Hi. Here's a patch that adds listdir support. Enjoy!
Original issue reported on code.google.com by [email protected]
on 3 Nov 2009 at 1:50
Attachments:
If given hostname "default" and port 0, libhdfs will read the right values
from hadoop config. Attached patch makes the script to "just work" for most
of the users.
Original issue reported on code.google.com by [email protected]
on 4 Nov 2009 at 8:38
Attachments:
The following will crash the jvm, as long the as the path exists:
fs = pyhdfs.connect(host, port)
l = pyhdfs.listdir(fs, '/doesnotexist')
This really does not seem to be a bug in libpyhdfs, but I figured I'll file
this ticket so anyone else looking for this problem sees this.
I have no idea how hadoop can trigger a jvm segfault.
This is the output:
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007ff0fadb0ee8, pid=29972, tid=140673005336304
#
# JRE version: 6.0_16-b01
# Java VM: Java HotSpot(TM) 64-Bit Server VM (14.2-b01 mixed mode linux-
amd64 )
# Problematic frame:
# V [libjvm.so+0x3d4ee8]
#
# An error report file with more information is saved as:
# /home/tv/src/libpyhdfs.git/test/hs_err_pid29972.log
#
# If you would like to submit a bug report, please visit:
# http://java.sun.com/webapps/bugreport/crash.jsp
#
A sample logfile is attached.
Original issue reported on code.google.com by [email protected]
on 3 Nov 2009 at 11:35
Attachments:
What steps will reproduce the problem?
1. hadoop fs -mkdir /empty
2. print pyhdfs.listdir(fs, '/empty')
What is the expected output? What do you see instead?
Expected [], got None.
This would let me differentiate between errors and empty directories.
Attached patch to fix this.
Original issue reported on code.google.com by [email protected]
on 4 Nov 2009 at 8:28
Attachments:
I need connect the hdfs-server. How do I set username and groupname? Thank you
Original issue reported on code.google.com by [email protected]
on 24 Sep 2014 at 6:59
Including jars, hadoop config and a prebuilt shared lib is extremely
unhelpful:
1. it actually prevents using this package with an actual Hadoop setup,
e.g. looking up the real config file
2. putting the files at a hardcoded /usr/local location prevents installing
the project with a --prefix= (distutils aborts with a sandbox error).
3. using the same .so isn't going to work across architectures
4. using a .so or jars you just download over the net is a security risk
Attaching a suggested patch dont-bundle.diff that just recommends using the
Cloudera debs.
Setting CLASSPATH inside the C source is also a bad idea, as it embeds the
version numbers of the jars in it, and they just won't ever match what the
person running the thing will have. Attaching patch no-classpath.diff that
removes that. Sadly, as running Java just never really got nicely
integrated with the operating system, and Hadoop provides no easy mechanism
for finding the right paths, the caller will have to set the CLASSPATH. But
at least that can happen outside of C, in Python or shell, and it can be
done so that it matches the runtime environment.
Original issue reported on code.google.com by [email protected]
on 23 Oct 2009 at 2:47
Attachments:
see https://issues.apache.org/jira/browse/HDFS-596
Original issue reported on code.google.com by kofreestyler
on 28 Oct 2009 at 5:17
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.