vasild / cpp-ipfs-http-client Goto Github PK
View Code? Open in Web Editor NEWIPFS C++ HTTP API client library
License: MIT License
IPFS C++ HTTP API client library
License: MIT License
When compiling with
cmake -DCMAKE_BUILD_TYPE=Release
the compilation fails with
Scanning dependencies of target ipfs-api
[ 5%] Building CXX object CMakeFiles/ipfs-api.dir/src/client.cc.o
[ 10%] Building CXX object CMakeFiles/ipfs-api.dir/src/http/transport-curl.cc.o
/tmp/nix-build-cpp-ipfs-api-2016-11-26.drv-0/cpp-ipfs-api-3b64e6e4520e87f14fee4216b4b5184c6f97a47a-src/src/http/transport-curl.cc: In lambda function:
/tmp/nix-build-cpp-ipfs-api-2016-11-26.drv-0/cpp-ipfs-api-3b64e6e4520e87f14fee4216b4b5184c6f97a47a-src/src/http/transport-curl.cc:293:5: error: 'replace_body' was not declared in this scope
replace_body = "";
^
Background:
This happens with the latest commit of cpp-ipfs-api when building for NixOS (currently an older version is packaged) and the build system uses the CMake default target Release
for non-debug builds (Debug
being the default for debug builds).
I suspect the following code, is maybe causing down-graded performance:
std::stringstream body;
body.str(); // <--- here
Is a stringstream
indeed slow in converting 'large' data to a string?
Another cause could be in ParseJson
:
Json::parse(input);
Additional debugging and performance measurements are required!
I see it takes 6ms to execute a simple swarm/peers request. Sure, this specific call could take longer. Yet other API calls are executed (far) below 1ms. And the difference with curl
command doesn't show much time differences between those different API calls.
The main difference is the body response size.
Just a thought experiment. Again, does requires follow-up research.
The following code demonstrate that the ObjectData function is not thread safe and throws std::runtime_error exception.
#include <thread>
#include <ipfs/client.h>
int main()
{
auto hash = "QmcXneAawME7d3vmV3x95p3WVxczeTid7LpJW8DcH4sA5c";
ipfs::Client ipfs("localhost", 5001);
std::thread t1([&hash, &ipfs]() { std::string data; ipfs.ObjectData(hash, &data); });
std::thread t2([&hash, &ipfs]() { std::string data; ipfs.ObjectData(hash, &data); });
t1.join();
t2.join();
return (0);
}
On MacOS:
libc++abi.dylib: terminating with uncaught exception of type std::runtime_error: Failed initialization
ipfs::Client::Client(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, long)' myprog.cc:(.text+0x78): undefined reference to
ipfs::Client::~Client()'Please help me!
I think better to use Make directly.
Makefile:
#
# IPFS C++ HTTP API client library
#
VER_MAJOR := 0
VER_MINOR := 4
VER_PATCH := 0
VERSION := $(VER_MAJOR).$(VER_MINOR).$(VER_PATCH)
HEADERS := include/ipfs/client.h include/ipfs/http/transport-curl.h include/ipfs/http/transport.h
TESTPROGS := block dht files key object swarm config generic name pin
CXXFLAGS = -Wall -Wpedantic -Wextra -Werror -Os -std=gnu++11 -I./include
LDLIBS = ./libipfs-http-client.a -lcurl
.PHONY: testprogs lib all clean
all: lib testprogs
lib: libipfs-http-client.a libipfs-http-client.so.$(VERSION)
libipfs-http-client.a: src/client.o src/http/transport-curl.o
$(AR) rc libipfs-http-client.a src/client.o src/http/transport-curl.o
libipfs-http-client.so.$(VERSION): src/client.o src/http/transport-curl.o
$(CXX) $(CXXFLAGS) -shared -Wl,-soname,libipfs-http-client.so.$(VER_MAJOR) -o libipfs-http-client.so.$(VERSION) src/client.o src/http/transport-curl.o -lcurl
ln -sf libipfs-http-client.so.$(VERSION) libipfs-http-client.so.$(VER_MAJOR)
ln -sf libipfs-http-client.so.$(VER_MAJOR) libipfs-http-client.so
testprogs: $(addprefix test/,$(TESTPROGS))
$(addprefix test/,$(TESTPROGS)):%:%.cc include/ipfs/client.h include/ipfs/test/utils.h libipfs-http-client.a
$(CXX) $(CXXFLAGS) $< $(LDLIBS) -o $@
src/client.o: src/client.cc $(HEADERS)
$(CXX) $(CXXFLAGS) -fPIC -o src/client.o -c src/client.cc
src/http/transport-curl.o: src/http/transport-curl.cc $(HEADERS) include/ipfs/test/utils.h
$(CXX) $(CXXFLAGS) -fPIC -o src/http/transport-curl.o -c src/http/transport-curl.cc
clean:
$(RM) src/client.o src/http/transport-curl.o libipfs-http-client.a libipfs-http-client.so* $(addprefix test/,$(TESTPROGS))
Tested in Debian Stretch.
Hi, thank you for making this library.
I plan on using it in an app i'm making could not find a way to get transfer progress incase of file download/upload.
Your response will highly appreciated.
Making multiple calls to client.ObjectPut method throwing up " Failed initialization: easy handle already used in multi handle
" error. Looks like transport curl not able to handle multiple calls at a time.
Any idea about this error message ?
FYI: If we make sequential calls it works fine. no issues at all
Hi! We are considering renaming the IPFS Client libraries, please read more at ipfs/ipfs#374 and comment if you are onboard.
I'm testing your project right now.
Could be nice to create a pkg-config file like in:
https://github.com/sebastiandev/zipper/blob/master/zipper.pc.cmakein
Or for Makefile:
https://github.com/Lecrapouille/MyMakefile/blob/master/Makefile.macros#L165
sudo make install will install it:
https://github.com/Lecrapouille/MyMakefile/blob/master/Makefile.macros#L181
https://github.com/Lecrapouille/MyMakefile/blob/master/Makefile.macros#L63
(not sure this is working well for Mac)
So in your readme instead of:
g++ -std=c++11 -I/path/to/header -L/path/to/lib -lipfs-http-client myprog.cc -o myprog
Could be replaced by:
g++ -std=c++11 myprog.cc -o myprog `pkg-config --cflags --libs ipfs-http-client`
In addition in your command -lipfs-http-client
is misplaced and -lcurl
is missing. The pkg-config will contain the -lcurl
code
Hi,
When I turn-off IPFS daemon on purpose, it should throw an runtime error as expected.
Nevermind, I should catch the error at client.FilesGet()
for example. This client Object creation, client("localhost", ...)
, always goes OK, and doesn't check the connection yet.
Regards,
Melroy van den Berg
#include <ipfs/client.h>
int main()
{
ipfs::Client ipfs("localhost", 5001);
ipfs::Client ipfsCopy(ipfs); // Segmentation fault
return (0);
}
Hiya :)
In the IPFS org we’ve created a new group for maintainers of client libraries. This will make it easier for us to loop you into issues that may cause problems or necessitate changes in your client library.
If you could let us know the github usernames of all the active maintainers we can add them to the group and note them in the clients table.
Also, we have a great IPFS weekly newsletter. A few days before each edition goes out a new pull request is created in the repo. If you have any announcements related to your client library feed free to log a new issue in the repo or just comment on an upcoming newsletter pull request.
From Kubo 0.28, most of DHT RPC commands are removed (except for query
) since they were already deprecated.
Instead, use /api/v0/routing
. At the moment, it should be a drop-in replacement.
Hi,
I just got this library up and running only to realise that it seems to connect to an existing local node. I was expecting it to host a local node also.
Am I correct in this? Do I need to host my own local server then use this library to connect to my local server?
It would be helpful to update the documentation with this information.
Hi,
I am trying to use cpp-ipfs-http-client with Infura. The code works fine with a local ipfs daemon (ipfs::Client client("localhost", 5001), but the port seems closed when using Infura. Here are the diferent configurations I tried:
ipfs::Client client("https://ipfs.infura.io", 5001); ipfs::Client client("ipfs.infura.io", 5001); ipfs::Client client("ipfs.infura.io", 5001, "https");
The terminal issues connection failed errors.
By any chance, did you try using Infura or similar tools smoothly with cpp-ipfs-http-client?
Thanks for your help !
The following two test cases take up 99% of the execution time:
Which won't help short development cycles in a DevOps cycle and fast CI/CD.
Let's see if we can make the two mentioned testcases faster.
Console output:
3/10 Test #63: test_dht ......................... Passed 104.44 sec
7/10 Test #67: test_name ........................ Passed 80.46 sec
Regards,
Melroy van den Berg
Install the project...
-- Install configuration: ""
-- Installing: /usr/local/lib/libipfs-http-client.a
-- Installing: /usr/local/include/ipfs/client.h
-- Installing: /usr/local/include/ipfs/http/transport.h
-- Installing: /usr/local/include/nlohmann/json.hpp
But nlohmann/adl_serializer.hpp is not copied while present in ./build/_deps/json-src/include/nlohmann/adl_serializer.hpp
Therefore this exemple https://github.com/vasild/cpp-ipfs-http-client#usage does not compile:
#include <nlohmann/adl_serializer.hpp> not found
Same for the whole folder.
Edit: -I ./build/_deps/json-src/include/ fixing my compilation issue but this line:
install(FILES ${json_SOURCE_DIR}/include/nlohmann/json.hpp DESTINATION include/nlohmann)
should be fixed
This is a notice I'm filling in repo of every HTTP Client I can find.
Feel free to close it if this project already works fine with go-ipfs 0.5
go-ipfs 0.5 will block GET
commands on the API port (ipfs/kubo#7097), requiring every command (RPC) to be sent as HTTP POST
request.
See API reference docs: https://docs.ipfs.io/reference/api/http/
This is potentially a breaking change,
double check if this project uses POST
for every RPC call.
Download links for v0.5-rc* are available at ipfs/kubo#7109
You can also test using an ephemeral Docker container:
$ docker run --rm -it --net=host ipfs/go-ipfs:v0.5.0-rc1
Hello,
I am looking for some help concerning the FilesAdd command.
I have installed the cpp-ipfs-http-client and I am able to successfully retrieve the FIlesGet element from the test file: retrieved contents are displayed in the console as expected.
However, the FilesAdd command from the test file fails: no output is displayed.
Here is the code:
ipfs::Client client("localhost", 5001); ipfs::Json add_result; client.FilesAdd( {{"foo.txt", ipfs::http::FileUpload::Type::kFileContents, "abcd"},{"bar.txt", ipfs::http::FileUpload::Type::kFileName, "../compile_commands.json"}}, &add_result); std::cout << "FilesAdd() result:" << std::endl << add_result.dump(2) << std::endl;
Did you already run into this issue? Am I doing something wrong?
Thank you for your help or any tip !
Try to get links of new object (hash: QmdfTbBqBPQ7VNxZEYEj14VmRuZBkqFbiwReogJgS1zR1n) with IPFS using curl:
curl "http://localhost:5001/api/v0/object/links?arg=QmdfTbBqBPQ7VNxZEYEj14VmRuZBkqFbiwReogJgS1zR1n"
IPFS returns:
{"Hash":"QmdfTbBqBPQ7VNxZEYEj14VmRuZBkqFbiwReogJgS1zR1n"}
There is no "Links" field in the response.
The method Client::ObjectLinks of cpp-ipfs-api throws an exception as it tries to access to the "Links" value
Hi,
Not all methods are implemented.
So its stated that there are methods missing in the README. But its not written down which methods are actually missing.
Bottom-line: What is the current state of the project? And much is implemented? And finally, what pieces needs to be done?
Maybe also add a link to the Core API documentation (from JS-IPFS) as a reference: https://github.com/ipfs/js-ipfs/tree/master/docs/core-api
Regards,
Melroy
While creating a Client object (or possibly later as well), I would like to set a time-out. Similar to ipfs-http-client
:
https://github.com/ipfs/js-ipfs/tree/master/packages/ipfs-http-client#global-timeouts
Luckily there is a common option to set timeout
server-side, meaning you don't need to handle time-outs client side.
Let's say you try to access a file that doesn't exist (yet) in IPFS:
curl -X POST "http://127.0.0.1:5001/api/v0/cat?arg=zb2rhe5P4gXftAwvA4eXQ5HJwsER2owDyS9sKaQRRVQPn93bA"
Introducing an additional query string (named timeout
with a value of a string, including 's' for seconds for example):
curl -X POST "http://127.0.0.1:5001/api/v0/cat?arg=zb2rhe5P4gXftAwvA4eXQ5HJwsER2owDyS9sKaQRRVQPn93bA&timeout=3s"
Similar behaviour shouldn't be hard to implement in CPP-IPFS client. Just add timeout=<value>
to the URL requests.
Regards,
Melroy van den Berg
We are currently using the cURL curl_easy_perform() function within our library. Which works fine and is thread-safe.
However, when using threading to call IPFS http client, you want to be able to cancel those threads. pthread_cancel()
should NOT be used, since that is not implemented under Windows (even when using the POSIX thread for win32 - MinGW for example). And thus not cross-platform implemented. Moreover, pthread_cancel
under Linux is not guaranteed to stop the thread, depending on the cancellation points/.. and basically on best effort basis.
Anyway, the best way to stop/cancel a running thread in a "native" way is to actually to be able to communicate with the thread using a shared resource. For example using an atomic boolean to stop the thread. The thread should be able to react on the boolean change, allowing the thread to stop itself.
Pseudo code, just a fast sketch:
# Main thread
std::atomic<bool> stop_thread(false);
[...]
// Stop the thread
stop_thread = true
thread.join()
stop_thread = false
# Thread code
while(..) {
if (stop_thread)
return;
}
Now the issue with cURL is that the curl_easy_perform()
function is a blocking call, and can't be easily cancelled or terminated. Instead I propose to use the cURL Multi API instead. Like the curl_multi_perform(), curl_multi_poll() and curl_multi_remove_handle() functions.
The cURL Multi API is select/poll based, allowing to rewrite the code in such a way you are able to cancel the cURL request within a thread, at any given moment.
See also the FAQ on curl.se - How do I stop an ongoing transfer?
If you are using the multi interface, you can also stop a transfer by removing the particular easy handle from the multi stack at any moment you think the transfer is done or when you wish to abort the transfer.
I underline "or when you wish to abort the transfer".
This solution would automatically solve the cross-platform issue, of canceling Windows Threads as well using for example such an atomic boolean to stop the thread from the main thread, at any given moment.
Personally, I'm facing those issues with my own app using this cpp-ipfs-http-client, which I try to cross-compile towards Windows.
@vasild PELASE, any feedback or ideas are welcome! I know, this requires quite some changes in the code. But I think it's feasible and would really help me out!
Instead of Travis, we could actually start using Github Actions to achieve the same thing.
Maybe it's a better integration with Github.
Let me know if you agree or not Vasil.
Regards,
Melroy van den Berg
Hi,
I noticed you wrapped the files cat
API command with the method name FilesGet
:
void Client::FilesGet(const std::string& path, std::iostream* response) {
http_->Fetch(MakeUrl("cat", {{"arg", path}}), {}, response);
}
Which will get the file via cat and returns the data via iostream as shown above.
However, FilesGet
call will conflict with the actual get
HTTP API call as well:
https://docs.ipfs.io/reference/http/api/#api-v0-get
Proposal: Fix the misalignment of interface names better sooner than later. Rename FilesGet
to FilesCat
. And FilesGet
can be implemented to download the file directly to disk (proving the arg
as path argument).
This will be conform the spec and even in-line with the API CLI.
This change will however impact devs who upgrade to a newer version of cpp-ipfs-http-client, it isn't nice. But again, I rather do this change now then later.
An alternative is: Keep the current names. And add FilesDownload
to use the get
API call. Which is, as expected, not inline with the spec.
@vasild Please, let me know what you think.
Regards,
Melroy van den Berg
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.