Comments (16)
@prasenjithaty you can use your customized async http client with the config to set retry for requests. We have not supported other low level retry such as the interceptors but we can possibly enhance it to make it work like that by enhancing the task worker and the requesting worker
Another way to set general retry is to only retry on those failed requests with another parallel task only targeting. (we can start the 2nd parallel task in another thread by polling a hashmap storing only those failed the 1st attempt)
from parallec.
@prasenjithaty can you see if this helps? http://www.parallec.io/docs/submit-task/#apis-on-variable-replacement-for-heterogeneous-requests
from parallec.
Hi @jeffpeiyt,
The example you shared is for heterogeneous requests to different hosts. How about doing the same with a single host? I'm not really able to figure out a way of doing this. Also javadoc on ParallelClient#setReplace* methods would be pretty useful for users.
Thanks.
from parallec.
but put the $VAR_NAME inside of the string in setHttpEntityBody
if you scroll down a bit you can see:
Different requests to the same target host
Here is the example of hitting 2 different APIs to the same target host. $ZIP is the variable being replaced. setReplaceVarMapToSingleTargetSingleVar(String variable, List replaceList, String uniformTargetHost) is the API to use. There are more complex replacements APIs available in javadoc. Complete sample code is here.
http://www.parallec.io/userdata/sample_weather_48824.txt
http://www.parallec.io/userdata/sample_weather_95037.txt
pc.prepareHttpGet("/userdata/sample_weather_$ZIP.txt")
.setReplaceVarMapToSingleTargetSingleVar("ZIP",
Arrays.asList("95037","48824"), "www.parallec.io")
.setResponseContext(responseContext)
.execute(new ParallecResponseHandler() {...}...
from parallec.
but put the $VAR_NAME inside of the string in setHttpEntityBody
^^ was the right pointer.
Thanks. Got it working now. Now, how can I collect all the responses?
from parallec.
Great! Can you check examples from https://github.com/eBay/parallec-samples/tree/master/sample-apps/src/main/java/io/parallec/sample/app/http ?
http://www.parallec.io/docs/track-status/
from parallec.
I already checked those. But, I want to set the key in the map dynamically. Maybe fileURL
from my first post in the request body
from parallec.
@prasenjithaty is your question on aggregating/collecting responses or setting the variables in different requests?
from parallec.
My question is to collect responses with the key being a field from my request body.
from parallec.
I thought I figured out with the below code, but it breaks during unmarshalling
.execute(new ParallecResponseHandler() {
public void onCompleted(ResponseOnSingleTask res,
Map<String, Object> responseContext) {
String responseContent = res
.getResponseContent();
TaskRequest taskRequest = res.getRequest();
LOGGER.info("post data = {}", taskRequest.getPostData());
LOGGER.info("post body = {}", taskRequest.getRequestContent());
MetadataRequest metadataRequest = JAXB.unmarshal(taskRequest.getRequestContent(), MetadataRequest.class);
LOGGER.info("!!Response: {}, TargetHost: ", responseContent, res.getHost());
responseContext.put(metadataRequest.getFileUrl(), responseContent);
}
});
However if, I write responseContext.put(UUID.randomUUID().toString(), responseContent);
I'm able to collect the responses.
from parallec.
Will take a look later today
from parallec.
@prasenjithaty Can you log/print the response content rather unmarshal the request? https://github.com/eBay/parallec-samples/blob/master/sample-apps/src/main/java/io/parallec/sample/app/http/HttpBasicMinimumApp.java#L43
For this question My question is to collect responses with the key being a field from my request body.
I suggest you pass the response context into the handler with a hashmap. In the response handler to update the hashmap with key as the field, value is the response ( or the part you need from the response).
Then when you finish ( outside of the handler) to check on this hashmap from the context again to parse the aggregated data
here is an example: https://github.com/eBay/parallec/blob/master/src/test/java/io/parallec/core/main/http/ParallelClientHttpBasicTest.java#L105
from parallec.
@jeffpeiyt log/printing the response content prints the responses perfectly fine. It breaks only when I try to unmarshal the request.
And I've passed response context as a hashmap into the handler as given in the example. if you see my first post on Ln.6 that is what I've done.
from parallec.
Figured out the issue. Problem was while unmarshalling. Got it working finally. Thanks for your help @jeffpeiyt
from parallec.
@prasenjithaty great! Thanks for trying parallec.io!
from parallec.
@jeffpeiyt does the library support an easy way of retrying requests? Like the interceptors in OkHttp? https://github.com/square/okhttp/wiki/Interceptors
from parallec.
Related Issues (20)
- Tasks keep running HOT 5
- Https Issues HOT 4
- How to create task with m2m request HOT 2
- Check url protocol in setting up target host HOT 3
- Creating API getter HOT 2
- Which parameter is to control the NettyAsyncHttpProvider? HOT 3
- NPE in ChunkedWriteHandler HOT 3
- Error when build Jar file w/ this library HOT 4
- May I know how to call an api and get json response using java HOT 1
- Time out error HOT 2
- Ping response time HOT 2
- Tcpclient can't flush the command HOT 1
- Allow sending parallel gRPC requests HOT 1
- Adding a Scala example HOT 1
- Multiple Global Configs HOT 1
- Bulk job of POST requests HOT 1
- Getting time out after 14 seconds HOT 1
- multiple hosts execute different ssh command concurrently
- ParallelTaskBuilder.execute hangs forever waiting for a task completion in sync mode after resources released HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from parallec.