NNG Usage about goose HOT 7 CLOSED

Thanks for the feedback, comments, and questions!

I'll review in more depth over the next few days, and try to apply all your feedback. My usage of nng isn't a performance bottleneck (Goose primarily uses nng to synchronize statistics, the actual load tests are managed locally even in a distributed test), but of course I'm happy to optimize where possible. At a quick glance, I expect to resolve 1-3 quickly (thanks), but likely will wait to address 4 until there's a new stable release, whenever that is.

My biggest confusion when I started using nng was not finding a way for n clients to initiate a connection to 1 server, and for that 1 server to push messages individually to any 1 of those specific clients at will. My understanding is that this is not a limitation of your crate however, but instead is the design of nng itself. I was ultimately able to work around this by having each client check in with the server regularly (pushing statistics upstream every 15 seconds).

Because of the dependency on CMake I already wrapped my usage of nng in a feature -- thus, anyone needing Gaggle support (distributed load testing) would need it for a really large test and IMO would be able to figure out how to install a newer version of CMake. It doesn't impact most users of Goose, who won't need distributed load testing as Goose is already respectably fast. From a single server.

As I work through your comments I'll try and provide additional feedback.

from goose.

Regarding #1, I've removed the unnecessary Mutexes (they slipped in when I originally cleaned up some unsafe code, I appreciate the review!).

I've been unable to cleanly address #2 and #3 however, I seem to always need to start with a Vec and then covert it to a message for it to compile and work. Any further tips there would be greatly appreciated -- re-opening until those issues are solved.

from goose.

(I didn't see this until now - sorry)

Is it the issue with serializing or deserializing? Serializing, Message implements Write so that should just work with serde_cbor::to_writer. Deserializing, Message derefs to &[u8] which implements Read, so serde_cbor::from_reader, should work. If it doesn't, I might have assumed Deref will cover that case when it doesn't and I should implement Read directly.

from goose.

If I try and write directly to the message (instead of to the buffer) I end up with the following error:

the trait `std::convert::From<&nng::message::Message>` is not implemented for `nng::message::Message`

Trying to solve that I run into issues with ownership.

If I try and remove the calls to as_slice() I see the following errors:

the trait `std::convert::From<std::vec::Vec<u8>>` is not implemented for `nng::message::Message`
the trait `std::io::Read` is not implemented for `nng::message::Message`
the trait `std::io::Read` is not implemented for `&nng::message::Message`

from goose.

If I try and write directly to the message (instead of to the buffer) I end up with the following error:

It looks like you're trying to pass in a reference to the Message. The following compiles for me (manager.rs@429):

// Send vector of client initializers to worker.
let mut message = Message::new().unwrap();
match serde_cbor::to_writer(&mut message, &clients) {
	Ok(_) => (),
	Err(e) => {
		error!("failed to serialize client initializers: {}", e);
		std::process::exit(1);
	}
}
info!(
	"sending {} clients to worker {}",
	clients.len(),
	workers.len()
);
match server.try_send(message) {
	Ok(_) => (),
	Err((_, e)) => match e {
		Error::TryAgain => {
			if ACTIVE_WORKERS.load(Ordering::SeqCst) == 0 {
				info!("all workers have exited");
				break;
			}
		}
		_ => {
			error!("communication failure: {:?}", e);
			std::process::exit(1);
		}
	},
}

If I try and remove the calls to as_slice() I see the following errors:

My bad - I forgot that I didn't impl Read for Message. I modeled it after Vec<u8>, so the following works but is really just shorthand for what you were already doing (manager.rs@250):

let requests: HashMap<String, GooseRequest> =
	serde_cbor::from_reader(&msg[..]).unwrap();

from goose.

@neachdainn Thanks! I'm at a loss as to why I couldn't get that working, it's pretty straightforward. I'm pushing a PR now to fix, and greatly appreciate your feedback/help!

As an aside, I still aim to look into the SSL question at some point too. I can say that NNG has been rock-solid for me -- I set up a 100-Worker Gaggle, all pushing statistics up to the Manager as fast as they can load pages, and I've never seen an error in the networking logic.

from goose.

I can say that NNG has been rock-solid for me -- I set up a 100-Worker Gaggle, all pushing statistics up to the Manager as fast as they can load pages, and I've never seen an error in the networking logic.

I can't take credit for anything except the safe AIO and the wrapper but I'll pass it along!

from goose.