Is Container#attach streaming working for you ? about docker-api HOT 26 CLOSED

Example usage:

d = Docker::Container.get('f7758e4a811d')
d.attach do |*a|
  p a
end

from docker-api.

The problem is that exconn (the http library used by docker-api) uses a quite large buffer when reading. To make attach interactive i do this:

@conn_interactive = Docker::Connection.new(docker_url, {:chunk_size => 1})
container = Docker::Container.create(..., @conn_interactive)
container.attach do |_fd, chunk|
  $stdout.write(chunk)
end

Remember to only use @conn_interactive for interactive thing as downloading or uploading big files will be really slow with a 1 byte buffer. I have some more code if you want to do line oriented things even when reading small chunks (like indenting the output etc).

from docker-api.

@phemmer nice!

@tlunter does it make sense to have #attach default to :chunk_size => nil?

from docker-api.

@bfulton ack, sorry, I deleted my comment. I forgot I had modified excon :-(

Removed my patch, trying to solve again.

from docker-api.

Ah. No worries! Let us know how you make out.

from docker-api.

What does it mean to have nil as max_len argument for read_nonblock? can't find anything in the documentation about it.

@bfulton i guess people assume attach to be "interactive" (i did) so it probably make sense to change it just for attach.

If efficiency matters I think the largest chunk size we can have and still make it interactive is 8, see https://github.com/dotcloud/docker/blob/master/utils/stdcopy.go (StdWriterPrefixLen)

from docker-api.

@wader That comment was a mistake on my part. I forgot I had patched excon elsewhere.

Yeah, there's no way to do this without patching excon. I've gone through that Excon::Socket#read method dozens of times, there is no way to get the desired behavior.

I really don't understand the design either. It looks like it's trying to behave like IO.read as it always blocks until the requested bytes have been read, but I don't understand why it's been complicated with all the read_nonblock/rescue stuff. Simply calling read and rescuing EOFError would achieve the same result with a fraction of the code.

At this point, I see 2 potential ways of proceeding:

1. Excon modifies that read method so that it doesn't block if the :nonblock option is set.
   This option will be a backwards incompatible and a very drastic change from the method's previous behavior.
2. Monkey patch the method to replace it with one that obeys the :nonblock option.
   Monkey patching is just bad, but as option 1 will likely be a significant problem, it may be the only option.

Any other options?

EDIT: 3 options:
3. use :chunk_size => 1 and read byte by byte. This is just horribly inefficient, I would prefer monkey patching over this option.

from docker-api.

@phemmer thanks.

A default-interactive #attach is important, so I would be in favor of adding :chunk_size => 1 (or 8 if we can get away with it as @wader suggests) to the default #attach opts, along with a comment referencing excon/excon/issues/350 so we can revisit if that issue is resolved.

@tlunter thoughts?

from docker-api.

I can't see :chunk_size => 8 being a sure thing. I can see how 1 would be inefficient since it's going down to Excon and back to Docker-API Message every byte, but I think that would honestly be the easiest.

It seems like socket.read(length) will read until EOF or until that size, so there really is no other way besides setting length to 1 or Excon's :nonblock is changed.

We can definitely set :chunk_size => 1 as the default to that, or at least specify in the README that if this method should be interactive in your application that this will need to be used. Chunk size can be set on a per request basis and isn't needed on a full connection. Perhaps we open up attach to a :chunk_size parameter and use a sensible larger default to prevent this from being too slow?

I don't want applications that expect larger responses to be bogged down by the :chunk_size => 1.

from docker-api.

@tlunter To be clear i meant 8 only for attach but i guess it wont make much difference to 1, both are quite inefficient but my guess is that it is not big deal if the log is mean to be used "interactively". If people do use attach to get very large logs in batch, maybe there should be two methods, e.g. attach and attach_interactive to make it clear?

from docker-api.

I can see attach_interactive being the route we should go down.

from docker-api.

ok, so I started digging into excon looking to solve this issue on that side. Conclusion: I'm not certain this is something excon should handle.

Because of excon/excon#350, I originally thought the issue was with that Excon::Socket#read method, but now I'm not sure it is. Yes, I think the method is ugly, and the #readline method even more so, but changing the way the methods behave isn't a good fix for the issue.
The methods are behaving as they were designed to. You're supposed to pass in a :chunk_size and your callback is supposed to get exactly that many bytes back, no more, no less (unless the connection closes).

Now if the server were using chunked transfer encoding, the callback would get called for every chunk received. But the docker API doesn't use chunked transfer encoding, it becomes a raw stream. As such, Excon has no clue how many bytes should be in each "chunk" that gets sent to the callback, so it reads however many bytes it was told to by the :chunk_size parameter.
The only way to solve this in excon would be to either pass the socket back to the caller in which the caller can call #read or #readpartial on the socket directly, or add a :chunk_size_min parameter which reads until it has chunk_size_min bytes and then calls the callback.

So basically, it's not a bug, it's just the way it was designed.
So now I would like to ask, why is docker-api using Excon? It seems like most of the requests being made are pretty basic. Why not extend Docker::Connection to handle the http request, and then it could handle the raw stream as it wants to.

I personally dislike the #attach_interactive idea. The docker-api gem puts a focus on the methods behaving like their API call counterparts. This would change that as #attach would behave radically different from the API call it's named after, and instead you'd have #attach_interactive which behaves properly.
While I think solving this so that #attach behaves like it should in all cases, if the behavior must be changed, #attach with no arguments should behave as the container/attach API call is documented, and then if you want to get large chunks, add a parameter to the method call, and not create a separate method.

from docker-api.

@phemmer I agree that #attach should be interactive, and that if we need a buffered version it should be a different method, or different opts. Abandoning excon seems excessive since the Docker API is mostly a normal HTTP API. What do you think about grabbing the low-level socket used in #attach through reflection?

from docker-api.

I'm using the @conn_interactive = Docker::Connection.new(docker_url, {:chunk_size => 1}) fix with success. Just wondering if there's been any progress since?

from docker-api.

I've added a MR to Excon to support chunking/streams since I need it for docker-api and another project on top. Works like a charm for me with docker.attach ;)
See: excon/excon#378

from docker-api.

@arlimus, great! Let us know when to bump excon.

from docker-api.

@bfulton Will do, hope it gets in ;)

from docker-api.

@arlimus, great to see excon/excon#378 merged. Looking forward to closing this soon.

from docker-api.

Excon 0.34.0 was released yesterday with this fix.

from docker-api.

@arlimus can you validate that attach streams properly on master now that we're requiring excon 0.34.0?

from docker-api.

@bfulton I've been trying to use attach with an HTTP streaming endpoint and, even on master, all of the output of my Docker containers gets buffered and output all at once at the end of my streaming endpoint.

from docker-api.

For some reason, doing the chunk_size: 1 trick didn't work for me either. Perhaps something else is going on with my application? https://github.com/davidcelis/runmygi.st/blob/master/config.ru#L53-L60

from docker-api.

@davidcelis yes, streaming is working, and you should not need the chunk_size hack.

My guess is that the runtime you're testing with is setting buffered stdout. If that's true, try specifying 'Tty' => true when you create the container. If that doesn't work, you might try adding unbuffer (from expect) to your script.

from docker-api.

Agh, you're right. Sorry about that. I was using a Ruby script as my test, and when I add a $stdout.flush in there, it works as expected.

from docker-api.

Great, glad it's working!

from docker-api.

So, just to report back.... I actually still require the :chunk_size hack. I forgot that I still had it in my codebase. But when it's removed, even with my Ruby script's buffering corrected, the output is not streamed.

from docker-api.