trusche / httplog Goto Github PK
View Code? Open in Web Editor NEWLog outgoing HTTP requests in ruby
License: MIT License
Log outgoing HTTP requests in ruby
License: MIT License
I don't want to remove the gem from the Gemfile but want to turn off http logging temporarily. How do I do that?
Thanks!
Hi, why is rack required for this gem? It can be used without web app at all, right? I have problems with some legacy app:
Bundler could not find compatible versions for gem "rack":
In Gemfile:
httplog (= 1.0.1) was resolved to 1.0.1, which depends on
rack (~> 2.0)
sprockets (~> 2.11.0) was resolved to 2.11.3, which depends on
rack (~> 1.0)
I have a project with aws-sdk gem. Are their logs also visible through httplog? What does aws-sdk gem use to make http requests?
Otherwise:
httplog-1.4.3/lib/httplog/configuration.rb:53:in `initialize': uninitialized constant HttpLog::Configuration::JSON (NameError)
This seems to be quite an unnecessary dependency.
Faraday is a common lib for making background HTTP request, especially when playing with REST API.
I'm trying to track X-Sendfile HTTP headers going around from one app to another...
Using excon (0.19.1), i get:
.rvm/gems/ruby-1.9.2-p320@rails3/gems/httplog-0.2.3/lib/httplog/adapters/excon.rb:15:in alias_method': undefined method
request_kernel' for class `Excon::Connection' (NameError)
When running the below script I get
D, [2017-04-12T15:43:07.738060 #29603] DEBUG -- : [httplog] Sending: GET https://example.com/bwtem
D, [2017-04-12T15:43:07.738148 #29603] DEBUG -- : [httplog] Data:
D, [2017-04-12T15:43:07.738366 #29603] DEBUG -- : [httplog] Connecting: example.com:443
/usr/local/share/gems/gems/httplog-0.99.2/lib/httplog/http_log.rb:55:in `log_status': uninitialized constant #<Class:HttpLog>::Rack (NameError)
from /usr/local/share/gems/gems/httplog-0.99.2/lib/httplog/adapters/httpclient.rb:28:in `do_get_block'
from /usr/local/share/gems/gems/httpclient-2.8.3/lib/httpclient.rb:1019:in `block in do_request'
from /usr/local/share/gems/gems/httpclient-2.8.3/lib/httpclient.rb:1133:in `protect_keep_alive_disconnected'
from /usr/local/share/gems/gems/httpclient-2.8.3/lib/httpclient.rb:1014:in `do_request'
from /usr/local/share/gems/gems/httpclient-2.8.3/lib/httpclient.rb:1104:in `follow_redirect'
from /usr/local/share/gems/gems/httpclient-2.8.3/lib/httpclient.rb:854:in `request'
from /usr/local/share/gems/gems/httpclient-2.8.3/lib/httpclient.rb:743:in `get'
from ./test.rb:25:in `<main>'
#!/usr/bin/ruby
require 'httpclient'
require 'pp'
require 'kconv' #seemed to be needed due to a bug..
require 'rubyntlm' # probably not needed directly?
require 'openssl'
require 'tempfile'
require 'nokogiri'
require 'mechanize'
require 'awesome_print'
require 'httplog' # require this *after* your HTTP gem of choice
user = 'test'
pass = 'test'
uri = URI 'https://example.com/bwtem'
domain = 'win.example.com'
cookie_file = Tempfile.new('cookie_jar.txt')
client = HTTPClient.new(default_header: {"User-Agent" => 'Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:52.0) Gecko/20100101 Firefox/52.0'})
client.set_cookie_store(cookie_file)
client.set_auth(nil, user, pass)
client.ssl_config.verify_mode = nil
r = client.get(uri, :follow_redirect => true)
Hi,
I've run into a problem of PDF dumping into log file. What do you think about making it configurable like:
HttpLog.configure do |config|
config.log_response_filter = proc do |content_type|
content_type =~ /^text/ ||
content_type =~ /^application/ && content_type != 'application/octet-stream'
end
I'd make a PR.
Any thoughts about supporting the http library? I don't believe it has any default logger support.
It could be nice to be able to run it where it's most useful : in production.
If you have an issue with an external API but can't log it from your application (with a feature switch or whatever) the usage is very reduce.
my 2 cents.
D, [2014-08-26T13:08:48.821528 #6500] DEBUG -- : [httplog] Connecting: example.com:443
D, [2014-08-26T13:08:49.292332 #6500] DEBUG -- : [httplog] Sending: GET https://example.com
D, [2014-08-26T13:08:49.335191 #6500] DEBUG -- : [httplog] Status: 200
D, [2014-08-26T13:08:49.335292 #6500] DEBUG -- : [httplog] Benchmark: 0.042724323 seconds
As can be seen in the above log excerpt (URL changed) the benchmarking does not include the connection time between the first and second lines. Is that intentional? Using httparty.
When using Google Drive API gem, for some reason the encoding is set to gzip, but the body is actually text/json.
So the httplog crashes on that:
Zlib::GzipFile::Error: not in gzip format
/Users/maris/.rvm/gems/ruby-2.4.1/gems/httplog-1.0.2/lib/httplog/http_log.rb:83:in `initialize'
/Users/maris/.rvm/gems/ruby-2.4.1/gems/httplog-1.0.2/lib/httplog/http_log.rb:83:in `new'
/Users/maris/.rvm/gems/ruby-2.4.1/gems/httplog-1.0.2/lib/httplog/http_log.rb:83:in `log_body'
/Users/maris/.rvm/gems/ruby-2.4.1/gems/httplog-1.0.2/lib/httplog/adapters/httpclient.rb:31:in `do_get_block'
/Users/maris/.rvm/gems/ruby-2.4.1/gems/httpclient-2.8.3/lib/httpclient.rb:1019:in `block in do_request'
/Users/maris/.rvm/gems/ruby-2.4.1/gems/httpclient-2.8.3/lib/httpclient.rb:1133:in `protect_keep_alive_disconnected'
/Users/maris/.rvm/gems/ruby-2.4.1/gems/httpclient-2.8.3/lib/httpclient.rb:1014:in `do_request'
/Users/maris/.rvm/gems/ruby-2.4.1/gems/httpclient-2.8.3/lib/httpclient.rb:1104:in `follow_redirect'
/Users/maris/.rvm/gems/ruby-2.4.1/gems/httpclient-2.8.3/lib/httpclient.rb:854:in `request'
/Users/maris/.rvm/gems/ruby-2.4.1/gems/google-api-client-0.20.1/lib/google/apis/core/http_command.rb:291:in `execute_once'
/Users/maris/.rvm/gems/ruby-2.4.1/gems/google-api-client-0.20.1/lib/google/apis/core/http_command.rb:104:in `block (2 levels) in execute'
/Users/maris/.rvm/gems/ruby-2.4.1/gems/retriable-3.1.1/lib/retriable.rb:61:in `block in retriable'
/Users/maris/.rvm/gems/ruby-2.4.1/gems/retriable-3.1.1/lib/retriable.rb:57:in `times'
/Users/maris/.rvm/gems/ruby-2.4.1/gems/retriable-3.1.1/lib/retriable.rb:57:in `retriable'
/Users/maris/.rvm/gems/ruby-2.4.1/gems/google-api-client-0.20.1/lib/google/apis/core/http_command.rb:101:in `block in execute'
/Users/maris/.rvm/gems/ruby-2.4.1/gems/retriable-3.1.1/lib/retriable.rb:61:in `block in retriable'
/Users/maris/.rvm/gems/ruby-2.4.1/gems/retriable-3.1.1/lib/retriable.rb:57:in `times'
/Users/maris/.rvm/gems/ruby-2.4.1/gems/retriable-3.1.1/lib/retriable.rb:57:in `retriable'
/Users/maris/.rvm/gems/ruby-2.4.1/gems/google-api-client-0.20.1/lib/google/apis/core/http_command.rb:93:in `execute'
/Users/maris/.rvm/gems/ruby-2.4.1/gems/google-api-client-0.20.1/lib/google/apis/core/base_service.rb:360:in `execute_or_queue_command'
/Users/maris/.rvm/gems/ruby-2.4.1/gems/google-api-client-0.20.1/generated/google/apis/drive_v2/service.rb:1290:in `list_files'
./app/lib/utils/remote_file_system/google_drive.rb:42:in `block in files_in_folder'
/Users/maris/.rvm/gems/ruby-2.4.1/gems/google-api-client-0.20.1/lib/google/apis/core/base_service.rb:51:in `block in initialize'
/Users/maris/.rvm/gems/ruby-2.4.1/gems/google-api-client-0.20.1/lib/google/apis/core/base_service.rb:53:in `block in initialize'
/Users/maris/.rvm/gems/ruby-2.4.1/gems/google-api-client-0.20.1/lib/google/apis/core/base_service.rb:64:in `block in each'
/Users/maris/.rvm/gems/ruby-2.4.1/gems/google-api-client-0.20.1/lib/google/apis/core/base_service.rb:63:in `loop'
/Users/maris/.rvm/gems/ruby-2.4.1/gems/google-api-client-0.20.1/lib/google/apis/core/base_service.rb:63:in `each'
./spec/lib/remote_file_system/google_drive_spec.rb:19:in `block (3 levels) in <top (required)>'
I know it may be better to wait for Google to fix the place where it decodes the body earlier, to reset the encoding or something, but it might never happen.
So I suggest to fix this place in code to handle the cases when the encoding is set to "gzip", but body is actually not gzipped anymore.
> Host: localhost:2080
> User-Agent: curl/7.47.0
> Accept: */*
>
* HTTP 1.0, assume close after body
< HTTP/1.0 200 OK
< Server: DAV/0.9.11 Python/3.5.2
< Date: Wed, 26 Sep 2018 08:44:24 GMT
< Content-type: application/pdf
< Transfer-Encoding: chunked
< Date: Wed, 26 Sep 2018 08:44:24 GMT
< DAV: 1,2
< Keep-Alive: timeout=15, max=86
< Last-Modified: Tue, 28 Aug 2018 13:10:16 GMT
* HTTP/1.0 connection set to keep alive!
< Connection: Keep-Alive
< Content-Length: 17896
< Content-Type: application/pdf
<
In this case HTTP gem provides Content-Type as an array and text_based?
handles it incorrectly.
In development.rb & production.rb of rails app I have:
Rails.application.configure do
HttpLog.configure do |config|
config.logger = Rails.logger
end
But running rails s
in dev or prod it explodes with:
/Users/drnic/Projects/dingo-projects/dingo-standalone-hub/config/environments/development.rb:9:in `block in <top (required)>': undefined method `configure' for HttpLog:Module (NoMethodError)
from /Users/drnic/.rvm/gems/ruby-2.3.3/gems/railties-5.0.1/lib/rails/railtie.rb:209:in `instance_eval'
from /Users/drnic/.rvm/gems/ruby-2.3.3/gems/railties-5.0.1/lib/rails/railtie.rb:209:in `configure'
from /Users/drnic/Projects/dingo-projects/dingo-standalone-hub/config/environments/development.rb:1:in `<top (required)>'
from /Users/drnic/.rvm/gems/ruby-2.3.3/gems/activesupport-5.0.1/lib/active_support/dependencies.rb:293:in `require'
from /Users/drnic/.rvm/gems/ruby-2.3.3/gems/activesupport-5.0.1/lib/active_support/dependencies.rb:293:in `block in require'
from /Users/drnic/.rvm/gems/ruby-2.3.3/gems/activesupport-5.0.1/lib/active_support/dependencies.rb:259:in `load_dependency'
from /Users/drnic/.rvm/gems/ruby-2.3.3/gems/activesupport-5.0.1/lib/active_support/dependencies.rb:293:in `require'
from /Users/drnic/.rvm/gems/ruby-2.3.3/gems/railties-5.0.1/lib/rails/engine.rb:600:in `block (2 levels) in <class:Engine>'
from /Users/drnic/.rvm/gems/ruby-2.3.3/gems/railties-5.0.1/lib/rails/engine.rb:599:in `each'
from /Users/drnic/.rvm/gems/ruby-2.3.3/gems/railties-5.0.1/lib/rails/engine.rb:599:in `block in <class:Engine>'
from /Users/drnic/.rvm/gems/ruby-2.3.3/gems/railties-5.0.1/lib/rails/initializable.rb:30:in `instance_exec'
from /Users/drnic/.rvm/gems/ruby-2.3.3/gems/railties-5.0.1/lib/rails/initializable.rb:30:in `run'
from /Users/drnic/.rvm/gems/ruby-2.3.3/gems/railties-5.0.1/lib/rails/initializable.rb:55:in `block in run_initializers'
from /Users/drnic/.rvm/rubies/ruby-2.3.3/lib/ruby/2.3.0/tsort.rb:228:in `block in tsort_each'
from /Users/drnic/.rvm/rubies/ruby-2.3.3/lib/ruby/2.3.0/tsort.rb:350:in `block (2 levels) in each_strongly_connected_component'
from /Users/drnic/.rvm/rubies/ruby-2.3.3/lib/ruby/2.3.0/tsort.rb:422:in `block (2 levels) in each_strongly_connected_component_from'
from /Users/drnic/.rvm/rubies/ruby-2.3.3/lib/ruby/2.3.0/tsort.rb:431:in `each_strongly_connected_component_from'
from /Users/drnic/.rvm/rubies/ruby-2.3.3/lib/ruby/2.3.0/tsort.rb:421:in `block in each_strongly_connected_component_from'
from /Users/drnic/.rvm/gems/ruby-2.3.3/gems/railties-5.0.1/lib/rails/initializable.rb:44:in `each'
from /Users/drnic/.rvm/gems/ruby-2.3.3/gems/railties-5.0.1/lib/rails/initializable.rb:44:in `tsort_each_child'
from /Users/drnic/.rvm/rubies/ruby-2.3.3/lib/ruby/2.3.0/tsort.rb:415:in `call'
from /Users/drnic/.rvm/rubies/ruby-2.3.3/lib/ruby/2.3.0/tsort.rb:415:in `each_strongly_connected_component_from'
from /Users/drnic/.rvm/rubies/ruby-2.3.3/lib/ruby/2.3.0/tsort.rb:349:in `block in each_strongly_connected_component'
from /Users/drnic/.rvm/rubies/ruby-2.3.3/lib/ruby/2.3.0/tsort.rb:347:in `each'
from /Users/drnic/.rvm/rubies/ruby-2.3.3/lib/ruby/2.3.0/tsort.rb:347:in `call'
from /Users/drnic/.rvm/rubies/ruby-2.3.3/lib/ruby/2.3.0/tsort.rb:347:in `each_strongly_connected_component'
from /Users/drnic/.rvm/rubies/ruby-2.3.3/lib/ruby/2.3.0/tsort.rb:226:in `tsort_each'
from /Users/drnic/.rvm/rubies/ruby-2.3.3/lib/ruby/2.3.0/tsort.rb:205:in `tsort_each'
from /Users/drnic/.rvm/gems/ruby-2.3.3/gems/railties-5.0.1/lib/rails/initializable.rb:54:in `run_initializers'
from /Users/drnic/.rvm/gems/ruby-2.3.3/gems/railties-5.0.1/lib/rails/application.rb:352:in `initialize!'
from /Users/drnic/Projects/dingo-projects/dingo-standalone-hub/config/environment.rb:5:in `<top (required)>'
from /Users/drnic/Projects/dingo-projects/dingo-standalone-hub/config.ru:3:in `require_relative'
from /Users/drnic/Projects/dingo-projects/dingo-standalone-hub/config.ru:3:in `block in <main>'
from /Users/drnic/.rvm/gems/ruby-2.3.3/gems/rack-2.0.1/lib/rack/builder.rb:55:in `instance_eval'
from /Users/drnic/.rvm/gems/ruby-2.3.3/gems/rack-2.0.1/lib/rack/builder.rb:55:in `initialize'
from /Users/drnic/Projects/dingo-projects/dingo-standalone-hub/config.ru:in `new'
from /Users/drnic/Projects/dingo-projects/dingo-standalone-hub/config.ru:in `<main>'
from /Users/drnic/.rvm/gems/ruby-2.3.3/gems/rack-2.0.1/lib/rack/builder.rb:49:in `eval'
from /Users/drnic/.rvm/gems/ruby-2.3.3/gems/rack-2.0.1/lib/rack/builder.rb:49:in `new_from_string'
from /Users/drnic/.rvm/gems/ruby-2.3.3/gems/rack-2.0.1/lib/rack/builder.rb:40:in `parse_file'
from /Users/drnic/.rvm/gems/ruby-2.3.3/gems/rack-2.0.1/lib/rack/server.rb:318:in `build_app_and_options_from_config'
from /Users/drnic/.rvm/gems/ruby-2.3.3/gems/rack-2.0.1/lib/rack/server.rb:218:in `app'
from /Users/drnic/.rvm/gems/ruby-2.3.3/gems/railties-5.0.1/lib/rails/commands/server.rb:59:in `app'
from /Users/drnic/.rvm/gems/ruby-2.3.3/gems/rack-2.0.1/lib/rack/server.rb:353:in `wrapped_app'
from /Users/drnic/.rvm/gems/ruby-2.3.3/gems/railties-5.0.1/lib/rails/commands/server.rb:124:in `log_to_stdout'
from /Users/drnic/.rvm/gems/ruby-2.3.3/gems/railties-5.0.1/lib/rails/commands/server.rb:77:in `start'
from /Users/drnic/.rvm/gems/ruby-2.3.3/gems/railties-5.0.1/lib/rails/commands/commands_tasks.rb:90:in `block in server'
from /Users/drnic/.rvm/gems/ruby-2.3.3/gems/railties-5.0.1/lib/rails/commands/commands_tasks.rb:85:in `tap'
from /Users/drnic/.rvm/gems/ruby-2.3.3/gems/railties-5.0.1/lib/rails/commands/commands_tasks.rb:85:in `server'
from /Users/drnic/.rvm/gems/ruby-2.3.3/gems/railties-5.0.1/lib/rails/commands/commands_tasks.rb:49:in `run_command!'
from /Users/drnic/.rvm/gems/ruby-2.3.3/gems/railties-5.0.1/lib/rails/commands.rb:18:in `<top (required)>'
from /Users/drnic/Projects/dingo-projects/dingo-standalone-hub/bin/rails:9:in `require'
from /Users/drnic/Projects/dingo-projects/dingo-standalone-hub/bin/rails:9:in `<top (required)>'
from /Users/drnic/.rvm/gems/ruby-2.3.3/gems/spring-2.0.1/lib/spring/client/rails.rb:28:in `load'
from /Users/drnic/.rvm/gems/ruby-2.3.3/gems/spring-2.0.1/lib/spring/client/rails.rb:28:in `call'
from /Users/drnic/.rvm/gems/ruby-2.3.3/gems/spring-2.0.1/lib/spring/client/command.rb:7:in `call'
from /Users/drnic/.rvm/gems/ruby-2.3.3/gems/spring-2.0.1/lib/spring/client.rb:30:in `run'
from /Users/drnic/.rvm/gems/ruby-2.3.3/gems/spring-2.0.1/bin/spring:49:in `<top (required)>'
from /Users/drnic/.rvm/gems/ruby-2.3.3/gems/spring-2.0.1/lib/spring/binstub.rb:31:in `load'
from /Users/drnic/.rvm/gems/ruby-2.3.3/gems/spring-2.0.1/lib/spring/binstub.rb:31:in `<top (required)>'
from /Users/drnic/Projects/dingo-projects/dingo-standalone-hub/bin/spring:15:in `require'
from /Users/drnic/Projects/dingo-projects/dingo-standalone-hub/bin/spring:15:in `<top (required)>'
from bin/rails:3:in `load'
from bin/rails:3:in `<main>'
Ideas?
The blog post link that you have linked for more information and details on httplog is broken.
Sorry if this is not the place to report the issue, but thought you would like to know.
Hi there,
I would like to toggle certain URLs without restarting my app server.
I have a admin panel to enable/disable logging for certain URLs and we would like to do that in runtime, maybe allowing a block in the whitelist/blacklist pattern?
HttpLog.configure do |config|
config.url_whitelist_pattern = lambda do
Regexp.union(AdminVendor.pluck(:endpoint).map { Regexp.escape _1 })
end
end
Httplog logs kind of memory pointer instead of the payload\data\body itself.
D, [2017-10-17T18:42:06.013390 #53965] DEBUG -- : [httplog] Sending: POST https://address.domain/api_method
D, [2017-10-17T18:42:06.013533 #53965] DEBUG -- : [httplog] Data: #<HTTP::Request::Body:0x007f93735632c8>
Here is my code for this
payload = { :param_1 => param_1.to_s, :param_2 => param_2.to_s, :param_3 => param_3.to_s}.to_json
HTTP.headers("id" => ID, :"User-Agent" => user_agent).post(url), :body => payload)
When I run the example from the readme:
RestClient.post 'http://example.com/resource', {param1: 'one', nested: {param2: 'two'}
I get the following log:
D [2018-01-03T15:53:21.839364] #4acdbe6b: [httplog] Connecting: example.com:80
D [2018-01-03T15:53:22.002725] #4acdbe6b: [httplog] Sending: POST http://example.com:80/resource
D [2018-01-03T15:53:22.003110] #4acdbe6b: [httplog] Data:
D [2018-01-03T15:53:22.334041] #4acdbe6b: [httplog] Status: 404
D [2018-01-03T15:53:22.334462] #4acdbe6b: [httplog] Benchmark: 0.33069 seconds
D [2018-01-03T15:53:22.334764] #4acdbe6b: [httplog] Response:
D [2018-01-03T15:53:22.334764] #4acdbe6b: <?xml version="1.0" encoding="iso-8859-1"?>
D [2018-01-03T15:53:22.334764] #4acdbe6b: <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
D [2018-01-03T15:53:22.334764] #4acdbe6b: "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
D [2018-01-03T15:53:22.334764] #4acdbe6b: <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
D [2018-01-03T15:53:22.334764] #4acdbe6b: <head>
D [2018-01-03T15:53:22.334764] #4acdbe6b: <title>404 - Not Found</title>
D [2018-01-03T15:53:22.334764] #4acdbe6b: </head>
D [2018-01-03T15:53:22.334764] #4acdbe6b: <body>
D [2018-01-03T15:53:22.334764] #4acdbe6b: <h1>404 - Not Found</h1>
D [2018-01-03T15:53:22.334764] #4acdbe6b: </body>
D [2018-01-03T15:53:22.334764] #4acdbe6b: </html>
Note that the data is empty. Tested in rest-client 2.0.2, httplog 1.0.0.
Is there any option/flag to set on to log the query parameters too for GET requests?
Hi devs,
Do I have to use this gem with Rails/Rack programs? Or anything special have to be done if I want to use the gem with a standalone ruby program.
Here's a small program that demonstrates the issue:
# demo.rb
require 'http'
require 'httplog'
HTTP.get('http://example.org')
Running this program with ruby demo.rb
yields:
D, [2017-07-19T14:53:48.658561 #74766] DEBUG -- : [httplog] Sending: GET http://example.org/
D, [2017-07-19T14:53:48.658660 #74766] DEBUG -- : [httplog] Data:
D, [2017-07-19T14:53:48.658702 #74766] DEBUG -- : [httplog] Connecting: example.org:80
/Users/shou/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/httplog-0.99.6/lib/httplog/http_log.rb:56:in `log_status': uninitialized constant #<Class:HttpLog>::Rack (NameError)
from /Users/shou/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/httplog-0.99.6/lib/httplog/adapters/http.rb:27:in `block in <class:Client>'
from /Users/shou/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/http-2.2.2/lib/http/client.rb:41:in `request'
from /Users/shou/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/http-2.2.2/lib/http/chainable.rb:75:in `request'
from /Users/shou/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/http-2.2.2/lib/http/chainable.rb:19:in `get'
from demo.rb:4:in `<main>'
I just want to inspect the traffic over a very simple ruby program, but HttpLog then yields this exception. What can I do to circumvent it? Thank you very much!
There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.
File: renovate.json
Error type: The renovate configuration file contains some invalid settings
Message: Configuration option
bundler.enabled should be boolean. Found: "true" (string)
When adding httplog 0.99.3
to a Rails 5 application and then setting logger to the one for Rails (as specified in the README), this error is presented:
undefined method `log' for nil:NilClass
Configuration in config/environments/development.rb
is as follows:
HttpLog.configure do |config|
config.logger = Rails.logger
end
Backtrace:
httplog (0.99.3) lib/httplog/http_log.rb:33:in `log'
httplog (0.99.3) lib/httplog/http_log.rb:38:in `log_connection'
httplog (0.99.3) lib/httplog/adapters/net_http.rb:36:in `connect'
/Users/kevin/.rvm/rubies/ruby-2.3.3/lib/ruby/2.3.0/net/http.rb:863:in `do_start'
/Users/kevin/.rvm/rubies/ruby-2.3.3/lib/ruby/2.3.0/net/http.rb:852:in `start'
/Users/kevin/.rvm/rubies/ruby-2.3.3/lib/ruby/2.3.0/net/http.rb:1398:in `request'
httplog (0.99.3) lib/httplog/adapters/net_http.rb:21:in `block in request'
/Users/kevin/.rvm/rubies/ruby-2.3.3/lib/ruby/2.3.0/benchmark.rb:308:in `realtime'
httplog (0.99.3) lib/httplog/adapters/net_http.rb:20:in `request'
newrelic_rpm (4.1.0.333) lib/new_relic/agent/instrumentation/net.rb:33:in `block in request_with_newrelic_trace'
newrelic_rpm (4.1.0.333) lib/new_relic/agent.rb:438:in `disable_all_tracing'
newrelic_rpm (4.1.0.333) lib/new_relic/agent/instrumentation/net.rb:32:in `request_with_newrelic_trace'
skylight (1.3.0) lib/skylight/probes/net_http.rb:26:in `request'
faraday (0.11.0) lib/faraday/adapter/net_http.rb:80:in `perform_request'
faraday (0.11.0) lib/faraday/adapter/net_http.rb:38:in `block in call'
faraday (0.11.0) lib/faraday/adapter/net_http.rb:85:in `with_net_http_connection'
faraday (0.11.0) lib/faraday/adapter/net_http.rb:33:in `call'
faraday (0.11.0) lib/faraday/request/url_encoded.rb:15:in `call'
faraday (0.11.0) lib/faraday/rack_builder.rb:139:in `build_response'
faraday (0.11.0) lib/faraday/connection.rb:377:in `run_request'
oauth2 (1.3.1) lib/oauth2/client.rb:99:in `request'
oauth2 (1.3.1) lib/oauth2/client.rb:146:in `get_token'
oauth2 (1.3.1) lib/oauth2/strategy/auth_code.rb:30:in `get_token'
omniauth-oauth2 (1.4.0) lib/omniauth/strategies/oauth2.rb:89:in `build_access_token'
omniauth-oauth2 (1.4.0) lib/omniauth/strategies/oauth2.rb:73:in `callback_phase'
It looks like the version.rb was bumped to 0.2.4 about 11 months ago which is inconsistent with what seems to be the release containing 0.2.4 code. Installing via gem install httplog
installs the code reflected around v0.2.3 but showing v0.2.4, possibly because of a RubyGems caching issue.
I attempted to use the blacklist feature but it wasn't working and noticed that the code I had was ~v2.1.3. Installing via git: <...> correctly installed the latest code.
May require a version bump?
signature of #create_socket has changed in httpclient between 2.6.0.1
httpclient/session.rb: def create_socket(site)
and 2.7.1
httpclient/session.rb: def create_socket(host, port)
httplog is not compatible with the most up to date signature.
httplog is licensed as MIT, but the colorize runtime dependency is licensed as GPL 2.0. Would you consider changes to httplog that remove the dependency or replacing it with an alternative (like rainbow) so that code that can't comply with GPL can use httplog?
Hello!
Sometimes there is an error type:
log writing failed. "\xC2" from ASCII-8BIT to UTF-8
"\xC2" - example. There are other characters
It was possible to find out what it is in response.
Can I use the following approach or is it something fraught?
msg.force_encoding('UTF-8')
# httplog/lib/httplog/http_log.rb
module HttpLog
...
class << self
...
def log(msg)
severities = Hash[*Logger::Severity.constants.enum_for(:each_with_index).collect{ |s, i| [i, s] }.flatten]
severity = severities[options[:severity]].to_s.downcase
options[:logger].send(severity, colorize(LOG_PREFIX + msg.force_encoding('UTF-8')))
end
...
end
...
end
Upon initial reading it seems like blacklisting is a way to tell httplog not to log a http request if the URL matches the blacklist. I gave this a try with httprb, and when I blacklist a URL, it seems like it doesn't send the request at all.
Which leads me to the question - is the blacklist only supposed to prevent logging of the request, or to prevent the request itself being sent?
httplog/lib/httplog/adapters/http.rb
Lines 11 to 40 in 9a853a9
When I read the source here above, there is no corresponding else
for if HttpLog.url_approved?(req.uri)
, so it seems like the request is not sent at all. Is this the correct behavior?
When I enable this gem, the HTTP gem does not return any data. So basically the following request returns an empty response.
content = HTTP.get(URL).body.to_s
Versions I am using -
ruby '2.7.0'
gem 'httplog', '1.4.0'
gem 'rails', '6.0.2.1'
gem 'http', '4.3.0'
This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.
These updates have all been created already. Click a checkbox below to force a retry/rebase of any.
Gemfile
.ruby-version
ruby 3.3.2
Congrats on the 1.6.0 release! Will there be a changelog entry added to highlight what's included? Thanks.
Hi,
I'm unable to see HTTP logs on my Rails 7 app.
I added version 1.5.0 of the gem in the :development group of my Gemfile and added the following to my /config/environments/development.rb :
HttpLog.configure do |config|
config.logger = Rails.logger
end
When I make a test http call from the ruby console using Ethon, the call succeeds but I get the following error message which points here:
/Users/<me>/.rvm/gems/ruby-3.0.3@hubflo/gems/httplog-1.5.0/lib/httplog/adapters/ethon.rb:33:in `perform': undefined method `[]' for nil:NilClass (NoMethodError)
When the test http call is made using Httparty, I don't get any error message. However regardless of the http gem used, no logs are displayed in my server logs.
I don't know what I'm missing. Thanks for your help!
Hello,
We encountered an Illegal seek (Errno::ESPIPE)
exception while doing some chunked transfer encoding with a pipe-io as body.
The exception is raised when the body is rewinded. While a pipe-io responses to rewind
it actually can't be rewinded.
httplog/lib/httplog/adapters/http.rb
Line 37 in ae519ee
In the source of the http.rb gem is a comment as to what happens when rewind is called on a pipe-io. Maybe that explains it a little better. :) https://github.com/httprb/http/blob/a0f540ffd03b29bf35bd5476b82a20fb8c3b88b8/lib/http/request/body.rb#L57
require 'http'
require 'httplog'
rd, wr = IO.pipe
wr.write('a chunk')
wr.close
HTTP.post('https://www.sumcumo.com', body: rd, headers: { 'Transfer-Encoding' => 'chunked' })
Don't raise an exception while chunked transfer encoding with pipe-io as body is used.
Have a great day! :)
Eric
I noticed that binary response body is logged although when I looked at the README and CHANGELOG there was a change to stop logging binary data.
As it now supports HTTPClient I suggest the title "Log outgoing HTTP requests" :)
Hello here!
Since Ruby 2.7 and frozen_string_literal, there is now more and more frozen strings used on Ruby code and gems.
Currently httplog is not compatible with such string, some calls crash if frozen strings met.
At least I spot a crash here because String#force_encoding
modify the string inplace and so raise a FrozenError: can't modify frozen String: ""
exception.
I hit this trouble through https://github.com/mrkamel/swift_client then https://github.com/jnunemaker/httparty then httplog:
FrozenError: can't modify frozen String: ""
from vendor/bundle/ruby/2.7.0/gems/httplog-1.4.3/lib/httplog/http_log.rb:332:in `force_encoding'
from vendor/bundle/ruby/2.7.0/gems/httplog-1.4.3/lib/httplog/http_log.rb:332:in `utf_encoded'
from vendor/bundle/ruby/2.7.0/gems/httplog-1.4.3/lib/httplog/http_log.rb:137:in `parse_body'
from vendor/bundle/ruby/2.7.0/gems/httplog-1.4.3/lib/httplog/http_log.rb:102:in `log_body'
from vendor/bundle/ruby/2.7.0/gems/httplog-1.4.3/lib/httplog/http_log.rb:46:in `call'
from vendor/bundle/ruby/2.7.0/gems/httplog-1.4.3/lib/httplog/adapters/net_http.rb:16:in `request'
from net/http.rb:1485:in `block in request'
from net/http.rb:933:in `start'
from net/http.rb:1483:in `request'
from vendor/bundle/ruby/2.7.0/gems/httplog-1.4.3/lib/httplog/adapters/net_http.rb:12:in `block in request'
from benchmark.rb:308:in `realtime'
from vendor/bundle/ruby/2.7.0/gems/httplog-1.4.3/lib/httplog/adapters/net_http.rb:11:in `request'
from vendor/bundle/ruby/2.7.0/gems/httparty-0.16.2/lib/httparty/request.rb:143:in `perform'
from vendor/bundle/ruby/2.7.0/gems/httparty-0.16.2/lib/httparty.rb:563:in `perform_request'
from vendor/bundle/ruby/2.7.0/gems/httparty-0.16.2/lib/httparty.rb:536:in `head'
from vendor/bundle/ruby/2.7.0/gems/httparty-0.16.2/lib/httparty.rb:629:in `head'
from vendor/bundle/ruby/2.7.0/bundler/gems/swift_client-72798338ec47/lib/swift_client.rb:193:in `request'
from vendor/bundle/ruby/2.7.0/bundler/gems/swift_client-72798338ec47/lib/swift_client.rb:126:in `head_object'
I currently bypass the crash with this monkey patch:
module Fixtures
module HttpLog
def utf_encoded(data, *args)
data = data.dup if data.frozen?
super(data, *args)
end
end
end
HttpLog.singleton_class.prepend Fixtures::HttpLog
Perhaps such protection may be added in other places in the code to be more robust to frozen strings.
It would be useful to be able to tell if any HTTP requests were made after an execution (for example to check if any tests are not using stubs for HTTP)
Under certain circumstances, a Encoding::CompatibilityError is raised (see backgrace below). This seems to occur only:
Backtrace:
Encoding::CompatibilityError: incompatible character encodings: UTF-8 and ASCII-8BIT
from /home/daniel/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/gems/activesupport-4.2.4/lib/active_support/core_ext/uri.rb:15:in `gsub'
from /home/daniel/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/gems/activesupport-4.2.4/lib/active_support/core_ext/uri.rb:15:in `unescape'
from /home/daniel/.rbenv/versions/2.2.0/lib/ruby/2.2.0/uri/common.rb:125:in `unescape'
from /home/daniel/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/gems/httplog-0.2.11/lib/httplog/http_log.rb:96:in `log_data'
from /home/daniel/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/gems/httplog-0.2.11/lib/httplog/adapters/net_http.rb:17:in `request'
from /home/daniel/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/gems/honeybadger-2.0.12/lib/honeybadger/plugins/net_http.rb:21:in `block in request_with_honeybadger'
from /home/daniel/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/gems/activesupport-4.2.4/lib/active_support/notifications.rb:164:in `block in instrument'
from /home/daniel/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/gems/activesupport-4.2.4/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
from /home/daniel/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/gems/activesupport-4.2.4/lib/active_support/notifications.rb:164:in `instrument'
from /home/daniel/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/gems/honeybadger-2.0.12/lib/honeybadger/plugins/net_http.rb:20:in `request_with_honeybadger'
from /home/daniel/.rbenv/versions/2.2.0/lib/ruby/2.2.0/net/http.rb:1376:in `block in request'
from /home/daniel/.rbenv/versions/2.2.0/lib/ruby/2.2.0/net/http.rb:853:in `start'
from /home/daniel/.rbenv/versions/2.2.0/lib/ruby/2.2.0/net/http.rb:1374:in `request'
from /home/daniel/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/gems/httplog-0.2.11/lib/httplog/adapters/net_http.rb:21:in `block in request'
from /home/daniel/.rbenv/versions/2.2.0/lib/ruby/2.2.0/benchmark.rb:303:in `realtime'
from /home/daniel/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/gems/httplog-0.2.11/lib/httplog/adapters/net_http.rb:20:in `request'
Is there any chance to get support for log4r gem?
When using the Ethon adapter and when the response_headers
is an empty string, this code produces nil
httplog/lib/httplog/adapters/ethon.rb
Lines 29 to 31 in 730c301
which then causes a crash when calling map
on it:
httplog/lib/httplog/adapters/ethon.rb
Line 40 in 730c301
I really need httplog to run in production in order to resolve a nasty issue that only happen on production server. I put it in the default group and add this initializer HttpLog.options[:logger] = Rails.logger
but cannot receive the http log in production.log. Please advice?
Gemfile.lock: httplog (1.1.0)
When I use the example in the README:
config.color = {color: :black, background: :yellow}
I get the following error:
HTTPLOG CONFIGURATION ERROR: {:color=>:black, :background=>:yellow} is not a valid color
Full initialization:
if Rails.env.development?
require 'httplog'
HttpLog.configure do |config|
# Enable or disable all logging
config.enabled = true
# You can assign a different logger
config.logger = Rails.logger
# I really wouldn't change this...
config.severity = Logger::Severity::DEBUG
# Tweak which parts of the HTTP cycle to log...
config.log_connect = true
config.log_request = true
config.log_headers = false
config.log_data = true
config.log_status = true
config.log_response = true
config.log_benchmark = true
# ...or log all request as a single line by setting this to `true`
config.compact_log = false
# Prettify the output - see below
config.color = false
# Limit logging based on URL patterns
config.url_whitelist_pattern = /.*/
config.url_blacklist_pattern = nil
end
end
Is there a way to filter the responce data (lambda/block)?
I want to filter the special data before sending it to the log.
According to Usage we need to use
require 'httplog'
But I've not added that require (only added the gem), yet I see httplog
logs in my log.
Is adding the require not necessary?
I'm seeing binary data spewing through our logs. [httplog] Data: PNG
is my clue. We're running v1.5.0 and I can see in readme etc that binary data shouldn't be logged.
Current initialization is:
HttpLog.configure do |config|
config.log_connect = false
config.log_request = true
config.log_headers = false
config.log_data = true
config.log_status = true
config.log_response = false
config.log_benchmark = false
config.compact_log = false
end
Any ideas or protips on how to stop this? Thanks in advance 👍
2021-06-22T02:39:59.332232+00:00 app[worker.1]: D, [2021-06-22T02:39:59.275059 #4] DEBUG -- : [httplog] Data: �PNG
2021-06-22T02:39:59.332235+00:00 app[worker.1]:
2021-06-22T02:39:59.332239+00:00 app[worker.1]:
2021-06-22T02:39:59.332271+00:00 app[worker.1]: �v���;��������|w��������������;��g��;������v�s�B:�����;e��_���{���o}����x<�Vp��K�/n�������v\k�}�p��;����>>:��7�=u�����'�T
2021-06-22T02:39:59.332274+00:00 app[worker.1]: O���,�"zbqlq|�����U=�8lq���<C����� ��p�!��q�M�,2
��%�y~�I�%���vz����H���z1��$�=��S����X���U}������>����x���'u�de������������`�E{�xg|.o�q�?�
���^N�;i���wr�H�eM�
2021-06-22T02:39:59.332279+00:00 app[worker.1]: ��r���>I��e������;j�hy��rQ��x���pi�I�C�;VF��K^
...
Update: config.log_data = false
seems to have quietened it down. I wonder what my peers were previously hoping to see with this set to true.
This could be my mistake, but I am getting this error after adding
gem 'httplog'
to my development group
uninitialized class variable @@options in HttpLog
rails4.0.1 with ruby2.1.0
Hello All,
I have installed httplog through gemfile. Everything is working fine but i want to get all the logs in a file rather than outputted directly in the terminal.
Could someone give me a help?
Regards
I have tried this in my development.rb
HttpLog.configure do |hconfig|
hconfig.logger = Rails.logger
end
I did a google search and found this:
Procyon::Application.configure do
...
config.after_initialize do
HttpLog.configure do |hconfig|
hconfig.logger = Rails.logger
end
end
end
ruby 2.2.6
rails 3.2.22.5
httplog 0.99.7
Neither one works. This is the error I get:
/.rvm/gems/ruby-2.2.6/gems/railties-3.2.22.5/lib/rails/commands/runner.rb:54:in `eval': undefined method `log' for #<ActiveSupport::BufferedLogger:0x000000042f0428> (NoMethodError)
from /home/pixie/.rvm/gems/ruby-2.2.6/gems/httplog-0.99.7/lib/httplog/http_log.rb:35:in `log'
from /home/pixie/.rvm/gems/ruby-2.2.6/gems/httplog-0.99.7/lib/httplog/http_log.rb:40:in `log_connection'
from /home/pixie/.rvm/gems/ruby-2.2.6/gems/httplog-0.99.7/lib/httplog/adapters/net_http.rb:36:in `connect'
from /home/pixie/.rvm/rubies/ruby-2.2.6/lib/ruby/2.2.0/net/http.rb:863:in `do_start'
from /home/pixie/.rvm/rubies/ruby-2.2.6/lib/ruby/2.2.0/net/http.rb:852:in `start'
from /home/pixie/.rvm/rubies/ruby-2.2.6/lib/ruby/2.2.0/net/http.rb:1375:in `request'
from /home/pixie/.rvm/gems/ruby-2.2.6/gems/httplog-0.99.7/lib/httplog/adapters/net_http.rb:21:in `block in request'
from /home/pixie/.rvm/rubies/ruby-2.2.6/lib/ruby/2.2.0/benchmark.rb:303:in `realtime'
from /home/pixie/.rvm/gems/ruby-2.2.6/gems/httplog-0.99.7/lib/httplog/adapters/net_http.rb:20:in `request'
from /home/pixie/.rvm/rubies/ruby-2.2.6/lib/ruby/2.2.0/net/http.rb:1398:in `send_entity'
from /home/pixie/.rvm/rubies/ruby-2.2.6/lib/ruby/2.2.0/net/http.rb:1186:in `post'
from /home/var/www/procyon/v5/lib/optimalblue.rb:21:in `oauth'
from (eval):1:in `<top (required)>'
from /home/pixie/.rvm/gems/ruby-2.2.6/gems/railties-3.2.22.5/lib/rails/commands/runner.rb:54:in `eval'
from /home/pixie/.rvm/gems/ruby-2.2.6/gems/railties-3.2.22.5/lib/rails/commands/runner.rb:54:in `<top (required)>'
from /home/pixie/.rvm/gems/ruby-2.2.6/gems/railties-3.2.22.5/lib/rails/commands.rb:64:in `require'
from /home/pixie/.rvm/gems/ruby-2.2.6/gems/railties-3.2.22.5/lib/rails/commands.rb:64:in `<top (required)>'
from script/rails:6:in `require'
from script/rails:6:in `<main>'
This gem is great but I would like to SELECTIVELY use it in my RSPEC tests with a key. For example, I might have some examples where I want to see the log, such as
it "lets me test the API connection to make sure it's working", show_log: do
...
where the show_log
key in the example would enable httplog and without it it would not be enabled.
Why httplog gem is not recommended to use for production environment, can you please explain about this briefly?
2.7.0 :006 > uri = URI('http://www.google.com')
2.7.0 :007 > Net::HTTP.get(uri)
Traceback (most recent call last):
1: from (irb):7
NoMethodError (undefined method `log' for nil:NilClass)
I can't get this work? why is it?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.