I think I’vs found a bug in httparty. When XML is parsed, escaped XML-characters (eg. ampersand) in attribute values are not replaced.

The method REXMLUtilityNode#translate_xml_entities is only used to unescape text values, the attribute values are left plain.

Example:
variant[‘url’] # “http://…/image.jpg?attr=val&attr2=val2”, not “http://…/image.jpg?attr=val&attr2=val2”.

What do you think?

Thanks and ciao,

der Flo

Right now, if you normalize_param for an array, like :foo, [‘bar’, ‘baz’], you’ll get “foo[]=bar&foo[]=baz”.

I’m pretty sure this is a convention that Rails uses. Regardless of its origin, not all services follow this convention. I’m wrapping the Sunlight API, and it expects a query more like “foo=bar&foo=baz”.

I could monkey patch Hash#normalize_param, but that seems less ideal. Maybe HTTParty could expose a normalization hook? Maybe something so awesome I can’t even fathom yet?

Reported by Josh Nichols

I think it may be a Crack problem, but if I make a get request returning the following XML (with format :xml set) the second 'package' appears inside the first, instead of as a sibling. (YAML output follows)
(The xml is from the Remote Control plugin used with JDownloader or JDownloader.org)

XML seems valid to me, looks like a parsing problem.

-- XML data --

-- End XML --

-- YAML of HTTParty interpretation --

package:
package_percent: "77.38"
package_todo: 83.62 MB
package_ETA: 00:-1
package_linkstotal: "4"
package_size: 369.72 MB
package:
package_percent: "0.00"
package_todo: 367.97 MB
package_ETA: 00:-1
package_linkstotal: "4"
package_size: 367.97 MB
package_linksinprogress: "0"
package_speed: 0 KB/s
package_name: More Rapidshare things
package_loaded: 0 KB
package_id: "1"
package_linksinprogress: "0"
package_speed: 0 KB/s
package_name: My Rapidshare things
package_loaded: 286.10 MB
package_id: "0"
-- End YAML --

Just did gem install for 0.5.2 and you are missing a lib for the require on line 10 of
/usr/lib/ruby/gems/1.8/gems/httparty-0.5.2/bin/../lib/httparty.rb

require dir + 'httparty/net_digest_auth'

missing from the lib dir:

/usr/lib/ruby/gems/1.8/gems/httparty-0.5.2/lib
/usr/lib/ruby/gems/1.8/gems/httparty-0.5.2/lib/httparty
/usr/lib/ruby/gems/1.8/gems/httparty-0.5.2/lib/httparty/request.rb
/usr/lib/ruby/gems/1.8/gems/httparty-0.5.2/lib/httparty/cookie_hash.rb
/usr/lib/ruby/gems/1.8/gems/httparty-0.5.2/lib/httparty/module_inheritable_attributes.rb
/usr/lib/ruby/gems/1.8/gems/httparty-0.5.2/lib/httparty/core_extensions.rb
/usr/lib/ruby/gems/1.8/gems/httparty-0.5.2/lib/httparty/response.rb
/usr/lib/ruby/gems/1.8/gems/httparty-0.5.2/lib/httparty/parser.rb
/usr/lib/ruby/gems/1.8/gems/httparty-0.5.2/lib/httparty/exceptions.rb
/usr/lib/ruby/gems/1.8/gems/httparty-0.5.2/lib/httparty.rb

I noticed you created a wrapper class HTTParty::Response that returns the same params as HTTPResponse but without the benefit of some of the methods like value() and comparison ability to things like HTTPSuccess, HTTPClientError, etc, so I have to hand roll some exception handling code to mimic this behavior rather than using the built in goodness from http/net.

Would it be possible to simply return the HTTPResponse rather than wrapping it?

This program illustrates:

require 'rubygems'
gem 'httparty', '0.5.0'
require 'httparty'

class Pub
  include HTTParty
  base_uri "http://pub.com"
end

class FoxAndHound < Pub
  base_uri "http://foxandhound.com"
end

class DragonsLair < Pub
end

Pub.base_uri("http://pub.com/v2")

puts Pub.base_uri         # => http://pub.com/v2
puts FoxAndHound.base_uri # => http://foxandhound.com
puts DragonsLair.base_uri # => http://pub.com

I would have expected:

puts Pub.base_uri         # => http://pub.com/v2
puts FoxAndHound.base_uri # => http://foxandhound.com
puts DragonsLair.base_uri # => http://pub.com/v2

0.4.3
Tumblr::User.new('VALID_EMAIL', 'VALID_PWD') => #<Tumblr::User:0x7fedb8b43dd8 @tumblr=LOTS_OF_TUMBLR_VARS

0.4.5
Tumblr::User.new('VALID_EMAIL', 'VALID_PWD') => #<Tumblr::User:0x36b40ec ....@tumblr="tumblr"

Note @tumblr just equals the string 'tumblr' in 4.5

Anyone have a pointer for why that might be?

I'm not sure where I can ask this question, so apologies if this clutters the issue list a bit.

Where/How can I set a timeout for a request? Given the below class, I thgouth I could put it in the options hash along with my :basic_auth settings, but that doesn't seem to affect how long it takes before timing out.

class MyRequest
  include HTTParty

  def initialize(u, p)
    @auth = {:username => u, :password => p}
  end

  def get(path)
    options = {:basic_auth => @auth, :timeout => 120000}  # I didn't know if it would be ms or seconds 
    self.class.get(path, options)
  end

end

Anyone know how to do this?

/Library/Ruby/Gems/1.8/gems/httparty-0.4.3/lib/httparty/core_extensions.rb:9: warning: method redefined; discarding old write

/Library/Ruby/Gems/1.8/gems/httparty-0.4.3/lib/httparty/request.rb:18: warning: method redefined; discarding old path=

/Library/Ruby/Gems/1.8/gems/httparty-0.4.3/lib/httparty/request.rb:26: warning: instance variable @reDIrect not initialized

The examples that use pp to display the response object give the error

undefined method inspect' for classHTTParty::Response' (NameError)

because inspect is undefined in BasicObject. Restoring an inspect method to HTTParty::Response would be helpful (and easier than modifying all the examples).

HappyMapper 0.2.2 (commit 545ebcd8a8b6dec66a526244321fc1dfb171de05) libxml 0.9.8

If an object including HappyMapper has multiple child elements (via has_one or has_many) and a default namespace, an exception is raised when processing the second child element.

I’ve recreated the issue in the specs, see attached diff.

It seems trying to assign a default namespace prefix to a node fails in libxml, but HappyMapper should probably not try to do this anyway.

I’ve fixed this ticket with http://github.com/lightningdb/happymapper/commit/ac6510bcec0314435d29ae092e857e965c0e1c28

The issue is that libxml falls over if the default prefix is assigned more than once. Seems to be a bug with libxml, but likewise it is probably something that happymapper should avoid anyway.

So before assigning the default prefix, we check whether a default prefix has already been assigned.

I’ve updated the tests to reflect this situation, by adding an extra ‘has_one’, which invokes the nested parsing that caused the error in the first place. I moved the Address class mapper definition up and added an address node to the product (I realise this doesn’t make ‘domain’ sense, but it proves the issue and the fix).

Hopefully John approves and can pull into the main happymapper?

There needs to be some kind of option to specify a timeout on calls.

Example patch to HTTParty::Request#http:

http://e-huned.com/2009/06/11/monkey-patch-httparty-to-use-a-timeout/
http://gist.github.com/130298

Here's a way to reproduce it. I have a patch to add a test that also reproduces it and a possible fix. Will send a pull request.

require 'httparty'
r1 = HTTParty.get("http://orgmode.org/org.html")
opts = {"if-modified-since" => r1.headers["last-modified"].first}
r2 = HTTParty.get("http://orgmode.org/org.html", :headers => opts)
URI::InvalidURIError: bad URI(is not URI?):
from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/uri/common.rb:436:in split' from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/uri/common.rb:485:inparse'
from /Library/Ruby/Gems/1.8/gems/httparty-0.5.0/lib/httparty/request.rb:29:in path=' from /Library/Ruby/Gems/1.8/gems/httparty-0.5.0/lib/httparty/request.rb:136:inhandle_response'
from /Library/Ruby/Gems/1.8/gems/httparty-0.5.0/lib/httparty/request.rb:59:in perform' from /Library/Ruby/Gems/1.8/gems/httparty-0.5.0/lib/httparty.rb:201:inperform_request'
from /Library/Ruby/Gems/1.8/gems/httparty-0.5.0/lib/httparty.rb:157:in get' from /Library/Ruby/Gems/1.8/gems/httparty-0.5.0/lib/httparty.rb:233:inget'
from (irb):7

I'm curious what folks think of this. In the following code, the base_uri is the entire URI because the method is specified in a parameter. So I need to get "", :query => ..., which is sort of ugly. See below:

class Flickr
  include HTTParty
  base_uri "http://api.flickr.com/services/rest/"
  default_params :api_key => ENV['WHOSE_FLICKR_API_KEY']

  class << self
    def photo(id)
      get("", :query => {:method => "flickr.photos.getInfo", :photo_id => id})
    end

    def user(id)
      get("", :query => {:method => "flickr.people.getInfo", :user_id => id})
    end
  end
end

What if get, if the first parameter isn't a string, just uses the base_uri as the entire URI to perform a GET request on? Curious if other folks would find this useful...

Cheers,
Alex

I wrote a small Twitter httparty client that used it's search features.

In development on my machine, 0.4.4 and 0.4.5 worked the same... When I deployed the application to Heroku, 0.4.4 works but 0.4.5 causes Twitter to return a 400 Bad Request.

I feel like a bad bug reporter since I don't have any sample code, it was basically the same as your twitter example but built for the search.

I would imagine it's something to do with how it builds the request. Not sure if anything has changed recently, but for some reason, whatever is machine specific in the request generation is causing errors on Heroku.

Steps to reproduce:

1. Used this: http://gist.github.com/178266
2. With 0.4, it returned results in dev mode and on Heroku.
3. With 0.4.5 it returns results in dev, but not on Heroku.

Expected:
I should get twitter search results.

Actual:
I get a 400 Bad Request

The fb fql api has a required param named query. This will slip through and not get properly processed by httparty

Issue

Including HTTParty in a class that inherits from ActiveRecord::Base and is the parent of a child class produces a TypeError “can't dup NilClass” on creation of a new instance of the child class:

TypeError: can't dup NilClass
    from /usr/local/lib/ruby/gems/1.8/gems/activerecord-2.3.8/lib/active_record/base.rb:2219:in `dup'
    from /usr/local/lib/ruby/gems/1.8/gems/activerecord-2.3.8/lib/active_record/base.rb:2219:in `scoped_methods'
    from /usr/local/lib/ruby/gems/1.8/gems/activerecord-2.3.8/lib/active_record/base.rb:2223:in `current_scoped_methods'
    from /usr/local/lib/ruby/gems/1.8/gems/activerecord-2.3.8/lib/active_record/base.rb:2206:in `scoped?'
    from /usr/local/lib/ruby/gems/1.8/gems/activerecord-2.3.8/lib/active_record/base.rb:2474:in `send'
    from /usr/local/lib/ruby/gems/1.8/gems/activerecord-2.3.8/lib/active_record/base.rb:2474:in `initialize'
    from (irb):2:in `new'
    from (irb):2

Steps to reproduce

1. Create new Rails project:

   rails test

2. Generate parent model:

   script/generate model Foo

3. Verify new instances of Foo create without issue:

   rake db:migrate
script/console
f = Foo.new # => #<Foo…>
quit

4. Generate child model:

   script/generate model Bar

5. Verify new instances of Bar create without issue:

   rake db:migrate
script/console
b = Bar.new # => #<Bar…>
quit

6. Require HTTParty in Foo:

   require 'httparty'

   class Foo < ActiveRecord::Base
include HTTParty
end

7. Verify new instances of Foo, Bar can still be created (repeat steps 3, 5 but without migration).

8. Change parent of Bar to Foo:

   class Bar < Foo
end

9. Create new instances of Foo, then Bar:

   script/console
f = Foo.new # => #<Foo…>
b = Bar.new # => TypeError

System configuration

Issue confirmed on 2 different operating systems using 3 different environments:

• Ruby 1.9.1 p378 / Rails 2.38 / HTTParty 0.6.0
• Ruby 1.8.7 p174 / Rails 2.3.4 / HTTParty 0.5.2, then 0.6.0
• Ruby 1.8.7 p174 / Rails 2.3.8 / HTTParty 0.6.0

HTTParty’s JSON parser seems to not handle backslashes properly. A script that demonstrates can be found here: http://pastie.org/377389

Briefly, HTTParty::Parsers::JSON.decode(‘{"chokes": “on this \”}’) raises an HTTParty::Parsers::JSON::ParseError, when services like JSONLint (jsonlint.com) deems {"chokes": “on this \”} to be valid JSON.

httparty has a runtime rubygems dependency which makes it hard to use in a non-RubyGems environment, e.g. when vendored.

I'm not sure how to fix these lines but perhaps checking for the "gem" command and falling back to just requiring crack on its own is enough?

For example:

gem 'crack', '>= 0.1.1' if respond_to? :gem
require 'crack'

Lines in question: http://github.com/jnunemaker/httparty/blob/master/lib/httparty.rb#L4-6

As far as I can tell, the module HTTParty has no instance methods. You include it, and it turns around and calls extend on a different module. And then it sets class-level instance variables, class-level inheritable attributes, and so forth.

Is it just a personal preference? What if we would prefer the module to operate on an instance level instead? IE: instantianting an agent class that configures the httparty attributes.

Tim

It took me opening up the gem source to even find the :body option for a post request since it wasn't in the rdoc or the examples (unless I overlooked it), perhaps some documentation could be added on what options you can pass for each method.

I'd volunteer myself, but I'm just starting with httparty (clearly :))

given:
class EnterpriseService
include HTTParty
format :plain

  def shared_function
    "some value"
  end
end

class MyService < EnterpriseService
  base_uri "some.server:4000/my_service"
end

class YourService < EnterpriseService
  base_uri "some.server:4000/your_service"
end

when using in development mode:

  $ script/console development
  >> MyService.base_uri
  => "some.server:4000/my_service"
  >> YourService.base_uri
  => "some.server:4000/your_service"

when using in production mode:

  $ script/console production
  >> MyService.base_uri
  => "some.server:4000/your_service"
  >> YourService.base_uri
  => "some.server:4000/your_service"

problem:

when I include HTTParty in a class that I'm planning on subclassing (in order to share functions and HTTParty), each subclass works fine when running in development mode. Once I switch to production, each subclass winds up with the same base_uri as the last class to get loaded (in alphabetical order).

The culprit seems to be "config.cache_classes = true"

I'm not entirely sure how to fix this or if it's even possible to fix in HTTParty itself, but for now, my workaround has been to simply include HTTParty and specify the HTTParty options separately in each subclass rather than doing it only once in the main class.

if I have format :xml and get a response with something like this in it:

\url\
\url\

the hash created is {'poster' => ['url', 'url']} and the sizes are lost.
Would it make sense to have it parsed something like this:

{'poster' => [{'size'=>'original', 'poster'=>'url'}, {'size'=>'mid', 'poster'=>'url']}
that way no data is lost. Open to other suggestions.

HTTParty is broken with Ruby 1.9.2

HTTParty.get("http://google.com") gives you:

NameError: uninitialized constant HTTParty::Response::Object
    from /lib/httparty/response.rb:41:in `class'
    from /lib/httparty/response.rb:49:in `inspect'

The Rails 3 Release notes under ActiveResource says that: "Ruby 1.9.2: URI.parse and .decode are deprecated and are no longer used in the library"

Also I checked the news for Ruby 1.9.2 and found:
URI

• new methods:

  URI.encode_www_form
  URI.decode_www_form
  URI.encode_www_form_component
  URI.decode_www_form_component
  

• Obsoleted methods:

  URI.decode
  URI.encode
  URI.escape
  URI.unescape
  

  FYI: ruby -v => ruby 1.9.2dev (2010-04-28 trunk 27532) [i686-linux]

The nil returned by HTTParty.get when the server returns an empty response is a bit strange. It is of class NilClass, but doesn't behave as you'd expect. See below for a sample irb session demonstrating the problem. In particular, note that the "if" statement evaluates to true even though you'd expect it to evaluate to false, and id returns a different value to object_id.

response = HTTParty.get("http://www.tianamo.com/elmo/getrelated_broadway.php", :query => {:query => "rif skos soic", :count => 5})

=> nil

puts "Response not nil" if response

Response not nil

=> nil

response.object_id

=> 4

response.id

=> 2156740480

A request for http://www.myurl.com/sample turns into something like http://www.myurl.com/sample?&&&&& before the request is fired off

This isn't a problem for many sites, but it turns out to be a problem for mine.

Suggested fix is in the request class:
query_string_parts << options[:default_params].to_params unless options[:default_params].nil?
should be
query_string_parts << options[:default_params].to_params unless options[:default_params].empty? (since it's an empty hash by default)

Thanks! Should be just a quick fix :)

Fixed with this patch

http://github.com/ddollar/httparty/commit/9c08f68503864d06aea05177ce0f46a437822f26

The docs for post claim:

Simple post with full url using :query option,

which gets set as form data on the request.

Foo.post('http://foo.com/resources', :query => {:bar => 'baz'})

I tried posting in this format and stepped through with ruby-debug and the query param gets tacked on to the uri path but not assigned to the body.

In the case of trying this post:
Foo.post( 'http://api.facebook.com/restserver.php', :query => { :monkeys => '12' } )

I get this EOFError:
/ruby-1.8.6-p383/lib/ruby/1.8/net/protocol.rb:133:in sysread': end of file reached (EOFError) from /Users/jesse/.rvm/ruby-1.8.6-p383/lib/ruby/1.8/net/protocol.rb:133:inrbuf_fill'
from /Users/jesse/.rvm/ruby-1.8.6-p383/lib/ruby/1.8/timeout.rb:62:in timeout' from /Users/jesse/.rvm/ruby-1.8.6-p383/lib/ruby/1.8/timeout.rb:93:intimeout'
from /Users/jesse/.rvm/ruby-1.8.6-p383/lib/ruby/1.8/net/protocol.rb:132:in rbuf_fill' from /Users/jesse/.rvm/ruby-1.8.6-p383/lib/ruby/1.8/net/protocol.rb:116:inreaduntil'
from /Users/jesse/.rvm/ruby-1.8.6-p383/lib/ruby/1.8/net/protocol.rb:126:in readline' from /Users/jesse/.rvm/ruby-1.8.6-p383/lib/ruby/1.8/net/http.rb:2020:inread_status_line'
from /Users/jesse/.rvm/ruby-1.8.6-p383/lib/ruby/1.8/net/http.rb:2009:in read_new' ... 6 levels... from /Users/jesse/.rvm/gems/ruby/1.8.6/gems/httparty-0.4.5/lib/httparty/request.rb:41:inperform'
from /Users/jesse/.rvm/gems/ruby/1.8.6/gems/httparty-0.4.5/lib/httparty.rb:171:in perform_request' from /Users/jesse/.rvm/gems/ruby/1.8.6/gems/httparty-0.4.5/lib/httparty.rb:152:inpost'
from httparty_bug.rb:10

In 6857f55 blank? was changed to nil?. If options[:default_params] is an empty Array, query_string(uri) will return a empty string, which will then cause the ? to be appended to the uri.

This isn't a big issue, however it makes checking for URL calls (via webmock) more difficult, as there's always this extra "?" in the query.

Right now the redirect logic resends the original query parameters on any redirects. I think in nearly all cases this is incorrect. When a 302 is encountered after a POST, it makes no sense to resend the original query parameters (I think this is also true of DELETE and PUT).

I couldn’t find an authoritative answer on GET, but I believe that the Location header returned in the 302 response is to be honored in its entirety and exclusively, that is without glomming additional query parameters.

Unfortunately I’ve run into several web services (esp. Java ones for some reason) that send 302s to canonicalize all traffic to using trailing-slashes. For example GET /foo/bar → 302 /foo/bar/

In this case, the practical thing to do is to probably resend the query parameters. I think the backing services probably need to be fixed to set Location to something like /foo/bar/?dude=true when /foo/bar?dude=true is sent.

Anyway…I think there’s a bug there.

Would it make sense for :format => :xxxx or header 'content-type' => xxxx to also set the format of the request being sent into xxxx format.

So for exmaple, if :format => :json, the request sent would be formatted in json.

To avoid conflict, a new option might be create. Something like :request_format.

The following code gives a ‘stack level too deep’ error message when run on the attached data file:

```
require ‘rubygems’
require ‘happymapper’

class Node
include HappyMapper
element :node_name, String
has_many :node, ::Node
end

class Taxonomy
include HappyMapper
element :taxonomy_name, String
has_many :node, ::Node
end

class Taxonomies
include HappyMapper
has_many :taxonomy, ::Taxonomy
end

file_contents = File.read(‘worldguide_data/taxonomy_short.xml’)
taxonomies = Taxonomies.parse(file_contents)

/opt/local/lib/ruby/gems/1.8/gems/jnunemaker-happymapper-0.1.5/lib/happymapper.rb:93:in `create_collection’: stack level too deep (SystemStackError)
from /opt/local/lib/ruby/gems/1.8/gems/jnunemaker-happymapper-0.1.5/lib/happymapper.rb:93:in `each’
from /opt/local/lib/ruby/gems/1.8/gems/jnunemaker-happymapper-0.1.5/lib/happymapper.rb:93:in `create_collection’
from /opt/local/lib/ruby/gems/1.8/gems/jnunemaker-happymapper-0.1.5/lib/happymapper.rb:107:in `inject’
from /opt/local/lib/ruby/gems/1.8/gems/jnunemaker-happymapper-0.1.5/lib/happymapper.rb:90:in `each’
from /opt/local/lib/ruby/gems/1.8/gems/jnunemaker-happymapper-0.1.5/lib/happymapper.rb:90:in `inject’
from /opt/local/lib/ruby/gems/1.8/gems/jnunemaker-happymapper-0.1.5/lib/happymapper.rb:90:in `create_collection’
from /opt/local/lib/ruby/gems/1.8/gems/jnunemaker-happymapper-0.1.5/lib/happymapper.rb:79:in `parse’
from /opt/local/lib/ruby/gems/1.8/gems/jnunemaker-happymapper-0.1.5/lib/happymapper/item.rb:29:in `from_xml_node’
… 12965 levels…
from /opt/local/lib/ruby/gems/1.8/gems/jnunemaker-happymapper-0.1.5/lib/happymapper.rb:90:in `inject’
from /opt/local/lib/ruby/gems/1.8/gems/jnunemaker-happymapper-0.1.5/lib/happymapper.rb:90:in `create_collection’
from /opt/local/lib/ruby/gems/1.8/gems/jnunemaker-happymapper-0.1.5/lib/happymapper.rb:79:in `parse’
from test.rb:22
```

It would be nice. Someone want to do it? Yay!

In net-http, when you lookup a header in the headers hash, it overrides the method and downcases the key argument before grabbing the value from the internal data structure. HTTParty should do the same thing.

Right now, for example, doing this: response.headers['Cache-Control'] gives you a nil even though there is a "Cache-Control" header in the HTTP response. response.headers['cache-control'] works, however.

If no one disagrees that this is a bug that needs fixing, I'll take a crack at fixing it in a fork and issue a pull request.

Encountering an odd error under Snow Leopard regarding a superclass mismatch when trying to run rake db:migrate:

** Invoke db:migrate (first_time)
** Invoke environment (first_time)
** Execute environment
rake aborted!
superclass mismatch for class BlankSlate
/opt/local/lib/ruby/gems/1.9.1/gems/httparty-0.4.4/lib/httparty/core_extensions.rb:6:in <top (required)>' /opt/local/lib/ruby/gems/1.9.1/gems/activesupport-2.3.3/lib/active_support/dependencies.rb:156:inrequire'
/opt/local/lib/ruby/gems/1.9.1/gems/activesupport-2.3.3/lib/active_support/dependencies.rb:156:in block in require' /opt/local/lib/ruby/gems/1.9.1/gems/activesupport-2.3.3/lib/active_support/dependencies.rb:521:innew_constants_in'
/opt/local/lib/ruby/gems/1.9.1/gems/activesupport-2.3.3/lib/active_support/dependencies.rb:156:in require' /opt/local/lib/ruby/gems/1.9.1/gems/httparty-0.4.4/lib/httparty.rb:203:in<top (required)>'
/opt/local/lib/ruby/gems/1.9.1/gems/activesupport-2.3.3/lib/active_support/dependencies.rb:156:in require' /opt/local/lib/ruby/gems/1.9.1/gems/activesupport-2.3.3/lib/active_support/dependencies.rb:156:inblock in require'
/opt/local/lib/ruby/gems/1.9.1/gems/activesupport-2.3.3/lib/active_support/dependencies.rb:521:in new_constants_in' /opt/local/lib/ruby/gems/1.9.1/gems/activesupport-2.3.3/lib/active_support/dependencies.rb:156:inrequire'
/opt/local/lib/ruby/gems/1.9.1/gems/twitter-0.6.15/lib/twitter.rb:11:in <top (required)>' /opt/local/lib/ruby/gems/1.9.1/gems/activesupport-2.3.3/lib/active_support/dependencies.rb:156:inrequire'
/opt/local/lib/ruby/gems/1.9.1/gems/activesupport-2.3.3/lib/active_support/dependencies.rb:156:in block in require' /opt/local/lib/ruby/gems/1.9.1/gems/activesupport-2.3.3/lib/active_support/dependencies.rb:521:innew_constants_in'
/opt/local/lib/ruby/gems/1.9.1/gems/activesupport-2.3.3/lib/active_support/dependencies.rb:156:in require' /opt/local/lib/ruby/gems/1.9.1/gems/rails-2.3.3/lib/rails/gem_dependency.rb:208:inload'
/opt/local/lib/ruby/gems/1.9.1/gems/rails-2.3.3/lib/initializer.rb:307:in block in load_gems' /opt/local/lib/ruby/gems/1.9.1/gems/rails-2.3.3/lib/initializer.rb:307:ineach'
/opt/local/lib/ruby/gems/1.9.1/gems/rails-2.3.3/lib/initializer.rb:307:in load_gems' /opt/local/lib/ruby/gems/1.9.1/gems/rails-2.3.3/lib/initializer.rb:164:inprocess'
/opt/local/lib/ruby/gems/1.9.1/gems/rails-2.3.3/lib/initializer.rb:113:in run' /Users/jamesthompson/Development/RubyBayou-Website/config/environment.rb:9:in<top (required)>'
/opt/local/lib/ruby/gems/1.9.1/gems/activesupport-2.3.3/lib/active_support/dependencies.rb:156:in require' /opt/local/lib/ruby/gems/1.9.1/gems/activesupport-2.3.3/lib/active_support/dependencies.rb:156:inblock in require'
/opt/local/lib/ruby/gems/1.9.1/gems/activesupport-2.3.3/lib/active_support/dependencies.rb:521:in new_constants_in' /opt/local/lib/ruby/gems/1.9.1/gems/activesupport-2.3.3/lib/active_support/dependencies.rb:156:inrequire'
/opt/local/lib/ruby/gems/1.9.1/gems/rails-2.3.3/lib/tasks/misc.rake:4:in block in <top (required)>' /opt/local/lib/ruby/gems/1.9.1/gems/rake-0.8.7/lib/rake.rb:636:incall'
/opt/local/lib/ruby/gems/1.9.1/gems/rake-0.8.7/lib/rake.rb:636:in block in execute' /opt/local/lib/ruby/gems/1.9.1/gems/rake-0.8.7/lib/rake.rb:631:ineach'
/opt/local/lib/ruby/gems/1.9.1/gems/rake-0.8.7/lib/rake.rb:631:in execute' /opt/local/lib/ruby/gems/1.9.1/gems/rake-0.8.7/lib/rake.rb:597:inblock in invoke_with_call_chain'
/opt/local/lib/ruby/1.9.1/monitor.rb:190:in mon_synchronize' /opt/local/lib/ruby/gems/1.9.1/gems/rake-0.8.7/lib/rake.rb:590:ininvoke_with_call_chain'
/opt/local/lib/ruby/gems/1.9.1/gems/rake-0.8.7/lib/rake.rb:607:in block in invoke_prerequisites' /opt/local/lib/ruby/gems/1.9.1/gems/rake-0.8.7/lib/rake.rb:604:ineach'
/opt/local/lib/ruby/gems/1.9.1/gems/rake-0.8.7/lib/rake.rb:604:in invoke_prerequisites' /opt/local/lib/ruby/gems/1.9.1/gems/rake-0.8.7/lib/rake.rb:596:inblock in invoke_with_call_chain'
/opt/local/lib/ruby/1.9.1/monitor.rb:190:in mon_synchronize' /opt/local/lib/ruby/gems/1.9.1/gems/rake-0.8.7/lib/rake.rb:590:ininvoke_with_call_chain'
/opt/local/lib/ruby/gems/1.9.1/gems/rake-0.8.7/lib/rake.rb:583:in invoke' /opt/local/lib/ruby/gems/1.9.1/gems/rake-0.8.7/lib/rake.rb:2051:ininvoke_task'
/opt/local/lib/ruby/gems/1.9.1/gems/rake-0.8.7/lib/rake.rb:2029:in block (2 levels) in top_level' /opt/local/lib/ruby/gems/1.9.1/gems/rake-0.8.7/lib/rake.rb:2029:ineach'
/opt/local/lib/ruby/gems/1.9.1/gems/rake-0.8.7/lib/rake.rb:2029:in block in top_level' /opt/local/lib/ruby/gems/1.9.1/gems/rake-0.8.7/lib/rake.rb:2068:instandard_exception_handling'
/opt/local/lib/ruby/gems/1.9.1/gems/rake-0.8.7/lib/rake.rb:2023:in top_level' /opt/local/lib/ruby/gems/1.9.1/gems/rake-0.8.7/lib/rake.rb:2001:inblock in run'
/opt/local/lib/ruby/gems/1.9.1/gems/rake-0.8.7/lib/rake.rb:2068:in standard_exception_handling' /opt/local/lib/ruby/gems/1.9.1/gems/rake-0.8.7/lib/rake.rb:1998:inrun'
/opt/local/lib/ruby/gems/1.9.1/gems/rake-0.8.7/bin/rake:31:in <top (required)>' /opt/local/bin/rake:19:inload'
/opt/local/bin/rake:19:in `

warning: HTTParty depends on version 0.1.7 of crack, not 0.1.6.

Getting this warning with HTTParty 0.6.0 running Ruby 1.9.1 and Ruby 1.8.7, even though I only have crack 0.1.7 installed. Httparty works greate in both environments, but the warning is annoying ;)

One spec fails to run in 1.9.x, due to crack being unable to parse the twitter.json fixture file. This issue has already been reported for crack.

Here's the output of running the specs with ruby 1.9.1:

$ rake spec
(in /home/phiggins/work/httparty/httparty)
/home/phiggins/.rvm/gems/ruby-1.9.1-p378/gems/jeweler-1.4.0/lib/jeweler/commands/check_dependencies.rb:13:Warning: Gem::Dependency#version_requirements is deprecated and will be removed on or after August 2010.  Use #requirement
All dependencies seem to be installed.
warning: HTTParty depends on version 0.1.7 of crack, not 0.1.6.
Example disabled: should allow hashes to be accessed with dot notation
Example disabled: should allow nested hashes to be accessed with dot notation

HTTParty::Request
- should not attempt to parse empty responses
- should not fail for missing mime type

HTTParty::Request initialization
- sets parser to HTTParty::Parser
- sets parser to the optional parser

HTTParty::Request#format
- should return the correct parsing format

HTTParty::Request options
- should use basic auth when configured
- should use digest auth when configured

HTTParty::Request#uri query strings
- does not add an empty query string when default_params are blank

HTTParty::Request http
- should use ssl for port 443
- should not use ssl for port 80
- uses ssl for https scheme with default port
- uses ssl for https scheme regardless of port

HTTParty::Request http PEM certificates when scheme is https
- should use a PEM certificate when provided
- should verify the certificate when provided

HTTParty::Request http PEM certificates when scheme is not https
- does not assign a PEM
- should not verify a certificate if scheme is not https

HTTParty::Request http PEM certificates debugging
- calls #set_debug_output when the option is provided
- does not set_debug_output when the option is not provided

HTTParty::Request http when setting timeout
- does nothing if the timeout option is a string
- sets the timeout to 5 seconds

HTTParty::Request#format_from_mimetype
- should handle text/xml
- should handle application/xml
- should handle text/json
- should handle application/json
- should handle text/javascript
- should handle application/javascript
- returns nil for an unrecognized mimetype
- returns nil when using a default parser

HTTParty::Request parsing responses
- should handle xml automatically
- should handle json automatically
- should handle yaml automatically
- should include any HTTP headers in the returned response

HTTParty::Request parsing responses with non-200 responses
- should return a valid object for 4xx response
- should return a valid object for 5xx response

HTTParty::Request parsing responses with non-200 responses 3xx responses
- returns a valid object for 304 not modified
- redirects if a 300 contains a location header
- returns the Net::HTTP response if the 300 does not contain a location header

HTTParty::Request a request that redirects once
- should be handled by GET transparently
- should be handled by POST transparently
- should be handled by DELETE transparently
- should be handled by PUT transparently
- should be handled by HEAD transparently
- should be handled by OPTIONS transparently
- should keep track of cookies between redirects
- should update cookies with rediects
- should keep cookies between rediects
- should make resulting request a get request if it not already
- should not make resulting request a get request if options[:maintain_method_across_redirects] is true

HTTParty::Request a request that redirects infinitely
- should raise an exception

HTTParty::Request with POST http method
- should raise argument error if query is not a hash

HTTParty::Request argument validation
- should raise argument error if basic_auth and digest_auth are both present
- should raise argument error if basic_auth is not a hash
- should raise argument error if digest_auth is not a hash

HTTParty::Response
- returns response headers
- should send missing methods to delegate
- should be able to iterate if it is array
- allows headers to be accessed by mixed-case names in hash notation
- returns a comma-delimited value when multiple values exist
- responds to hash methods

HTTParty::Response initialization
- should set the Net::HTTP Response
- should set body
- should set code
- should set code as a Fixnum

HTTParty::Parser.SupportedFormats
- returns a hash

HTTParty::Parser.call
- generates an HTTParty::Parser instance with the given body and format
- calls #parse on the parser

HTTParty::Parser.formats
- returns the SupportedFormats constant
- returns the SupportedFormats constant for subclasses

HTTParty::Parser.format_from_mimetype
- returns a symbol representing the format mimetype
- returns nil when the mimetype is not supported

HTTParty::Parser.supported_formats
- returns a unique set of supported formats represented by symbols

HTTParty::Parser.supports_format?
- returns true for a supported format
- returns false for an unsupported format

HTTParty::Parser#parse
- attempts to parse supported formats
- returns the unparsed body when the format is unsupported
- returns nil for an empty body
- returns nil for a nil body

HTTParty::Parser#supports_format?
- utilizes the class method to determine if the format is supported

HTTParty::Parser#parse_supported_format
- calls the parser for the given format

HTTParty::Parser#parse_supported_format when a parsing method does not exist for the given format
- raises an exception
- raises a useful exception message for subclasses

HTTParty::Parser parsers
- parses xml with Crack
- parses json with Crack
- parses yaml
- parses html by simply returning the body
- parses plain text by simply returning the body

HTTParty::CookieHash#add_cookies with a hash
- should add new key/value pairs to the hash
- should overwrite any existing key

HTTParty::CookieHash#add_cookies with a string
- should add new key/value pairs to the hash
- should overwrite any existing key

HTTParty::CookieHash#add_cookies with other class
- should error

HTTParty::CookieHash#to_cookie_string
- should format the key/value pairs, delimited by semi-colons
- should not include client side only cookies

HTTParty AllowedFormats deprecated
- warns with a deprecation message
- returns HTTPart::Parser::SupportedFormats

HTTParty base uri
- should have reader
- should have writer
- should not modify the parameter during assignment

HTTParty#normalize_base_uri
- should add http if not present for non ssl requests
- should add https if not present for ssl requests
- should not remove https for ssl requests
- should not modify the parameter

HTTParty headers
- should default to empty hash
- should be able to be updated
- uses the class headers when sending a request
- overwrites class headers when passing in headers

HTTParty headers with cookies
- utilizes the class-level cookies
- adds cookies to the headers
- adds optional cookies to the optional headers

HTTParty cookies
- should not be in the headers by default
- should raise an ArgumentError if passed a non-Hash
- should allow a cookie to be specified with a one-off request

HTTParty cookies when a cookie is set at the class level
- should include that cookie in the request
- should pass the proper cookies when requested multiple times
- should allow the class defaults to be overridden

HTTParty cookies in a class with multiple methods that use different cookies
- should not allow cookies used in one method to carry over into other methods

HTTParty default params
- should default to empty hash
- should be able to be updated

HTTParty default timeout
- should default to nil
- should support updating

HTTParty debug_output
- stores the given stream as a default_option
- stores the $stderr stream by default

HTTParty basic http authentication
- should work

HTTParty digest http authentication
- should work

HTTParty parser
- should set parser options
- should be able parse response with custom parser
- raises UnsupportedFormat when the parser cannot handle the format
- does not validate format whe custom parser is a proc

HTTParty format
- should allow xml
- should allow json
- should allow yaml
- should allow plain
- should not allow funky format
- should only print each format once with an exception
- sets the default parser
- does not reset parser to the default parser

HTTParty#no_follow
- sets no_follow to false by default
- sets the no_follow option to true

HTTParty#maintain_method_across_redirects
- sets maintain_method_across_redirects to true by default
- sets the maintain_method_across_redirects option to false

HTTParty with explicit override of automatic redirect handling
- should fail with redirected GET
- should fail with redirected POST
- should fail with redirected DELETE
- should fail with redirected PUT
- should fail with redirected HEAD
- should fail with redirected OPTIONS

HTTParty with multiple class definitions
- should not run over each others options

HTTParty two child classes inheriting from one parent
- does not modify each others inherited attributes

HTTParty#get
- should be able to get html
- should be able parse response type json automatically (FAILED - 1)
- should be able parse response type xml automatically
- should not get undefined method add_node for nil class for the following xml
- should parse empty response fine
- should accept http URIs
- should accept https URIs
- should raise an ArgumentError on URIs that are not http or https
- should raise an InvalidURIError on URIs that can't be parsed at all

1)
Crack::ParseError in 'HTTParty#get should be able parse response type json automatically'
Invalid JSON string                                                                                                                
/home/phiggins/.rvm/gems/ruby-1.9.1-p378/gems/crack-0.1.7/lib/crack/json.rb:14:in `rescue in parse'
/home/phiggins/.rvm/gems/ruby-1.9.1-p378/gems/crack-0.1.7/lib/crack/json.rb:12:in `parse'
/home/phiggins/work/httparty/httparty/lib/httparty/parser.rb:116:in `json'
/home/phiggins/work/httparty/httparty/lib/httparty/parser.rb:136:in `parse_supported_format'
/home/phiggins/work/httparty/httparty/lib/httparty/parser.rb:103:in `parse'
/home/phiggins/work/httparty/httparty/lib/httparty/parser.rb:66:in `call'
/home/phiggins/work/httparty/httparty/lib/httparty/request.rb:171:in `parse_response'
/home/phiggins/work/httparty/httparty/lib/httparty/request.rb:166:in `handle_response'
/home/phiggins/work/httparty/httparty/lib/httparty/request.rb:59:in `perform'
/home/phiggins/work/httparty/httparty/lib/httparty.rb:278:in `perform_request'
/home/phiggins/work/httparty/httparty/lib/httparty.rb:230:in `get'
/home/phiggins/work/httparty/httparty/lib/httparty.rb:310:in `get'
/home/phiggins/work/httparty/httparty/spec/httparty_spec.rb:438:in `block (3 levels) in <top (required)>'
/home/phiggins/.rvm/gems/ruby-1.9.1-p378/gems/rspec-1.3.0/lib/spec/example/example_methods.rb:40:in `instance_eval'
/home/phiggins/.rvm/gems/ruby-1.9.1-p378/gems/rspec-1.3.0/lib/spec/example/example_methods.rb:40:in `block in execute'
/home/phiggins/.rvm/rubies/ruby-1.9.1-p378/lib/ruby/1.9.1/timeout.rb:44:in `timeout'
/home/phiggins/.rvm/gems/ruby-1.9.1-p378/gems/rspec-1.3.0/lib/spec/example/example_methods.rb:37:in `execute'
/home/phiggins/.rvm/gems/ruby-1.9.1-p378/gems/rspec-1.3.0/lib/spec/example/example_group_methods.rb:214:in `block in run_examples'
/home/phiggins/.rvm/gems/ruby-1.9.1-p378/gems/rspec-1.3.0/lib/spec/example/example_group_methods.rb:212:in `each'
/home/phiggins/.rvm/gems/ruby-1.9.1-p378/gems/rspec-1.3.0/lib/spec/example/example_group_methods.rb:212:in `run_examples'
/home/phiggins/.rvm/gems/ruby-1.9.1-p378/gems/rspec-1.3.0/lib/spec/example/example_group_methods.rb:103:in `run'
/home/phiggins/.rvm/gems/ruby-1.9.1-p378/gems/rspec-1.3.0/lib/spec/runner/example_group_runner.rb:23:in `block in run'
/home/phiggins/.rvm/gems/ruby-1.9.1-p378/gems/rspec-1.3.0/lib/spec/runner/example_group_runner.rb:22:in `each'
/home/phiggins/.rvm/gems/ruby-1.9.1-p378/gems/rspec-1.3.0/lib/spec/runner/example_group_runner.rb:22:in `run'
/home/phiggins/.rvm/gems/ruby-1.9.1-p378/gems/rspec-1.3.0/lib/spec/runner/options.rb:152:in `run_examples'
/home/phiggins/.rvm/gems/ruby-1.9.1-p378/gems/rspec-1.3.0/lib/spec/runner/command_line.rb:9:in `run'
/home/phiggins/.rvm/gems/ruby-1.9.1-p378/gems/rspec-1.3.0/bin/spec:5:in `<main>'

Finished in 0.373480374 seconds

157 examples, 1 failure
rake aborted!
Command /home/phiggins/.rvm/rubies/ruby-1.9.1-p378/bin/ruby -I"lib:lib:spec"  "/home/phiggins/.rvm/gems/ruby-1.9.1-p378/gems/rspec-1.3.0/bin/spec" "spec/httparty/request_spec.rb" "spec/httparty/response_spec.rb" "spec/httparty/parser_spec.rb" "spec/httparty/cookie_hash_spec.rb" "spec/httparty_spec.rb" --options spec/spec.opts failed

(See full trace by running task with --trace)

It'd be nice if default params did the same functionality for POST requests, ie add the default params to the body instead of the query_string.

class A
  include HTTParty
  headers 'referer' => 'http://mysite.com'
end

When inspect options within HTTParty::Request#perform_actual_request, the headers hash always contained :headers=>{"cookie"=>""}.
I've temporarily fixed this by overriding #perform_request and removing the call to #process_cookies

class A
  include HTTParty
  headers 'referer' => 'http://mysite.com'

  def self.perform_request(http_method, path, options) #:nodoc:
    # process_cookies(options)
    Request.new(http_method, path, default_options.dup.merge(options)).perform
  end
end

In lib/httparty.rb, there is no class method to specify a default timeout value for all calls. In lib/httparty/request.rb, the private method http looks for the :timeout key in options for setting http read timeout. Hence, using default_params class method for specifying default timeout does not work.

Currently, after including HTTParty in the class, we use

self.class.default_options[:timeout] = <some timeout value>

to set timeouts. But this seems more like a hack, as we are using instance variables defined by HTTParty. Is there a cleaner/better way out which we are missing?

Something similar to how twitter gem handles it would probably work.

I'm getting an intermittent REXML::ParseException when hitting the Technorati API, and it's very difficult to understand what's going on when you don't have access to the raw response body. There should be a way to get access to that when things go wrong.

Hi,

I've just tested to use the option digest_auth on two sites but it doesn't work.
Error is : "HTTP Digest: Access denied.\n"
I don't know why :(

Pierre

I have the following mapping for the attached xml:

```
module FamilySearch
class AlternateIds
include HappyMapper

end
```

Notice the AlternateIds class declares a has_many with type String, which should assign ids as a collection of Strings. However, ids ends up being a String of the first object.

I have a failing spec in my fork that represents this situation:
http://github.com/jimmyz/happymapper/commit/496365548e20a15707a46be9a756b1035301773e

Got following error tryingto follow included examples: NameError: uninitialized constant HTTParty::Parser.

Seems I can't get the examples to work: pp response throws and exception (from the twitter.rb example):

/ruby-1.8.7-p174/lib/ruby/1.8/pp.rb:258:in `method': 
  undefined method `inspect' for class `HTTParty::Response' (NameError)

It would be nice to have 'Accept-Encoding' => 'gzip, deflate' and have it deflate automatically. carsonmcdonald seems to have fixxed it, http://github.com/carsonmcdonald/httparty/commits/

When no_follow is set in the request params, a HTTParty::RedirectionTooDeep error will immediately result, even before the request is sent.

In the existing codebase, the xpath for non-primitive elements is from the downcased classname of the non-primitive class. This works well, but there are several cases where some more flexibility would be good.

For instance, consider this XML:

```

```

There is an obvious problem with this xml: the q1, q2, q3, q4 elements should be ‘quarter’ elements, with an attribute containing the quarter number. However, we usually don’t have control of the format of external xml, so we just need to work with it. Anyway, we might map these with the following classes:

```
class Quarter
include HappyMapper

end

class Details
include HappyMapper

end

class Game
include HappyMapper

end
```

The problem is that the elements q1, q2, q3 and q4 will not be able to be found using the current HappyMapper implementation, since the xpath used will be ‘quarter’, which will find game/details/quarter in each case. Adding :tag => ‘q1’ etc will not help, since the tag name is derived from the class name.

We can fix this and allow mapping classes to be resuable. Instead of using the classname as the default tag name, we can use the following finders, in order:
1. specified tag (e.g. :tag => ‘q1’)
2. name of element (e.g. from has_one :q1)
3. tag_name (derived from class name by default)

Using this method, the Game class changes to:

```
class Game
include HappyMapper

end
```

This is a bit of a change, but the additional flexibility and reusability has worked really well for me in a variety of situations. In the existing specs, I only had to make one change for everything to pass:

```
class Radar
include HappyMapper

1. OLD: has_many :places, Place
2. NEW:
   has_many :places, Place, :tag => :place
   end
   ```

I’ve made the changes needed in my fork (http://github.com/lightningdb/happymapper/tree/master, commits ce59623df1ce0b16bbbee89e6fac2b2473756a43 and 3251509d1e68fb652227a42b1e5593ee8bc19d5b), and updated the gemspec version to 0.3.0 to reflect that the change might need minor changes to existing mappings. I think the flexibility is worth it though.

Any thoughts or questions?