copiousfreetime / stickler Goto Github PK

a tool to organize and maintain an internal gem distribution server

License: MIT License

Ruby 88.24% CSS 9.99% HTML 1.76%

stickler's Issues

(Incompatibility with) Rack 1.3.1

Stickler doesn't work with rack 1.3.1 & ruby 1.9.2, the gem index fails to build due to somewhere in the middleware stack calling .each on a string. This means stickler appears to work (can be pushed to, viewed in browser) but will fail to work as a bundler source. I don't have time to track this down right now, but it works fine with rack 1.2.3, so hopefully this will help others with this issue.

Daemonize does not work in v2.4.0

Starting stickler-server with '--daemonize' does not actually start the server

% stickler-server start --daemonize /tmp/stickler-test

Right now, we're resorting to using nohup or screen to get around this.

413 Request Entity Too Large error

When pushing a gem, for example webrat-0.7.3.gem, I receive a 413 HTTP error. Am I setting something up wrong somewhere?

$ stickler push webrat-0.7.3.gem 
Pushing gem(s) to http://gems.site.net/ ...
  /Users/justinsmestad/Documents/api/vendor/cache/webrat-0.7.3.gem -> ERROR: Failure pushing /Users/justinsmestad/Documents/api/vendor/cache/webrat-0.7.3.gem to remote repository : response code => 413, response message =>     '<html>
<head><title>413 Request Entity Too Large</title></head> 
<body bgcolor="white">
<center><h1>413 Request Entity Too Large</h1></center>
<hr><center>nginx/1.0.10</center>
</body>
</html>
'

Relicense stickler for MIT

Make stickler MIT so its easy for https://github.com/bundler/gemstash to incorporate. See rubygems/gemstash#4

Known contributors - checked means approved

Add serverside code hooks based upon operation.

from @jsmestead.

Add the facility so that when someone pushes/yanks a gem from stickler you can have a script/class executed on the server. Also webhooks.

Add in an 'unmirror' or similar command for removing a gem from the mirror repo

Either use the 'DELETE' or similar method to remove a gem and its spec from the mirror repo.

when mirroring or pushing a gem, have stickler pull in the dependencies automatically for mirroring

Running stickler command-line against ruby-1.9.3-p327 fails with NoMethodError

This worked just fine with the previous version of stickler (2.2.4). It looks like you're rescuing with a LoadError, but it's a NoMethodError that's being raised instead. Am I right to assume you want this to still work with version 1.9.3 for a while?

kinsersh@kinsersh-VirtualBox:~/sandbox/blueprint-deploy (inline-acceptance-test-mgr *)$ stickler push pkg/blueprint-deploy-0.0.409.gem --server http://myserver:6789
Pushing gem(s) to http://myserver:6789/ ...
/home/kinsersh/sandbox/blueprint-deploy/pkg/blueprint-deploy-0.0.409.gem -> /home/kinsersh/.rvm/gems/ruby-1.9.3-p327/gems/stickler-2.4.0/lib/stickler/gem_container.rb:26:in load_container': undefined methodnew' for Gem::Package:Module (NoMethodError)
from /home/kinsersh/.rvm/gems/ruby-1.9.3-p327/gems/stickler-2.4.0/lib/stickler/gem_container.rb:12:in initialize' from /home/kinsersh/.rvm/gems/ruby-1.9.3-p327/gems/stickler-2.4.0/lib/stickler/repository/remote.rb:279:innew'
from /home/kinsersh/.rvm/gems/ruby-1.9.3-p327/gems/stickler-2.4.0/lib/stickler/repository/remote.rb:279:in specification_from_gem_file' from /home/kinsersh/.rvm/gems/ruby-1.9.3-p327/gems/stickler-2.4.0/lib/stickler/repository/remote.rb:275:inspeclite_from_gem_file'
from /home/kinsersh/.rvm/gems/ruby-1.9.3-p327/gems/stickler-2.4.0/lib/stickler/repository/remote.rb:82:in push' from /home/kinsersh/.rvm/gems/ruby-1.9.3-p327/gems/stickler-2.4.0/lib/stickler/client/push.rb:39:inblock in run'
from /home/kinsersh/.rvm/gems/ruby-1.9.3-p327/gems/stickler-2.4.0/lib/stickler/client/push.rb:35:in each' from /home/kinsersh/.rvm/gems/ruby-1.9.3-p327/gems/stickler-2.4.0/lib/stickler/client/push.rb:35:inrun'
from /home/kinsersh/.rvm/gems/ruby-1.9.3-p327/gems/stickler-2.4.0/bin/stickler:47:in <top (required)>' from /home/kinsersh/.rvm/gems/ruby-1.9.3-p327/bin/stickler:19:inload'
from /home/kinsersh/.rvm/gems/ruby-1.9.3-p327/bin/stickler:19:in <main>' from /home/kinsersh/.rvm/gems/ruby-1.9.3-p327/bin/ruby_noexec_wrapper:14:ineval'
from /home/kinsersh/.rvm/gems/ruby-1.9.3-p327/bin/ruby_noexec_wrapper:14:in `

kinsersh@kinsersh-VirtualBox:~/sandbox/blueprint-deploy (inline-acceptance-test-mgr *)$ ruby -e 'puts RUBY_DESCRIPTION'
ruby 1.9.3p327 (2012-11-10 revision 37606) [x86_64-linux]

kinsersh@kinsersh-VirtualBox:~/sandbox/blueprint-deploy (inline-acceptance-test-mgr *)$ cat /etc/issue
Ubuntu 13.04 \n \l

kinsersh@kinsersh-VirtualBox:~/sandbox/blueprint-deploy (inline-acceptance-test-mgr *)$ ruby -rubygems -e "require 'stickler'; puts Stickler::VERSION"
2.4.0

stickler error under Windows Server 2008

When attempting to initialize a stickler repository on a machine running Windows Server 2008 R2 Standard and Ruby 1.9.2, the following error occurs:

C:/Ruby192/lib/ruby/gems/1.9.1/gems/sinatra-1.0/lib/sinatra/base.rb:753:in `read': Invalid argument - <internal:lib/rubygems/custom_require> (Errno::EINVAL)
    from C:/Ruby192/lib/ruby/gems/1.9.1/gems/sinatra-1.0/lib/sinatra/base.rb:753:in `inline_templates='
    from C:/Ruby192/lib/ruby/gems/1.9.1/gems/sinatra-1.0/lib/sinatra/base.rb:706:in `set'
    from C:/Ruby192/lib/ruby/gems/1.9.1/gems/sinatra-1.0/lib/sinatra/base.rb:715:in `block in enable'
    from C:/Ruby192/lib/ruby/gems/1.9.1/gems/sinatra-1.0/lib/sinatra/base.rb:715:in `each'
    from C:/Ruby192/lib/ruby/gems/1.9.1/gems/sinatra-1.0/lib/sinatra/base.rb:715:in `enable'
    from (__DELEGATE__):2:in `enable'
    from C:/Ruby192/lib/ruby/gems/1.9.1/gems/sinatra-1.0/lib/sinatra.rb:7:in `<top (required)>'
    from <internal:lib/rubygems/custom_require>:29:in `require'
    from <internal:lib/rubygems/custom_require>:29:in `require'
    from C:/Ruby192/lib/ruby/gems/1.9.1/gems/stickler-2.0.2/lib/stickler/middleware/local.rb:1:in `<top (required)>'
    from <internal:lib/rubygems/custom_require>:29:in `require'
    from <internal:lib/rubygems/custom_require>:29:in `require'
    from C:/Ruby192/lib/ruby/gems/1.9.1/gems/stickler-2.0.2/lib/stickler/middleware/gemcutter.rb:3:in `<top (required)>'
    from <internal:lib/rubygems/custom_require>:29:in `require'
    from <internal:lib/rubygems/custom_require>:29:in `require'
    from C:/Ruby192/lib/ruby/gems/1.9.1/gems/stickler-2.0.2/lib/stickler/server.rb:3:in `<top (required)>'
    from <internal:lib/rubygems/custom_require>:29:in `require'
    from <internal:lib/rubygems/custom_require>:29:in `require'
    from C:/Ruby192/lib/ruby/gems/1.9.1/gems/stickler-2.0.2/lib/stickler.rb:10:in `<top (required)>'
    from <internal:lib/rubygems/custom_require>:29:in `require'
    from <internal:lib/rubygems/custom_require>:29:in `require'
    from C:/Ruby192/lib/ruby/gems/1.9.1/gems/stickler-2.0.2/bin/stickler:3:in `<top (required)>'
    from C:/Ruby192/bin/stickler:19:in `load'
    from C:/Ruby192/bin/stickler:19:in `<main>'

Reproduce with:

gem install stickler
cd c:\
mkdir stickler

Then:
stickler

or
stickler help

Or any other stickler command, actually. :)

[EDIT: Realized everything was bombing, not just repository setup.]

stickler push fails to push the gems

I am trying to get my head around why we are unable to push gems to our gem repository running using stickler.

It looks like as soon I update Excon to 0.50.1, I start running into this issue, and it works fine with Excon 0.49.0.

Error that i get:

/opt/mydir/ruby-2.2.2/gems/excon-0.50.1/lib/excon/middlewares/expects.rb:6:in response_call': Expected([200]) <=> Actual(404 Not Found) (Excon::Error::NotFound) from /opt/mydir/ruby-2.2.2/gems/excon-0.50.1/lib/excon/middlewares/response_parser.rb:8:inresponse_call'
from /opt/mydir/ruby-2.2.2/gems/excon-0.50.1/lib/excon/connection.rb:389:in response' from /opt/mydir/ruby-2.2.2/gems/excon-0.50.1/lib/excon/connection.rb:253:inrequest'
from /opt/mydir/ruby-2.2.2/gems/stickler-2.4.2/lib/stickler/repository/remote.rb:296:in resource_request' from /opt/mydir/ruby-2.2.2/gems/stickler-2.4.2/lib/stickler/repository/remote.rb:268:inremote_uri_exist?'
from /opt/mydir/ruby-2.2.2/gems/stickler-2.4.2/lib/stickler/repository/remote.rb:264:in remote_gem_file_exist?' from /opt/mydir/ruby-2.2.2/gems/stickler-2.4.2/lib/stickler/repository/remote.rb:83:inpush'
from /opt/mydir/ruby-2.2.2/gems/stickler-2.4.2/lib/stickler/client/push.rb:39:in block in run' from /opt/mydir/ruby-2.2.2/gems/stickler-2.4.2/lib/stickler/client/push.rb:35:ineach'
from /opt/mydir/ruby-2.2.2/gems/stickler-2.4.2/lib/stickler/client/push.rb:35:in run' from /opt/mydir/ruby-2.2.2/gems/stickler-2.4.2/bin/stickler:47:in<top (required)>'
from /opt/mydir/ruby-2.2.2/bin/stickler:23:in load' from /opt/mydir/ruby-2.2.2/bin/stickler:23:in

Use 'version' sort for web view

Version list in web view appears to be lexicographic when it should be version/numeric

Use error exit codes when a command fails/errors

We noticed that when we were using stickler to push gems that even when there is an error, the exit code is 0, after rescuing the exception, we would like to exit with an error code of 1.

gem specification warnings on 2.1.3 gemspec

$ gem install stickler
Invalid gemspec in [/Users/nathan/.rvm/gems/ruby-1.8.7-p334/specifications/stickler-2.1.3.gemspec]: invalid date format in specification: "2011-07-24 00:00:00.000000000Z"
$ gem env
RubyGems Environment:
  - RUBYGEMS VERSION: 1.8.6
  - RUBY VERSION: 1.8.7 (2011-02-18 patchlevel 334) [i686-darwin10.6.0]
  - INSTALLATION DIRECTORY: /Users/nathan/.rvm/gems/ruby-1.8.7-p334
  - RUBY EXECUTABLE: /Users/nathan/.rvm/rubies/ruby-1.8.7-p334/bin/ruby
  - EXECUTABLE DIRECTORY: /Users/nathan/.rvm/gems/ruby-1.8.7-p334/bin
  - RUBYGEMS PLATFORMS:
    - ruby
    - x86-darwin-10
  - GEM PATHS:
     - /Users/nathan/.rvm/gems/ruby-1.8.7-p334
     - /Users/nathan/.rvm/gems/ruby-1.8.7-p334@global
  - GEM CONFIGURATION:
     - :update_sources => true
     - :verbose => true
     - :benchmark => false
     - :backtrace => false
     - :bulk_threshold => 1000
     - "gem" => "--no-ri --no-rdoc --no-user-install"
  - REMOTE SOURCES:
     - http://rubygems.org/

Allow a bundler Gemfile to be used as input for 'mirror' command

Instead of explicitly pulling in the gems necessary automatically, add in support of mirroring via Gemfile input.

implement the gemcutter dependency api

2.4.0

How can I install 2.4.0 of this gem by source? There is no gemspec to install it from source.

Need to be able to specify the context path for stickler-server

We have a haproxy instance that does url re-writing, but stickler server is hard-coded to fetch gems off of /gems (i.e. sticklerserver:6789/gems). We want to be able to specify the context path as an option to be able to do this: http://sticklerserver:6789/[some_context_path]/gems and be able to view the website off of http://sticklerserver:6789/[some_context_path]

This is useful for organizations that use haproxy with url rewriting.

Authentication support

All operations should have the ability to be authenticated against some system.

Add support for Ruby 2.0

.bundle/gems/ruby/2.0.0/gems/stickler-2.3.0/lib/stickler/repository/local.rb:5:in `require': cannot load such file -- rubygems/format (LoadError)
from .bundle/gems/ruby/2.0.0/gems/stickler-2.3.0/lib/stickler/repository/local.rb:5:in `<top (required)>'
from .bundle/gems/ruby/2.0.0/gems/stickler-2.3.0/lib/stickler/repository.rb:9:in `require'
from .bundle/gems/ruby/2.0.0/gems/stickler-2.3.0/lib/stickler/repository.rb:9:in `<top (required)>'
from .bundle/gems/ruby/2.0.0/gems/stickler-2.3.0/lib/stickler.rb:12:in `require'
from .bundle/gems/ruby/2.0.0/gems/stickler-2.3.0/lib/stickler.rb:12:in `<top (required)>'

Stickler server returns all gemspecs on the server

I've been running stickler to manage the gems of the company I work for and as we started to have more an more gems added to stickler we saw the gem install command getting very slow. Running bundle install --vendor, I can see that the stickler returns all the gemspec in the server, and that's why it task a long time to finish fetching the gems because at every install we are passing on all the gems metadata in the server.
Is is a configuration problem? Thanks.

stickler mirror does not work

When I do a:

stickler mirror --server http://localhost:6789/ --gem-version 3.0.3 rails

I get this:

ERROR -> can't modify frozen string

This is a bug in the spec_lite.rb, line 20:

@version = Gem::Version.new( version )

This apparently causes my RubyGems 1.3.7 running on Ruby 1.9.2 to fail with a "can't modify frozen string" error. There are 2 ways I fixed it (both strange and hacky):

@version = version #Gem::Version.new( version )

OR run stickler like:

stickler mirror --server http://localhost:6789/ --gem-version=3.0.3 rails

And after these hacks, I further ran into this issue:

undefined method 'first' for "http://production.cf.rubygems.org/gems/rails-3.0.3.gem":String

Resourceful is buggy in that it tries to follow redirects and assumes that the response "Location" header is always an array. I suggest that Stickler would ditch the use of Resourceful gem (use CURL or Net::Http).

Expose the 'delete' command

Internally a DELETE method for a gem will delete it from the system, add this functionality tot he client

Add support for X-Sendfile or similar capability for serving up the .gem and .spec files.

Additional Gem API from gemcutter to consider

http://guides.rubygems.org/rubygems-org-api/

GET - /api/v1/versions/[GEM NAME].(json|xml|yaml)
GET - /api/v1/downloads/top.(json|xml|yaml)
GET - /api/v1/downloads.(json|xml|yaml)
GET - /api/v1/downloads/all.(json|xml|yaml)
GET - /api/v1/downloads/[GEM NAME]-[GEM VERSION].(json|xml|yaml)
GET - /api/v1/versions/[GEM NAME]-[GEM VERSION]/downloads.(json|yaml)
GET - /api/v1/versions/[GEM NAME]-[GEM VERSION]/downloads/search.(json|yaml)?from=[START DATE]&to=[END DATE]

stickler on windows issues

I'm not sure if stickler is supposed to run on windows or not. It comes with .cmd wrappers for the stickler and stickler-server commands, but that could just be auto-gen code.

Anyway, after patching some binmodes into local.rb
tempfile = Tempfile.new( "uploaded-gem.", temp_dir )
tempfile.binmode
tempfile.write( io.read )

and

def install_gem( spec, io )
  File.open( full_path_to_gem( spec ) , "w+" ) do |of|
    of.binmode
    io.each do |str|
      of.write( str )
    end
  end
end

and "deactivating" syslog in the logable.rb it seems to work.

But the question is, should it...?

Add in mirroring proxy.

Add in an ability for a the mirror middleware to also do intermediate proxying when a gem is requested. Just pass through the request and the put the gem that was downloaded into the mirror repo.

latest_specs did not show all platforms for a gem

installing a gem using latest_specs index sould result in the wrong gem getting installed. This is because in latest_specs it should have the latest for each platform available.

Mirroring Gemfile.lock always skips java gems.

As an example, one of the entries in my Gemfile.lock is

json (1.8.3-java)

I can mirror this individually using --gem-version and --platform, but this gem (and any other -java gems) will always be ignored if I try to mirror the whole Gemfile.lock. Passing --platform=java does not seem to affect anything.

check on 'uninitalised constant in initialize. Gem::RemoteFetcher (NameError)'

On Windows :

When trying 'stickler add gem' from the wrong directory (the
repositories parent say) I got an error 'uninitalised constant in
initialize. Gem::RemoteFetcher (NameError)'; when running the command
from the repository directory nothing was displayed but also nothing
happened.

I needed to add "require 'rubygems/remote_fetcher'" to the Source_group.rb file and then everything worked ok (from the repository directory).

yank and unyank commands do not work for platform gems

Steps to reproduce:

stickler yank baz --platform java --gem-version 3.1.4
stickler unyank baz --platform java --gem-version 3.1.4

Extract out all the functionality of Sticker::Server into its own Middleware

In order to help with embedding stickler in other rack based applications, extract out the core stickler repository gem api into its own configurable middleware. /c @jbarnette

detect and ignore duplicates

Uploading a gem directory twice will create duplicates in the web ui. Is it simple enough to ignore duplicate versions?

Add a Label to the gems to show which repository that gem is in local/mirror etc.

specify the required_rubygems_version

As seen in #35 a minimum rubygems version is now required. Make sure to set this.

Update depenedencies to fix fattr/main issue

Mirror functionality should be able to use a proxy to get to rubygems

At the moment it looks like stickler mirror does not work with an http proxy. For some folks behind firewalls, this is an issue and needs to be addressed.

Mirror errors when no gem-version supplied

Using ruby 2.1.5p273 (2014-11-13 revision 48405) [x86_64-darwin14.0] and Stickler version 2.4.1 on Mac OS X 10.10.1:

$ stickler mirror deep_merge --server http://my.mirror
/Users/michaelsmith/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/stickler-2.4.1/lib/stickler/client/mirror.rb:37:in `block in parse'
/Users/michaelsmith/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/stickler-2.4.1/lib/stickler/client.rb:33:in `block in parse'
/Users/michaelsmith/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/trollop-2.0/lib/trollop.rb:740:in `with_standard_exception_handling'
/Users/michaelsmith/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/stickler-2.4.1/lib/stickler/client.rb:31:in `parse'
/Users/michaelsmith/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/stickler-2.4.1/lib/stickler/client/mirror.rb:31:in `parse'
/Users/michaelsmith/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/stickler-2.4.1/lib/stickler/client/mirror.rb:80:in `run'
/Users/michaelsmith/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/stickler-2.4.1/bin/stickler:55:in `<top (required)>'
/Users/michaelsmith/.rbenv/versions/2.1.5/bin/stickler:23:in `load'
/Users/michaelsmith/.rbenv/versions/2.1.5/bin/stickler:23:in `<main>'
ERROR -> undefined local variable or method `lock' for #<Stickler::Client::Mirror:0x007f9022491588>

This appears to be because the lock variable doesn't exist, and should be gemfile_lock.

Stickler push reported file non-readable when the file actually didn't exist

When I tried to sticker push my gem with a typo in the gem file name, I got "[file name] must be readable". It took me sometime to figure out in fact the file name actually didn't exist.
It would be nice if the error message reflects the issue more accurately such as "[file name] doesn't exists."

Add 'unyank' support

add an 'unyank' command to put the .spec back in place if it has been yanked.

copiousfreetime / stickler Goto Github PK

stickler's Issues

Known contributors - checked means approved

Recommend Projects

Recommend Topics

Recommend Org