abrader / abrader-gms Goto Github PK

View Code? Open in Web Editor NEW

26.0 26.0 32.0 294 KB

Git management system types and providers for Puppet utilizing Github, Gitlab, Stash APIs

License: Apache License 2.0

Ruby 100.00%

abrader-gms's People

Stargazers

Watchers

abrader-gms's Issues

Repo needs a README

perhaps still some code examples from http://github.com/acidprime/r10k

Is it possible to modify Gitlab Settings-> Outbound Requests -> Allow requests to the local network from hooks and services?

Allow requests to the local network from hooks and services is now disabled by default and will cause all custom hooks to fail inside the private network:

https://docs.gitlab.com/ee/security/webhooks.html

Is it possible to modify this setting programatically so that poor souls like me don't spend hours search for why their webhooks suddenly break?

Webhook for merges as well as pushes?

Is there a way to modify the API call to create a webhook entry that fires on merge events as well as push events?

I'd be willing to submit a pull request if it's possible and not implemented, but I'm not quite sure where to look - your provider, the r10k module webhook itself, etc. Is this something that you've ever investigated?

Add parameter for disabling SSL verification

For r10k users who use the ssl webhook , they will need to disable ssl verification on github to make use of this t&p when using their control repository

I think this is done with the config blob
https://developer.github.com/v3/repos/hooks/#edit-a-hook

Gitlab only allows a single SSH key to be used once on a given repo as a Deploy Key

Per a disussion with @natemccurdy I need to review an issue he encountered when the git_deploy_key resource was used with GitLab and the title varied but key was the same. Error received was:

{"message":{"fingerprint":["has already been taken"]}

unexpected token

I'm getting unexpected token and I don't know what's wrong. I did everything as documented but doesn't seem to help. Could you please assist? Full errror below.

Error: /Stage[main]/Profile::Puppet::R10k/Git_webhook[github_post_receive]: Could not evaluate: 751: unexpected token at '<html><body>You are being <a href="https://github.com/login">redirected</a>.</body></html>'

  class { 'r10k::webhook::config':
    enable_ssl      => false,
    protected       => false,
    use_mcollective => false,
  }

  class { 'r10k::webhook':
    use_mcollective => false,
    user            => 'root',
    group           => 'root',
    require         => Class['r10k::webhook::config'],
  }

  git_webhook { 'github-nz_post_receive':
    ensure             => present,
    webhook_url        => "https://${::fqdn}:8088/payload",
    token              => 'TOKEN',
    project_name       => 'myuser/myrepo',
    server_url         => 'https://github.com',
    disable_ssl_verify => true,
    provider           => 'github',
  }

Add Gerrit support

There's a few customers using Gerrit, especially if they're using openstack, as it's what OS uses for codereview.

This should be the ticket to create the Key:
https://gerrit-review.googlesource.com/Documentation/rest-api-accounts.html#add-ssh-key

Not sure about the hooks

Errors with GitLab 8.15

After upgrading GitLab to 8.15, I'm now getting this error from the git_deploy_key and git_webhook calls in pe_code_manager_webhook::code_manager:

Could not evaluate: no implicit conversion of String into Integer

I've checked, and my API key hasn't changed, so it seems like possibly some API change is breaking things?

Puppet.newtype is deprecated

In PE 2015.2.0 and puppet 4.2.1, I'm getting the following deprecation warning when using the git_deploy_key type.

Stash: external-hooks - 'safe_path' needs to be set

When using Stash (Bitbucket Server) and the External-Hooks add-on, the git_webhook type is unable to modify (create/change) the parameters for the external-hooks add-on because the safe_path attribute needs to be set when hitting the API.

-X PUT -d '{"exe":"debug.sh", "params":"-a foo", "safe_path":true}'

Specifically, if using a Stash user account that only has admin rights to a project/repo, then the exe executable must be in $STASH_HOME/external-hooks, and the path must be relative to that directory. When doing that, safe_path must be set to true or else the API will not allow the change.

If using a Stash user account that has global admin rights (like the default 'admin' user), then safe_path doesn't need to be managed and the exe does not have to be in the $STASH_HOME/external-hooks directory. In which case an absolute path to the exe should be used.

Wrong rights on lib/puppet/provider/gms_webhook/github.rb

Hey,

Got this error after upgrading to 1.0.2 :

Error: /File[/opt/puppetlabs/puppet/cache/lib]: Failed to generate additional resources using 'eval_generate': Error 400 on SERVER: Permission denied - /etc/puppetlabs/code/environments/production/modules/gms/lib/puppet/provider/gms_webhook/github.rb

Adding missing read right on others solves it :

chmod o+r /etc/puppetlabs/code/environments/production/modules/gms/lib/puppet/provider/gms_webhook/github.rb

Cheers!

Update documentation for GitLab user minimum permissons

In order for the GitLab token to be able to create deploy keys, the associated user must be a 'master' or 'owner' on the GitLab server.

See: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/permissions/permissions.md#permissions

[Feature] Support for stash in git_webhook

Any plans to include support for Stash?

http URL validation is too restrictive

The regex here:

https://github.com/abrader/abrader-gms/blob/master/lib/puppet/type/git_webhook.rb#L36

will fail if someone uses http://user:password@server:8080/payload

I am wondering is something more like

^(https?:\/\/).*:?.*\/?$ would still work but not be as finicky with possible permutations

Document that at least the gitlab provider creates new webhooks when the url changes

As far as I can tell the namevar for gitlab is the url, so when I change the url I end up with two webhooks. There's no way around that since gitlab doesn't allow naming the webhook but it should be documented that this is the behavior.

Also, if we allowed purging un-managed webhooks then that would make for a good example to avoid orphaned webhooks.

missing tag 1.0.3

I'm having a hard time debugging an issue with gms, and it's a little more confusing because there's no 1.0.3 tag. It appears that 1.0.3 is (presently) the master/HEAD as per metadata.json.

Could you please tag the appropriate commit with 1.0.3?

thanks

I should not be required to use a fqdn for my gms server, I would like to use the short name

abrader-gms/lib/puppet/type/git_deploy_key.rb

Lines 49 to 50 in fd47b54

 unless value =~ /^(https?:\/\/)?([\da-z\.-]+)\.([a-z\.]{2,6})([\/\w \.-]*)*\/?$/ 

 raise(Puppet::Error, "Git server URL must be fully qualified, not '#{value}'")

Compile error - no such file to load puppet_x/gms/type

Having a difficult time compiling a catalog on puppetserver 2.4.0 (FOSS). I'm sure that gms 1.0.3 worked previously, because I upgraded to fix the minor permissions issue in 1.0.2.

A puppet run throws an error compiling the catalog containing gms:

Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Evaluation Error: Error while evaluating a Resource Statement, Could not autoload puppet/type/git_deploy_key: no such file to load -- puppet_x/gms/type at /etc/puppetlabs/code/environments/qa/site/profile/manifests/puppetmaster.pp:29:5 on node <puppetmaster>

The reference to line 29 is this block in the puppetmaster profile:

    git_deploy_key { 'add_deploy_key_to_puppet_control':
      provider      => 'gitlab',
      ensure        => present,
      name          => "r10k-$::fqdn",
      path          => '/etc/puppetlabs/puppet/r10k-id_rsa.pub',
      token         => hiera('gitlab_api_token'),
      project_name  => 'puppet/control-repo',
      server_url    => 'http://gitlab.mycorp.com',
    }

From looking at the puppetserver logs, it looks like the issue is the require line in gms/lib/puppet/type/git_deploy_key.rb

The top of the stacktrace points here

2016-07-07 10:27:26,907 ERROR [qtp803704776-67] [puppetserver] Puppet Could not autoload puppet/type/git_deploy_key: no such file to load -- puppet_x/gms/type
org/jruby/RubyKernel.java:1072:in `require'
/opt/puppetlabs/server/apps/puppetserver/puppet-server-release.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:54:in `require'
/etc/puppetlabs/code/environments/lab/modules/gms/lib/puppet/type/git_deploy_key.rb:1:in `(root)'
...

Just in case I messed something up, I wiped gms from the modules directory and wiped r10k's cache (and redeploying the env), and also tried unpacking the module from a github-downloaded zip file into the modules directory. Same result.

It seems as if maybe there's a ruby search path issue here, but I'm not advanced enough yet to understand how to fix it.

Access token requirements should be documented

I think it would helpful to have a quick note for at least github that you can create access tokens with limited abilities for security to accomplish only webhooks

see this screen shot

git_deploy_key with gitlab will not work when project is on a sub-subgroup

When project is on a sub-subgroup the key deployment not working.
i think the second / is not htmlized
revelant part of :

puppet agent -t -d

opening connection to <my_gitlab_server>:443...
opened
starting SSL for <my_gitlab_server>:443...
SSL established
<- "GET /api/v3/projects/infra-projects%2Fhubapplicatif/web HTTP/1.1\r\nAccept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3\r\nAccept: */*\r\nUser-Agent: Ruby\r\nContent-Type: application/json\r\nPrivate-Token: <my_private_token>\r\nConnection: close\r\nHost: <my_gitlab_server>\r\n\r\n"
-> "HTTP/1.1 404 Not Found\r\n"
-> "Server: nginx\r\n"
-> "Date: Wed, 20 Sep 2017 12:38:53 GMT\r\n"
-> "Content-Type: application/json\r\n"
-> "Content-Length: 25\r\n"
-> "Connection: close\r\n"
-> "Cache-Control: no-cache\r\n"
-> "Vary: Origin\r\n"
-> "X-Request-Id: b540fa1c-afcf-4b0b-ab95-be856e807bef\r\n"
-> "X-Runtime: 0.015513\r\n"
-> "\r\n"
reading 25 bytes...
-> "{\"error\":\"404 Not Found\"}"
read 25 bytes
Conn close

in /api/v3/projects/infra-projects %2F hubapplicatif / web the second / between hubapplicatif and web must be %2F

add puppet 6 support

The metadata.json file limits this module to only working with versions prior to puppet 6. Please add support for puppet 6.

git_webhook doesn't change hook after it's been created

On GitLab (Github untested), the git_webhook resource doesn't modify the webhook if it already exists:

git_webhook { 'web_post_receive_webhook' :
ensure => present,
webhook_url => 'http://puppetserver.example.com:8088/payload',
token => hiera('gitlab_api_token'),
project_name => 'puppet/control-repo',
server_url => 'http://gitlab.example.com',
provider => 'gitlab',
disable_ssl_verify => false,
}

This successfully creates the webhook, but if you change:
disable_ssl_verify => true
the webhook on Gitlab doesn't get updated.

Add disable_ssl_verify to the gitlab provider

It appears that gitlab added this as a feature of webhooks recently

undefined local variable or method `get_token'

# puppet --version
4.8.2

When trying to create a git_webhook resource, get an error:

# puppet apply githook.pp
Notice: Compiled catalog for hub.chepkov.lan in environment production in 0.31 seconds
Error: /Stage[main]/Main/Git_webhook[web_post_receive_webhook]: Could not evaluate: undefined local variable or method `get_token' for Git_webhook[web_post_receive_webhook](provider=github):Puppet::Type::Git_webhook::ProviderGithub
Notice: Applied catalog in 0.22 seconds

Gitlab API v3 is no longer supported

OS: Centos 7.5.1804
Module version: v1.0.3
Puppet: v5.5.1
Gitlab: gitlab-ee-11.0.0-ee.0.el7.x86_64

Debug output from failed puppet run.

opening connection to gitlab.company.local:443...
opened
starting SSL for gitlab.company.local:443...
SSL established
<- "GET /api/v3/projects/0/keys HTTP/1.1\r\nAccept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3\r\nAccept: */*\r\nUser-Agent: Ruby\r\nContent-Type: application/json\r\nPrivate-Token: PjbcSe-D_zxHynDQGNzc\r\nConnection: close\r\nHost: gitlab.company.local\r\n\r\n" 
-> "HTTP/1.1 410 Gone\r\n" 
-> "Server: nginx\r\n" 
-> "Date: Tue, 26 Jun 2018 04:41:02 GMT\r\n" 
-> "Content-Type: application/json\r\n" 
-> "Content-Length: 62\r\n" 
-> "Connection: close\r\n" 
-> "Cache-Control: no-cache\r\n" 
-> "Vary: Origin\r\n" 
-> "X-Request-Id: 50890cbc-5be0-4a1f-9003-e7ad1357268c\r\n" 
-> "X-Runtime: 0.008501\r\n" 
-> "\r\n" 
reading 62 bytes...
-> "{\"error\":\"API V3 is no longer supported. Use API V4 instead.\"}" <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
read 62 bytes
Conn close
Debug: gitlab_deploy_key::api_call: REST API GET Response: #<Net::HTTPGone 410 Gone readbody=true>

Gitlab only accepts a deploy key to one project

If you have one master that will be accessing two Gitlab projects (e.g. puppet_control and puppet_hiera), Gitlab will only let you create a deploy key to one of the two projects. The other project gets (in the UI) a section called: "Deploy keys from projects you have access to" that shows the server key that was added to the first project and has a button to click to enable it on the second as well.

If attempting to have matching deploy key resources to both projects, the following error occurs:

Error: gitlab_deploy_key::create: gitlab_deploy_key::create: #<Net::HTTPBadRequest 400 Bad Request readbody=true>
Error: /Stage[main]/Profile::Pe::Master/Git_deploy_key[add_deploy_key_to_hieradata]/ensure: change from absent to present failed: gitlab_deploy_key::create: gitlab_deploy_key::create: #<Net::HTTPBadRequest 400 Bad Request readbody=true>

The simple workaround is to go into the UI and click the button, but I wanted to raise it in case someone knew an easy way to extend the provider to handle this case.

Please cut a forge release

Now there is a README, please cut a forge release.

stash provider for git_deploy_key won't create keys if none exist

Adding any public key to the repo and the provider works just fine. This is for the stash provider.

It appears that nil/false handling is a little weird in the exists function? It looks like get_key_id returns false, which isn't nil so creating calling create never happens.

I can provide additional information, or logging if necessary.
Thank you for your work on this module.

Getting "can't convert String into Integer" errors on git_webhook and git_deploy_key

Trying to automate r10k-related SSH keys and webhooks in our PE 3.7/3.8 environment that has control repos on GitHub Enterprise. Both git_deploy_key and git_webhook return "Could not evaluate: can't convert String into Integer" with the following code (anonymized):

git_deploy_key { 'r10k_add_deploy_key_to_puppet_control_repo':
  ensure       => present,
  name         => $::fqdn,
  path         => '/root/.ssh/id_rsa.pub',
  token        => '65ba6770857c9f29d76ba5149da2c697fb39827e',
  project_name => 'Puppetdev/control',
  server_url   => 'https://github.example.net/api/v3',
  provider     => 'github',
}
git_webhook { 'r10k_post_receive_webhook' :
  ensure             => present,
  webhook_url        => 'https://puppet:[email protected]:8088/payload',
  token              => '65ba6770857c9f29d76ba5149da2c697fb39827e',
  project_name       => 'Puppetdev/control',
  server_url         => 'https://github.example.net/api/v3',
  disable_ssl_verify => true,
  provider           => 'github',
}

Grateful for any pointers. I'm not good enough with Ruby yet to try to debug the types and providers.

0.0.8 tarball has bum permissions on git_webhook.rb

When installing this from the forge git_webhook.rb ends up being 400 which makes puppet unhappy. Looking in the tarball it's 600 which really isn't any better.

Would you mind rolling a new tarball with the correct permissions on that file?

Thanks!

Migrate module to voxpupuli

Hey @abrader as it seems from #44 you are not actively supporting this module anymore, but we need some important fixes in it.
Therefore I propose to migrate this module to voxpupuli, if you would like to have help with the process let me know.

New Feature: Support the Bitbucket Server Web Post Hooks Plugin

Currently, the bitbucket/stash webook only supports the External Hooks plugin, and even that has some issues (#25).

It would be good if this module also supported the following webhook plugin which is officially supported by Puppet's Code Manager:
https://marketplace.atlassian.com/plugins/com.atlassian.stash.plugin.stash-web-post-receive-hooks-plugin/server/overview

	unless value =~ /^(https?:\/\/)?([\da-z\.-]+)\.([a-z\.]{2,6})([\/\w \.-])\/?$/
	raise(Puppet::Error, "Git server URL must be fully qualified, not '#{value}'")

abrader / abrader-gms Goto Github PK

abrader-gms's People

Stargazers

Watchers

Forkers

abrader-gms's Issues

puppet agent -t -d

Recommend Projects

Recommend Topics

Recommend Org