I am a newbie here, so please bear with me. I am observing this error on a CentOS 6.7 with ruby 1.8.7 (2013-06-27 patchlevel 374) [x86_64-linux], Puppet 3.8.3, and puppet-staging v1.0.6:

Error loading fact /etc/puppet/modules/staging/lib/facter/staging_windir.rb: /etc/puppet/modules/staging/lib/facter/staging_windir.rb:2: syntax error, unexpected ':', expecting kEND
confine osfamily: :windows
^

What am I doing wrong? Is the above being caused by the ruby version?

Thank you in advance for any pointers.

New users should be directed to the much preferred puppet-archive module

Currrently, you cannot pass in the strip parameter to the staging::deploy class, so it cannot be passed down. You have to stage the file and extract it in two steps if you wish to use strip.

Modulefile 'name' property should be 'nanliu-staging'

A couple thoughts on some download error handling that could potentially be added to this module...

1. in file.pp, remove file when wget/curl exit with non-zero?
   $http_get = "curl ... || rm -f ${name}"
2. use exec's tries attribute, in conjunction with curl's -C - and wget's --continue to resume broken transfers?

we've implemented item 1 since even a failed download will create the file and result in the file not being redownloaded.

I could submit a pull request for either route if interested.

Commands like wget can fail to download a file and create a file with 0 length instead
The next time agent runs it will see the file and won't retry to download it again.
I made an attempt to create a workaround in #42 , but it's not compatible with Windows

FreeBSD doesn't have the directory '/opt' by default. When running the module against FreeBSD, we should move the staging directory under /var/tmp as that's the suggest persistent temporary storage.

This module does not have a release on the forge. Can I go ahead and do this?

If we are happy to do this, @nanliu could you deprecate nanliu/staging with a 999.999.999 release, and request the transfer the puppet approved status to puppet/staging ?

In manifests/file.pp, there is a default path specified for Exec which overrides anything set by the user in site.pp.

Unfortunately, this breaks on Solaris, at least in my environment. Solaris 10 does not ship with curl, so I have added it with the pkgutil provider. This places it the curl binary in /opt/csw/bin/curl.

From looking at the code, the workaround is to put a symlink in /usr/local/bin/curl pointing to /opt/csw/bin/curl, but I'd rather not if I can avoid it.

Can the default Exec path either be parameterized or removed?

It would be nice to have an option to delete the staged temporary file after it has been deployed.

Currently, if a file exists, unzip will prompt, which is usually the wrong thing. So the -o -f flags should be passed, optionally controlled with $overwrite and $freshen or whatever.

When a password used with an https file source has characters that contain special meaning to the shell, various errors can occur. For example, if the char sequence !2 exists in a password (which it happened to for mine) bash interprets that as the 2nd command in my history and interpolates that into the curl command.

Sample Puppet code:

staging::file { 'somefile':
    source => 'https://host/path/to/file',
    username => 'myuser',
    password => 'password!2',
    target   => '/tmp/myfile',
}

Executing this will result in one of two results, depending on whether you have run at least 2 shell commands prior to executing Puppet. I've only sampled the resulting curl commands in the examples here.

[root@rpilproxy vagrant]# curl  -f -L -o /tmp/myfile -u user:password!2 https://host/path/to/file
bash: !2: event not found

...or it will output the following if ls -la happened to be the 15th command in my history...

[root@rpilproxy vagrant]# curl  -f -L -o /tmp/myfile -u user:password!15 https://host/path/to/file
curl  -f -L -o /tmp/myfile -u user:passwordls -la https://host/path/to/file

curl: (6) Couldn't resolve host 'host' # <- this particular error isn't significant, the curl command will still attempt to run the new command with the history interpolation

I believe the solution to be simply wrapping the username:password in the curl request with single quotes.

.tar.gz and .tgz are supported, but a single gzipped file is not

@nanliu can we get a new release to include the strict_variables support?

Any chance of getting a new release?

Hi everybody,

I would like to deprecate this module in favour of our archive module: https://github.com/voxpupuli/puppet-archive

Our archive module is well maintained and tested. Years ago it was created as the successor of staging, but staging was never deprecated. I do not see a value in a second module with a similar featureset. @voxpupuli/collaborators let me know what you think about it.

Hi

I've ran into this issue recently

Error: Could not retrieve catalog from remote server: Error 500 on SERVER: {"message":"Server Error: Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Error while evaluating a Resource Statement, Duplicate declaration: File[/opt/jboss3/.staging/jboss] is already declared in file /etc/puppetlabs/code/environments/jboss_master/modules/staging/manifests/file.pp:39; cannot redeclare at /etc/puppetlabs/code/environments/jboss_master/modules/staging/manifests/file.pp:39

This code has been working for months on end with no issues.

Affected Puppet, Ruby, OS and module versions/distributions

• Puppet: puppetserver-2.5.0-1.el7.noarch
• Ruby:
• Distribution: Centos
• Module version: master

How to reproduce (e.g Puppet code you use)

staging::file { "jboss-${::jboss::version}.zip":
source => "${::jboss::download_url}/jboss-${::jboss::version}.zip",
}

What are you seeing

Duplicate resource declaration

What behaviour did you expect instead

Create the .staging directory.

Output log

Any additional information you'd like to impart

I've checked the code for file.pp and changed it locally on my side
from:
if ! defined(File[$staging_dir]) { file { $staging_dir: ensure=>directory, } }
to:
ensure_resource('file', $staging_dir, { ensure => directory } )

Which fixes the issue.

Would it be possible to change the above to use ensure_resource which is part of stdlib?

Thanks
Shaun

Hi guys,

I tried to override the $target path for a call to staging::file. What I found was that nothing was happening until I specified the full path, including filename.

The documentation for staging::file indicates:

$target      = undef, #: the target staging directory, if unspecified ${staging::path}/${caller_module_name}

Looking at the source below from file.pp you can see if the $target is specified, $target_file is set to the submitted value which, in my case, is a folder. If $target is not specified, $target_file is set to ${staging_dir}/${name} (${name} would be the resource title here I would imagine?):

  if $target {
    $target_file = $target
    $staging_dir = staging_parse($target, 'parent')
  } else {
    $staging_dir = "${staging::path}/${subdir}"
    $target_file = "${staging_dir}/${name}"

    if ! defined(File[$staging_dir]) {
      file { $staging_dir:
        ensure=>directory,
      }
    }
  }

The reason nothing was happening (no file copy and no error) was due to your Idempotent check. I was creating the folder in another task. Your check would fire, checking if the "folder" existed, and move on.

Hope this makes sense,

Michael

Affected Puppet, Ruby, OS and module versions/distributions

• Puppet: 4.6.1
• Ruby: 2.0.0p384
• Distribution: Ubuntu 14.04
• Module version: 1.0.7 (per the MySQL module requirement: link)

How to reproduce (e.g Puppet code you use)

I have a Puppet manifest that loads the MySQL tuner like this:

class { '::mysql::server::mysqltuner': }

What are you seeing

This appears when I run puppet agent -t:

/Stage[main]/Mysql::Server::Mysqltuner/File[/usr/local/bin/mysqltuner]  Could not evaluate: Could not retrieve information from environment development source(s) file:/mysql/mysqltuner-v1.3.0

At that moment, this appears in the /var/log/puppetlabs/puppetserver/puppetserver.log log:

2016-08-23 19:49:28,414 WARN  [qtp1119244362-278] [puppetserver] Puppet Unknown variable: '::staging::path'. at /etc/puppetlabs/code/environments/development/modules/mysql/manifests/server/mysqltuner.pp:44:21

What behaviour did you expect instead

No issue in applying the manifest

Output log

N/A

Any additional information you'd like to impart

Please let me know if there's more info you'd like.

The staging module currently has an unzip_opts parameter which will get passed along to the unzip exec when run. This would be useful for tar (and probably other extraction methods as well). It would be especially useful in situations where developers ship configuration files in their archives which would then overwrite running configuration files.

I made a very quick, but insufficient run of puppet-staging under Puppet 4 and so far everything looks good. Is puppet-staging compatible with Puppet 4 or can be made compatible with? The version requirement says that Puppet 4 is currently not supported.

When running spec tests for compiling on Linux/OSX it complains about fully qualified path:

       error during compilation: Parameter path failed on File[C:\staging]: File paths must be fully qualified, not 'C:\staging' at ...

Has anyone found solution to this issue?
Might need to add some Windows specific spec tests to this module.

Would like to use your staging module, but I'm running puppet in standalone mode and it sounds like staging requires puppetmaster and hiera.

in https://github.com/nanliu/puppet-staging/blob/master/manifests/params.pp

lines 20-22 the mode, group and owner are defined which breaks folder permissioning and creates C:\ProgramData\staging but nothing can write or access it.

error I get:

rr: /Stage[main]/Firmware::Windows_gen8/Staging::File[hp_bl460_gen8_bios_12.20.2013.zip]/File[C:\ProgramData\staging
rmware]/ensure: change from absent to directory failed: Could not set 'directory on ensure: Permission denied - C:/Pr
amData/staging/firmware at /opt/puppet/environments/planet_firmware/modules/staging/manifests/file.pp:39
notice: /Stage[main]/Firmware::Windows_gen8/Staging::File[hp_bl460_gen8_bios_12.20.2013.zip]/Exec[C:\ProgramData\stag
/firmware/hp_bl460_gen8_bios_12.20.2013.zip]: Dependency File[C:\ProgramData\staging/firmware] has failures: true

If I try to set a staging directory by using target => in staging:file I get a different error:

err: Failed to apply catalog: Parameter cwd failed: cwd must be a fully qualified path at /opt/puppet/environments/pl
t_firmware/modules/staging/manifests/file.pp:93

@nanliu Is this a known issue for older versions of puppet for Windows? Does this work in the current version?

cc: @rismoney @jumanjiman

Affected Puppet, Ruby, OS and module versions/distributions

• Puppet: 2018.1.5
• Ruby:
• Distribution:
• Module version:

How to reproduce (e.g Puppet code you use)

puppet agent -t

What are you seeing

source_permissions in file.pp is raising a deprecation warning

What behaviour did you expect instead

no deprecation warning

Output log

Any additional information you'd like to impart

Hi,

The way I understand the code a custom fact staging_http_get should return a binary that needs to be used to download the file. But this search is based on PATH environment variable without considering $staging::exec_path and therefore can't find wget, which is in /usr/sfw/bin on Solaris.

Source URLs are given to curl/wget unescaped, so shell metacharacters break the command.

I'm attempting to add functionality to this to handle the usecase of:

class myclass (
  $version = "1.0"
){

  staging::file { "file-${version}.tar.gz":
    source => "http://host.com/path/file-${version}.tar.gz";
  }

  staging::extract { "file-${version}.tar.gz":
    target => "/path/to",
    creates => "directory",
    require => Staging::File["file-${version}.tar.gz"];
  }
}

As soon as I switch the $version to "2.0", i'd like for the new file to be downloaded (ok so far), but then be extracted in a way which replaces entirely the existing (1.0) "/path/to/directory". This can be done with a "tar xzvf --recursive-unlink" option, but trying to navigate around the creates/onlyif/unless logic and execute extract based on a subscribe to the Staging::File becomes difficult.

Any suggestions for using this module for this usecase, or contributions which would help satisfy would be welcome!

For two reasons:

• Connecting to server via HTTPS doesn't mean the target file was not tampered with and checking checksum can prevent it
• If target file is either "in middle of uploading" or incomplete for some other reason (corrupted disk etc.) it should not load it.

Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Duplicate declaration: File[/opt/staging] is already declared in file /etc/puppetlabs/puppet/environments/production/modules/staging/manifests/init.pp:28; cannot redeclare at /opt/puppet/share/puppet/modules/pe_staging/manifests/init.pp:25 on node hostname.internal
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run

I'm using Puppet 3.8.7 on Debian 4 & 5 machines (don't ask...). Puppet uses Ruby 1.8.7 on these machines which causes a syntax error with this module:

Error loading fact /var/lib/puppet/lib/facter/staging_windir.rb: /var/lib/puppet/lib/facter/staging_windir.rb:2: syntax error, unexpected ':', expecting kEND
  confine osfamily: :windows
                   ^

Could the JSON-style syntax be replaced with the old hash rocket syntax to make this module work on older environments again?

It would be useful if we could rename the output directory upon extraction. This way, we could specify a parameter to name the directory that is created when we extract a file.

• There is no existing issue or PR that addresses this problem

Optional, but makes our lives much easier:

• The issue affects the latest release of this module at the time of
  submission

Affected Puppet, Ruby, OS and module versions/distributions

Puppet: 3.8.4
Ruby: 1.8.7
OS: CentOS 6.7
Module: puppet-staging 2.0.0

What are you seeing

Error loading fact /var/lib/puppet/lib/facter/staging_windir.rb: /var/lib/puppet/lib/facter/staging_windir.rb:2: syntax error, unexpected ':', expecting kEND
confine osfamily: :windows
^

What behaviour did you expect instead

Not an error?

How did this behaviour get triggered

We used to use nanliu-staging and changed to puppet-staging due to a module dependency

Output log

sudo puppet agent -t --noop
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Info: Loading facts
Error loading fact /var/lib/puppet/lib/facter/staging_windir.rb: /var/lib/puppet/lib/facter/staging_windir.rb:2: syntax error, unexpected ':', expecting kEND
confine osfamily: :windows
^
Info: Caching catalog for somehost
Info: Applying configuration version '1465851765'
Notice: Finished catalog run in 10.45 seconds

Any additional information you'd like to impart

This change seems to be the issue: d3c1016

When using this module to run my tests with strict variables, I typically get the below error:

Undefined variable "::path"; Undefined variable "path" at confluence/spec/fixtures/modules/staging/manifests/extract.pp:53 on node merritt-l.home
# ./spec/classes/confluence_spec.rb:15:in `block (3 levels) in <top (required)>'

It does not seem to be correctly scoped. PR to follow shortly.