dg / ftp-deployment Goto Github PK

I really don't like storing my ftp credentials in the code. Is there any way for providing credentials on request?

The after jobs are not run when there's nothing to synchronize - I completely understand this behavoir, but could there be some afterAlways implemented?

My use case:

'before' => [
    'local: composer install --no-dev'
],
'after' => [
    'local: composer install'
],

I don't like uploading dev requierements to production server. And yes, doing deployment when there's nothing to do is kind of edge-case. Does this make sense at all? TY

Error: Cannot upload file /.deployment, number of retries exceeded. in …/Deployment.php on 248

made me wonder what did I wrong. Only the attempt to upload a file via an ordinary FTP client revealed:

Server said: STOR denied: quota exceeded: used 205775 of 100000 upload files
STOR: notice: quota reached: used 205775 of 100000 upload files… Error -160: could not start data transfer

FTP deployment

Started at [2013/02/01 13:54]
Config file is /Users/.../deployment/deployment.ini

Deploying my site
Connecting to server
Loaded remote .htdeployment file
Scanning files in /Users/...
..
Fatal error: Call to undefined function fnmatch() in /Users/.../deployment/libs/Deployment.php on line 411

$ php --version
PHP 5.4.11 (cli) (built: Jan 26 2013 11:02:57)
Copyright (c) 1997-2013 The PHP Group
Zend Engine v2.4.0, Copyright (c) 1998-2013 Zend Technologies

$ php-config
Usage: /usr/local/php/bin/php-config [OPTION]
Options:
--prefix [/Users/jozef/Applications/php-5.4.11]
--includes [-I/Users/jozef/Applications/php-5.4.11/include/php -I/Users/jozef/Applications/php-5.4.11/include/php/main -I/Users/jozef/Applications/php-5.4.11/include/php/TSRM -I/Users/jozef/Applications/php-5.4.11/include/php/Zend -I/Users/jozef/Applications/php-5.4.11/include/php/ext -I/Users/jozef/Applications/php-5.4.11/include/php/ext/date/lib]
--ldflags [ -L/Users/jozef/Downloads/bin/build/lib -L/Users/jozef/Downloads/bin/build//lib -L/Users/jozef/Applications/mysql-5.5.29//lib]
--libs [ -lmysqlclient_r -lldap -llber -liconv -liconv -lfreetype -lpng -lz -ljpeg -lcurl -lz -lm -lxml2 -lz -liconv -lm -lcurl -lxml2 -lz -liconv -lm -lmysqlclient_r -lxml2 -lz -liconv -lm -lxml2 -lz -liconv -lm -lxml2 -lz -liconv -lm -lxml2 -lz -liconv -lm -lxml2 -lz -liconv -lm ]
--extension-dir [/Users/jozef/Applications/php-5.4.11/lib/php/extensions/no-debug-zts-20100525]
--include-dir [/Users/jozef/Applications/php-5.4.11/include/php]
--man-dir [/Users/jozef/Applications/php-5.4.11/php/man]
--php-binary [/Users/jozef/Applications/php-5.4.11/bin/php]
--php-sapis [ apache2handler cli cgi]
--configure-options [--prefix=/Users/jozef/Applications/php-5.4.11 --without-pear --with-apxs2=/Users/jozef/Applications/Apache2/bin/apxs --with-mysql=/Users/jozef/Applications/mysql-5.5.29/ --with-pdo-mysql=/Users/jozef/Applications/mysql-5.5.29//bin/mysql_config --with-png-dir=/Users/jozef/Downloads/bin/build/ --with-jpeg-dir=/Users/jozef/Downloads/bin/build/ --with-freetype-dir=/Users/jozef/Downloads/bin/build/ --with-curl=/Users/jozef/Downloads/bin/build/ --with-libxml-dir=/Users/jozef/Downloads/bin/build/ --with-zlib --with-gd --with-mhash --with-ldap --with-iconv --enable-mbstring --enable-ftp --enable-exif --enable-bcmath --enable-soap --enable-shared=no --enable-static=yes]
--version [5.4.11]
--vernum [50411]

Hi David,
I have question, If is posible ignore some directories in purge directories.
Like:

purge[] = temp
purge[] = !temp/sessions

I need clear temp, but don't sessions directory

Is it bug or feature? Deployment process take a long time because of that...

I can create patch if you like.

Would be possible to highlight the information about current mode? I think this is pretty crucial information and should be higher and maybe highlighted. As well as inform about live mode, not just about the test mode.

Scenario: I'm always using test mode if I'm deploying bigger updates before I do the final upload. Just to check whether I'm not deleting anything wrong. In case I forgot to run the deploy in test mode would be good to let me know this so I can stop the upload.

Screen: http://d.pr/i/15Q4S

I am implementing replacement of SshServer, by using pure PHP SSH2/SFTP library phpseclib, because PHP 5.6 PECL has no php_ssh2.dll. The thing is that I cant find a perfect solution for including the phpseclib (composer require phpseclib/phpseclib) in Deployment. Should I make it optional composer requirement? And how to include it in SshServer.php? Deployment does not use composer autoloader, so now I have hardcoded hack in the file: include DIR . '/../../../../autoload.php'; which is ugly and non-systematic.

1. deploy
2. add pattern to ignore-list
3. deploy

Result:
Items matching the new patten are deleted (on the remote server).

I expected them to be ignored on both sides.

Hi,
thank you for nice tool, works almost perfectly. I have only one (maybe) bug. When I try to upload to root of FTP (remote = ftp://user:[email protected]/), script falls down with message:

Error: /folder/ není souborem ani adresářem

When I try to upload to some subfolder, there is no problem. I have tried adding or removing trailing slash, but withou success. Until now, I was using this amazing tool for uploads to my prod. server, there was no need to upload to root of ftp... But now I'm maintaining 3rd party site and this is required... Is it bug or did I have overlooked something? I'm using linux (Debian Jessie, i386, PHP 5.6.0RC2)

Is there a chance to add rollback capability for example when an error occurred we should be able to
run a command like : deployment config.in --rollback=version

if ($entry == '.' || $entry == '..' || isset($disallow[$entry])) {
    continue;

The variable $disallow is undefined. I think it can be removed from the code.

Hi David,
is there a way to bypass the password via commandline, instead of write it down in deployment file ?

Thanks in advance,
Michael

Hi,
in FTP class is set passive mode:

https://github.com/dg/ftp-deployment/blob/master/Deployment/libs/Ftp.php#L66

It would be better when it will be optional, because I have a problem on some FTP with this mode. When I comment it all its OK.

Some servers (eg. FileZilla) refuse to rename the file when it already exists. Its "product made" behavior that could be sometimes changed. Related discussion - http://forum.filezilla-project.org/viewtopic.php?f=6&t=23373.

In that case we get an error and all the temporary files are left on the server. How to solve it?

1. detect this behavior and refuse to continue

2. rollback all the changes

3. detect and optionally use ftp_delete & ftp_rename

4. seems the best to me

It would be nice to support reversed synchronization as well.

For example, if we are working in team (and we are not using git), I can download the newest files from FTP, edit them and then deploy to FTP..

SshServer::purge - is this some kind of limitation of SSH itself or it just has not been implemented yet?

Normally the script uses an unencrypted connection. I was able to make it use secure connection by modifying Deployment/libs/Ftp.php line 64 from:
$this->connect($parts['host']...
to:
$this->ssl_connect($parts['host']...

This seems like an ugly hack though. Is there some better way to do it? Am I missing something? Thanks in advance.

Hello,

Would it also be possible to have my login details and ftp address split to separate lines instead of a single remote address?

[my site] ; There may be more than one section
; remote FTP server
; remote = ftp://user:[email protected]/directory
user = "user"
pass = "secretpassword"
host = "ftp.example.com/directory"

Hey, I'm getting this error when trying to use ftp-deployment to upload to my website's SFTP server. I tested uploading with the sftp CLI client and it showed that sftp itself is working. The server's probably configured in a way ftp-deployment isn't expecting. I'm using the latest (Github) version of ftp-deployment. I'm puzzled by it. Here are my config file and outlog log:

_deployment.ini:

log = _deployment.log

remote = sftp://USERNAME:PASSWORD@URL

local = ./_site

preprocess = no

ignore = "misc"

_deployment.log:

Started at [2016/09/24 15:28]
Config file is { DIRECTORY }/_deployment.ini

Deploying 
Live mode
Connecting to server
Remote .htdeployment file not found
Scanning files in  { DIRECTORY }/_site
Creating remote file .htdeployment.running
Error: ssh2_sftp_mkdir failures.

Deployment\SshException: ssh2_sftp_mkdir failures. in  { DIRECTORY }/ftp-deployment/src/Deployment/SshServer.php:226
Stack trace:
#0  { DIRECTORY }/ftp-deployment/src/Deployment/SshServer.php(132): Deployment\SshServer->protect('ssh2_sftp_mkdir', Array)
#1  { DIRECTORY }/ftp-deployment/src/Deployment/Deployer.php(139): Deployment\SshServer->createDir('/')
#2  { DIRECTORY }/ftp-deployment/src/Deployment/CliRunner.php(93): Deployment\Deployer->deploy()
#3  { DIRECTORY }/ftp-deployment/src/deployment.php(22): Deployment\CliRunner->run()
#4  { DIRECTORY }/ftp-deployment/deployment(3): require(' { DIRECTORY }...')
#5 {main}

I tried uncommenting line 139 in Deployer.php (maybe ftp-deployment thinks it can create directories which already exist, while the server doesn't like that?) and got a different error:

_deployment.log:

Started at [2016/09/24 15:43]
Config file is { DIRECTORY }/_deployment.ini

Deploying 
Live mode
Connecting to server
Remote .htdeployment file not found
Scanning files in  { DIRECTORY }/_site
Creating remote file .htdeployment.running
Error: fwrite() expects parameter 1 to be resource, boolean given

Deployment\SshException: fwrite() expects parameter 1 to be resource, boolean given in { DIRECTORY }/ftp-deployment/src/Deployment/SshServer.php:224
Stack trace:
#0 { DIRECTORY }/ftp-deployment/src/Deployment/SshServer.php(92): Deployment\SshServer->protect(Object(Closure))
#1{ DIRECTORY }/ftp-deployment/src/Deployment/Deployer.php(140): Deployment\SshServer->writeFile('/tmp/deployment...', '/.htdeployment....')
#2 { DIRECTORY }/ftp-deployment/src/Deployment/CliRunner.php(93): Deployment\Deployer->deploy()
#3 { DIRECTORY }/ftp-deployment/src/deployment.php(22): Deployment\CliRunner->run()
#4 { DIRECTORY }/ftp-deployment/deployment(3): require('{ DIRECTORY }...')
#5 {main}

Is there anything you see that could be the obvious cause? I'm willing to look into it myself but I don't know the code as well and I'm a bit stumped currently. It's quite important to me because I used ftp-deployment to easily update my site whenever I changed it (it's great!), and doing it manually in sftp is such a pain it's really not doable.

I'm not sure if this should happen according to sftp spec, but I had to make a patch for myself.

In case someone else had this problem, merge this commit:
tkafka/ftp-deployment@dg:master...master

That's not working.

[my site] ; There may be more than one section
; log file (defaults to config file with extension .log)
log = ...

Because of this line..

$logger = new Logger(empty($config['log']) ? preg_replace('#\.\w+$#', '.log', $options['config']) : $config['log']);

In $config is under key 'my site' whole options. So $config[log] is everytime unset.

Readme could answer or link answers to:

• how to set up a post-push hook in version-control-system or if it's a good idea at all
• pairing with Nette - setting maintenance mode, clearing caches, and other best practices

In new version I am not possible to delete directory temp/cache. It died on 273 in FtpServer.
In version 1.3 it was OK.

Deleting:
Cleaning temp/cache
Error: Remove directory operation failed. in ..\libs\FtpServer.php on 273

I found, that in 1.4 is not possible to use a Passive mod. Maybe this is problem?

on https://github.com/dg/ftp-deployment/blob/master/Deployment/libs/Ftp.php#L67

Is possible to remove it or it is there for better security?

Is there a way to simulate environments using this library?I mean I want to run something like
deployment --env=staging

@dg As I understand from this library I can use different config files for every environment for example :
deployment staging.ini
deployment production.ini
deployment development.ini --t

but in this case I have to repeat same options for every environment like ignore option for example

Any suggestion ? I really like this library

in folder there is many files, that shouldnt be ignored, but there is also lot of subdirectories that should be ignored, how to do it?

I've tried login data and I have password that ends with exclamation mark (!) so it looks something like that: ftp://user:[email protected]/directory

and it throws:

Error: syntax error, unexpected '!' in /var/www/html/***/ftp_deploy/deployment.ini on line 2

I've got a problem with deployment caused by different error message returned by FileZilla server. It is becouse server returns File not found message instead of hardcoded No such file or directory.

Have got any idea how to solve it?

Example output:

c:\dev\www\example>libs/bin/deployment.bat deploy/production.ini
Started at [2014/06/25 23:35]
Config file is C:\dev\www\example\deploy\production.ini

Deploying my site
Connecting to server
Remote .htdeployment file not found
Scanning files in ./deploy
....
Uploading:
(1 of 3) /production.ini
(2 of 3) /production.log
(3 of 3) /.htdeployment

Renaming:
(1 of 3) Renaming /path/to/example/deploy/production.ini
Error: File not found in C:\dev\www\example\libs\dg\ftp-deployment\Deployment\libs\FtpServer.php on 273

On my server I got error Cannot create directory '\'. when attempting upload files.
This problem occurs immediately after Uploading: appears in the log.

I've found the problem here where dirname() function is used.

I'm running on Windows and my FTP server doesn't accept backslash as a dir.

@dg
exception 'Deployment\FtpException' with message '1 matches total' in /opt/lampp/htdocs/ftpwrapper/src/Deployment/FtpServer.php:303

Stack trace:

#0 [internal function]: Deployment\FtpServer->Deployment{closure}(2, 'ftp_rename(): 1...', '/opt/lampp/htdo...', 306, Array)

#1 [internal function]: ftp_rename(Resource id #5766, '/lib/Mage/Backu...', '/lib/Mage/Backu...')

#2 /opt/lampp/htdocs/ftpwrapper/src/Deployment/FtpServer.php(306): call_user_func_array('ftp_rename', Array)

#3 /opt/lampp/htdocs/ftpwrapper/src/Deployment/FtpServer.php(290): Deployment\FtpServer->protect('ftp_rename', Array)

#4 /opt/lampp/htdocs/ftpwrapper/src/Deployment/FtpServer.php(145): Deployment\FtpServer->ftp('rename', '/lib/Mage/Backu...', '/lib/Mage/Backu...')

#5 /opt/lampp/htdocs/ftpwrapper/src/Deployment/Deployer.php(286): Deployment\FtpServer->renameFile('/lib/Mage/Backu...', '/lib/Mage/Backu...')

#6 /opt/lampp/htdocs/ftpwrapper/src/Deployment/Deployer.php(153): Deployment\Deployer->renamePaths(Array)

#7 /opt/lampp/htdocs/ftpwrapper/src/Deployment/CliRunner.php(94): Deployment\Deployer->deploy()

#8 /opt/lampp/htdocs/ftpwrapper/src/deployment.php(23): Deployment\CliRunner->run()

#9 /opt/lampp/htdocs/ftpwrapper/deployment(3): require('/opt/lampp/htdo...')

#10 {main}

NOTE: We are facing this above exception issue while renaming files after upload via system so could you please help us to sort out this issue.

You can reply at below given email address.

Email: [email protected]

Regards
Narendra Kumar Dangi

Hi,
Currently, any command sent to the server requires SITE EXEC
For example,

before[] = remote: mkdir somefolder
before[] = remote: cd somefolder

But, some commands like above could be executed without such extension, so, would nice to have an option to run standard ftp commands like

before[] = ftp: mkdir somefolder
before[] = ftp: cd somefolder

I'm not sure if this is viable from your side, but my proposal is add new ftp method to the Server interface, and then perform

...
 elseif ($m[1] === 'ftp') {
  $out = $this->server->ftp($m[2]);
...

Or perhaps, use a small list with the few standard ftp commands and switch to the ftp method.

Thanks

Please, add possibility to set public/private key file when trying to login via SSH key ssh2_auth_pubkey_file.

After update to PHP7 on Windows I tried to use ftps instead of ftp protocol since openssl is bundled in php, but deployment with ftps protocol doesn't work as expected.

If I run deployment via ftp, it works normally:

Loaded remote .htdeployment file
Scanning files in ...
Ignoring ....
Creating remote file .htdeployment.running

Uploading:
...

But if I run it with ftps i get strange error Error: Ok to send data. Also, the log falsely says that remote .htdeployment does not exist. File .htdeployment.running is created.

Remote .htdeployment file not found
Scanning files in ...
Ignoring ...
Creating remote file .htdeployment.running 
Error: Cannot upload file C:\Users\...\AppData\Local\Temp\deployment\dep189C.tmp, number of retries exceeded. Error: Ok to send data.

PHP 7.0.8, ftp-deployment 2.5, Wedos hosting

I haven't found any correct way of changing this value. I'm using Windows and the default one 'java' does not work (raises ERROR_FILE_NOT_FOUND), but if I provide absolute path C:\\Program Files\\Java\\jre8\\bin\\java.exe, it works. I think It cannot be done right now unless hardcoding the Preprocessor::javaBinary value. Neither including java directory in $Path does work.

Is there a option/flag to prevent the status indicator prints? I'm running this script via GitLab CI and my deployment log looks something like this:

I think you get the idea. The log gets very long and unreadable because of this.

In context of Nette app, with index.php like:

<?php

if(file_exists(__DIR__ . '/../.deployment')) {
    require '.maintenance.php';
}

// Let bootstrap create Dependency Injection container.
$container = require __DIR__ . '/../app/bootstrap.php';

// Run application.
$container->application->run();

the deployment script could write a temporary file in root directory eg .deployment and after deployment is finished remove this file.

Při použití SFTP mi to hlásí chybu:

Error: dir(): Unable to open ssh2.sftp://Resource id #17/www/example.com/
www/temp/cache/.delete54a67b575d74a on remote host in \dg-ftp-deployment\libs\SshServer.php on 159

Divné je to, že tato složka ale vůbec neexistuje.
Přijde mi to, jako když to složku smaže, ale pak jí to chce mazat ještě jednou.
Díky tomu, že to vyhodí chybu to proces nedokončí.
Např. v dané složce zbyly ještě 3 .delete..... všechny mají správná práva a chmod 755.

Vyřešil jsem to tak, že jsem v SshServer#159 nahradil dir( za @dir( a následující while obalil do if ($iterator) { ... }

Udělal jsem několik uploadů, aktualizací a po této úpravě nikdy nebyl problém.

Hi/Ahoj David,
will it there be a select option in config to select only some of directories to preprocess with minifier of js and css.

preprocess_dirs = "/web/js"

Or is there some "hard" way how to do it now?
Thank you for nice work : )

If I have two sections in ini file, I have problem with the "local" option. There is used an relative path to previous settings, not to base path.

example:
[first]
local = myApp

[second]
local = data

get: BASEPATH/myApp/data
expected: BASEPATH/data

Yuicompressor is available via composer package.

https://packagist.org/packages/nervo/yuicompressor

Ftp-deployment could suggest this package. What do you think?

in some cases , we might already have an active project in our server and we want to integrate this tool with our current porject so we will need to generate the .htdeployment file only and upload to our server and next time we deploy only changed files will be uploaded .

Any idea ?

Deployer.php#L250 => filters were applied.

It is possible to add something like 'logDir'?
This param will be optional. Over this param will be posibble to save log file in to different directory than is ini file.

Actualy I have one folder where i have ini files for every project (prepared .bat) & logger is creating mess in this folder. :(

I was testing ftp deployment yesterday on two computers, strange thing happened, one of computers out of nowhere started to stuck at compressing one jquery plugin. (nothing was changed between deploys)

When I try to compress that jquery plugin in cmd, everything works well.

Does someone experienced a similar issue? Testing it a couple of hours now, but no bigger progress so far.

Thanks

As far as I can tell, file paths which are md5-ed are absolute.

That means when we develop in team and members are deploying to the same ftp, everyone is still rewriting all project files although nothing has changed.

Is there a reason for absolute paths?

Hi @dg! First of all, thanks for your great job. I'm using it in all my projects and want to contribute with all my efforts. My question is: is there any way to run a command in the server? The after[] jobs will only run commands on client and access an url in the server, correct? I really wanted to run console commands on server. Thanks.

If I misspell configuration of local path in development.ini file, I get an error with message:

Error: chdir(): Invalid argument (errno 22) in ...\Deployment.php on 99

which needs to be inspected manually in code. There should be local path existence testing.