What do you think about an command to run Unit- and Functional tests that are configured in composer.json similar to ./typo3cms install:generatepackagestates - Command? So testing would also be more consistent if you can also use the ./typ3cms-Command.

This could be helpful for deployment purposes. If someone else would like to see this feature i could kickstart it in the next days.

I want to have the script available centrally. eg. /usr/bin

Unfortunately the require statement uses DIR which resolves to /usr/bin then. It should rather use the working directory (or allow to pass this as CLI arguement)

What do you think?

I often have the problem that I run into timeouts or a php memory limit when using the upgrade wizard in the install tool.
Could be very nice if I could run the steps from this tool.

Maybe some inspiration to get from:
https://github.com/czenker/t3ext-cli_update_wizard

requires #2 to be finished first

Using TYPO3 6.2.10, just installed typo3_console from git master, last commit is d6e3491.

~/public_html$ ./typo3cms
TYPO3 Console 1.0.7
usage: ./typo3cms

See './typo3cms help' for a list of all available commands.

~/public_html$ ./typo3cms help
No command could be found that matches the command identifier "extbase:help:help".

Enter "./typo3cms help" for an overview of all available commands
or "./typo3cms help " for a detailed description of the corresponding command.
~/public_html$ ./typo3cms help cache:flush
No command could be found that matches the command identifier "extbase:help:help".

Enter "./typo3cms help" for an overview of all available commands
or "./typo3cms help " for a detailed description of the corresponding >command.

If that matters: typo3_src is linked to /var/lib/typo3/6.2 -> typo3_src-6.2.10, but even linking it locally didn't help.

There are extensions that do provide such functionality with the scheduler but It would make a lot of sense to integrate a frontend Crawler into this extension as it aims to provide a set of cache options. Why not this also as it speeds up the frontend if invoked every 24 Hrs.

So as scheduler task. Which should then be invocable from this script.

sys_log takes a lot of space and sometimes is the biggest table in the database.

What about a command like:

cleanup:sys_log:1m

delets all entries older then 1month

or:

cleanup:sys_log:10000

delets all entries over 10000 lines.

thanks for typo3_console!

The existing configuration:removebypath is already quite useful for automatic setup but what is missing are configuration:getbypath and configuration:setbypath to complete the set.

Example scenario:

./typo3cms install:setup --non-interactive ...
encryption_key=$(./typo3cms configuration:getbypath SYS/encryptionKey) # Move to external file later
./typo3cms configuration:setbypath SYS/encryptionKey placeholder # Prevent re-generation by install tool
./typo3cms configuration:removebypath SYS/encryptionKey

The ConfigurationManager already provides getConfigurationValueByPath as well as setLocalConfigurationValueByPath which should make this straightforward.

Features backend:lock and backend:unlock are pretty cool to lock "everyone" out of the backend (even users with administrator privileges).

It would also be nice to have a command to lock the backend for non-admin users only (but allow access for admins). This can be done by adding/removing the adminOnly configuration in file LocalConfiguration.php (where 0 = no restriction, 1 = admin logins only, 2 = no BE users at all, -1 = no BE, no Install Tool).

I understand, manipulating LocalConfiguration.php is a little bit more complicated than writing/deleting file typo3conf/LOCK_BACKEND (and a completely different logic in the source code of EXT:typo3_console), but maybe just an additional parameter would be a user-friendly approach? E.g. typo3cms backend:lock --admin-only

I am too lazy to browse through my setup to find out all the tags :)

So I need a tool to shows me the list of tags I might flush.

Cheers,
Masi

When the typo3cms script is run by php the path in usage is ok:

See '/usr/bin/php typo3cms help <command identifier>'

But when run as a shell script is is wrong:

# $PWD is /home/user/public_html/example
# script called as ./typo3cms
See '/home/user/public_html/example/./typo3cms ./typo3cms help <command identifier>'

The responsible method seems to be $this->request->getCallingScript().

Tested on Ubuntu Precise w/PHP 5.5 with typo3 6.2.11 and typo3_console 1.0.7.

On a clean TYPO3 6.2 installation and following the steps provided for install this typo3_console on Unix systems, I got this error:

$ php ./typo3/cli_dispatch.phpsh extbase extension:install typo3_console
ERROR: No backend user named "_cli_lowlevel" was found!

Acording with #40, this was already solved.

Is there a recommended permission setup for the user _cli_lowlevel? I haven't found anything in the documentation.

Would make a lot of sense as you do not have to use a providerextensions and/or use the backend to add a set of t3d data to the cms.

If one uses typo3cms globally it means the script itself might not exist in the web root directory itself.

However https://github.com/helhum/typo3_console/blob/master/Scripts/typo3cms.php
always sets PATH_thisScript to the typo3cms in the web root.

Therefore the checkMainPathsExist will fail if "typo3cms" is not also in the web root path which kind of removes the "point" in being able to use typo3cms globally.

I just wrote a small task that installs all activated TYPO3 Extensions (based on composer.json "active-packages") by using TYPO3's InstallUtility.

Is that something that is usefull in general?

As already mentioned in the Database Compare Ticket (#13) this does not only update the database schema but also creates required folders and stuff.

I've prepared a Gist for this. https://gist.github.com/sascha-egerer/ea67084e3082b37c552a

At first I thought that the command would activate all packes listed in "require" and "require-dev". After a glance at the code I added an extra[active-packages] section to my composer.json. And boom...

typo3cms happily removed all extensions that haven't been listed including all Core extensions not flagged as required. This is IMHO a rather unfortunate behavior.

I suggest that only extensions managed by composer.json are handled by this command. Furthermore the command should take a --no-dev option to work like Composer when it comes to "require-dev".

Sidenote: perhaps the section should be renamed to active-typo3-packages to avoid possible name clashes with other scripts.

Masi

The unit tests currently use a generic Tests\Unit namespace, this should be fixed to use Helhum\Typo3Console\Tests\Unit instead.

There should be a shell command that can be ran to update the TYPO3 database according to the things usually done via the install tool. For the command line however it should only apply new and updated fields/tables and not rename or remove anything.

Hi helhum,

I have tried to install your extension in typo3/ext. Typo3 is complaining about a missing packet class.
Installed in typo3conf/ext, the extension works fine.
The core version is 6.2.12

Full error message:

PHP Fatal error: Uncaught exception 'TYPO3\Flow\Package\Exception\CorruptPackageException'
with message
'The package "typo3_console" does not contain a valid package class. Check if the file "/wwwroot/typo3/ext/typo3_console/Classes/Package.php" really contains a class called "TYPO3\CMS\Typo3Console\Package".' in
/wwwroot/typo3/typo3_src-6.2/typo3/sysext/core/Classes/Package/PackageFactory.php:61
Stack trace:
#0 /usr/share/typo3/typo3_src-6.2/typo3/sysext/core/Classes/Package/PackageManager.php(460): TYPO3\CMS\Core\Package\PackageFactory->create('/var/www/dev-mg...', 'typo3/ext/typo3...', 'typo3_console', 'Classes/', '')
#1 /usr/share/typo3/typo3_src-6.2/typo3/sysext/core/Classes/Package/PackageManager.php(282): TYPO3\CMS\Core\Package\PackageManager->registerPackagesFromConfiguration()
#2 /usr/share/typo3/typo3_src-6.2/typo3/sysext/core/Classes/Package/PackageManager.php(130): TYPO3\CMS\Core\Package\PackageManager->loadPackageStates()
#3 /usr/share/typo3/typo3_src-6.2/typo3/sysext/core/Classes/Core/Bootstrap.php( in /usr/share/typo3/typo3_src-6.2/typo3/sysext/core/Classes/Package/PackageFactory.php on line 61

We should automatically create the _cli_lowlevel user, when the extension is installed.
We can take the code from the scheduler as template how to do that:

\TYPO3\CMS\Scheduler\Controller\SchedulerModuleController::checkSchedulerUser()
\TYPO3\CMS\Scheduler\Controller\SchedulerModuleController::createSchedulerUser()

The ./typo3cms install:generatepackagestates command deactivates partOfFactoryDefault extensions because it seems the following if statement

    public function generatePackageStatesCommand($removeInactivePackages = FALSE) {
        $installationPackages = $this->getPackagesFromRootComposerFile();
        foreach ($this->packageManager->getAvailablePackages() as $package) {
            if (
                in_array($package->getPackageKey(), $installationPackages)
                || $package->isProtected()
                || ($package instanceof Package && $package->isPartOfMinimalUsableSystem())

missing a || ($package instanceof Package && $package->isPartOfFactoryDefault()) condition.

In case debug output happens, a HTML table structure is shown on the console which is bad.

Instead the logging framework should be used with an attached console writer

For instance:
cd /home/foo
/var/www/typo3cms help

Will result in:
Could not find typo3cms.php, make sure your path is within a TYPO3 installation and/or EXT:typo3_console is installed.

Reason since this change:
ebcb2d0

the directory where the file itself is, is not checked anymore.

As https://github.com/FluidTYPO3/builder aims into the same direction and has already some very cool stuff inside it would make sense to think about the possibility to merge these two projects and concentrate the menpower. perhaps @NamelessCoder has also something to say about the idea?!

By renaming configuration:removebypath to configuration:removelocal we could clearly indicate that this only affects values stored in the LocalConfiguration.

The calculation of the path for typo3cms.php fails with symlinked extension.

I symlinked the extension from /Volumes/HDD/Users/sok/Development/TYPO3/Extensions/git/typo3_console/into /Volumes/HDD/Users/sok/Sites/TYPO3/www.coreapi.dev/http/typo3conf/ext/typo3_console. This leads to the following error:

./typo3cms extension:install realurl

Warning: require(/Volumes/HDD/Users/sok/Development/TYPO3/Extensions/git/typo3_console/Scripts/typo3conf/ext/typo3_console/Scripts/typo3cms.php): failed to open stream: No such file or directory in /Volumes/HDD/Users/sok/Development/TYPO3/Extensions/git/typo3_console/Scripts/typo3cms on line 8

Call Stack:
    0.0082     237608   1. {main}() /Volumes/HDD/Users/sok/Development/TYPO3/Extensions/git/typo3_console/Scripts/typo3cms:0


Fatal error: require(): Failed opening required '/Volumes/HDD/Users/sok/Development/TYPO3/Extensions/git/typo3_console/Scripts/typo3conf/ext/typo3_console/Scripts/typo3cms.php' (include_path='.:/Volumes/HDD/Users/sok/pear/share/pear') in /Volumes/HDD/Users/sok/Development/TYPO3/Extensions/git/typo3_console/Scripts/typo3cms on line 8

Call Stack:
    0.0082     237608   1. {main}() /Volumes/HDD/Users/sok/Development/TYPO3/Extensions/git/typo3_console/Scripts/typo3cms:0

A quick workaround was to replace the __DIR__ by $_SERVER['PWD'] in typo3_console/Scripts/typo3cms

#!/usr/bin/env php
<?php

/**
 * Bootstrap for the command line
 */
$__self_dir = __DIR__;
require $_SERVER['PWD'] . '/typo3conf/ext/typo3_console/Scripts/typo3cms.php';

Error:

The package "typo3-console" does not contain a valid package class. Check if the file "......./typo3conf/ext/typo3-console/Classes/Package.php" really contains a class called "typo3-console\Package".

my setup:

"repositories": [
    { "type": "composer", "url": "http://composer.typo3.org/" },
    { "type": "vcs", "url": "https://github.com/helhum/typo3_console.git" }
  ],
"require": {
    "helhum/typo3-console": "*",
  },

when i change the name in composer.json to "helhum/typo3_console": "*", and change the composer file like here #46 it works.

Install tool code cannot handle that case, so we must fail with a reasonable error message (see also #45)

This is most likely due to disable Extbase caching during a certain setup step.

It may help to execute the database import command in a different context

Following the approach of CoreAPI let console bring DB-compare functionality - or should the compare be in Install tool and just started by console CommandController?

See https://github.com/TYPO3-coreapi/ext-coreapi/blob/master/Classes/Service/DatabaseComparator.php and https://github.com/TYPO3-coreapi/ext-coreapi/blob/master/Classes/Service/DatabaseCompareReal.php

The Service layer currently provides an Exception which derives from TYPO3\CMS\Fluid\Exception. This should be fixed to use a custom internal exception instead.

Currently it is not possible to select an existing database by executing the command:

php typo3cms install:databaseSelect --database-name=typo3 --database-create=0

Expected Result
Setting in LocalConfiguration should contain

...
    'DB' => array(
        ...
        'database => 'typo3',
    ),
...

Can you confirm that?
I think that is related to this change in the TYPO3 Core
TYPO3/typo3@ced1f40

Even ./typo3cms didn't run any more because of a broken extension (yes, my fault, but nevertheless...) I manually removed it from PackageStates.php.

But ./typo3cms still didn't run. So I did a "rm -rf typo3temp/*" and then the backend came back. I didn't think of the InstallTool though.

If you ask me: I'd love to see those two function in "typo3_console".

Summary

For a straight forward installation of a new typo3 instance it would make sense to move/add some basic functionalities to the typo3cms script while it is no extension at this point. Just a standalone script which provides basic functionalities. Once the core is fetched and the installation is done it shall add the _cli_lowlevel user automatically and install itself as fully functional extension.

Especially extension installation/deinstallation should be detached from any cms bindings because if you have any problem in your local cms installation it should be still possible to disable and enable extensions and clear cache and cachefolders. Also the switching back to a working preset of extensions would be a benefitical feature .

functionalities to include

static optional configuration file

read a optional static configuration file for some static settings depending on the environment [php Path,backupPath,extensionsToInstallPresets[asNamedArraysToSwitchBetween], ]

install:getCore

./typo3cms install:getCore [flavour git[with automatic checkout]/sourceforge] [symlinked/nonSymlinked]

Either an installation from git, sourceforge, (composer?) or individual resource should be possible

install:updateCore

./typo3cms install:updateCore [flavour git[with automatic checkout]/sourceforge] [symlinked/nonSymlinked] toVersion

install:getExtensions

./typo3cms install:getExtensions [flavour git/TER/Composer] [checkout/Version] [alternativePaths]

• Either an installation from git, sourceforge, (composer?) or individual resource should be possible
• Fallbackswitch for extension:install with "--g" option to this function?

extension:uninstall

./typo3cms extension:uninstall [withDeleteOption]

Should be detached from any CMS bindings as this makes an inaccessible cms still managable for inexpierienced users.

cache:wipeTempfolder

./typo3cms cache:wipeTempfolder

Should be detached from any CMS bindings as this makes an inaccessible cms still managable for inexpierienced users.

database:mysqlDump

./typo3cms database:mysqlDump

is a very common task which should be included here. Makes moving and snapshotting installations an ease.

database:mysqlDumpImport

./typo3cms database:mysqlDumpImport

is a very common task which should be included here. Makes moving and importing installations an ease.

maintenance:fullBackup

./typo3cms maintenance:fullBackup

This function shall invoke the the database:xxxDump and run a compression on all folders in docroot as default, optional only with provided array. Naming is the t3installation_timestamp.[tar.gz/zip] .Makes moving and snapshotting installations an ease.

optional frontend GUI

If this script had a GUI for initial installation it would make the whole installation even more fun for inexpierienced users. Imagine: Get this miniscript -> Open in Browser -> Enter some options -> Backendlogin or installtool for more individualized configuration. -> Done

Would make installation easy as hell

When running ./typo3cms cache:flush I expect all caches to be flushed, but it does not flush tables cf_extbase_typo3dbbackend_queries
Not sure about the others tables cf_extbase_typo3dbbackend_*

I don't know if there is an existing extension for this commands.
Maybe we'll write an own extension but I think it's better to have this commands in one extension.

The current setup has two requirements/expectations to successfully execute the typo3cms executable:

1. typo3 directory/symlink on the same level (execution)
2. typo3conf directory/symlink on the same level (installation)

This makes it impossible to have a web root with minimal exposure (index.php + typo3conf) and execute typo3cms in a directory above the web root. (E.g. for delivering a more secure frontend ;-) )

What I am currently trying to achieve is basically something like this:

Packages/
composer.+
typo3_src/
typo3cms
Web/
  fileadmin/, typo3temp/, uploads/
  typo3/
  typo3conf/
  index.php

For typo3cms to find the core in this situation, maybe the Composer autoloader could be used. This would even allow the typo3/cms package to live in Packages/ and still be found.

As topic says: EM displays an "OK" message about the typo3cms script having been copied to the site root, but the script is not copied.

Additional note: typo3conf/ext/typo3_console is a symlink to ~/Repositories/typo3_console. I checked the relative path starting from the actual path of the repository as well - the script was not copied to there (which is expected; the script would be written in a directory without write permissions). So it appears to be that errors from writing the script is never caught.

OSX Yosemite, PHP 5.5.14, TYPO3 7.1.

Trying a setup as described in the administrator manual yields the following error:

Class Helhum\Typo3Console\Composer\InstallerScripts is not autoloadable, can not call post-update-cmd script

The same error occurs with CMS 6.2.9 as well as 6.2.10-rc1. What have I done wrong? My composer manifest:

{
  "repositories": [
    {
      "type": "composer",
      "url": "http://composer.typo3.org/"
    }
  ],
  "name": "pagemachine/typo3-cms-distribution",
  "description" : "TYPO3 CMS Distribution",
  "license": "GPL-2.0+",
  "config": {
    "vendor-dir": "Packages/Libraries",
    "bin-dir": "bin"
  },
  "scripts": {
    "post-update-cmd": "Helhum\\Typo3Console\\Composer\\InstallerScripts::postUpdateAndInstall",
    "post-install-cmd": "Helhum\\Typo3Console\\Composer\\InstallerScripts::postUpdateAndInstall"
  },
  "require": {
    "composer/installers": "~1.0",
    "typo3/cms": "6.2.10-rc1",
    "typo3-ter/typo3-console": "~1.0.2"
  },
  "extra": {
    "installer-paths": {
      "html/typo3conf/ext/{$name}": [
        "type:typo3-cms-extension"
      ]
    }
  }
}

If user feedback is put into flashMessageQueue, it should be written to the console.

Current master has two calls to initializeClassLoader in ConsoleBootstrap.php
One if there is a composer classLoader present and one where there is not composer classLoader present.

Furthermore, typo3cms.php only checks for typo3_src composer autoload.php but not for the one in libraries.
Also it shouldn't really matter as initializeComposerClassLoader is called too in baseSetup.

Not sure how to best fix this and keep backwards compatbility with 6.2, for a composer install 7.3.1 it is enough to remove both calls but I don't think that is the case with e.g. 6.2
Maybe checking if the ComposerClassLoader class exists?

When you upload the extension (downloaded it as t3x from typo3.org) manually by using the extension manager, the typo3cms command is not copied to the website root path.

I do ./typo3cms cache:flush followed by ./typo3cms cache:warmup and get "Segmentation fault (core dumped)"

We can easily install extensions but we have to download manually new extensions from TER in the EM prior to execute install command.

It would be nice to have a command to download extension from TER and another one to remove extension in the typo3conf/ext directory.

Hi,

I've tested this behaviour with different instances and with a fresh installation of 6.2.9. There must be something wrong with the cache clearing api.
I've made a test-bash script which calls the cache clearing in a infinte loop

!/bin/bash

while :
do
./typo3cms cache:flush
sleep 1

done

While the script is running the frontend is not accessible anymore.

Sometimes I get this error:
Fatal error: Interface 'Psr\Log\LoggerInterface' not found in /srv/www/htdocs/dev/typo3_src/typo3/sysext/core/Classes/Log/Logger.php on line 24
PHP Fatal error: Interface 'Psr\Log\LoggerInterface' not found in /srv/www/htdocs/dev/typo3_src/typo3/sysext/core/Classes/Log/Logger.php on line 24

Am I doing smth wrong or can you reproduce it? When I use EXT:coreapi which the same infinite loop it works like a charm.

Could you add a composer.json?