owncloud / testing Goto Github PK

View Code? Open in Web Editor NEW

3.0 65.0 4.0 1.03 MB

🔧 app for testing ownCloud

License: GNU Affero General Public License v3.0

PHP 51.99% Makefile 2.04% Gherkin 13.52% Starlark 32.46%

owncloud-app

testing's People

Contributors

Stargazers

Watchers

Forkers

iliaskabden saw-jan stjordanis

testing's Issues

add an occ command to mass-create users

copy code from owncloud/core#30572 to the testing app

Acceptance tests failing after core changes

https://drone.owncloud.com/owncloud/testing/1020

core PR owncloud/core#37332 changed various acceptance test code and steps.

Refactor acceptance tests so they pass.

Why do we escapeshellarg the parts of occ commands?

In lib/Occ.php function execute() we escapeshellarg the parts of the command to be executed.

We make exceptions to that for parts of occ commands that we know we need to send aalready enclosed in quotes, like:

--display-name='User One'
' '
'A value that has space in it'

It gradually gets more complicated to let through these string patterns and escapeshellarg everything else.

If you enable the testing app, and you know the admin password, then you can already do lots of nasty things to your system. If you know the admin password, then you may also have command line access to your system anyway - so you can type any command that you like.

So why do we try to protect occ command execution from command injection?

Undefined index: value

While running some password_policy tests locally I noticed my PHP dev server output:

...
[Wed Mar 18 17:29:32 2020] 192.168.1.22:35444 [200]: /status.php
[Wed Mar 18 17:29:32 2020] 192.168.1.22:35446 [200]: /status.php
[Wed Mar 18 17:29:32 2020] 192.168.1.22:35448 [200]: /status.php
[Wed Mar 18 17:29:32 2020] 192.168.1.22:35450 [200]: /status.php
[Wed Mar 18 17:29:32 2020] 192.168.1.22:35452 [207]: /remote.php/webdav/folder-to-share
[Wed Mar 18 17:29:32 2020] Undefined index: value at /home/phil/git/owncloud/core/apps-external/testing/lib/Config.php#102
[Wed Mar 18 17:29:32 2020] 192.168.1.22:35458 [200]: /ocs/v2.php/apps/testing/api/v1/apps
[Wed Mar 18 17:29:32 2020] 192.168.1.22:35460 [200]: /ocs/v2.php/apps/testing/api/v1/apps
[Wed Mar 18 17:29:33 2020] 192.168.1.22:35466 [200]: /ocs/v2.php/apps/testing/api/v1/occ
[Wed Mar 18 17:29:33 2020] 192.168.1.22:35474 [200]: /ocs/v2.php/apps/testing/api/v1/opcache
[Wed Mar 18 17:29:33 2020] 192.168.1.22:35476 [200]: /ocs/v2.php/apps/testing/api/v1/occ
[Wed Mar 18 17:29:33 2020] 192.168.1.22:35480 [200]: /ocs/v2.php/apps/testing/api/v1/opcache
[Wed Mar 18 17:29:34 2020] 192.168.1.22:35484 [200]: /ocs/v2.php/apps/testing/api/v1/lockprovisioning
[Wed Mar 18 17:29:34 2020] 192.168.1.22:35488 [200]: /ocs/v1.php/apps/testing/api/v1/apps
[Wed Mar 18 17:29:34 2020] 192.168.1.22:35490 [200]: /ocs/v2.php/apps/testing/api/v1/occ
[Wed Mar 18 17:29:34 2020] 192.168.1.22:35492 [200]: /ocs/v2.php/apps/testing/api/v1/opcache
...

Something is calling the testing app to set some setting(s) and is not sending value

Investigate.

(Probably this will be something in run.sh or a BeforeScenario of the ordinary test environment setup - ~~likely it needs fixing in core - just a guess~~ - it does not happen when I run a core scenario, so next guess is something in password_policy BeforeScenario)

Add Helper for automatic definition of skeleton folder

Currently the skeleton directory is set from external via run.sh - https://github.com/owncloud/core/blob/master/tests/acceptance/run.sh#L596-L625

It makes sense to have a specific helper (api) that accept 3 values:

api
ui
"path"

So we can configure it from the outside if needed, but when api/ui is defined, it assumes that we use the folder provided from the location where the testing app resides

Provide a installable tarball that contains the latest changes

For installing the app ( on docker images ) - ownCloud expects to find a tarfile with

testing
  - dir1 
  - dir2

But github provides everything in a top folder. We thus require a build app that can be used to install it in a docker image

Possible Solutions

move the codebase into a sub-folder (easiest )
create a release pipeline

increaseFileIDsBeyondMax32bits triggers "ORA-00942: table or view does not exist"

Stack trace:

Doctrine\DBAL\Driver\OCI8\OCI8Exception: ORA-00942: table or view does not exist
#8 /drone/src/lib/composer/doctrine/dbal/lib/Doctrine/DBAL/Driver/OCI8/OCI8Exception.php(33): fromErrorInfo
#7 /drone/src/lib/composer/doctrine/dbal/lib/Doctrine/DBAL/Driver/OCI8/OCI8Statement.php(245): execute
#6 /drone/src/lib/composer/doctrine/dbal/lib/Doctrine/DBAL/Driver/OCI8/OCI8Connection.php(116): query
#5 /drone/src/lib/composer/doctrine/dbal/lib/Doctrine/DBAL/Connection.php(852): executeQuery
#4 /drone/src/lib/private/DB/Connection.php(187): executeQuery
#3 testing/lib/BigFileID.php(46): increaseFileIDsBeyondMax32bits
#2 /drone/src/lib/private/legacy/api.php(179): call
#1 /drone/src/lib/private/Route/Router.php(342): match
#0 /drone/src/ocs/v1.php(63): null
Doctrine\DBAL\Exception\TableNotFoundException: An exception occurred while executing 'SELECT * from oc_filecache where fileid='2147483647'':

ORA-00942: table or view does not exist
#7 /drone/src/lib/composer/doctrine/dbal/lib/Doctrine/DBAL/Driver/AbstractOracleDriver.php(58): convertException
#6 /drone/src/lib/composer/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php(128): driverExceptionDuringQuery
#5 /drone/src/lib/composer/doctrine/dbal/lib/Doctrine/DBAL/Connection.php(855): executeQuery
#4 /drone/src/lib/private/DB/Connection.php(187): executeQuery
#3 testing/lib/BigFileID.php(46): increaseFileIDsBeyondMax32bits
#2 /drone/src/lib/private/legacy/api.php(179): call
#1 /drone/src/lib/private/Route/Router.php(342): match
#0 /drone/src/ocs/v1.php(63): null

Most probably we should have used the query builder for this method instead of plain sql which fails on oracle

Helper for fetching ssl certificates

While exploring real black-box testing with federated servers, I encountered the issue, that our container runs with self-signed certificates.

This is an issue when wanting to setup federated sharing, as the self-signed root certificate is not known to the owncloud trying to open a ssl connection.

Ref: owncloud/core#32413

We could establish a helper, from which we could fetch the ssl certificate.

Create cli acceptance tests

For the new testing:createusers cli command introduced in PR #85

Create Pull Request template

In order to release the app, we currently need to move the latest tag forward (ref: https://github.com/owncloud/testing#publish-latest-version-as-github-release ).

In order for people to be reminded, lets create PR templates so its not forgotten.

Add more test coverage for the testing app

PR #27 introduced the first unit tests.

The testing app is getting more endpoints and complexity. It would be useful to have tests that test it, rather than just relying on the "external" tests that use it.

Testing app requires federation app to be enabled

It looks like PR #101 added features to the testing app that require the federation app to be enabled.

check if this is true (disable federation and then try to run some ordinary acceptance tests that should run without federation)
think of a way to adjust the testing app so that it can do its job, and still work cleanly (maybe with a little less functionality) when the federation app is disabled
think if there are other core apps that the testing app depends on, and what we could do about that
document whatever dependencies are unavoidable

CI failing on trusted server tests

e.g. https://drone.owncloud.com/owncloud/testing/724/16/9

  Scenario Outline: Add new trusted server using the testing api                                # /var/www/owncloud/testrunner/apps/testing/tests/acceptance/features/apiTestingApp/trustedServer.feature:4
    Given using OCS API version "<ocs-api-version>"                                             # FeatureContext::usingOcsApiVersion()
    When the administrator adds url "http://new-oc.com" as trusted server using the testing API # FeatureContext::theAdministratorAddsUrlAsTrustedServerUsingTheTestingApi()
    Then the HTTP status code should be "<http-status>"                                         # FeatureContext::theHTTPStatusCodeShouldBe()
    And the HTTP reason phrase should be "<http-reason-phrase>"                                 # FeatureContext::theHTTPReasonPhraseShouldBe()
    And the OCS status code should be "<ocs-status>"                                            # OCSContext::theOCSStatusCodeShouldBe()
    And url "http://new-oc.com" should be a trusted server                                      # FeatureContext::urlShouldBeATrustedServer()

    Examples:
      | ocs-api-version | ocs-status | http-status | http-reason-phrase |
      | 1               | 201        | 201         | Created            |
      | 2               | 201        | 201         | Created            |
        HTTP status code is not the expected value
        Failed asserting that 200 matches expected '201'.

and later fails.

Some status codes seem to have changed.

Needs investigation to understand what changed and why, then fix code or tests.

Need ability to freeze uploads

This is needed to be able to test things like "job status" owncloud/core#32414 and also the ability to test behavior in case of PHP timeouts.

occ commands with spaces in quoted strings are not parsed correctly

Try to use the testing app to do an occ command like:

user:add user0 --password-from-env --display-name='User One' --email='[email protected]'

The testing app separates the command parts by whitespace, then escapes each part. The problem is that you get parts ```--display-name='UserandOne'`` - and then there is trouble.

The regex needs tto be smarter so that is sees --display-name='User One' as a single match.

Add getAppValue and getAppValues functionality

Currently we can setAppValue setAppValues deleteAppValue and deleteAppValues.

Code that wants to getAppValue or getAppValues is currently doing a remote occ command to display and return each app value that it cares about. It will be much more efficient to provide a get method where we already have a post and a delete method.

Ability to specify OCC command

Right now the occ command is tied to be executed via php console.php.

This however ignores that for example the docker container has a occ wrapper to ensure certain things.

Additionally with #12 we sort of lose the server environment variables. Later scoped helpers to retain them are only able to retain little.

UniqueConstraintViolationException inside log when adding trusted servers using testing api

Sample log from https://drone.owncloud.com/owncloud/phoenix/7515/24/13:

{"reqId":"2gbt7aFeIjaM3FErZKjb","level":3,"time":"2020-01-03T06:48:02+00:00","remoteAddr":"172.23.0.10","user":"admin","app":"PHP","method":"POST","url":"\/ocs\/v2.php\/apps\/testing\/api\/v1\/trustedservers?format=json","message":"Doctrine\\DBAL\\Exception\\UniqueConstraintViolationException: An exception occurred while executing 'INSERT INTO `oc_trusted_servers` (`url`, `url_hash`) VALUES(?, ?)' with params [\"http:\\\/\\\/owncloud\", \"668b5fcda851fe516fef14e82973beffe32f385a\"]:\n\nSQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '668b5fcda851fe516fef14e82973beffe32f385a' for key 'url_hash' at \/var\/www\/owncloud\/server\/lib\/composer\/doctrine\/dbal\/lib\/Doctrine\/DBAL\/Driver\/AbstractMySQLDriver.php#55"}

Analysis

This happens because we never check if the trusted server is already added. We just catch this error and return happily (which we should). The better way would be to check if the trusted server exists in the DB and then, make a decision (i.e. add or do nothing).
This pollutes the log, and could make debugging harder when checking CI failures.

Remove steps moved to core

Some steps for testing configkeys were moved to core in this PR
They need to be removed from the testing app

Basic CI pipeline

Establish a basic CI pipeline for:

check that code is working (phpstan // phan )
check that the code follows owncloud-codestyle

Fix failing notifications test

The last 2 nights e.g.
https://drone.owncloud.com/owncloud/testing/543/109

--- Failed scenarios:

    /var/www/owncloud/apps/testing/tests/acceptance/features/apiTestingApp/notifications.feature:31

50 scenarios (49 passed, 1 failed)
358 steps (356 passed, 1 failed, 1 skipped)
9m32.70s (15.06Mb)

  Scenario Outline: Testing app can create notifications for user                                  # /var/www/owncloud/apps/testing/tests/acceptance/features/apiTestingApp/notifications.feature:11
    Given using OCS API version "<ocs-api-version>"                                                # FeatureContext::usingOcsApiVersion()
    When the administrator creates a notification with the following details using the testing API # TestingAppContext::theAdministratorCreatesANotification()
      | key         | value                      |
      | subject     | lorem_subject              |
      | message     | lorem_message              |
      | user        | user0                      |
      | object_type | local_share                |
      | link        | www.lorem-notification.com |
      | object_id   | 47                         |
    Then user "user0" should have 1 notifications                                                  # NotificationsCoreContext::userNumNotifications()
    And the last notification of user "user0" should match these regular expressions               # NotificationsCoreContext::matchNotificationRegularExpression()
      | subject     | /^lorem_subject$/              |
      | message     | /^lorem_message$/              |
      | link        | /^www.lorem-notification.com$/ |
      | object_type | /^local_share$/                |
      | object_id   | /^47$/                         |

    Examples:
      | ocs-api-version |
      | 1               |
      | 2               |
        Failed asserting that actual size 2 matches expected size 1.

Display banner on every page

In case someone somehow is running daily-qa-stable10 or some version of it containing the testing app, as this is insecure, show a banner on every page.

Could either be a yellow notification or inject some flashy DOM element in the header.

Add php-cs-fixer

To be consistent with other apps.

Part of owncloud/QA#582

Fix steps changed in core

some steps were changed in core in this PR
Some steps needs to be changed in this app in order for CI to pass

Fix steps changed in core

some acceptance test steps related to user creation were changed in core. They need to be fixed in order for CI to pass.

Add new routes for handling trusted servers in owncloud

Currently there is no API or OCC command to administrate trusted federation server in owncloud. So for the acceptance tests to be able to handle the trusted servers properly we need to add new routes to the testing api. That way we will have new testing api for adding removing and listing the trusted servers in owncloud.

Related issue owncloud/core#34742