drevops / behat-screenshot Goto Github PK

https://github.com/AlexSkrypnyk/scaffold/releases/tag/0.12.1

Do not update CircleCI config used for testing. But update all other GHAs.

Also, please note that this project uses Symfony coding standard, which includes 4 spaces as indentation for PHP scripts.

Having followed the setup instructions and added a screenshot step as desecribed to my feature test, no screenshot is saved, regardless of the directory that is specified by dir: in behat.yml, and no error is printed either to terminal or PHP error log.

The step Then I save screenshot passes and is printed green in terminal output. I have tried using /tmp and my home directory as values of dir: but no files there are created (either on screenshot step succeed, or on step fail).

Shouldn't there be error output if there was a problem?

My behat.yml:

default:
    autoload:
        - %paths.base%/tests/features/bootstrap
    suites:
        default:
            paths:
                - %paths.base%/tests/features
            contexts:
                - IntegratedExperts\BehatScreenshotExtension\Context\ScreenshotContext
                - FeatureContext:
                    base_url: http://local.pl3
                    db_user: foo
                    db_password: foo
                    db_name: foo
                    admin_username: foo
                    admin_password: foo

    extensions:
        Behat\MinkExtension:
            # Change to you custom url:
            base_url:  'http://local.pl3'
            show_auto: true
            show_cmd: epiphany %s
            sessions:
                default:
                    #goutte: ~
                    selenium2: ~
                    
        IntegratedExperts\BehatScreenshotExtension:
            dir: /tmp
            fail: true
            purge: false

As a means to share step workflows with testers, I knocked together this extremely crude addition to ScreenshotContext this morning. It captures a screenshot of each step of a tagged scenario (probably erring on the side of too many screenshots, really), and means a tester can see the output frame by frame.

From here it would not be far to output a video artefact.

  use FeatureTrait, ScenarioTrait;

  /**
   * Record every step with a screenshot, if tagged with 'screenshot-vcr'.
   *
   * @AfterStep
   */
  public function recordStepScreenshots($event)
  {
    $tagName = 'screenshot-vcr';
    /** @var \Behat\Behat\Hook\Scope\BeforeStepScope $event */
    if (!$this->getScenario()->hasTag($tagName) && !$this->getFeature()->hasTag($tagName)) {
      return;
    }
    $driver = $this->getSession()->getDriver();
    $stepText = $event->getStep()->getText();
    $stepLine = $event->getFeature()->getFile() . '-' . $event->getStep()->getLine();
    $fileName = $this->makeFileName('html', preg_replace('/[^a-zA-Z0-9-]+/', '-', $stepLine . '-' . $stepText));
    try {
      $data = $driver->getContent();
    } catch (DriverException $exception) {
      // Do not do anything if the driver does not have any content - most
      // likely the page has not been loaded yet.
      return;
    }
    $this->saveScreenshotData($fileName, $data);
    try {
      $data = $driver->getScreenshot();
      // Preserve filename, but change the extension - this is to group
      // content and screenshot files together by name.
      $fileName = substr($fileName, 0, -1 * strlen('html')).'png';
      $this->saveScreenshotData($fileName, $data);
    } catch (UnsupportedDriverActionException $exception) {
      // Nothing to do here - drivers without support for screenshots
      // simply do not have them created.
    }
  }

Any thoughts on the approach @AlexSkrypnyk ? I'd love to contribute this in a way that makes sense alongside the rest of the project, or perhaps you know of existing solutions I'm not aware of.

Happy to turn into a PR if you want, but I don't feel it's at that level of quality yet 😁

(I mistakenly opened this at drevops/behat-steps#69 initially 🙄)

The extension works fine with any headed browser, but when I try to use it combined with a headless browser, my browser driver (in my case gecko) always says Error: no DISPLAY environment variable specified
When I disable this plugin in my behat.yml everything works, but with this plugin, I cannot run my behat tests anymore. So any idea how to fix this issue?

Please document the effect and available options for fail and purge parameters.

Similar to drevops/behat-steps#134, we need to update all step definitions provided by this package to adhere to consistent BDD language.

Current way of setting settings

default:
  suites:
    default:
      contexts:
        - IntegratedExperts\BehatScreenshot\ScreenshotContext:
          -
            dir: %paths.base%/screenshots
            fail: true
            purge: false
        - FeatureContext     

Proper way of setting settings

default:
  suites:
    default:
      contexts:
        - FeatureContext     
        - IntegratedExperts\BehatScreenshot\ScreenshotContext
      extensions:
          IntegratedExperts\BehatScreenshot\ScreenshotContext:
            dir: %paths.base%/screenshots
            fail: true
            purge: false

When i add new function in my .feature it will be saved in below not in featurecontext.php
u C:\behat\vendor\integratedexperts\behat-screenshot\src\IntegratedExperts\BehatScreenshotExtension\Context\ScreenshotContext.php - I click the element "Client Login" definition added

Tests fail the Selenium test currently.

I see the same error locally as well - all other tests pass, then feature "Selenium screenshots" hangs at "Ensure that screenshots for Selenium driver can be captured." then times out with "" after a few minutes wait.

Feature: Selenium screenshots
  
  Ensure that screenshots for Selenium driver can be captured.

  ┌─ @BeforeScenario # DrevOps\BehatScreenshotExtension\Context\ScreenshotContext::beforeScenarioInit()
  │
  ╳  Could not open connection: Could not start a new session. New session request timed out 
  ╳  Host info: host: '3b446d376eed', ip: '192.168.112.3'
  ╳  Build info: version: '4.13.0', revision: 'ba948ece5b*'
  ╳  System info: os.name: 'Linux', os.arch: 'amd64', os.version: '6.2.0-34-generic', java.version: '11.0.20.1'
  ╳  Driver info: driver.version: unknown (Behat\Mink\Exception\DriverException)
  │
  @phpserver @javascript
  Scenario: Capture a screenshot using Selenium driver          # tests/behat/features/selenium.feature:6
    When I am on the screenshot test page                       # FeatureContext::goToScreenshotTestPage()
    And save screenshot                                         # DrevOps\BehatScreenshotExtension\Context\ScreenshotContext::iSaveScreenshot()
    Then file wildcard "*.selenium.feature_8.png" should exist  # FeatureContext::assertFileShouldExist()
    And file wildcard "*.selenium.feature_8.html" should exist  # FeatureContext::assertFileShouldExist()
    And save 800 x 600 screenshot                               # DrevOps\BehatScreenshotExtension\Context\ScreenshotContext::iSaveSizedScreenshot()
    Then file wildcard "*.selenium.feature_11.png" should exist # FeatureContext::assertFileShouldExist()
    And save 1440 x 900 screenshot                              # DrevOps\BehatScreenshotExtension\Context\ScreenshotContext::iSaveSizedScreenshot()
    And file wildcard "*.selenium.feature_13.html" should exist # FeatureContext::assertFileShouldExist()

--- Skipped scenarios:

    tests/behat/features/selenium.feature:6

18 scenarios (17 passed, 1 skipped)
130 steps (122 passed, 8 skipped)
5m14.69s (14.19Mb)

Same result in CircleCI: https://app.circleci.com/pipelines/github/xurizaemon/behat-screenshot/13/workflows/3a5a3502-a19f-4da3-89ee-cdc59164ce75/jobs/14

• config param purge: 1
• env var BEHAT_SCREENSHOT_PURGE=1, overrides config param, so that one-time command could be ran:

BEHAT_SCREENSHOT_PURGE=1 vendor/bin/behat

• update readme file.

minkphp/MinkSelenium2Driver#373

• drevops/behat-screenshot:1.3.0 requires symfony/http-client:^7.0
• drupal/drupal-extension:5.0.0 requires symfony/http-client:^4.4 || ^5 || ^6
• Composer cannot resolve this

Command failed: composer update drevops/behat-screenshot:1.3.0 drevops/behat-steps:2.2.0 drupal/ds:3.19.0 drupal/migrate_conditions:2.1.1 drupal/migrate_devel:2.3.0 drupal/migrate_plus:6.0.2 drupal/migrate_sandbox:1.1.5 drupal/raven:5.0.8 drupal/rdf:2.1.1 drupal/views_bulk_operations:4.2.6 drupal/yaml_editor:1.2.0 drush/drush:12.4.4 ergebnis/composer-normalize:2.42.0 phpro/grumphp-shim:2.5.0 phpspec/prophecy-phpunit:2.2.0 --with-dependencies --no-ansi --no-interaction --no-scripts --no-autoloader --no-plugins
Loading composer repositories with package information
Dependency drupal/drupal-extension is also a root requirement. Package has not been listed as an update argument, so keeping locked at old version. Use --with-all-dependencies (-W) to include root dependencies.
Updating dependencies
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - Root composer.json requires drevops/behat-screenshot ^1.3.0 -> satisfiable by drevops/behat-screenshot[1.3.0].
    - drevops/behat-screenshot 1.3.0 requires symfony/http-client ^7.0 -> found symfony/http-client[v7.0.0, ..., v7.0.5] but these were not loaded, likely because it conflicts with another require.

Use the option --with-all-dependencies (-W) to allow upgrades, downgrades and removals for packages currently locked to specific versions.

Output above generated with drupal/drupal-extension:5.0.0-rc1 installed, same issue with v5.0.0.

www-data@396ff6d5e641:/app$ composer require -W integratedexperts/behat-screenshot
Using version ^0.7.3 for integratedexperts/behat-screenshot
./composer.json has been updated
Running composer update integratedexperts/behat-screenshot --with-all-dependencies
Loading composer repositories with package information
Updating dependencies
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - drupal/core-recommended is locked to version 9.1.8 and an update of this package was not requested.
    - drupal/core-recommended 9.1.8 requires symfony/http-client-contracts v2.3.1 -> found symfony/http-client-contracts[v2.3.1] but it conflicts with another require.

Use the option --with-all-dependencies (-W) to allow upgrades, downgrades and removals for packages currently locked to specific versions.

Installation failed, reverting ./composer.json and ./composer.lock to their original content.

Think it's this one:

www-data@396ff6d5e641:/app$ composer show symfony/finder
name     : symfony/finder
descrip. : Finds files and directories via an intuitive fluent interface
keywords : 
versions : * v5.2.4

composer.json:

{
    "name": "catalyst-it/ci-basic",
    "type": "project",
    "description": "Drupal project for ci-basic, based on drupal/recommended-project",
    "homepage": "https://www.drupal.org/project/drupal",
    "license": "GPL-2.0-or-later",
    "require": {
        "catalyst-it/example-package": "^0.0.3",
        "composer/installers": "^1.9",
        "drupal/core-composer-scaffold": "^9.1",
        "drupal/core-project-message": "^9.1",
        "drupal/core-recommended": "^9.1",
        "drush/drush": "^10.5"
    },
    "conflict": {
        "drupal/drupal": "*"
    },
    "require-dev": {
        "ergebnis/composer-normalize": "^2.13"
    },
    "config": {
        "gitlab-domains": [
            "gitlab.catalyst.net.nz"
        ],
        "sort-packages": true
    },
    "extra": {
        "composer-exit-on-patch-failure": true,
        "drupal-scaffold": {
            "allowed-packages": [
                "catalyst-it/drupal-cove-hosted"
            ],
            "locations": {
                "web-root": "web/"
            }
        },
        "enable-patching": true,
        "installer-paths": {
            "drush/Commands/contrib/{$name}": [
                "type:drupal-drush"
            ],
            "web/core": [
                "type:drupal-core"
            ],
            "web/libraries/{$name}": [
                "type:drupal-library"
            ],
            "web/modules/contrib/{$name}": [
                "type:drupal-module"
            ],
            "web/modules/custom/{$name}": [
                "type:drupal-custom-module"
            ],
            "web/profiles/contrib/{$name}": [
                "type:drupal-profile"
            ],
            "web/profiles/custom/{$name}": [
                "type:drupal-custom-profile"
            ],
            "web/themes/contrib/{$name}": [
                "type:drupal-theme"
            ],
            "web/themes/custom/{$name}": [
                "type:drupal-custom-theme"
            ]
        },
        "patchLevel": {
            "drupal/core": "-p2"
        }
    },
    "repositories": {
        "drupalorg": {
            "type": "composer",
            "url": "https://packages.drupal.org/8"
        },
        "catalyst-it/drupal": {
            "type": "composer",
            "url": "https://gitlab.catalyst.net.nz/api/v4/group/36/-/packages/composer/packages.json"
        }
    },
    "minimum-stability": "stable",
    "prefer-stable": true,
    "support": {
        "chat": "https://www.drupal.org/node/314178",
        "docs": "https://www.drupal.org/docs/user_guide/en/index.html"
    }
}

This is a follow-up task from #37

Need to move CI to GHA and update to not use Docker for development.

Currently the screen size gets set to 1440 x 990 pixels for all screenshots.

https://github.com/integratedexperts/behat-screenshot/blob/37f2c9c43122c188ce604d1555968b6deaa422dc/src/IntegratedExperts/BehatScreenshotExtension/Context/ScreenshotContext.php#L79-L86

It would be wonderful to use this extension to make responsive screenshots as well.

Easiest probably would be to make this part of the config in the behat.yml. Maybe something like adding a new size key to fetch the desired width and height:

default:
  suites:
    default:
      contexts:
        - IntegratedExperts\BehatScreenshotExtension\Context\ScreenshotContext
        - FeatureContext
  extensions:
    IntegratedExperts\BehatScreenshotExtension:
      dir: %paths.base%/screenshots
      fail: true
      purge: false
      size:
        width: 320
        height: 568 

But an even nicer feature would be to have the ability to pass this as an option directly from the step. Like for example:

  Given I am on "http://google.com"  
  Then I save [320, 568] screenshot

or

  Given I am on "http://google.com"  
  Then I save [320, 568] responsive screenshot

This is to easily distinguish screenshots of failed tests from screenshots triggered by using step definition.

Different to #28 (which I read to mean, specifying the output filename in the stepdefinition, eg Then I save screenshot to my-screenshot.png).

It would be nice to have control over the screenshot filename generation when the filename is not specified.

My use case would be the ability to remove the time-based uniqueness, since then the screenshot filename is based on the feature / step (or failing step), and you can hit reload in the browser to see updated HTML / PNG after a test run. Currently this workflow requires re-opening the newly generated file 😁

Just a wishlist item - looking at makeFileName() we'd need to introduce some interpolation of values like @featureFile, @stepLine to a template string I expect.

  extensions:
    IntegratedExperts\BehatScreenshotExtension:
      dir: %paths.base%/screenshots
      fail: true
      purge: false
      # eg ...
      filenamePattern: @time-@prefix@featureFile@stepLine.@ext

Currently (per #53) I'm seeing some issues with the test configuration of this project.

PRs fail on CircleCI at the step the configuration depends on Docker Compose. The fix for this appears to be the steps in Installing and using Docker Compose.

Important: Remove all conflicting config code.

Also, please update readme file.

Pseudo-code:

  /**
   * Init values required for snapshots.
   *
   * @BeforeStep
   */
  public function beforeScenarioScreenshotInit(BeforeStepScope $scope) {
    $this->screenshotScenarioStartedTimestamp = microtime(TRUE);
    $this->screenshotScope = $scope;
    $paths = $scope->getSuite()->getSetting('paths');
    $this->screenshotDir = getenv('BEHAT_SCREENSHOT_DIR') ? getenv('BEHAT_SCREENSHOT_DIR') : reset($paths) . '/screenshots';
  }

  /**
   * Make screenshot filename.
   *
   * Format: micro.seconds_title_of_scenario_trimmed.ext.
   *
   * @param string $ext
   *   File extension without dot.
   *
   * @return string
   *   Unique file name.
   */
  protected function makeScreenshotFileName($ext) {
    $fileName = basename($this->screenshotScope->getFeature()->getFile());
    $stepLine = $this->screenshotScope->getStep()->getLine();

    return sprintf('%s.%s_[%s].%s', $this->screenshotScenarioStartedTimestamp, $fileName, $stepLine, $ext);
  }

test feature

@testapi
Feature: Behat screenshots

  Ensure that Behat is capable of taking screenshots.

  @phpserver
  Scenario: Make HTML screenshot of the test page
    Given I am on the screenshot test page
    When I save screenshot
    Then file wildcard "*.api\.screenshot\.feature_\[9\]\.html" should exist

```****

I don't know if you have seen this, but I came across this while searching for ways to grab videos of out tests because sometimes it would be helpful to just see things going through... and we can get some of this with paid services, but want to see if I can do it with chrome and the likes of CircleCI

It looks like it may be possible? and I I'm not sure if this is very simular to what you are doing or not cause I haven't dug through your code, but see https://medium.com/@anchen.li/how-to-do-video-recording-on-headless-chrome-966e10b1221

it's a javascript implementation but maybe we can do something in PHP? It might even be a whole new project... not sure.

Symfony only support ^5.4 || ^6.3 || ^7.0

I can make a PR to add Symfony 7, but what do you think of dropping support for unsupported versions of Symfony?

It'd be nice to have control of the filename from behat.yml configuration, and interpolate configured variables (or template string) with values from the step at hand.

In this way people could perhaps generate more informative screenshot naming, eg that included the feature filename, step line, time (accounting for repeated line entries from Examples: tables), success/fail state and other useful context.

I'm warming to the current layout, as it happens - but I think it'd be nice to be able to rearrange?

This would be a useful addition for things like #40 too.

Previously opened in error at drevops/behat-steps#70

 ┌─ @BeforeScenario # IntegratedExperts\BehatScreenshotExtension\Context\ScreenshotContext::beforeScenarioInit()
  │
  ╳  Fatal error: Call to a member function window() on null (Behat\Testwork\Call\Exception\FatalThrowableError)
  │

This happens due to the fact that driver's session has not been started and latest versions of Mink are re-using existing sessions (do not start it every time). We are using the driver directly, so have to replicate the same mechanism.

            if ($driver instanceof Selenium2Driver) {
                if (!$driver->isStarted()){
                    $driver->start();
                }
                $this->getSession()->resizeWindow(1440, 900, 'current');
            }

This is to allow providing a screenshot name from the step definition itself:
When I save screenshot with name "myname".

This would override any name settings set in other places.

Capturing screenshots in full-screen is not supported by Selenium natively, so we have to capture screenshot and scroll the screen and then glue screenshots together.

Hello,

is there a way to save a screenshot of the whole page in stead of the "fold" only and to save also the html?

I tried using

IntegratedExperts\BehatScreenshotExtension: dir: %paths.base%/screenshots fail: true purge: false png: true html: true

with no luck :)

thanks for the great extension!

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

This repository currently has no open or pending branches.

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository

Looking at #30 today I noticed an inconsistency in behaviour (I think).

https://github.com/integratedexperts/behat-screenshot/blob/master/src/IntegratedExperts/BehatScreenshotExtension/Context/ScreenshotContext.php#L73-L92

This checks instanceof Selenium2Driver and fires $driver->start() then does a resize. However, not all drivers that support screenshots are Selenium2Driver (eg DMore/chrome-mink-driver's ChromeDriver.

So ChromeDriver retains resized dimensions between runs (until Chrome exits), while Selenium is consistent. That's not a problem for me - but to implement initial size from behat.yml config in that function for drivers other than Selenium2, we'd need to have that function accommodate those other drivers too.

Need to find a way to do it in such a way that a user would see a clear message about the cause of this. Note that this is not about the browser, but the Selenium.