d8-contrib-modules / cloudflare Goto Github PK

Requirements

The new D8 Purge module has a plugin architecture to integrate different CDNs. https://github.com/nielsvm/purge

• Implement plugin classes to integrate CloudFlare with Purge.
• Integrate the CloudFlare Config forms so that they are displayed by Purge.
• Test that editing a node results in an automated purge at edge. Confirm by looking at cache headers. Until varnish is fixed at acquia end-to-end will not work.

Implementation

See PR here

• #29

It is possible to authenticate with the CloudFlare API without having defined a website at CloudFlare. The first part of the configuration form accept the users credentials but the zone configuration form displays a fatal error:

Fatal error: Call to a member function getZoneId() on null in src/Form/ZoneSelectionForm.php on line 130

This adds some defensive coding practises around the getZoneId method call to ensure that there are zones available before attempting the zone id lookup.

Only the first thing listed below is actually expected. The rest is stuff you don't need, unless you use the "build system" included in this module. Drupal modules don't include build systems.

I guess it's because somehow require-dev is being used instead of require, but still, most unpleasant.

> Drupal\composer_manager\Composer\Command::update
Loading composer repositories with package information
Updating dependencies (including require-dev)
  - Installing d8-contrib-modules/cloudflarephpsdk (dev-master e250a89)
    Downloading: 100%         

  - Installing phing/phing (2.11.0)
    Downloading: 100%         

  - Installing squizlabs/php_codesniffer (2.3.4)
    Downloading: 100%         

  - Installing drupal/coder (8.2.3)
    Cloning fcda9b7e7cc8dc741ce01dfbeb44fd6c4cf3cde8

  - Installing sebastian/git (1.2.0)
    Downloading: 100%         

  - Installing symfony/finder (v2.7.4)
    Downloading: 100%         

  - Installing theseer/fdomdocument (1.6.1)
    Downloading: 100%         

  - Installing sebastian/finder-facade (1.2.0)
    Downloading: 100%         

  - Installing phploc/phploc (2.0.6)
    Downloading: 100%         

  - Installing sebastian/phpcpd (2.0.2)
    Downloading: 100%         

  - Installing pdepend/pdepend (1.1.3)
    Downloading: 100%         

  - Installing phpmd/phpmd (1.5.1)
    Downloading: 100%         

phing/phing suggests installing phpdocumentor/phpdocumentor (Documentation Generator for PHP)
phing/phing suggests installing pear/versioncontrol_svn (A simple OO-style interface for Subversion, the free/open-source version control system)
phing/phing suggests installing pear/versioncontrol_git (A library that provides OO interface to handle Git repository)
phing/phing suggests installing pear/archive_tar (Tar file management class)
phing/phing suggests installing tedivm/jshrink (Javascript Minifier built in PHP)
Writing lock file
Generating autoload files
> Drupal\Core\Composer\Composer::preAutoloadDump
> Drupal\Core\Composer\Composer::ensureHtaccess
> Drupal\composer_manager\Composer\Command::rewriteAutoload

Business Requirements

• As a developer/sitebuilder I want a stable release of the CloudFlare module that provides cache-clearing functionality.

Technical Requirements

• Review composer.json to make sure that dependencies are appropriately pinned.
• Audit the readme.md
• Tag a beta release.
• Get a release on D.O
• Add beta to packagist

I am not sure about the reason behind the customization of composer.json and tests, I

1. synced latest changes from github to drupal.org
2. cleanup composer.json requirements (causing issues like #59 )
3. Converted all simpletest/webtests to PHPUnit.

We have all tests green on drupal.org. Could we continue development there, please?

Requirements

As a user installing CloudFlare in front of my site I want to be able to confirm that CloudFlare is caching both static and dynamic resources.

• Have a field for a static path: default to something standard like robots.txt
• Have a field for a dynamic path: default to homepage.
• Store these settings in CMI. Follow the config settings in the existing module as an example.
• Update CloudFlarePHPSdk with a diagnostic class to accept a path, baseurl, and curl the site,
  {{curl -SLIXGET https://adam.weingarten.me/test-post}}
  we are looking for {{CF-Cache-Status: HIT}}
  remember that you may need to curl 2x in-order to get a cache hit.
• Expose the test in Drupal.
• Incorporate in hook_requirements

Business Requirements

• As a developer I need to quickly know that the latest stable combination of D8 Core, Composer Manager and CloudFlare work.

Technical Requirements

• Write automated tests to test the module.

Business Requirements

• As a developer I want to use a drupal module that uses PHP best practices like dependency injection.

Technical Requirements

• Research Symphony and Drupal Services. See if it makes sense to implement API endpoints as symphony services
• Update module to use service instead of raw calls to PHPSDK
• Add any phpunit tests as necessary.

Background

This module currently very much uses a D7 pattern, of using \Drupal::config() (~ variable_get()) and then instantiating objects all over the place.

Instead of exposing services that have their configuration injected, and those services can then be injected again, without every piece of code needing to retrieve the configuration and instantiate such an object again with that configuration.

Clear example of this: https://github.com/wimleers/cloudflare/commit/3281a7d74033b3ee2ab13834cd2bf24e327c62bc#diff-a35ab1b58293fa8fcf5e19b41a5f79e8L104.

currently the help is stubbed out and needs to be filled in.

Business Requirements

• As a contributor to the cloudflare module I want to be able to write automated tests using the best frameworks provided by Drupal Core.

Technical Requirements

• Submit a core patch for BrowserTestBase so that it will bootstrap the correct autoloader for Drupal.
  BrowserTestBase assumes that the autoloader is in docroot/core/vendor however composer manager moves this to docroot/vendor. This seems to cause problems.

Background Error

Drupal\cloudflare\Tests\CloudFlareConfigSettingsTest::testGoTo PHPUnit_Framework_Exception: PHP Fatal error: Cannot redeclare composerRequireDrupal8() (previously declared in /var/www/docroot/vendor/composer/autoload_real.php:56) in /var/www/docroot/core/vendor/composer/autoload_real.php on line 59 Fatal error: Cannot redeclare composerRequireDrupal8() (previously declared in /var/www/docroot/vendor/composer/autoload_real.php:56) in /var/www/docroot/core/vendor/composer/autoload_real.php on line 59

The purger module relies on careful calibration for calculating the purger capacity. Need to test the various purge types at cloudflare and update "getTimeHint" to the longest common denominator.

Requirements

• Getting cloudflare setup the first time is a little tricky. Update the update.txt instructions with details.
• Instructions for setting up SSL.
• Setting up page rule: Out of the box it only caches static resources you need to configure a page ruleset to cache everything in order for it to cache page

I have a brand clean new installation of Drupal8 on Centos 7 running with php7 and apache (it complained 5.6 was being deprecated this month)
Installation of the cloudflare modules (as based on the https://www.drupal.org/project/cloudflare page).
Problem is when i go to the settings page im given "The website encountered an unexpected error. Please try again later." and the below error in the error_log.

I am new to Drupal, installing the module to get familiar with it and help a friend. From what i can see EmailValidator is installed.

I can dump my composer show if this helps.

Error: Class 'Drupal\\Component\\Utility\\EmailValidator' not found in /var/www/html/drupal/modules/cloudflare/src/Form/SettingsForm.php on line 92 #0 /var/www/html/drupal/core/lib/Drupal/Core/DependencyInjection/ClassResolver.php(28): Drupal\\cloudflare\\Form\\SettingsForm::create(Object(Drupal\\Core\\DependencyInjection\\Container))\n#1 /var/www/html/drupal/modules/ctools/src/Wizard/FormWizardBase.php(238): Drupal\\Core\\DependencyInjection\\ClassResolver->getInstanceFromDefinition('Drupal\\\\cloudfla...')\n#2 /var/www/html/drupal/core/lib/Drupal/Core/Form/FormBuilder.php(202): Drupal\\ctools\\Wizard\\FormWizardBase->getFormId()\n#3 /var/www/html/drupal/core/lib/Drupal/Core/Form/FormBuilder.php(224): Drupal\\Core\\Form\\FormBuilder->getFormId(Object(Drupal\\cloudflare\\Wizard\\ConfigWizard), Object(Drupal\\Core\\Form\\FormState))\n#4 /var/www/html/drupal/modules/ctools/src/Wizard/WizardFactory.php(54): Drupal\\Core\\Form\\FormBuilder->buildForm(Object(Drupal\\cloudflare\\Wizard\\ConfigWizard), Object(Drupal\\Core\\Form\\FormState))\n#5 /var/www/html/drupal/modules/ctools/src/Controller/WizardFormController.php(78): Drupal\\ctools\\Wizard\\WizardFactory->getWizardForm(Object(Drupal\\cloudflare\\Wizard\\ConfigWizard), Array, false)\n#6 [internal function]: Drupal\\ctools\\Controller\\WizardFormController->getContentResult(Object(Symfony\\Component\\HttpFoundation\\Request), Object(Drupal\\Core\\Routing\\RouteMatch))\n#7 /var/www/html/drupal/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(123): call_user_func_array(Array, Array)\n#8 /var/www/html/drupal/core/lib/Drupal/Core/Render/Renderer.php(582): Drupal\\Core\\EventSubscriber\\EarlyRenderingControllerWrapperSubscriber->Drupal\\Core\\EventSubscriber\\{closure}()\n#9 /var/www/html/drupal/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(124): Drupal\\Core\\Render\\Renderer->executeInRenderContext(Object(Drupal\\Core\\Render\\RenderContext), Object(Closure))\n#10 /var/www/html/drupal/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(97): Drupal\\Core\\EventSubscriber\\EarlyRenderingControllerWrapperSubscriber->wrapControllerExecutionInRenderContext(Array, Array)\n#11 /var/www/html/drupal/vendor/symfony/http-kernel/HttpKernel.php(151): Drupal\\Core\\EventSubscriber\\EarlyRenderingControllerWrapperSubscriber->Drupal\\Core\\EventSubscriber\\{closure}()\n#12 /var/www/html/drupal/vendor/symfony/http-kernel/HttpKernel.php(68): Symfony\\Component\\HttpKernel\\HttpKernel->handleRaw(Object(Symfony\\Component\\HttpFoundation\\Request), 1)\n#13 /var/www/html/drupal/core/lib/Drupal/Core/StackMiddleware/Session.php(57): Symfony\\Component\\HttpKernel\\HttpKernel->handle(Object(Symfony\\Component\\HttpFoundation\\Request), 1, true)\n#14 /var/www/html/drupal/core/lib/Drupal/Core/StackMiddleware/KernelPreHandle.php(47): Drupal\\Core\\StackMiddleware\\Session->handle(Object(Symfony\\Component\\HttpFoundation\\Request), 1, true)\n#15 /var/www/html/drupal/core/modules/page_cache/src/StackMiddleware/PageCache.php(99): Drupal\\Core\\StackMiddleware\\KernelPreHandle->handle(Object(Symfony\\Component\\HttpFoundation\\Request), 1, true)\n#16 /var/www/html/drupal/core/modules/page_cache/src/StackMiddleware/PageCache.php(78): Drupal\\page_cache\\StackMiddleware\\PageCache->pass(Object(Symfony\\Component\\HttpFoundation\\Request), 1, true)\n#17 /var/www/html/drupal/core/lib/Drupal/Core/StackMiddleware/ReverseProxyMiddleware.php(47): Drupal\\page_cache\\StackMiddleware\\PageCache->handle(Object(Symfony\\Component\\HttpFoundation\\Request), 1, true)\n#18 /var/www/html/drupal/core/lib/Drupal/Core/StackMiddleware/NegotiationMiddleware.php(52): Drupal\\Core\\StackMiddleware\\ReverseProxyMiddleware->handle(Object(Symfony\\Component\\HttpFoundation\\Request), 1, true)\n#19 /var/www/html/drupal/vendor/stack/builder/src/Stack/StackedHttpKernel.php(23): Drupal\\Core\\StackMiddleware\\NegotiationMiddleware->handle(Object(Symfony\\Component\\HttpFoundation\\Request), 1, true)\n#20 /var/www/html/drupal/core/lib/Drupal/Core/DrupalKernel.php(693): Stack\\StackedHttpKernel->handle(Object(Symfony\\Component\\HttpFoundation\\Request), 1, true)\n#21 /var/www/html/drupal/index.php(19): Drupal\\Core\\DrupalKernel->handle(Object(Symfony\\Component\\HttpFoundation\\Request))\n#22 {main}, referer:

Business Requirements

• As a user of cloudFlare, with a multi-zone account I want the module to purge the correct zone.

Background

Since free tier only has a single zone some shortcuts were taken which assume that there is only one zone.

Technical Requirements

• Talk through use-cases for multi-zone configurations.
• Setup a site with multiple CloudFlare zones.
• Test changing settings and confirm that the correct zone is updated.
• Test cache clearing and confirm that the correct zone is updated.
• Ensure that any multizone selects or UX are hidden in a single zone account.
• Add description text to explain what a zone is since its super confusing.

Business Requirements

• Implement the key module to store cloudflare credentials
• Update module dependencies
• Update the existing screen for entering creds.
• Add automated tests to ensure that credentials are set
• Ensure that the automated tests are executed by travis-ci

Technical Recommendation

• Consider using https://www.drupal.org/node/2469723
• For Travis CI builds consider looking at
  https://github.com/acquia-pso/project-template/blob/7.x/build/phing/build.xml

Hey.
I use Drupal 8 + Acquia + CloudFlare.
I want to set up the reverse proxy on my website, to return the correct client IP.

I changed the settings.php as follows.

$settings['reverse_proxy'] = TRUE;
$settings['reverse_proxy_addresses'] = array($_SERVER ['REMOTE_ADDR']);

But the function \Drupal::request()->getClientIp() keeps returning the CloudFlare IP, instead of the client ip.

How can I set up Drupal 8 + Acquia + CloudFlare to return the correct client IP?

There are a bunch of integration points that need to happen to allow end-to-end purging.

• Work with Acquia hosting to support cache-tag clearing on varnish.
• Work with CloudFlare to support 16k http cache headers currently only 255 bytes are supported.
• Work with CloudFlare to support high-traffic, high velocity cache clearing. Currently according to them "You may purge up to 30 tags in one API call, and make up to 200 purge API calls in every 24 hour period."

see
https://support.cloudflare.com/hc/en-us/articles/206596608-How-to-Purge-Cache-Using-Cache-Tags

When I input a valid key I get the error message:

Invalid Api Key: Key should be 37 chars long.

Perhaps that was for old api keys?

Description

The freetier_setup.md documentation specifies:

From the root of your site run install composer dependencies: composer require d8-contrib-modules/cloudflarephpsdk "1.0.0-alpha5"

When this command is run, Drupal core's composer.json is modified to add the new dependency. When a user then subsequently updates Drupal core; best case scenario they have to mentally account for that change and be sure it is not overwritten, worst case, they will unintentionally remove the dependency with the next core update. Is it not possible to add this dependency in a way that does not modify core?

Expected Result

I would expect the core-provided composer.json to not be modified.

Actual Result

Drupal core's composer.json is modified.

Proposed Resolution

Is it possible to include the dependency via a composer.json within the module itself?

Cloudflare doesn't natively support purging by domain. It understand paths, zones, and tags. A zone could have several domains. We want to be able to purge one domain and not the others.

Technical Requirements

• Add a domain tag to all page requests using base_url or the new analog in D8. Allow override by user for CF.
• Add support for purging by domain to the CF module using ^^ Purge will simply be a purge on ^^ cache tag.

Business Requirements

• As a site-builder I want granular permissions for controlling access to vital functionality

Technical Requirements

• Add a permission for accessing the global cache-clear tab
  /admin/config/development/cloudflare/cache-clear
• Add a permission for accessing the zone settings tab
  /admin/config/development/cloudflare/zone
• Make sure that the API creds tab is protected by standard admin permissions
  /admin/config/development/cloudflare
• Add automated tests to validate that ^^ works.

Requirements

• As a user I want the ability to manually purge tags on cloudflare.
• Update existing purge by path to allow purge by tag.
• Expose a dropdown with a list of tags that users can choose from.
• Add a radio to allow users to invoke internal drupal cache-clear framework OR cloudflare-purge directly.
• Add text explaining that the preferred method is to use purge but that this can be used for troubleshooting
• Add automated testing

Background

You should be able to follow the pattern here:
https://github.com/d8-contrib-modules/cloudflare/blob/master/src/Form/CloudFlareCacheClearForm.php

Business Requirements

[ ] As a developer using Drupal I expect REMOTE_ADDR to be the the end user's IP address NOT the cloudflare CDN's IP.

Technical Requirements

[ ] Rename the menu task from API Credentials to Config
[ ] Add admin functionality to enable this here: admin/config/services/cloudflare
[ ] Store the setting in CMI
[ ] Add code to update REMOTE_ADDR to HTTP_CF_CONNECTING_IP
[ ] Write automated tests

Background Reading

https://support.cloudflare.com/hc/en-us/articles/200170986-How-does-CloudFlare-handle-HTTP-Request-headers-

Also recommend looking at the D7 cloudflare modules for some ideas. DO NOT SLAVISHLY copy.

* 
* Replace $_SERVER['REMOTE_ADDR'] with $_SERVER['HTTP_CF_CONNECTING_IP'] header
* Properly flag $_SERVER['HTTPS'] and set the base-url correctly if cloudflare is using flexible SSL 
*/
function cloudflare_boot() {
  global $base_url;
  global $conf;

  // Assign a proper IP address for end-client connecting via cloudflare using CF-Connecting-IP header
  if (variable_get('cloudflare_cf_connecting_ip', FALSE)) {
    $trusted = FALSE;
    if (!empty($_SERVER['HTTP_CF_CONNECTING_IP'])) {
      // If an array of known reverse proxy IPs is provided, then trust
      // the CF header if request really comes from one of them or from a cloudflare IP.
      $reverse_proxy_addresses = variable_get('reverse_proxy_addresses', array());
      $connecting_ip = $_SERVER['REMOTE_ADDR'];

      if (in_array($connecting_ip, $reverse_proxy_addresses) || cloudflare_ip_address_in_range($connecting_ip)) {
        $_SERVER['ORIG_REMOTE_ADDR'] = $connecting_ip;
        $_SERVER['REMOTE_ADDR'] = $_SERVER['HTTP_CF_CONNECTING_IP'];
        $trusted = TRUE;
        $conf['reverse_proxy'] = FALSE; // Disable any furthur reverse proxy handling.
      }
    }
  }
  // If we are not configured to use CF-Connecting-IP then use X-Forwarded-For instead
  // This is the recommended method and is used by default.
  else {
    $conf['reverse_proxy'] = TRUE;
    // Remove CloudFlare IP addresses from X-Forwarded-For header. 
    // This ensures that the end-user IP address is never identified as a CloudFlare IP. 
    $reverse_proxy_header = variable_get('reverse_proxy_header', 'HTTP_X_FORWARDED_FOR');
    if (!empty($_SERVER[$reverse_proxy_header])) {
      $forwarded = explode(',', $_SERVER[$reverse_proxy_header]);
      $forwarded = array_map('trim', $forwarded);
      $good_ips = array();
      foreach ($forwarded as $ip) {
        if (!cloudflare_ip_address_in_range($ip)) {
          $good_ips[] = $ip;
        }
      }
      $_SERVER[$reverse_proxy_header] = implode(',', $good_ips);
    }
  }

  // Properly flag the request as a HTTPS request if CloudFlare's flexible SSL is being used
  if (!empty($_SERVER['HTTP_X_FORWARDED_PROTO']) && $_SERVER['HTTP_X_FORWARDED_PROTO'] == 'https' && empty($_SERVER['HTTPS'])) {
    if ($trusted || in_array($connecting_ip, $reverse_proxy_addresses) || ip_address_in_range($connecting_ip, cloudflare_ips())) {
      $_SERVER['HTTPS'] = 'on';
    }
  }

  // Check for AlwaysOnline spider and set appropriate cache-control headers
  if (!empty($_SERVER['HTTP_USER_AGENT'])  && $_SERVER['HTTP_USER_AGENT'] == CLOUDFLARE_ALWAYSONLINE_USER_AGENT) {
    drupal_add_http_header('Cache-Control', 'public, max-age=86400');
    drupal_add_http_header('Vary', 'User-Agent');
  }
}

Free Tier does not support tagging. It gives a cryptic 400 error. The module should detect when a user has a free tier account plus purge by tagging configured.

• Add some Drupal Set Message Warnings.
• Add Purger diagnostic.