I'm currently working on a project that uses a rather complex mix of standard, remote and custom datasources and noticed that supporting the execution conditions "Required Parameter" and "Forbidden Parameter" (just like the standard datasources do) would greatly improve the flexibility of the remote datasource.

It's not a bug, but would be a great addition – so I'm leaving this here as a reminder.

I entered a single backslash rather than a forward slash in the xpath field and rendered the backend with an error and I couldn't seem to access the data-sources page on the backend.

Does there need to be some sort of validation on that field at all? a single backslash blew it up :)

Just one observation.

remote datasource extension issue. It worked for a week and then all I could get was status code 0, even though my feed was set to cache and update every 240 minutes (4 hours). But, if I visit the same source url in my browser, the feed would pull up perfectly, but if Remote Datasource would fetch it, it would return an error with Status Code 0. And then I tried saving it to a static JSON file (with the content-type: application/json) and pulled in that static file and then I got the following notification…

Status code 200 was returned. Content-type: application/octet-stream

User Notice: GenericErrorHandler 1024: Cannot request a page callback without first specifying the page. on line 317 of file /Users/user1/Sites/www.site1.com/symphony/lib/core/class.administration.php

I can't uninstall this extension, I get a warning in the notification bar saying

Invalid argument supplied for foreach()

Neither server- nor symphony-logs tell me wich foreach exactly. Deleting the entry from sym_extensions uninstalls this extension correctly.

In some instances it would be nice to disable caching. For example if retrieving sensitive information through an API (or if a client instructs so).

Having no cache would ensure data is always fresh, and there are no copies on the server so if ever compromised no confidential data could be retrieved.

Current version always forces cache to be larger then 0; this could be disabled by either setting 0/-1 or else a separate check-box. Willing to implement myself as I need it for a project.

In preparation of an XML import using XML Importer's datasource branch, I created a data source named import-deutsch that fetches external XML. This data source either displays the error noted in #28 or the following:

    <import-deutsch valid="false" url="http://path/to/my.xml">
        <error>The <code>Mutex</code> class failed to acquire a lock.</error>
        <error>Transformed XML is invalid.</error>
        <errors>
            <item>loadXML(): Empty string supplied as input</item>
            <item>loadXML(): Empty string supplied as input</item>
        </errors>
        <raw-data></raw-data>
    </import-deutsch>

The given link is correct and provides valid XML.

Double quotes " inside the Included Elements field are breaking the backend:

Symphony Fatal Database Error: Unknown column 'namespace' in 'field list'

Single quotes work. This is on Symphony 2.6.0-alpha.

I'd have a small feature request; sometimes I'd like to output parameters from a remote datasource. I fully understand that outputting the right parameter might involve a lot of xPath computations etc. But it would be an interesting feature.

Anyway at the least would be interesting if we could have a delegate to plug into - and then let the extension developers worry about how to 'grab' the right output parameters for their need(s).

Not expecting anything immediate but would be interesting to have a healthy discussion this way one avoids 'custom' datasources where possible so when upgrading one does not risk incompatibility.

I've installed Remote Datasource to check all is well for an upcoming project and noticed that the input fields for this field are missing when I attempt to create a new Remote DS:

I know this probably hasn't been looked at yet, but thought I'd flag it now.

In PHP 5.6.x I get

<errors valid="false">
            <error>Data returned is invalid.</error>
            <item>loadXML(): Opening and ending tag mismatch: link line 8 and head in Entity, line: 11</item>
            <item>loadXML(): Opening and ending tag mismatch: meta line 6 and html in Entity, line: 22</item>
            <item>loadXML(): Opening and ending tag mismatch: head line 5 and rootelement in Entity, line: 22</item>
            <item>loadXML(): Premature end of data in tag html line 4 in Entity, line: 22</item>
            <item>loadXML(): Premature end of data in tag rootelement line 1 in Entity, line: 22</item>
</errors>

In PHP 7.0+, I get

<error>Status code 400 was returned. Content-type: text/html; charset=iso-8859-1</error>

It really bugs me out that I get a 400, because I know that the endpoint returns a 200.

I need to investigate.

cdata sections of xml datasources are not copied into the symphony page data xml. resulting in errors if cdata contains reserved xml character sequences. common use case for cdata are html fragments from WYSIWYG editors.

one solution may be the option to configure elements containing cdata and use this list in the
cdata-section-elements attribute of xsl:output.

a more radical solution could be to think if the extra xsl transformation of the datasoruce is realy nessesary. as far as i see it is only used to apply the xpath configured?

So I'm trying this extension because of an error with Dynamic XML in Symphony Core:

symphonycms/symphonycms#2202

I also posted on the original issue here:

http://www.getsymphony.com/discuss/thread/91558/

I enabled Remote Datasource and created a new datasource called Ustream. When trying to save I got this error:

Symphony Fatal Database Error: Unknown column '59a5a1d25df3cc4a4e34bb03ac2ffe20' in 'where clause'

An error occurred while attempting to execute the following query
DELETE FROM `sym_cache` WHERE `hash` = 59a5a1d25df3cc4a4e34bb03ac2ffe20

Backtrace
[/Users/jdsimcoe/Sites/atheycreek/symphony/lib/toolkit/class.mysql.php:506]
    MySQL->__error();
[/Users/jdsimcoe/Sites/atheycreek/symphony/lib/toolkit/class.mysql.php:695]
    MySQL->query();
[/Users/jdsimcoe/Sites/atheycreek/symphony/lib/toolkit/cache/cache.database.php:158]
    MySQL->delete();
[/Users/jdsimcoe/Sites/atheycreek/symphony/lib/toolkit/cache/cache.database.php:136]
    CacheDatabase->delete();
[/Users/jdsimcoe/Sites/atheycreek/symphony/lib/core/class.cacheable.php:75]
    CacheDatabase->write();
[/Users/jdsimcoe/Sites/atheycreek/extensions/remote_datasource/data-sources/datasource.remote.php:568]
    Cacheable->write();
[:0]
    RemoteDatasource::prepare();
[/Users/jdsimcoe/Sites/atheycreek/symphony/content/content.blueprintsdatasources.php:1239]
    call_user_func();
[/Users/jdsimcoe/Sites/atheycreek/symphony/content/content.blueprintsdatasources.php:1118]
    contentBlueprintsDatasources->__formAction();
[/Users/jdsimcoe/Sites/atheycreek/symphony/lib/toolkit/class.administrationpage.php:683]
    contentBlueprintsDatasources->__actionNew();
[/Users/jdsimcoe/Sites/atheycreek/symphony/lib/toolkit/class.administrationpage.php:646]
    AdministrationPage->__switchboard();
[/Users/jdsimcoe/Sites/atheycreek/symphony/lib/toolkit/class.administrationpage.php:444]
    AdministrationPage->action();
[/Users/jdsimcoe/Sites/atheycreek/symphony/lib/core/class.administration.php:239]
    AdministrationPage->build();
[/Users/jdsimcoe/Sites/atheycreek/symphony/lib/core/class.administration.php:442]
    Administration->__buildPage();
[/Users/jdsimcoe/Sites/atheycreek/symphony/lib/boot/func.utilities.php:231]
    Administration->display();
[/Users/jdsimcoe/Sites/atheycreek/symphony/lib/boot/func.utilities.php:207]
    symphony_launcher();
[/Users/jdsimcoe/Sites/atheycreek/index.php:14]
    symphony();
Database Query Log
[0.0003] SET character_set_connection = 'utf8', character_set_database = 'utf8', character_set_server = 'utf8';
[0.0002] SET CHARACTER SET 'utf8';
[0.0002] SET time_zone = '-07:00';
[0.0019] SELECT SQL_CACHE t1.name, t2.page, t2.delegate, t2.callback FROM `sym_extensions` as t1 INNER JOIN `sym_extensions_delegates` as t2 ON t1.id = t2.extension_id WHERE t1.status = 'enabled' ORDER BY t2.delegate, t1.name;
[0.0004] SELECT SQL_CACHE `session_data` FROM `sym_sessions` WHERE `session` = '4a0dd14761866559420cbd5a533d3170' LIMIT 1;
[0.0004] SELECT SQL_CACHE a.* FROM `sym_authors` AS `a` WHERE `username` = 'jdsimcoe' ORDER BY a.id ASC LIMIT 1;
[0.0003] UPDATE sym_authors SET `last_seen` = '2014-09-18 14:07:36' WHERE `id` = 4;
[0.0005] SELECT SQL_CACHE `name` FROM `sym_extensions` WHERE `status` = 'enabled';
[0.0004] SELECT SQL_CACHE * FROM `sym_extensions`;
[0.0004] SELECT SQL_CACHE `s`.* FROM `sym_sections` AS `s` ORDER BY `s`.`sortorder` asc;

Any ideas what is going on? It is weird that I'm getting the same SQL Errors in both places. Makes me think the issue is bigger than this extension but I wanted to raise it here as well since it occurred with this extension.

I'd like to fetch data from Twitter: is there a way to let this extension handle authorisation via oAuth?

Is there any way to access the cached data for a fulltext search? I guess there’s no straightforward way to use search index with external date?

A remote datasource doesn't seem to properly set other datasource-parameters that are used in the remote-datasource-url as dependencies.

This seems to be a bug, because the remote datasource won't work as expected if the needed datasource-parameter isn't available at the execution time of the remote-datasource.

Example

The url for my remote datasource looks like this:

 http://api.domain.com/downloads/{$ds-page.system-id}/

The datasource-parameter $ds-page.system-id is created by another datasource that's attached to the same page.

When trying to make this work I didn't get any results from the remote datasource because the parameter $ds-page.system-id wasn't available. I needed to manually add it as a dependency to the __construct-function to make the remote datasource work as expected:

public function __construct($env=NULL, $process_params=true)
{
    parent::__construct($env, $process_params);
    $this->_dependencies = array(
        '$ds-page.system-id'
    );
}

Setup:

• Symphony 2.7.2
• Remote Datasource 2.3.0

Not entering a start number gives an error:
Symphony Fatal Database Error: Column 'start_number' cannot be null

Solution:

• Making start number a required field
• Default value 1 for start number

I have the same error line twice:

loadXML(): Start tag expected, '<' not found in Entity, line: 1

Will investiguate more on that...

I'm getting an odd response from the Tender JSON API — it's returning a HTTP response code of 0 when called by this extension. Scooping the HTTP data, it looks like the connection is reset by a peer.

Calling the end point manually using curl works fine, so the problem does appear to lie in our code, not their service.

Any ideas?

Remote Data Source uses old .group classes for columns. It should use .two.columns > .primary.column / .secondary.column.

Sorry that I can't provide any more detail here, but this extension simply doesn't seem to work at all with Symphony 2.4, where it's supposed to replace the freshly removed Dynamic XML datasource if I remember correctly.

I also can't find anything about it in the migration guide.

There are no settings and when I try to save the datasource it just shows an error.

Any idea what went wrong here? Did someone at least make sure this extension actually works before removing the Dynamic XML Datasource from the core?

This extension aims to build upon the Dynamic XML datasource functionality provided in Symphony

The extension also says in the readme that it's actually build upon the core's Dynamic XML datasource, so I'm wondering how this is supposed to work and why it's been removed from the core in the first place.

In the light of upcoming XML Importer changes, it would be nice if this extension could read CSV data as well.

This is probably user error, but when I try save a remote datasource with an external URL for an xml file, it returns a 400 error.

Failed to load URL, status code 400 was returned.

If i edit the file directly and add the URL, I get this error within the debug output

<error>Status code 400 was returned. Content-type: text/html; charset=iso-8859-1</error>

The XML passes validation, and I have the XML format selected.

Is there something I'm missing? I thought it might be permissions, but playing with those didn't work. I tried using the URL that the site is on, and also uploading the file to another site I manage and trying that. Same result either way.

Noting that it works locally, pointing to a local URL. It's just the live instance.

Cheers!

Would it be possible to add YAML as a remote datasource type?

Hello,

first of all thank you for this extension! the new version has improved a lot!

I spend 2 days debugging to find this bug...
I use 2 different Remote_Datasource - one in XML and one in JSON Format - on the same page.

Line 9 of remote_datasource/data-sources/datasource.remote.php defines the $transformer class as static member.
Line 1092 of remote_datasource/data-sources/datasource.remote.php only loads a transformer for given format if no transformer was choosen yet.

The first datasource (XML) is executed - and the XML tranformer is loaded. The 2nd datasource (JSON) is executed, and resuses the XML tranformer - wich fails cause... doh! :)

Because the transformer is a static member, choice is made once per page based on the first format encountered.

i suggest making $transformer an array for formats

    public static function getTransformer($format)
    {
        if (is_array(self::$transformer) && array_key_exists($format, self::$transformer)) {
            return self::$transformer[$format];
        }

        if (!is_array(self::$transformer)) {
            self::$transformer = Array();
        }

        $transformerFile = EXTENSIONS . '/remote_datasource/lib/class.' . strtolower($format) . '.php';
        if (file_exists($transformerFile)) {
            $classname = require_once $transformerFile;
            self::$transformer[$format] = new $classname;
            return self::$transformer[$format];
        }

        throw new Exception("could not load transformer [$transformerFile] for format [$format]");
    }