singer-io / tap-bing-ads Goto Github PK

View Code? Open in Web Editor NEW

12.0 12.0 28.0 325 KB

A Singer.io tap for extracting data from the Bing Ads API

License: GNU Affero General Public License v3.0

Python 100.00%

tap-bing-ads's People

Contributors

Stargazers

Watchers

tap-bing-ads's Issues

Add logging for tracking identifier on failed reports

When we run a report and it fails, we should add logging of the "tracking identifier" so it can be passed along to Google's support:

https://docs.microsoft.com/en-us/bingads/reporting-service/reportrequeststatustype

An error occurred while generating the report. You will need to submit your report request again. If the request continues to fail, consider getting the tracking identifier from the response message and contacting support.

Change from login.live auth endpoints to Microsoft Identity Platform Endpoints

See @nick-mccoy's comment in #37 and https://singer-io.slack.com/archives/C8N89PPGF/p1585239856006800

Bing Ads V12 sunset

https://docs.microsoft.com/en-us/advertising/guides/migration-guide?view=bingads-13

https://blogs.msdn.microsoft.com/bing_ads_api/2019/04/23/announcing-general-availability-of-api-v13/

This seems like something which needs to be fixed ASAP. I've submitted a PR to deal with this issue. I think that removing Language from the top level Report Request is the only breaking change but I'd appreciate if someone can double check and confirm this.

tap-bing-ads does not support AudiencePerformanceReport

Bing Ads' documentation notes that this report, "[compares] delivery performance statistics by audience, ad group, campaign, or account attributes."

This report is not currently supported by the tap and as a result is excluded from the REPORT_WHITELIST

Stitch API error

Hi,

We keep getting the following error when running the Stitch tap:

"2018-03-01 09:12:55,722Z main - INFO Exit status is: Discovery succeeded. Tap failed with code 1 and error message: "Error running AdGroupPerformanceReport report". Target succeeded."

Any ideas?

Sean.

Impossible to set PK on Reports

This is an issue which I've noticed across a few taps but first here so I'll raise the issue here and take it from there.

In my use case I am pulling static reports from various taps and using a target to store them in a database. Because the reports are static I know what the primary key (PK) should be.

It would be good to be able to set the PK in a similar fashion as was just merged into tap-adwords (singer-io/tap-adwords#58)

This would involve changing this line to:

    singer.write_schema(report_stream.stream,
                                     report_schema,
                                     metadata.get(stream_metadata, (), 'tap-bing-ads.report-key-properties') or [])
   )

This change would, of course, need the relevant documentation additions.

Please do let me know if such a PR would be approved and I'll put it in ASAP.

'CountryOrRegion' cannot be deserialized when using GeographicPerformanceReport

The following error occurs when running the GeographicPerformanceReport:

2018-01-16 18:26:12,851Z    tap - CRITICAL Server raised fault: 'The formatter threw an exception while trying to deserialize the message: There was an error while trying to deserialize parameter https://bingads.microsoft.com/Reporting/v11:ReportRequest. The InnerException message was 'Invalid enum value 'CountryOrRegion' cannot be deserialized into type 'Microsoft.AdCenter.Advertiser.Reporting.Api.DataContracts.Enum.GeographicPerformanceReportColumn'. Ensure that the necessary enum values are present and are marked with EnumMemberAttribute attribute if the type has DataContractAttribute attribute.'.  Please see InnerException for more details.'
2018-01-16 18:26:12,869Z    tap - Traceback (most recent call last):
2018-01-16 18:26:12,870Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/suds/transport/http.py", line 82, in send
2018-01-16 18:26:12,870Z    tap -     fp = self.u2open(u2request)
2018-01-16 18:26:12,870Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/suds/transport/http.py", line 132, in u2open
2018-01-16 18:26:12,870Z    tap -     return url.open(u2request, timeout=tm)
2018-01-16 18:26:12,870Z    tap -   File "/usr/lib/python3.5/urllib/request.py", line 472, in open
2018-01-16 18:26:12,870Z    tap -     response = meth(req, response)
2018-01-16 18:26:12,871Z    tap -   File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response
2018-01-16 18:26:12,871Z    tap -     'http', request, response, code, msg, hdrs)
2018-01-16 18:26:12,871Z    tap -   File "/usr/lib/python3.5/urllib/request.py", line 510, in error
2018-01-16 18:26:12,871Z    tap -     return self._call_chain(*args)
2018-01-16 18:26:12,871Z    tap -   File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
2018-01-16 18:26:12,872Z    tap -     result = func(*args)
2018-01-16 18:26:12,872Z    tap -   File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default
2018-01-16 18:26:12,872Z    tap -     raise HTTPError(req.full_url, code, msg, hdrs, fp)
2018-01-16 18:26:12,872Z    tap - urllib.error.HTTPError: HTTP Error 500: Internal Server Error
2018-01-16 18:26:12,872Z    tap - 
2018-01-16 18:26:12,872Z    tap - During handling of the above exception, another exception occurred:
2018-01-16 18:26:12,873Z    tap - 
2018-01-16 18:26:12,873Z    tap - Traceback (most recent call last):
2018-01-16 18:26:12,873Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/suds/client.py", line 613, in send
2018-01-16 18:26:12,873Z    tap -     reply = self.options.transport.send(request)
2018-01-16 18:26:12,873Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/suds/transport/https.py", line 66, in send
2018-01-16 18:26:12,873Z    tap -     return HttpTransport.send(self, request)
2018-01-16 18:26:12,874Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/suds/transport/http.py", line 94, in send
2018-01-16 18:26:12,874Z    tap -     raise TransportError(e.msg, e.code, e.fp)
2018-01-16 18:26:12,874Z    tap - suds.transport.TransportError: Internal Server Error
2018-01-16 18:26:12,874Z    tap - 
2018-01-16 18:26:12,874Z    tap - During handling of the above exception, another exception occurred:
2018-01-16 18:26:12,875Z    tap - 
2018-01-16 18:26:12,875Z    tap - Traceback (most recent call last):
2018-01-16 18:26:12,875Z    tap -   File "tap-env/bin/tap-bing-ads", line 11, in <module>
2018-01-16 18:26:12,875Z    tap -     sys.exit(main())
2018-01-16 18:26:12,875Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_bing_ads/__init__.py", line 761, in main
2018-01-16 18:26:12,875Z    tap -     raise exc
2018-01-16 18:26:12,876Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_bing_ads/__init__.py", line 758, in main
2018-01-16 18:26:12,876Z    tap -     main_impl()
2018-01-16 18:26:12,876Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_bing_ads/__init__.py", line 751, in main_impl
2018-01-16 18:26:12,876Z    tap -     do_sync_all_accounts(account_ids, args.catalog)
2018-01-16 18:26:12,876Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_bing_ads/__init__.py", line 738, in do_sync_all_accounts
2018-01-16 18:26:12,876Z    tap -     sync_account_data(account_id, catalog, selected_streams)
2018-01-16 18:26:12,877Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_bing_ads/__init__.py", line 726, in sync_account_data
2018-01-16 18:26:12,877Z    tap -     sync_reports(account_id, catalog)
2018-01-16 18:26:12,877Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_bing_ads/__init__.py", line 719, in sync_reports
2018-01-16 18:26:12,877Z    tap -     sync_report(client, account_id, report_stream)
2018-01-16 18:26:12,877Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_bing_ads/__init__.py", line 699, in sync_report
2018-01-16 18:26:12,878Z    tap -     request_id = client.SubmitGenerateReport(report_request)
2018-01-16 18:26:12,878Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_bing_ads/__init__.py", line 64, in wrapper
2018-01-16 18:26:12,878Z    tap -     return service_method(*args, **kwargs)
2018-01-16 18:26:12,878Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/bingads/service_client.py", line 269, in __call__
2018-01-16 18:26:12,878Z    tap -     raise ex
2018-01-16 18:26:12,878Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/bingads/service_client.py", line 261, in __call__
2018-01-16 18:26:12,878Z    tap -     response = self.service_client.soap_client.service.__getattr__(self.name)(*args, **kwargs)
2018-01-16 18:26:12,879Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/suds/client.py", line 521, in __call__
2018-01-16 18:26:12,879Z    tap -     return client.invoke(args, kwargs)
2018-01-16 18:26:12,879Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/suds/client.py", line 581, in invoke
2018-01-16 18:26:12,879Z    tap -     result = self.send(soapenv)
2018-01-16 18:26:12,879Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/suds/client.py", line 619, in send
2018-01-16 18:26:12,880Z    tap -     description=tostr(e), original_soapenv=original_soapenv)
2018-01-16 18:26:12,880Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/suds/client.py", line 670, in process_reply
2018-01-16 18:26:12,880Z    tap -     raise WebFault(fault, replyroot)
2018-01-16 18:26:12,880Z    tap - suds.WebFault: Server raised fault: 'The formatter threw an exception while trying to deserialize the message: There was an error while trying to deserialize parameter https://bingads.microsoft.com/Reporting/v11:ReportRequest. The InnerException message was 'Invalid enum value 'CountryOrRegion' cannot be deserialized into type 'Microsoft.AdCenter.Advertiser.Reporting.Api.DataContracts.Enum.GeographicPerformanceReportColumn'. Ensure that the necessary enum values are present and are marked with EnumMemberAttribute attribute if the type has DataContractAttribute attribute.'.  Please see InnerException for more details.'

Add support for Campaign Performance Report

Bing Ads API has many additional reports available. We'd love to see the Campaign Performance Report added.

https://docs.microsoft.com/en-us/bingads/reporting-service/campaignperformancereportrequest

tap-bing-ads does not support ConversionPerformanceReport

Bing Ads' documentation notes that this report is used, "to understand which campaigns and keywords are leading an audience to complete a purchase or other conversion action."

This report is not currently supported by the tap and as a result is excluded from the REPORT_WHITELIST

'ascii' codec can't decode byte 0xef in position 0: ordinal not in range(128)

When using an ascii encoding the following error occurs. Should we explicitly set the encoding to utf-8 in the tap? There was a similar issue with the tap-fullstory as seen here: singer-io/tap-fullstory#1

2018-01-02 20:38:46,083Z    tap - CRITICAL 'ascii' codec can't decode byte 0xef in position 0: ordinal not in range(128)
2018-01-02 20:38:46,086Z    tap - Traceback (most recent call last):
2018-01-02 20:38:46,086Z    tap -   File "tap-env/bin/tap-bing-ads", line 11, in <module>
2018-01-02 20:38:46,086Z    tap -     sys.exit(main())
2018-01-02 20:38:46,086Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_bing_ads/__init__.py", line 696, in main
2018-01-02 20:38:46,086Z    tap -     raise exc
2018-01-02 20:38:46,086Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_bing_ads/__init__.py", line 693, in main
2018-01-02 20:38:46,087Z    tap -     main_impl()
2018-01-02 20:38:46,087Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_bing_ads/__init__.py", line 680, in main_impl
2018-01-02 20:38:46,087Z    tap -     do_sync_all_accounts(account_ids, args.catalog)
2018-01-02 20:38:46,087Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_bing_ads/__init__.py", line 667, in do_sync_all_accounts
2018-01-02 20:38:46,087Z    tap -     sync_account_data(account_id, catalog, selected_streams)
2018-01-02 20:38:46,087Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_bing_ads/__init__.py", line 655, in sync_account_data
2018-01-02 20:38:46,087Z    tap -     sync_reports(account_id, catalog)
2018-01-02 20:38:46,087Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_bing_ads/__init__.py", line 648, in sync_reports
2018-01-02 20:38:46,088Z    tap -     sync_report(client, account_id, report_stream)
2018-01-02 20:38:46,088Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_bing_ads/__init__.py", line 629, in sync_report
2018-01-02 20:38:46,088Z    tap -     report_time)
2018-01-02 20:38:46,088Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_bing_ads/__init__.py", line 558, in stream_report
2018-01-02 20:38:46,088Z    tap -     header_line = next(csv_file)[1:-1]
2018-01-02 20:38:46,088Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/encodings/ascii.py", line 26, in decode
2018-01-02 20:38:46,088Z    tap -     return codecs.ascii_decode(input, self.errors)[0]
2018-01-02 20:38:46,088Z    tap - UnicodeDecodeError: 'ascii' codec can't decode byte 0xef in position 0: ordinal not in range(128)

Add support for Goals and Funnels Report

Bing Ads API has many additional reports available. We'd love to see the Goals and Funnels report added.

https://docs.microsoft.com/en-us/bingads/reporting-service/goalsandfunnelsreportfilter

tap-bing-ads does not support ProductDimensionPerformanceReport

Bing Ads' documentation notes that this report, "aggregates the performance data by product category, custom label, title, and type for a specified time period. You can include details in the report such as impressions, clicks, and spend that you can use to identify whether or not the product dimensions are performing well."

This report is not currently supported by the tap and as a result is excluded from the REPORT_WHITELIST

singer-io / tap-bing-ads Goto Github PK

tap-bing-ads's People

Contributors

Stargazers

Watchers

Forkers

tap-bing-ads's Issues

Recommend Projects

Recommend Topics

Recommend Org