Comments (10)
mhogeweg
commented on September 26, 2024
hi David, we have seen this CSW also has started failing in Geoportal Server 1.2.7. I have reached out to GSA (they operate Data.gov).
from geoportal-server-harvester.
pandzel-zz
commented on September 26, 2024
As a hint, neither of these two calls deliver any record content:
Any other combination of possible values of ElementSetName, typenames, outputformat, etc. doesn't help.
from geoportal-server-harvester.
valentinedwv
commented on September 26, 2024
&resultType=results works. not pretty.
https://catalog.data.gov/csw?request=GetRecords&service=CSW&version=2.0.2&ElementSetName=full&typenames=csw:Record&resultType=results
from geoportal-server-harvester.
mhogeweg
commented on September 26, 2024
not to mention that these include many http-only links that will start to fail given the https-only policy in place for the federal government or when using Geoportal Server over https
from geoportal-server-harvester.
pandzel-zz
commented on September 26, 2024
Currently, harvester has a "Data.gov" dedicated input broker which allows to acquire metadata from that particular source. Since pycsw keeps failing, "Data.gov" broker is using a combination of CKAN and WAF methods to get job done. Broker itself requires no configuration beside giving a name; all other properties are optional.
from geoportal-server-harvester.
valentinedwv
commented on September 26, 2024
So if we want to filter based on an organization, how might we do that?
from geoportal-server-harvester.
tomkralidis
commented on September 26, 2024
cc @kalxas
@pandzel / @mhogeweg to clarify, what are the issues at hand? Is there a bug in how the CSW behaves? If there are bugs in pycsw please let us know and we can fix them accordingly.
from geoportal-server-harvester.
mhogeweg
commented on September 26, 2024
hi @tomkralidis. Geoportal Server can do federated search to CSW and there are users who want to do this. We're seeing intermittent success. Most common issue appears to be 403 responses.
This request shows the response error:
https://gptogc.esri.com/geoportal/rest/distributed?rid=local&ridName=This%20Site&rids=local%2CdataGov&searchText=water&start=1&max=10&orderBy=relevance&f=atom
Exception when Posting CSW query to https://catalog.data.gov/csw-all: HTTP Request failed: HTTP/1.1 403 Forbidden
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<TITLE>ERROR: The request could not be satisfied</TITLE>
</HEAD>
<BODY>
<H1>403 ERROR</H1>
<H2>The request could not be satisfied.</H2>
<HR noshade size="1px">
Request blocked.
<BR clear="all">
<HR noshade size="1px">
<PRE>
Generated by cloudfront (CloudFront)
Request ID: fZtHiItoRuNkv0Wx9yrLUUXENvYEatwAkc0nyjPlEzOFmkKi0b2eJg==
</PRE>
<ADDRESS></ADDRESS>
</BODY>
</HTML>
The request is blocked
from geoportal-server-harvester.
kalxas
commented on September 26, 2024
This is not a response from pycsw but from a proxy/caching layer in front of data.gov
from geoportal-server-harvester.
mhogeweg
commented on September 26, 2024
we know. but we haven't been able to get GSA to resolve this
from geoportal-server-harvester.
Related Issues (20)
- Harvester not removing content from geoportal that has been removed from source WAF HOT 5
- Item type of tiled image layers in ArcGIS Image not properly maintained when harvesting into ArcGIS Portal/Online HOT 1
- Harvester Issue to ArcGIS Portal - The size of each typeKeyword cannot be more than 256 characters
- Translation for AGOL/Portal HOT 1
- Harvester CKAN Broker Iterator Error for Data.gov
- Upgrading to 2.7 issue HOT 2
- Parse markdown to HTML in metadata XML
- Associate harvested metadata to existing sub-layers HOT 1
- Enable ArcGIS Online/Portal authentication in the harvester HOT 2
- Support for records in ISO 19115-3? HOT 2
- Enable layers option on ArcGIS Portal input broker. HOT 2
- Use title as output file name
- include reference to source metadata when publishing fails
- Harvest full XML from ArcGIS Server services and layers when available HOT 1
- Use ArcGIS Server layer metadata if available
- translate metadata when harvesting into geoportal
- translate locale information when harvesting to ArcGIS Online/Portal HOT 1
- support harvesting from OGC API: Records
- give CSW input broker option to switch http client
- include explicit sign out from web app HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from geoportal-server-harvester.