Comments (7)
@Jin-Sun-tts The error you posted is the same in this issue:
from data.gov.
https://github.com/GSA/catalog.data.gov/actions/workflows/snyk.yml
from data.gov.
Friday 10/23
https://github.com/GSA/data.gov/
Check Catalog Auto Tasks
-
DB-Solr Sync:
87 packages need to be removed from Solr
880 packages need to be updated/added to Solr
225 packages without harvest_object need to be mannually deleted
Finished 826s -
Tracking Update:
2023-10-27T16:40:20.87+0000 [APP/TASK/ckangeodatagovtracking-update-6664271318-2/0] OUT 2023-10-27 16:40:20,875 INFO [ckanext.geodatagov] 6854 package indexes to be rebuilt starting from 2023-10-25 00:00:00
from data.gov.
Monday 10/23
https://github.com/GSA/data.gov/
Check Catalog Auto Tasks
-
DB-Solr Sync:
0 packages need to be removed from Solr
1173 packages need to be updated/added to Solr
186 packages without harvest_object need to be mannually deleted
Finished 588s -
Tracking Update:
2023-10-23T07:39:17.47+0000 [APP/TASK/ckangeodatagovtracking-update-6610117038-1/0] OUT 2023-10-23 07:39:17,470 INFO [ckanext.geodatagov] 5883 package indexes to be rebuilt starting from 2023-10-20 00:00:00
Check Harvesting Emails
- Catalog:
Other
Two issues found from New Relic logs:
-
catalog web proxy error for url http://www.google-analytics.com/collect#4508
We have added www.google-analytics.com to the allow list, see the #4509.But in the ckanext-googleanalytics code, HTTP instead of HTTPS is used. We need convince ckan upstream to use HTTPS.
-
disable new dataset creation on catalog#4161
anything starts with /dataset/new is blocked as in /dataset/new-hampshire-and-southern-maine-ocean-uses-atlas.
Checked catalog, inventory production, works fine.
Also checked Solr leader and followers, all work as normal.
from data.gov.
Snyk vulnerabilities check failed at synk test
Running setup.py develop for ckanext_googleanalytics
above errors happened since Oct 4th which just after enable ckanext_googleanalytics
from data.gov.
Above snyk error were fixed, will double check tomorrow morning.
Check catalog auto tasks:
- DB-Solr Sync:
0 packages need to be removed from Solr
883 packages need to be updated/added to Solr
190 packages without harvest_object need to be mannually deleted
Finished 628s
Check Harvesting Emails
- Catalog:
Other
The two issues found on Monday also be fixed and confirmed from New Relic log.
from data.gov.
harvest run failed : GSA/catalog.data.gov#1121
2023-10-27T13:55:52.02+0000 [APP/TASK/ckanharvesterrun-6668092691-1/0] ERR ckan.lib.search.common.SearchIndexError: Solr returned an error: Solr responded with an error (HTTP 500): [Reason: Server error writing document id 7b08d7b851606bed7c34f955569a891c4c05cff9 to the index]
after restarted solr leader, harvester rebuild index works.
rerun db solr sync and tracking update after above rebuild index issue fixed.
from data.gov.
Related Issues (20)
- Clarify options for download json in inventory.data.gov HOT 2
- SNYK finding: SNYK-PYTHON-CRYPTOGRAPHY-7161587 HOT 3
- synk finding: SNYK-PYTHON-PYOPENSSL-7161590 HOT 1
- Reconfigure GTM to ingest org and publisher HOT 1
- DCAT-US Load Test in Cloud.gov HOT 1
- O+M 2024-06-10 HOT 4
- Harvesting2.0 conflicting name with CKAN
- Create Clear for Harvest2.0 Harvest source
- Migrate content from federation.data.gov to https://resources.data.gov/resources/us-data-federation/
- Redirect all url endpoints of federation.data.gov to resources.data.gov/resources/us-data-federation/ HOT 1
- H2.0 Production Release
- Save CKAN ID to HarvestRecord HOT 4
- Port Harvester UI elements to Flask App
- HTML meta tag content should be escaped HOT 1
- uswds-compile reverted back to [4.0.2](https://github.com/uswds/uswds-compile/commit/74ddb4ca8ce5fec113a152265d31fb2450d33d65) recently
- Fix harvest_source notification_emails field HOT 2
- Setup/cleanup mdTranslator mini-app HOT 6
- O+M 2024-06-17 HOT 1
- re-evaluate paging loading speed after ckan 2.11 release in catalog-next HOT 1
- O+M 2024-06-24 HOT 7
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from data.gov.