Implement dataset deletion on organization/harvest-source removal about data.gov HOT 4 OPEN

kicking this back to the board so we can focus on tickets that achieve our MVP-load-test

from data.gov.

CKAN has a bulk delete. This could prove helpful: https://github.com/ckan/ckanapi#bulk-delete

from data.gov.

The current API route for deleting a harvest source removes all related entries in cascaded tables such as job, record, and error. A CKAN remote call has been added to delete all datasets in the CKAN database associated with the package IDs (ckan_id) from the harvest_record table.

from data.gov.

Using ckan.action.dataset_purge(id=pkg_id) for 1000 datasets takes about 2 minutes.

However, with the CKAN API call ckan.action.bulk_update_delete(datasets=ckan_ids, org_id='xxx'), it only takes 25.77 seconds to process 1097 datasets.

from data.gov.