kanedata / find-that-charity Goto Github PK

Issue only seems to occur when charity is registered with both CCEW and OSCR.

@drkane if you check the branch at https://github.com/NICVA/find-that-charity/tree/es_query_yaml I've made a couple of changes that might make ongoing maintenance easier. Let me know if you like the look of it and I'll merge it into NICVA/master and submit a PR.

1. Reconfigured the ES search query json so that it is defined in it's own yaml file es_config.yaml which will be read in to the server command and converted to the json string. I find yaml easier to read and adapt than a json string, so good for ongoing development.
2. Added the CCEW, CCNI and OSCR charity number fields to the search query (see these lines). Means that a user can enter a known charity number in the search box and get a match.
3. Separated the ES search json for reconcile service, as it's likely that this will take different arguments than the web service version (e.g. names only). As with the above it's defined in its own yaml file recon_config.yml, and it's used in /reconcile GET and POST now instead of the previous search query.

From user

you mentioned that the service requires for NI charity numbers to be pre-pended with ‘NIC’. I’m afraid to say this runs contrary to how we record NI charity numbers (without the NIC) and the file required a bit of tweaking to make it work. I know that CCNI records charity numbers without the NIC and the Find that charity search works in a similar way (although the URLs for relevant charity pages all contain the ‘NIC’, oddly enough – e.g. https://findthatcharity.uk/charity/NIC100012). We actually have a data cleansing routine to remove ‘NIC’ from charity numbers too. It’s not the end of the world – it just adds an extra step to the process.

see https://github.com/OpenRefine/OpenRefine/wiki/Data-Extension-API

NI LA have LAN prefix here: https://dev.findthatcharity.uk/orgid/source/lan
It should be GB-LANI http://org-id.guide/list/GB-LANI

To allow autocorrelation/suggestion based on user input.

Would need to incorporate typeahead functionality.

For example of use (in jQuery): http://api.jqueryui.com/autocomplete/#option-source

https://www.elastic.co/guide/en/elasticsearch/reference/current/search-suggesters-completion.html

See implementation in similar service here:

• kanedata/find-that-postcode@38dc3be
• kanedata/find-that-postcode@f67b424

Search results / reconciliation suggestions put XI-GRID results higher than official registers
https://dev.findthatcharity.uk/?q=queen%27s+university+belfast&orgtype=all.

Local authority search results can show the GB-SHPE- results before the government register results.
https://dev.findthatcharity.uk/?q=%22manchester+city+council%22&orgtype=all

Search results / open refine suggestions should ideally prioritise IDs by org-id list % criteria or a defined ‘primary identity’ - eg a registered charities primary identity is charity, even if it is also registered as a company or social housing provider. Local authorities primary identity is local authority, even if they are also a social landlord or a research org.

• date of registration (how old)
• company number
• date removed (/is active)
• website
• org-id
• oscr number
• ccni number

• what it's for
• who it's for
• why use it instead of another tool
• roadmap/vision - future plans/ideas
• v0.2 plans
• how does it connect to 360 giving

I find that in these lines https://github.com/TechforgoodCAST/find-that-charity/blob/master/data_import/create_elasticsearch.py#L8-L28, because mapping is a tuple, that INDEXES[0]["mapping"][0] = args.es_type fails (tuples being immutable).

Changing mapping from a tuple to a [list] solves the problem and doesn't seem to create any problems down the line.

Perhaps as a result of reconciliation process?

Maybe just CICs and CLGs

Hi @drkane

I think you might be familiar with http://org-id.guide? What I propose is adding the prefixes so that we have org-id format identifiers for the relevant registers appearing on the charity's page and as an object in the reconciliation api.

• Company House: GB-COH-[company number]
• England & Wales: GB-CHC-[CCEW charity number]
• Scotland: GB-SC-[OSCR charity number]
• NI: GB-NIC-[CCNI charity number]

... where those exist.

P.S. great job on getting the finder online.

eg if a charity record hasn't been imported but the charity does exist, then scrape the data from charity commission site

https://github.com/drkane/find-that-charity/blob/master/static/js/components/Reconcile/Download/Link.jsx#L94

Ltd > Limited
& > and
Remove apostrophes '
The >
YMCA
Charities > Charity
Inc > Incorporated

"This website is supported by 360 giving"

https://github.com/OpenRefine/OpenRefine/wiki/Suggest-API

Does using quotation marks make a difference to the search?
How does the closeness of the name match relate to position in results vs weighting for the different lists.

Clarity about ownership

Including filtering organisations?

Moved to: https://github.com/drkane/find-that-charity-scrapers

Make sure removed from readme.md too

To the organisation info page

So we can spread the word about it

https://github.com/OpenRefine/OpenRefine/wiki/Data-Extension-API

Currently not possible to use elasticsearch versions above 2.x with dokku. See:

• dokku/dokku-elasticsearch#53
• dokku/dokku-elasticsearch#64

https://github.com/drkane/find-that-charity/blob/master/static/js/components/Reconcile/Download/Link.jsx#L111

Links

This section should be just for further links, as there are hyperlinks via numbers themselves. Re-name ‘Further links’ and move to lower on page.

Data Sources

Can this be combined with data source in boxes - eg link to access the data link or anchor point in table using this format on is page https://dev.findthatcharity.uk/about#data-sources

Locations

What is the source of this data - area of operation, or based on postcode? Not sure of the utility of having this on the record pages.

Just leaving this up as an issue that isn't working for me but might work for others, would be good to get feedback.

though /reconcile does define the preview window as per the OpenRefine Preview API, I can't get it to work (i.e. display the preview iframe on hover over a reconcile candidate) - though preview windows do work for other services (e.g. OpenCorporates).

{
	"name": "charitysearch",
	"identifierSpace": "http://rdf.freebase.com/ns/type.object.id",
	"schemaSpace": "http://rdf.freebase.com/ns/type.object.id",
	"view": {
		"url": "http://localhost:8080/charity/{{id}}"
	},
	"preview": {
		"url": "http://localhost:8080/preview/charity/{{id}}",
		"width": 430,
		"height": 300
	},
	"defaultTypes": [{
		"id": "/charity",
		"name": "charity"
	}]
}

and this preview itself also works fine outside of OpenRefine, e.g. http://localhost:8080/preview/charity/NIC100012

How to implement this?

FWIW, we scrape some data from the NI Charity Commission website - data that doesn't appear in the downloadable .csv (e.g. number of volunteers, trustees and employees, regions it operates and free text description of what charity does, how it meets public benefits test and its charitable purposes etc).

Morph.io scraper and data here https://morph.io/BobHarper1/charity-commish-ni

Might be useful for people interested text mining, also could provide more info to this search tool.