Google indexing appears to have stalled out at 16k of the 69k profiles.

• Submitted to Bing and Yandex (Yandex to improve visibility on DuckDuckGo)
• Run through Pagespeed to improve initial load time
• Convert title to h1 tag
• Add SEO friendly links to profiles (e.g. /profiles/bill-and-melinda-gates-foundation) :: Jekyll Redirect From
• Add structured data

Ignore. This Epic created in wrong repo.

• Research most used filters on large foundation online grants databases
• v0 Design w/ example data
• Algolia brainstorm + sign-off on branding
  • Add UTM links
  • Switch placeholder partnership logo for crisper version

Open bugs and must-haves:

• Add a no results template to autocomplete
• Ensure grants search does not autofocus

Blocked until figure out how to add v3 currentRefinement widget using separate buttons (not combined)

Use case:
User searches for another grantee or subject area (e.g. "City Year" or "climate change")
Wants to quickly see who has donated

Hypothesis:
On mobile, folks don't really want to be doing power searches. They want to simply browse who is funding whom. Perhaps, either:
a) Show a button/link that shows the facet count, then expands into a list (most helpful is the organization list, as visitors could then click through to the profile).
b) Allow them to send themselves the url w/ search parameters (alt: just use platform-native share sheets)

Long term:
Download / share list

If using InstantSearch connector:
Good example

Already using Cloudflare to encrypt user <=> Cloudflare connection, but Github Pages now offers an encrypted connection from GP origin.

https://blog.github.com/2018-05-01-github-pages-custom-domains-https/

• Update A records - remove old two and enter new four (docs)
• Cloudflare > Crypto > Set SSL - Select Full SSL (not strict)
• Remove existing custom domain in Github repo settings - be sure to click "Save" (docs)
• Re-add custom domain in settings to trigger the HTTPS setup process on Github's end
• Ensure CNAME automatically updated
• Check Enforce HTTPS (when available)
• In profiles repo, check Enforce HTTPS

Note: Had to turn off HTTP proxy at CloudFlare to trigger SSL cert provisioning at Github (otherwise never gets past initial "domain not properly configured" stage)

Github Pages IP Addresses (for A records)

185.199.108.153
185.199.109.153
185.199.110.153
185.199.111.153

Tests

• Ensure naked redirect works (grantmakers.io => www.grantmakers.io)
• Ensure HTTPS is forced (http://www.grantmakers.io => https://www.grantmakers.io
• Ensure no issues with profiles repo (https://www.grantmakers.io/profiles)

It appears the 2018 json index file still has not been finalized (as of May 1 2019).

2018_index.csv => 457,509 records (last updated 2019-02-21)
2018_index.json => 432,801 records (last updated 2019-02-13)

990-PF Filings
67,632 CSV
64,764 JSON
2,868 diff

Tracking here: awslabs/open-data-registry#288

User believes organization is a good fit.

The foundation's application process is detailed in latest 990-PF. Instructions reference a form, but user is unable to find the form as foundation does not have a website. Foundation does list a mailing address, presumably to request the form via mail?

Specific user request:
"It would be a great help if you were able to include a link to the form on your site."

Reference:
https://www.grantmakers.io/profiles/v0/396076956-gardner-foundation/

The IRS-provided Index object concatenates names if two lines exist.

Example
https://www.grantmakers.io/profiles/010215216/

Consider pulling names from Filings instead of Index.

Applies to search results and profile header names.

Source code
https://github.com/smartergiving/irs-990-fetch/blob/master/normalize.js#L6

e.g. Filing:

"BusinessName" : {
                    "BusinessNameLine1Txt" : "CHINA LIBRARY ASSOCIATION",
                    "BusinessNameLine2Txt" : "ALBERT CHURCH BROWN MEMORIAL LIBRAR"
                },

Index:

"OrganizationName" : "CHINA LIBRARY ASSOCIATION ALBERT CHURCH BROWN MEMORIAL LIBRAR",

// Moved to proper repo (profiles)

• Map out how a search on these fields might work (e.g. 990PF charitable activity + grantee purposes + SmarterGiving tags)

• Add UTM parameters from redirects (link)

Currently have link to MongoDB algolia.bson file on README and anyone can download full profile data by forking Github repo. Anything else requires a visitor to request the data via email, providing an extra barrier to folks who just want to tinker.

Provide direct downloads of all useful data, particularly in MongoDB bson format (or alternative), to allow for more hackers, researchers, journalists, and academics to more quickly run specialized queries.

The hasRecentGrants flag was always a proxy for if a foundation is still active or not.

We will remove that flag and create a new isActive flag based on a more definitive source (e.g. Pub 78 or the EO BMF).

Use existing IRS schema, a custom/modified schema, or existing industry standard (e.g. hGrant, IATI, 360Giving)?

hGrant JSON
360Giving
IATI

Background
Tim Davies

PDF format for taxes is a hit-and-miss sometimes especially for 2016 tax returns for Foundations.
However it works well for 2015 tax returns in pdf format.
Here is one example of hit-and-miss pdf (2016) for Ina Kay Foundation, Inc.

Up until recently, AWS Indexes used uppercase.
Noticed this exception:
https://www.grantmakers.io/profiles/v0/582303901-grace-and-hope-foundation-inc/

Need to explore if this is a one-off or if AWS has changed indexing format

Current scripts assume a user is starting from scratch and downloading the IRS Form 990 dataset for the first time. Given the size of the files and the frequency of updates by the IRS, it's time to create scripts that focus on updating existing MongoDB collections.

The IRS provides both a unique identifier (DLN) and a timestamp (LastUpdated) in the index files.

• Create update scripts - currently located in the smartergiving/irs-990-fetch repo
• Improve Algolia upload sequence (see below)
• Update profile pages on Github Pages

Algolia

Repo
https://github.com/smartergiving/irs-990-fetch/blob/master/gcf_http/syncWithAlgolia/index.js

Discussion
https://discourse.algolia.com/t/indexing-large-file-mongo-collection-using-api-node/2719

Data
Full dataset can be downloaded here
Warning - 1GB+ file size

Nice-to-have
Allowing it to work as a Google Cloud Function (2GB memory limit and 9 minute execution limit)

Sample Index object from AWS

{
        "EIN" : "208164731",
        "TaxPeriod" : "201506",
        "DLN" : "93491316013475",
        "FormType" : "990PF",
        "URL" : "https://s3.amazonaws.com/irs-form-990/201523169349101347_public.xml",
        "OrganizationName" : "PLATTE VALLEY LEGACY FOUNDATION INC",
        "SubmittedOn" : "2016-02-05",
        "ObjectId" : "201523169349101347",
        "LastUpdated" : "2016-03-21T17:23:53"
}

Some foundations use the BusinessName field instead of PersonName under key employees and officers listings.

Early years:
https://s3.amazonaws.com/irs-form-990/201321299349101257_public.xml

Recent years:
https://s3.amazonaws.com/irs-form-990/201623159349100522_public.xml

Note:
Edited the early year profile manually as temp fix

One major challenge searching through many grants via Grantmakers Search is that some Foundations who will only accept preselected charitable organization, and do not accept unsolicited request for funds. The Foundations would indicate such practice on 990, Part XV, Item #2.

We want to search for the Foundations that would accept unsolicited request for funds via Grantmakers Search.

Stretch goal - Allow fetch.js to run via cloud functions (e.g. AWS Lambda or Google Cloud Functions)

Immediate goal - reduce memory footprint

Consider making use of ES8 async/await

• Mobile - add facets
• Mobile - fixed navbar for grant search header
• Mobile - lazy loading instead of pagination
• Switch back to static list of grants if a) Algolia is over quota, b) throws an error, or c) initial search has no results
  • Hide Algolia search bar
  • Hide Algolia wrapper (stats/hits/facets)
  • Disable fixed header

- [ ] Consider adding grantee_name facet - useful for users searching for project fit (e.g. org X has been supported by foundation y for past five years...what projects over the years have been supported?)

If OrganizationName contains a hyphen, profile url contains the hyphen (e.g. foo---var), but links to those profiles exclude the hyphen (e.g. foo-bar).

Bad links exist in sitemap and Algolia search results

2,545 foundation names contain hyphens

Examples:
https://www.grantmakers.io/profiles/v0/133877532-leonard-friedland-charitable-foundation-co-cherry-bekaert-llp/

https://www.grantmakers.io/profiles/v0/133195928-pollock-foundation-co-oscar-pollock---ingalls-and-snyder/

Existing front-end framework potentially restricting to certain grantmaking institutions.

Move to Materialize

e.g. 04ARTS FOUNDATION
https://www.grantmakers.io/profiles/742730665/

• Create new bucket and turn requester pays option to 'on' (docs)
• Add additional step to deploy scripts to also send data to requester pays bucket
• Detailed README with instructions (including limitations of dataset)

Size of files (gzipped):
1.5GB Normalized filings (2011-current)
0.2GB Grants (2011-current)

Storage cost negligible (only $0.05 / mo...$0.026 / GB)
Network costs and egress fees are the concern (fee schedule)

Alternative:

1. Use G-Suite storage (API docs)
   Q: What are the consequences if a developer constantly hammers the endpoint(s) programmatically?

2. Use a cloud function to disallow programmatic access. Would allow good actors access, but prevent runaway costs from programmatic hammering of endpoint.

The problem:
v2 of the currentRefinements widget allowed for a button per each refinement selection. v3 buttons have one button per facet category (e.g. multiple selections per facet appear on a single button). The result is the button will overflow off the page. This is particularly prevalent on mobile.

Unless Algolia adds the templates option back to the currentRefinements widget, will require lots of additional connector code.

Adding to #someday for now as very few visitors use multiple facets on mobile.

The alternative is to simply hide currentRefinements on mobile (and perhaps add a filter count to direct user back to refinements slide-out)

May require a new UX strategy on mobile (e.g. hide top/intro element on focus)

Possibly related

Hey! This is an interesting and useful project. I noticed there's some potential bugs with the index page. There are repeating entries for some place entities. Puerto Rico and British Columbia both show up twice, for example:

I'm guessing these entries show up twice because either it's case sensitive (e.g. BRITISH COLUMBIA vs. British Columbia) or abbreviations are used in some but not in others (e.g. Puerto Rico vs. PR).

There's also some odd entries that are obviously not U.S. states, e.g. Foreign, Beijing, England, Ontario, etc. I would suggest ordering the U.S. states first, then list the odd entries, since I think most users would look for U.S. states first.

If an organizations changes their name from one tax year to the next, the url of the profile changes on Grantmakers.

Google eventually catches up, but the longer-term fix may be using the org name from the filing instead of the index. The index includes the second line of the business name (e.g. "co John Public").

• Standardize on single name-style (e.g. Index or Filing or ??)
• Create redirects if name has changed (307 redirects)
  • Quickly explore possible alternatives to Jekyll redirects (e.g. Netlify's _redirects feature)
  • Update scripts and re-run profiles
• Considering removing CO and ATTN (5,520 profiles)

Examples

New Name
https://www.grantmakers.io/profiles/v0/912154616-ronald-and-ann-williams-charitable-foundation-co-richard-newton/

Old Name
https://www.grantmakers.io/profiles/v0/912154616-ronald-and-ann-williams-charitable-foundation/

New Name
https://www.grantmakers.io/profiles/v0/510137200-san-jose-symphony-foundation/

Old Name
https://www.grantmakers.io/profiles/v0/510137200-san-jose-symphony-foundation-co-marc-rebboah-treasurer/

New Name
https://www.grantmakers.io/profiles/v0/800727539-turner-farm-foundation-inc-co-robert-g-edmiston-trustee/

Old Name
https://www.grantmakers.io/profiles/v0/800727539-turner-farm-foundation-inc-robert-g-edmiston-trustee/

Material Kit is now licensed under MIT, so item 4 in the FAQs can be simplified

Two recent 'no results' queries:

"a.	Enhanced Monitoring and Increased Protection of [protected for privacy] in [country - protected for privacy]"

"[ethnic population] health care [XYZ county], [state]"

Would be super cool to run through Google Natural Language API, parse, and send a suggestion (e.g. click this button to use our suggested filters)

FeedbackLite widget not generating many responses

Consider Intercom.io (or Drift.com, et al)

See also GetSiteControl

Allow foundations to quickly get a grants database online by creating a stand-alone repo.

If PersonNm has an XML attribute, an object is captured (see normalize.js). Profile template assumes Name is a string, not an object.

Example:
https://www.grantmakers.io/profiles/476261291/#people

Button currently shows 'invalid campaign code', likely due to them changing the ability to use custom buttons (?).

Help funders get the word out about RFPs

• FAB modal requires scroll on mobile (small only)
• FeatureDiscovery title bleeds over on some mobile devices (iPhone 6/6s/7, Galaxy S5, and smaller)

Example:
https://www.grantmakers.io/profiles/931307278/
https://s3.amazonaws.com/irs-form-990/201321299349101257_public.xml

child of grantmakers/profiles#23

• Add messaging to educate visitors that a) it might be found in the PDF version and b) we show exactly what is provided in the electronic 990PF

• Does the IRS publish the files electronically?

  • If so, add files to fetch script
  • If not, add messaging throughout stating grant detail is not available?

• Examples

  • https://www.grantmakers.io/profiles/366058062/
  • https://www.grantmakers.io/profiles/931208721/

Grantmakers.io was currently built entirely with the nonprofit fundraising professional in mind. Turns out many scholarship seekers are also using the site.

This could be as simple as adding the "grants purpose" field to search refinements combined with a brief tutorial that is scholarship specific.

TODOs

• Market research: I'm surprised this need isn't already being fulfilled via other mechanisms
• Define v0

• Blog post
• Social media messaging

e.g. by city/state

Primary goal is to improve SEO. Algolia search remains best method for exploration for users starting from the landing page.

Currently Profiles use Materialize
All other pages use Material Kit

Move to single front-end framework to allow for faster deploy of new pages

parent of #2
parent of #25
parent of grantmakers/grants-search#3

Ability to search through all 3.2m grants contained in electronic IRS 990-PF

Has Recent Grants is true if the organization's most recent electronically filed 990PF has any grants.

Need to consider organizations that used to file electronically, but no longer do (e.g. should grants from tax year 2015 really be considered as "recent"?).

Particularly important as more foundations move to DAFs (e.g. EIN 452523468)

See normalize.js

1. Input bleeds over on right edge
2. Navbar scrolls away if no text entered
3. Works as expected if input has text

UPDATE:
HTML5 autofocus attribute not supported by iOS...need new navbar approach on webkit mobile

Waffle.io service is shutting down May 16, 2019

Need to move to Github Pages (migration link) or alternative like Zenhub.

Decision: Moving to Zenhub

• Add goals (Issues)
• Add initiatives (Milestones)
• Consider adding Type: enhancement, Type: bug, Type: infrastructure to Issues
• How to integrate User Story / Job Story? (inspiration)
• Per above, consider adding a Github Issues template for Epics
• Add remaining Trello cards (board)
• Add new Epic that covers Add Charitable Activities...Enhance Search?
• Add Orchard label?