Clean some data, win some swag

The "St. Albnas" problem has become quite a meme on the internet. It seems to have originated here (h/t to Adam Sroka).

The meme captures a well known issue with data quality: Free-text fields aren't consistent!

Write some code (SQL, GPT, regex, whatever you want) that will accurately, precisely, and consistently converge all of the elements in the positives.txt file to the correct spelling: St. Albans.

The code should repeatably converge as many of the entries as possible into the correct spelling while also avoiding false positives (e.g. you can't just replace the entire string with St. Albans everytime by brute force). To ensure that your code avoids false positives, we've included a negatives.txt file. Your code should avoid converting any of these to St. Albans.

• To submit an entry, clone this repo, checkout a new branch, and submit a pull request.
• Use whatever language you want, whatever libraries you want, whatever. But it must be code and it must compile. (You can use GPT or any other LLM, but simple text GPT prompts will not be accepted!)
• Any entry that converts the elements by brute force (i.e. by individual string matching) will be rejected.
• Valid entries must include:
  • All necessary code to do the data cleansing
  • A README explaining how to run the code over the included .txt files.
  • A demonstration of the results you achieved (image, file, etc.) that should be repeatable
  • Optional: Include your Twitter handle in your GitHub profile if you'd like to be mentioned on Twitter.
• Submissions are due by Friday, July 21st at 5 PM GMT
• Tinybird employees may not participate

Aim for Accuracy (ACC), measured as (true positives + true negatives) / (all possibilities).

For example, a submission that accurately converts 15 out of the 17 elements in positives.txt and accurately ignores 24 out of the 25 elements in negatives.txt would score (15 + 24) / (17 + 25) = 92.8%

Submissions that correctly convert all 17 of the elements in positives.txt and none of the 25 elements in negatives.txt to St. Albans would score a 100%.

Let's be honest, this problem isn't that hard, but it should be fun! All you need to do is submit a working attempt, and you'll get $20 off at The Tinyshop. That's enough for a t-shirt, a coffee mug, or 2 sticker sheets!

Also, as an incentive to score well, we'll tweet the final leaderboard when this ends :).

• Submit a valid entry (see above)
• Star this repo
• Follow Tinybird on Twitter @tinybirdco and/or LinkedIn
• Share your submission on Twitter/LinkedIn using #stalbnashackathon (tag us too!)

Join our Community Slack!

It's a very simple project using Lucene to compare the texts against the canonical "St. Albans" ;) 100% of effectivity without reinventing the wheel XD