mouse-reeve / graph-books Goto Github PK

Test pairs of books that I think should be next to each other

• The Golem and the Ginni and The Good Thief
• Dune and Foundation
• The Forever War and Ender's Game
• Shade's Children and Under the Skin

It takes a million years and makes at minimum 3x too many queries.

this is some real old code

This means a total re-structuring of the database. But I think it's the right way to go.

I have to handle the bad/dupe data in the author field in order to address this. Also translators, editors, &c

It would be pretty sweet, right?

Right now it's not good for anything. Not sure if there's anything worth doing with it.

Presumably I can download it from github in the updateDatabase script, rather than storing a copy in this repo as well. It changes pretty often, after all.

Depth first search creates disjointed jumps when a branch is completed and a sibling branch starts.

It's ridiculous that it isn't a mapreduce job. Shouldn't be in this repo.

Writing to a file is sufficient

In the broader sense, rely less than the rest client.

Not just year published, but decade and maybe century

esp. for references, which won't produce any useful results when treated as a literal string.

Right now, it includes all list-type fields and ignores strings and integers. Which is pretty arbitrary.

So much new data!

It's data that is covered by the scraper, but for updating tags on existing items, it's WAY quicker.

There's too much in python file.

Use good-practice relationship names and include a field for grammatical names. Also consider adding a weight scale.

Publisher < Recommender < Author?

The scripts as they stand are only good for making a db from scratch

Might as well get it all from the scraper, rather than doing all this csv manipulation, since I'm poking around there anyway. Currently tags are coming from the csv, while character, original publication date, and locations are coming from the scraper.

Depends on mouse-reeve/book-scraper#2

With ISBN as a unique identifier field, so that if the database needs to be rebuilt from scratch, all this data isn't lost.

esp in the booksOnly graph creation. I'm pretty sure it's technically possible to do that entire monster in one horrifying query. maybe not one, but at least it could be less of a monster

They completely changes exports.

• CSV -> JSON
• Now includes purchase location, other fields that I previously had to scrape

Hardcoding numerical values to a list of presumed labels seems real bad.