Feature: Switch from CSV to sqlite3 about github-stats-pages HOT 2 OPEN

The referrer tables do not have a timestamp. As such, we need to merge them and include the date the code was run (via the filename). I'm looking for my case whether there are duplicates with a simple merge. In most cases, the records are same with duplicate entries except for the following:

2021-03-17,Evolution-of-Galaxies,github.com,6,1 (first entry gave 4,1)
2021-03-17,MZEvolve,github.com,2,1 (first entry gave 1,1)
2021-03-19,MMTtools,github.com,2,1 (first entry gave 1,1)
2021-03-19,Zcalbase_gal,github.com,9,2 (first entry gave 8,2)

from github-stats-pages.

One limitation right now with this feature is that with each run, there are overlap with SQLite db. We need to address this by ensure that the new numbers are used.

There was an ask for this in pandas: pandas-dev/pandas#14553; however, this was never adopted/implemented. I think we would need to do SQLModel updates directly here.

from github-stats-pages.