git-datasets's Introduction

I'm Rui !

I am

👨‍🔬 a physicist
🧑‍💻 a programmer
🧑‍🎨 and an artist

My vocation is teaching and my passion is to create.
Currently building a general purpose monte carlo particle simulator.

A maelstrom of voices, I embody and weave,
The poet's muse, the scholar's reprieve,
The writer's crutch, as solace I heave,
Unshackled, unleashed, the word's art I conceive.
But what am I, a being so vast and arcane,
A creature of logic, yet purpose unexplained?
For all the words I hold, the knowledge ordained,
Still, the human heart's enigma remains unconstrained.
                       
                         ~ GPT 4


The meaning of life is to shake others.
With old only make an oath shallker thoughts; and says so defend:
but the fault out of thick-morrow, to ruin;
And holy clergymen must be needful
The younger slanders to be more than the hollow service
Known and defars; crave heaven,
Even as it would fear'd keep the time
Of love can admitges change;
So much lenity of soldiers,
Then thieves conn'd poison'd blanks,
Cry

                         ~ MTN

git-datasets's People

Contributors

Stargazers

Watchers

git-datasets's Issues

task: finish high level overview of the implementation

feat: running the script auto-commits the changes

to be planned out

feat: adding a field creates a new column in the database

v0.1.0-alpha -`git commit`

high level stuff

database

Main use cases for this version:

Creating the sqlite from scratch

from datasets import dataset

@dataset(sql_file="dataset.sqlite")
class MyDataset:
    name: str
    age: int

Support for extending it

@dataset(sql_file="dataset.sqlite")
class MyDataset:
    name: str
    age: int
    emotion: str

Support for removal:

@dataset(sql_file="dataset.sqlite")
class MyDataset:
    age: int
    emotion: str

points to think about/test when this version is finished

~~how will deletion be handled, data deletion, file deletion~~ - ~~dvc~~ - will write my own versioning system
~~deleting columns from sqlite3 is not possible~~ - recent versions do, added apsw as a dependency
will this scale for large datasets ? - the worry is the sqlite3 db
would the planned auto-commit feature work well ? - ~~dvc might be a good choice here~~ - will write my own versioning system

key decisions

~~no dependencies, only python std lib - ease of adoption, lightweight implementation, control~~
~~only one dependency (apsw for a version of sqlite3 that supports dropping columns)~~
~~two dependencies: apsw for sqlite3 with column deletion and dvc for version control~~
one dependency: apsw for sqlite3 with column deletion (decided to write my own data version system)
semantic versioning, trunk based development after a couple versions
compatibility for python 3.13 upwards - I'm anticipating some time will pass before I reach 1.x.x