Generates data using faker and our meta data schemas. This package expects meta data that conforms to the ETL Manager schema repo here
from data_generator.data_generator import MetaFaker
meta = {
"columns": [
{
"name": "my_int",
"type": "int",
"minimum": 10,
"maximum": 20,
"nullable": True
},
{
"name": "my_character_enum",
"type": "character",
"enum": ["a", "b", "c"]
},
{
"name": "my_email",
"type": "character",
},
{
"name": "my_datetime",
"type": "datetime",
}
]
}
sc = {
"my_email": "email",
}
mf = MetaFaker(meta=meta, special_cols=sc)
mf.generate_row()Would return something like this:
{
'my_int': 18,
'my_character_enum': 'a',
'my_email': '[email protected]',
'my_datetime': '1988-05-24 10:00:57',
}
Can also write to jsonl or csv
#csv
mf.write_data_to_csv("test.csv", total_rows=10)
# jsonl
mf.write_data_to_jsonl("test.jsonl", 10)In the above example you could see that we stated that the my_email column should produce email-like strings in the following way.
...
sc = {
"my_email": "email",
}
...The sc parameter works for the key/value pair where the key is the column name (in the data to be generated) and the value is a string that is the name of the provider in the Faker package that is called under the hood of this package (this line shows how).. To find what other types of specific strings you can generate e.g. address, name, last_name, etc. Please look at Faker providers.
If you want to get the same data every time you run, set the seed.
mf = MetaFaker(meta=meta, special_cols=sc)
mf.seed = 888As of v0.0.4, this package defaults to British special characters where they are available (e.g. address). This can be changed by:
mf = MetaFaker(meta=meta, special_cols=sc, locale="en_US")For a full list of locales, see https://faker.readthedocs.io/en/master/locales.html.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent