Comments (1)
Something like this:
from sqlalchemy import create_engine
# etc....
def get_engine(db_name):
# Create the connection to the database
engine = create_engine(f'postgresql://postgres:postgres@db:5432/{db_name}')
return engine
def import_table(dbf_file, table_name, db_name):
try:
# Attempt to read the dbf file with utf-8 encoding
table = DBF(dbf_file, load=True)
df = pd.DataFrame(iter(table))
# # Print a preview of the dataframe
logging.info(f"Preview of '{table_name}':\n{df.head()}")
# # Convert DataFrame to SQL
engine = get_engine(db_name)
df.to_sql(table_name, con=engine, schema='import', if_exists='replace', index=False)
# --------------------------------------^^^^^^^^^^^^^--------------
except Exception as e:
# Broad exception handling to catch any other unexpected errors
logging.error(f"Unexpected error during import of file {dbf_file}: {e}. Investigation needed.")
will work, and import the table(s) in a schema named import
just create the schema in the database in your db before hand CREATE SCHEMA import IF NOT EXIST;
or some such before hand.
A
from dbfread.
Related Issues (20)
- error occured when parsing a 'N' type field which is mistakenly defined as ‘C'
- Generalize the --encoding-xlsx input option for other file types in in2csv
- UnicodeEncodeError for special charater HOT 2
- dbfread read hidden rows
- Error parsing dates HOT 1
- Replace test system HOT 1
- Return dict instead of OrderedDict as default from 3.7 and up HOT 1
- Better tests for field parsing HOT 2
- Create debug tools for inspecting unsupported or broken files
- Break up code into functions that can be composed in different ways
- Support for dataclasses? HOT 1
- Add support for reading from a (non-seekable) file object instead of a filename HOT 1
- New PyPI release and release-checklist.rst file
- Throwing exceptions if column name contains commas
- UnicodeDecodeError: 'charmap' codec can't decode byte 0x8d in position 65: character maps to <undefined> HOT 1
- UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 HOT 1
- DB4MemoFile field termination.
- ValueError: Field type I must have length 4 (was 0) HOT 1
- ValueError: could not convert string to float: b'60.00\x00\x00' HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dbfread.