18vijayb / erdrecognition Goto Github PK

View Code? Open in Web Editor NEW

0.0 0.0 0.0 162.52 MB

Python 100.00%

erdrecognition's People

Contributors

Watchers

erdrecognition's Issues

Use PyTesseract to read cut-out entities, relationships, and attributes

Run PyTesseract code on each file in the directory created from earlier task, and read text associated with image. Return a json file formatted in the issue from earlier.

Get the dataset, and split it into training and testing

Use Tensorflow to split the dataset. Also, add dataset to the repository

Update README

Fill out README with problem statement and overarching goals of the project

Remove text from dataset images

Recognizing Relationships on ERD

Make POST request to GCP endpoint for each image

Send each image to endpoint, retreive json, and count entities and images. Store it in the format as mentioned in the other issue

Recognizing Entities on ERD

The goal is to recognize entities based on ERD diagrams. Read up on TensorFlow to see what is necessary to recognize the "boxes" in each diagram

Install TensorFlow and deps locally

Convert each image to a json file as formatted below

{
  "filepath": "PATH_TO_FILE",
  "filename": "0001.jpg",
  "numEntities": 5,
  "numRelationships": 4,
  "numAttributes": 2,
  "entities": [
    {
      "name": "Food Item",
      "primaryKey": "Name",
      "numEntityAttributes": 2,
      "entityAttributes": [
        "Price", 
        "Number of Calories"
      ]
    }
  ],
  "relationships": [
    "Usual Side",
    "Supervises",
    etc.
  ],
  "attributes": [
    "String 1",
    "String 2",
    etc.
  ],
}

Associate start and endpoints with entities and relationships

Using the coordinates of endpoints, associate the entities with each endpoint, whether it is a start or end, as well as the relationships. This can be done using a normal distance comparison between each entity and a given point. N^2 time.

Extract images from bounding boxes specified in JSON using PIL

Extract entities, relationships, and attributes, into separate images for OCR with pytesseract.

End goal: Images should be extracted in a subdirectory named after the image name, located in the same directory

Label dataset for model

Annotate images for starting and ending points of lines

Annotate approximately 25 images on GCP and train the dataset
After training, evaluate the performance of the model by deploying it
If performance is not up to benchmark, annotate 25 more images and train until accuracy is achieved

Use OpenCV to connect the dots

Mask out all white from the image, and take a starting point and an ending point.
Is there a path between the two points?
If not, take another ending point. Repeat until a starting point has been paired with an ending point. Add this pairing to a json file formatted like so

[
  {
    "start": *COORDINATES FOR STARTING POINT*,
    "end": *COORDINATES FOR ENDING POINT*I,
  },
  etc.

Recognizing Attributes on ERD

Write clustering algorithm based on number of entities, relationships, and attributes

POST each picture to the GCP endpoint
Get the number of entities, attributes, and relationships
Associate the image name with the file
Perform K-means clustering algorithm on all images

End goal: Obtain a graph of all the clusters of images based on entities, attributes, and relationship.

18vijayb / erdrecognition Goto Github PK

erdrecognition's People

Contributors

Watchers

erdrecognition's Issues

Recommend Projects

Recommend Topics

Recommend Org