Giter Club home page Giter Club logo

Comments (9)

proffapt avatar proffapt commented on August 30, 2024 2

I will automate the shit out of it :D

from chillzone.

harshkhandeparkar avatar harshkhandeparkar commented on August 30, 2024 1

tag hi banado na jab release nahi karna toh ¯_(ツ)_/¯

Mujhe laga tag mein date nahi dikhti. Apparently dikhti hai. Chalega tag bhi.

from chillzone.

proffapt avatar proffapt commented on August 30, 2024

A tag is better, if not going for a release.. because writing release notes is pain in itself :/

from chillzone.

harshkhandeparkar avatar harshkhandeparkar commented on August 30, 2024

A tag is better, if not going for a release.. because writing release notes is pain in itself :/

We don't necessarily have to write release notes. Making a release creates a tag anyway, and we can keep track of changes.

from chillzone.

proffapt avatar proffapt commented on August 30, 2024

tag hi banado na jab release nahi karna toh ¯_(ツ)_/¯

from chillzone.

shikharish avatar shikharish commented on August 30, 2024

It is not possible to update data using a github actions workflow.

  • For 1st year timetable, the PDF parsing is not accurate for some pages and need to be fixed manually.
  • Even for other years, sometimes the slots format may change in ERP(like it did in Spring 24)

from chillzone.

harshkhandeparkar avatar harshkhandeparkar commented on August 30, 2024

It is not possible to update data using a github actions workflow.

* For 1st year timetable, the PDF parsing is not accurate for some pages and need to be fixed manually.

* Even for other years, sometimes the slots format may change in ERP(like it did in Spring 24)

In that case it would be good to open a PR using the workflow. The maintainers can manually review and make changes to the PR.

from chillzone.

shikharish avatar shikharish commented on August 30, 2024

The current workflow is:

  1. Get 1st year pdf from ERP
  2. Run update script(it parses pdf file to xlsx file)
  3. got errors in parsing
  4. check what is the error in xlsx file(usually a column is parsed twice)
  5. manually fix errors(in code not xlsx file)
  6. run update script again
  7. 1st year scraper run successfully and it creates a json file will all 1st year data
  8. for other years scraper, erp-login-go package is used and it would prompt for credentials and otp. After successful login, all data would be fetched. It updates the json file created in previous step.

from chillzone.

harshkhandeparkar avatar harshkhandeparkar commented on August 30, 2024

The current workflow is:

1. Get 1st year pdf from ERP

2. Run update script(it parses pdf file to xlsx file)

3. got errors in parsing

4. check what is the error in xlsx file(usually a column is parsed twice)

5. manually fix errors(in code not xlsx file)

6. run update script again

7. 1st year scraper run successfully and it creates a json file will all 1st year data

8. for other years scraper, erp-login-go package is used and it would prompt for credentials and otp. After successful login, all data would be fetched. It updates the json file created in previous step.

Many of these steps can be automated. If there is an error, the code can be updated and the rest of the workflow can still be automates.

from chillzone.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.