Comments (14)
maxachis
commented on August 11, 2024
1
Haven't heard from Eva on this since June. We may want to turn this over to someone else--possibly using Eva's existing code.
from city-scrapers-pitt.
bonfirefan
commented on August 11, 2024
Reference the old spider here
from city-scrapers-pitt.
ben-nathanson
commented on August 11, 2024
I'll take this issue. I have a simple proof-of-concept script written that just gives us each event's title and time. The original spider appears to point to an older version of the site. I should have a PR ready for this by the end of May.
from city-scrapers-pitt.
bonfirefan
commented on August 11, 2024
@ben-nathanson, can you collab with @guesschess on this?
from city-scrapers-pitt.
ben-nathanson
commented on August 11, 2024
Sure! We've gone over some existing code, discussed some of the technical quirks of navigating the port authority site, and will meet up as needed to work on this.
from city-scrapers-pitt.
ben-nathanson
commented on August 11, 2024
@guesschess is making good progress on this - we're at a working scraper script, the next steps are isort/flake8 formatting and writing test methods.
from city-scrapers-pitt.
ben-nathanson
commented on August 11, 2024
Please note that the website has changed to: https://www.portauthority.org/inside-Port-Authority/Port-Authority-Board/Board-Meeting-Information/
This change combined the events and documents into one nice comprehensive page
from city-scrapers-pitt.
guesschess
commented on August 11, 2024
@ben-nathanson What do we do about the end time? We don't know how long it will take.
from city-scrapers-pitt.
ben-nathanson
commented on August 11, 2024
If we don't have any specific information about the end time, and I think that is the case here, I would add three hours to the start time
(Edit): See below. Thanks Bonnie!
from city-scrapers-pitt.
guesschess
commented on August 11, 2024
@ben-nathanson Thanks!
from city-scrapers-pitt.
bonfirefan
commented on August 11, 2024
@guesschess you can leave the end time as None if there no end time. Glad you all are making progress!
from city-scrapers-pitt.
maxachis
commented on August 11, 2024
I will have a look at this one.
from city-scrapers-pitt.
vnbcs
commented on August 11, 2024
I think I'm finished. I have created a pull request 🚍🚍🚍🚍🚍🚍🚍🚍
from city-scrapers-pitt.
ben-nathanson
commented on August 11, 2024
Haven't heard from Eva on this since June. We may want to turn this over to someone else--possibly using Eva's existing code.
Sounds good to me.
from city-scrapers-pitt.
Related Issues (20)
- [Sentry] [pa_development] website changed HOT 4
- [Sentry] [alle_finance_dev] Outdated website HOT 1
- Protecting our Pipfile.lock file! HOT 1
- Add pre-commit hook for black HOT 1
- City Scrapers Calendar is not Rendering Events HOT 3
- [Sentry] [pa_liquorboard] local variable referenced before assignment HOT 3
- Spider: Pittsburgh Post-Gazette (Mixin Class) HOT 5
- Support integration testing locally
- Remove isort from docs
- Forbidden Link in Pittsburgh Public Schools HOT 1
- Strip HTML from pgh_public_schools descriptions
- Add new flag for virtual status HOT 3
- Spider: Pittsburgh Citizen Police Review Board (CPRB) HOT 1
- Possible bug with pipenv shell
- Scrapy Shell "Attribute Error" on Startup HOT 4
- Improving Documentation HOT 1
- Add VCR or Betamax libraries?
- BUG: no attribute 'base_url' on pitt_city_council
- BUG: test timings are in the past HOT 2
- Add Google Analytics to Website? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from city-scrapers-pitt.