Spider: Pittsburgh Zoning Board of Adjustment about city-scrapers-pitt HOT 5 OPEN

I'll take a crack at this one today, the PDFs look to be easily parsed out by pdfminer.six

from city-scrapers-pitt.

So I'll need some clarification on how we want to go about with this one, @ben-nathanson @bonfirefan.

Do we want to scrape every individual meeting? Because with what looks like at least 10 meetings per day, that could clog up the gears of City-scrapers by quite a bit. Or do we want to simply list the first one of them, which is 9:00AM?

from city-scrapers-pitt.

It's good that you're thinking about scale. In this example we will want to scrape every individual meeting.

I would also say that we're building on top of existing tools that were designed to handle significant workloads, so it should be a long time before we encounter issues around scaling.

from city-scrapers-pitt.

I think my concern is moreso that it might take up a lot of space on the calendar itself. Are we fine with having 10+ similar meetings listed on the same day in the calendar?

from city-scrapers-pitt.

@cheog was looking into this with certain python modules, and I'm looking into it with different python modules--in my case, PDFMiner.

from city-scrapers-pitt.