Giter Club home page Giter Club logo

atti-online's Introduction

Fetches documents and meta data

Requirements

  • Git (tested on 2.10.1)
  • NodeJS (tested on v6.7.0)
  • NPM (tested on 3.10.7)

Setup

Clone this repository:

git clone https://github.com/olistik/atti-online.git

Enter into the working directory:

cd atti-online

Install the dependencies (CasperJS):

npm install

Now tweak the hardcoded values within fetch.js:

  • address
  • label

And then run the script:

./node_modules/casperjs/bin/casperjs fetch.js

At the end, if everything went well, you should see a documents/ directory and a records.json database in the current directory.

Parameters

address: "http://www.e-desio.it/ULISS-e/utility/info/info01.aspx?pagina=ATTI&men_id=00.03"

Label

  • "Delibere di Giunta"
  • "Delibere di Consiglio"
  • "Determine dei dirigenti"
var searchParams = {
  year: "2016",
  range: {
    start: "1",
    end: "500"
  }
};
var documentsBasePath = "documents";

Batch

For some unknown reasons, attachments are not retrieved when the range is too high (1-250).

For this reason I've prepared a script that chains batches together:

./batch.sh

It currently performs 10 batches each covering the fetch of 50 records therefore covering a total amount of 500 records.

License

This project is licensed under the AGPLv3.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.