The project needs to have NodeJS installed and all its dependencies this can be done by running:

$npm install

The process for scrapping was the following:

• Given a list of universities in universtities.txt file (In this case from QS) they are searched using Scopus and Scival to retrive their id in both platforms using the command $node qs_bot.js > universities.csv (In our task, this had missing values that were later added manually)

• Then having universities.csv we used for $node sc_bot.js > result.jsonto grad all the attributes from Scopus and Scival and having it as a json file with raw attributes

• Finally the result.json was used for $node calc.js > Database.arffto have all the attributes with their calculations as specified in the assigment document

(The idea of doing the scrapping in different steps was to avoid being blocked by Scopus and Scival, and also having the raw attributes in case we need to have another type of calculation in the future)