To reproduce all the experiments of Lightor system

See here

Plots of experiments can be found here

In order to reproduce the same results as in paper, Python 3.5 is required. We strong recommend you to set up a virtualenv using Anaconda to reproduce experiments:

• Download and install Anaconda
• Create a virtualenv of Python 3.5 and activate it (details)

  > conda create -n py35 python=3.5
  > source activate py35 (for macOS and Linux in Terminal)
  > activate py35 (for Windows in Anaconda Prompt)

• Clone the repo

  > git clone https://github.com/sfu-db/Lightor_Exp
  > cd Lightor_Exp

• Install required packages and jupyter notebook

  > pip install -r requirements.txt
  > pip install jupyter

• Add py35 kernel to jupyter

  > pip install ipykernel
  > python -m ipykernel install --name py35

• Run jupyter notebook

  > jupyter notebook exp.ipynb

• Change the kernel in notebook menu: Kernel >> Change Kernel >> py35 and run the code blocks

• All crowdsourcing data has been anonymized. (e.g. replace Worker ID with W+number)
• paper