nachtm / cosmic-rai Goto Github PK

• Proton at high energy looks like iron at low energy
• Maybe we should only look at events with similar energy (ignore full spectrum)
  • if that works well, do s_125 analysis

It looks like in some cases, "dimensionality reduction" via techniques like PCA can improve the performance of classifiers.

Sklearn has a suite of tools for this: http://scikit-learn.org/stable/modules/decomposition.html#decompositions

I think the "cropping" of sensors that Micah has discussed, (to omit sensors that aren't hit for a given event) is basically also dimentionality reduction. In fact, that might be the best way of reducing attributes for this particular dataset. Rather than passing readings for all sensors on a given hit, just pass the 10 most important ones?

In the Data Exploration notebook (copied into data_prep.py) I'm wondering if its an error to use the variable i down in that if statement, outside of the for loop?

#build a name->index dict
def get_index_dict(sensors, direction_data=False):
    name_index_dict = {}
    for i in range(len(sensors)):
        name_index_dict[sensors[i]] = i
    i = len(sensors)
    if direction_data:
        name_index_dict['zenith'] = i
        name_index_dict['azimuth'] = i + 1
    return name_index_dict

Are there parameters we need to tune for each model we're using? Here's a start:
DNN:

• Number of hidden layers
• Number of nodes per layer

Are there parameters in SVM/any of the others that would be worth tuning?

• Rather than raw charge values, try difference between all sensors
• Use log of charge values

Quality control:

• Instead of looking at 'high gain' OR 'low gain' choose best value. Low gain unless cutoff reached.
• Make sure that fit status is okay
• Make sure at least 1 sensor is at least 6 (low energy event not important)
• Zenith less than 40
• Make sure center of event is not on the edge of array (see notes)

Notes:

• Core is the center of the event
• Reco is reconstructed

Right now, Feature Generation.ipynb has some nice stuff for generating features. At the same time, we're looking at using pandas. We'd like to get all of these into the general pipeline.

Frank says "In case you were curious, here's kind of a classic visualization. I think it uses log(charge) to determine the radius of the circle for the tank hit. Red v. Blue is a high v. low-gain DOM."

We could try this?

Wanna try more stuff?

• DNN on Tensor Flow
• SVM with sklearn
• Image classification stuff
• Bayesian Classifier?
• Random forest?