Giter Club home page Giter Club logo

openmore's Introduction

General

OpenMORe is a collection of Python modules for Model-Order-Reduction, clustering and classification.

How to cite

If you use OpenMORe for your publications, I kindly ask you to cite the following papers: If you use clustering.py and/or classification.py:

  • D’Alessio et al., "Adaptive chemistry via pre-partitioning of composition space and mechanism reduction." Combustion and Flame 211 (2020): 68-82.
  • D'Alessio et al., "Feature extraction and artificial neural networks for the on-the-fly classification of high-dimensional thermochemical spaces in adaptive-chemistry simulations." Data-Centric Engineering 2 (2021).

If you use model_order_reduction.py:

  • D'Alessio et al., "Analysis of turbulent reacting jets via principal component analysis." Data Analysis for Direct Numerical Simulations of Turbulent Combustion. Springer, Cham, 2020. 233-251.

Implemented techniques:

Model Order Reduction:

  • Principal Component Analysis (PCA)
  • Local PCA (LPCA)
  • Kernel PCA (KPCA)
  • Feature selection via PCA
  • Outlier removal via PCA
  • Data sampling

Clustering:

  • Local PCA (via Vector Quantization, unsupervised)
  • FPCA (via conditioning vector, supervised)
  • Spectral Clustering (unsupervised)

Utilities:

  • Multivariate data preprocessing
  • Varimax Rotation
  • Clustering solution evaluation
  • Fast algorithm for SVD

Requirements:

In order to use OpenMORe on your devices, the following requirements must be satisfied:

  • Python version >= 3.6
  • Numpy must be installed
  • Scipy must be installed
  • Matplotlib must be installed
  • Pandas must be installed
  • Latex must be installed (for the plots' labels)

Installation:

If the libraries requirements are satisfied, clone or download the repo. After that, go to the OpenMORe folder from your terminal (where the file setup.py is located) and type: python setup.py install.

Test:

It is possible to check if the installation process was successful running the tests. To do that, just type:

  • python -m unittest tests/test_PCA.py
  • python -m unittest tests/test_sampling.py
  • python -m unittest tests/test_clustering.py

If the tests have positive response, you should get a message like:

___________________________
Ran 4 tests in 0.113s

OK
___________________________

Usage:

If the tests are successful, you can now use OpenMORe. In the "examples" folder there are some pre-set cases, organized according to the final purpose (e.g., clustering, dimensionality-reduction, variables-selection, others). They are also fully commented to describe the required dictionary inputs. In the “data/reactive_flow” folder, there is a collection of data (from a CFD simulation of a turbulent reacting jet) to run the examples and test the code, while in "data/dummy_data" you can find relatively simple data sets to test the scripts' functionality. A detailed description of all the classes and functions is available in the detailed documentation and in the source code.

Documentation:

The official documentation is available in /OpenMORe/Documentation. It is strongly suggested to read it before using the software.

For any question or problem regarding the code you can write to me at the following address: [email protected]

openmore's People

Contributors

gdalessi avatar kamilazdybal avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

Forkers

bcocco xupeiwust

openmore's Issues

The system cannot find the path specified

hank you for your code, it's very useful to perform local PCA and easy to use 🙂 However, I have a problem in the code below and I don't know how to fix it.
My code is simply finding the best clusters for each number of eigenvectors (from 1 to the initial space). Then, I perform local pca on my different clusters and finally evaluate the reconstruction of my initial dataset. Here is my code :

import OpenMORe.clustering as clustering
n_clusters = 4
r2final=[]
r2clusters=[]

for q in range(1,test.shape[1]+1): #number of pc
settings_clustering = {
#centering and scaling options
"center" : True,
"centering_method" : "mean",
"scale" : True,
"scaling_method" : "auto",

#set the initialization method (random, observations, kmeans, pkcia, uniform)
"initialization_method"     : "random",

#set the number of clusters and PCs in each cluster
"number_of_clusters"        : n_clusters,
"number_of_eigenvectors"    : q,

#enable additional options:
"correction_factor"         : "off",    # --> enable eventual corrective coefficients for the LPCA algorithm:
                                        #     'off', 'c_range', 'uncorrelation', 'local_variance', 'phc_multi', 'local_skewness' are available

"classify"                  : False,    # --> call the method to classify a new matrix Y on the basis of the lpca clustering
"write_on_txt"              : False,    # --> write the idx vector containing the label for each observation
"evaluate_clustering"       : True,     # --> enable the calculation of indeces to evaluate the goodness of the clustering
"neighbors_number"          : 0,
}

# First we create the model and then we fit it to the dataset
model = clustering.lpca(test, settings_clustering)
labels_vqpca = model.fit()

r2final=[]
for k in range(n_clusters):
    mask = (labels_vqpca  == k)
    X_k = test[mask] # the mask is used to select the clusters with a specific label
    cnt_k = np.mean(X_k, axis=0)
    scl_k = np.std(X_k, axis=0)
    X0_k = (X_k - cnt_k)/scl_k # we need to center the data in each cluster
    pca.fit(X0_k)
    A_k = pca.components_.T
    Z_k = X0_k @ A_k
    X0_k_rec = Z_k[:,:q] @ A_k[:,:q].T
    X_k_rec = scl_k * X0_k_rec + cnt_k
    r2list = []
    for i in range(X_k.shape[1]): #reconstruction for each elements
        r2 = r2_score(X_k[:,i], X_k_rec[:,i])
        r2list.append(r2)
    r2final.append(r2list)
averages = [sum(elements) / len(elements) for elements in zip(*r2final)]  
r2clusters.append(averages)

However, I get this error which i don't really understand when calling model.fit():

FileNotFoundError: [WinError 3] The system cannot find the path specified: 'Clustering LPCA - 2023_05_05-144151'

Thank you for your help.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.