ivanolauriola / mklpy Goto Github PK

when I want to run the KerNET from the tutorial, I found that the versions of keras is important, since when i run the part of import code, the following problem occured:

SystemError: initialization of _pywrap_checkpoint_reader raised unreported exception

and after checking the Error, i found that it is because somthing about the version of keras and the version of tensorflow.

therefore, i want to ask for the versions of the packages you used.

File "./examples/test_training.py", line 75, "unexpected keyword argument 'estimator'"

Hello,

Is this code compatible with Python 3 ?

Would it be possible to have reference publications associated with algorithms ?

Hello, I'm running the testi_training.py, and change the input to a 3-class dataset, and I encounter a problem from these lines:
from MKLpy.algorithms import AverageMKL, EasyMKL, KOMD #KOMD is not a MKL algorithm but a simple kernel machine like the SVM
print ('training AverageMKL...', end='')
clf = AverageMKL().fit(KLtr,Ytr) #a wrapper for averaging kernels
print ('done')
print(clf.weights) #print the weights of the combination of base kernels
K_average = clf.ker_matrix #the combined kernel matrix

the clf.weights and K_average is both None. I'n not sure if clf.weights should be changed to clf.clf.weights, and I don't know what to do with K_average. Is this a bug or is it just the way the algorithm is? Thanks in advance for your kind answer.

Hi, thank you for your work on this useful tool!
When I using the sigmoid kernel from scikit-learn package in EasyMKL, the error info of ValueError: Rank(A) < p or Rank([P; A; G]) < n showed up. I am wondering whether the algorithm supports sigmoid kernel, which is not offered in MKLpy.pairwise, or not.

If the entire error information is needed, you can refer to this link.

It seems that there are some bugs for multi-classification with TwoStepMKL methods.
Please see the error messages as follows:

AttributeError Traceback (most recent call last)
in ()
----> 1 mkl.fit(MKtr,ytr)

1 frames
/usr/local/lib/python3.7/dist-packages/MKLpy/algorithms/base.py in fit(self, KL, Y)
87 if self.multiclass_ : # a multiclass wrapper is used in case of multiclass target
88 metaClassifier = OneVsOneMKLClassifier if self.multiclass_strategy == 'ovo' else OneVsRestMKLClassifier
---> 89 self.clf = metaClassifier(self.class(**self.get_params())).fit(self.KL,self.Y)
90 self.solution = self.clf.solution
91 else :

/usr/local/lib/python3.7/dist-packages/MKLpy/multiclass.py in fit(self, KL, Y)
38 for task in self.tasks:
39 mkl = self.mkl.class(**self.mkl.get_params())
---> 40 mkl.learner = mkl.learner.class(**mkl.learner.get_params())
41 mkl.learner.kernel = 'precomputed'
42

AttributeError: 'NoneType' object has no attribute 'get_params'

I know that cross_value_score function internally uses K-fold method, but I want to use leave one out cross-validation method. So my question is how can I do it?
Your suggestion might help me.

First off, I have to say, thank you for writing this library. It's great and I've found it helpful and will be citing it in my research. That said, onto the issue: in the function MKLpy.utils.misc.ideal_kernel, the call to Y.unique() is not general. Specifically, in working with the function MKL.metrics.alignment.alignment_yy, which calls the ideal_kernel function as a subroutine, I'm passing in an np.ndarray as my y parameter to alignment_yy. The np.ndarray object does not have a unique method, but torch.Tensor does, which I suspect is why it was written as a method of the object in ideal_kernel. This points to an issue where it is assumed the user will pass an array-like object to ideal_kernel but the code is not sufficiently general to support an np.ndarray. I see two possible solutions here:

1. Stick to PyTorch and instead of using array_like.unique() apply the function torch.unique() to the array-like object, which removes the assumption that the unique function is a method of the array-like object. Of course, type consistency would be an issue seeing as this would cast the array-like object to a Torch.tensor. Personally, I always expect the datatype being input and returned should be consistent with the user's expectation and in this instance I don't see a reason to expect not to receive an np.ndarray back.
2. Alternatively, within the function, enable duck typing by disambiguating the approach to computing the unique array on the basis of the specific array type passed in. Return the same type that was input. So, if it's a Torch.tensor return that, a numpy.ndarray, return that, etc.

As a final note, it wasn't until I came across this issue I became aware that PyTorch was being used in some places. PyTorch is wonderful, but I wonder if it's necessary? It's my understanding that letting NumPy do the heavy numeric lifting wouldn't present an issue for compatibility with PyTorch, but it seems the case is less so the other way around. Also, maybe I haven't explored MKLpy enough, but it seems to me that the automatic differentiation capabilities of PyTorch are only lightly used (of course, I may be wrong about this). If that's the case, it may be worth using a different, lighter weight library with a more NumPy-like API, like Jax. An example of a similar use of Jax is this kernel library, which leverages Scikit-learn, much as MKLpy does, too.

Happy to discuss further what a solution might look like and contribute to any updates that may be required.

DefaultCPUAllocator: not enough memory: you tried to allocate 5077901088 bytes.
Is there any ways to solve this problem.

where is simpleMKL

When i type "from MKLpy.algorithms import AverageMKL, EasyMKL, KOMD", "ImportError: DLL load failed: 找不到指定的模块" what should I fix?

hi,i just run the tutorial which is from https://mklpy.readthedocs.io/en/latest/examples/examples_loading/,
maybe some dataset i missed

When I try to run the files in the test folder, it gives me the following error.
Can you help me check what the problem is? Thank you.

runfile('E:/Anaconda3/Lib/site-packages/MKLpy/test/check_list.py', wdir='E:/Anaconda3/Lib/site-packages/MKLpy/test')
Traceback (most recent call last):

File "", line 1, in
runfile('E:/Anaconda3/Lib/site-packages/MKLpy/test/check_list.py', wdir='E:/Anaconda3/Lib/site-packages/MKLpy/test')

File "E:\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 880, in runfile
execfile(filename, namespace)

File "E:\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 102, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)

File "E:/Anaconda3/Lib/site-packages/MKLpy/test/check_list.py", line 46, in
from MKLpy.lists import kernel_list, HPK_generator, SFK_generator

File "E:\Anaconda3\lib\site-packages\MKLpy_init_.py", line 39, in
import algorithms,lists,metrics,test,utils

ImportError: No module named 'algorithms'

First of all, thank you for such useful tool!
I wonder that if I have multiple training and testing feature sets, how can I get the composite kernel of all sets to be used in model training and prediction?
Thank you in advance for your reply :)

Hi there,

I am considering using this package as part of my work, and would like to know if any of the developers have written any tests for it already? If so, let me know where they are?

I can also help write some, or improve the existing test suite. I want to coordinate with you if that's possible.

Looking forward to hear you, and use the well-tested version of this package.

Thanks!

Hello, if i have data from multipel MRI modalities,how can i combine multiple kernels with you code? thank you

In line 80 of file /MKLpy/algorithms/EasyMKL.py .

Hi,

I am working on a project related to multiple kernel learning and I wanted to use the MKLpy library. I am currently trying to run the small example given in the MKLpy paper on a dataset with 750 instances. In my program, I am creating a list of kernels using my custom kernel function. So, the shape of my training set is (11, 384, 600) where 11 is the number of kernels and (384, 600) is the size of each kernel (the kernel is obtained by the random fourier features). I am using this list of kernels to train an EasyMKL model from the library. Running this program gives me the error MKLpy.utils.exceptions.SquaredKernelError: K is not squared: torch.Size([384, 600]).
Can anyone help me out with this? I am not able to understand why it should be a squared matrix if the library allows any custom kernel.

Here is the code example for reference -

Hello every one, I am a new one to learn machine learning. And now I have a data base with 4 millions of datas and I am concerning this would be failed to be trained by MKLpy, I am woundering whether this project can provide online training method like partial_fit in sklearn?

I try to search for the best parameters of EasyMKL, so I build a module, which works well, to implement it.

def mkl_search(KL, y, param_grid, scoring='accuracy', seed=np.random.RandomState()):
    '''
    Search for best params of MKL classifier
    Return best score, params and post-trained model 
    '''
    best_params = {}
    best_score = 0
    for lam in param_grid['lam']:
        for C in param_grid['C']:
            svm = SVC(C=C)
            mkl = EasyMKL(lam=lam, learner=svm)
            scores = cross_val_score(KL, y, mkl, n_folds=5, scoring=scoring, random_state=seed)
            if np.mean(scores) > best_score:
                best_params['lam'] = lam
                best_params['C'] = C
                best_score = np.mean(scores)
    best_model = EasyMKL(lam=best_params['lam'], learner=SVC(C=best_params['C']))
    print('start training the best model of MKL...')
    best_model.fit(KL, y)
    print('training DONE!')
    return best_score, best_params, best_model

It will return a fitted mkl model, and I can run the code below smoothly.

param_grid = {
    'lam': np.linspace(0,1,11),
    'C': np.logspace(-4,2,7)
}
best_score, best_params, best_model = mkl_search(KL_train, y_train, param_grid)
print("best param (MKL): {}\nbest score: {}".format(best_params, best_score))
y_pred = best_model.predict(KL_test)
y_score = best_model.decision_function(KL_test)

When I run the best_model.fit(KL_train, y_train) in the command line instead of in the function, however, it will pop out an error like KeyError: 'verbose'. The complete error info can be found in this link.

It seems that all the MKL methods presented focus on classification, would it be possible to allow also MKL regression?

If I serendipitously try that, I get errors like "ValueError: Unknown label type: 'continuous'"