Error in quadprog::solve.QP(D, d, A, bzero) : matrix D in quadratic function is not positive definite! about giotto HOT 9 OPEN

Received another error related to quadprog::solve.QP when trying to deconvolve a larger spatial dataset and a larger reference.
Error in quadprog::solve.QP(D/sc, d/sc, A, bzero) : NA/NaN/Inf in foreign function call (arg 1)

Call to Giotto::runDWLSDeconv was same as before, but this time using a spatial dataset of 135 genes x 45978 pixels and a signature matrix of 19234 genes x 23 cell-types. Note that these genes were selected from a scRNA-seq reference using Seurat::FindAllMarkers(). If this being too large could be contributing to the issue, then perhaps some guidance on selecting marker genes in the signature matrix would be helpful.

Thank you and sorry for the trouble!

from giotto.

Hi, If this is a general question re positive definitive, you could try to use the following tip to see if this solves the problem. Alternatively, we could help you troubleshoot if you could send us the data in some way (for example, removing the gene and cell-type names so that the data remain private). https://stackoverflow.com/questions/39129253/error-in-solve-qp

from giotto.

gobject.tar.gz

Thanks for getting back to me. My question is less about positive definite matrices in general and more about why it is occurring in Giotto::runDWLSDeconv when larger matrices are being used.

To this post I have added a gzipped file of the giottoobj and enrich_matrix I have been using, both of which have been de-identified in terms of the cell and gene names.

Both files have worked in conjunction with other enrich files or gioobjects, respectively, but here when used together quadprog::solve.QP complains about matrix D in quadratic function is not positive definite!.

The command I have been using is:

giottoobj <- Giotto::runDWLSDeconv(gobject = giottoobj,
                                    expression_values = c('normalized'), # selects @norm_expr 
                                    logbase = 2,
                                    cluster_column = 'cluster_9', # 'leiden_clus'
                                    sign_matrix = enrich_matrix,
                                    n_cell = 50,
                                    cutoff = 2,
                                    name = NULL,
                                    return_gobject = TRUE)

If you have any insight into why a non-positive definite matrix is being generated using this combination of normalized ST counts and enrich matrix I would greatly appreciate it!

Thank you

from giotto.

Hi @bmill3r - I wasn't able to download the data you posted. (Edited) I noticed above you were using a binary matrix (for more information on expected input for runDWLSDeconv() please see this paper.
Please install the Giotto suite branch and create your signature matrix using the function makeSignMatrixDWLS() if using data from a processed Giotto Object, or makeSignMatrixDWLSfromMatrix() if your processed data is elsewhere. You can find more information about those functions in this script.

Can you give this a try and let us know if you are still having any problems?
Thanks!

from giotto.

Thanks for getting back to me,

I actually tried running Giotto::runDWLSDeconv() using both the binary signature matrix and the expression matrix of my scRNA-seq reference (it was not immediately clear what the sign_matrix parameter in runDWLSDeconv() needed to be).

In either case, I still receive the same error. Note that this error occurs when I use a signature matrix of 135 genes and 76 cell-types on a larger spatial dataset but not when I use a smaller spatial dataset. For instance, my smaller spatial dataset is 135 genes x 3072 pixels whereas my larger one is 135 genes x 13477 pixels. Conversely, if I use the larger spatial dataset (or the smaller one) but a smaller signature matrix of 135 genes and 9 cell-types, it works. So I don't think it is the spatial dataset or the signature matrix specifically but it is the combination of both the larger signature matrix and the larger spatial dataset.

If I use an even bigger spatial dataset, I get a different error: Error in quadprog::solve.QP(D/sc, d/sc, A, bzero) : NA/NaN/Inf in foreign function call (arg 1)

I am using Giotto version 1.0.4 if that helps.

I can try to send you the data another way if you are having trouble downloading from GitHub directly.

Thanks again

from giotto.

Hi @bmill3r, thanks for clarifying! Our Giotto suite branch includes the latest updates for the DWLS functions.
As previously mentioned, please install the Giotto suite branch. You can do so by running the following:
remotes::install_github("RubD/Giotto@suite")

From there, you can create your signature matrix either from data pre-processed with Giotto (makeSignMatrixDWLS()) or from your own data processed elsewhere (makeSignMatrixDWLSfromMatrix()).

Below, I have written an explanation for each parameter used in both of the above functions mentioned.

makeSignMatrixDWLS(gobject = Giotto Object, 
                   expression_values = expression values to use, 
                   reverse_log = default is set to TRUE, 
                   log_base = default is set to 2, 
                   sign_gene = marker genes to use, 
                   cell_type_vector = vector of cell type for each cell in your dataset (length = ncol(matrix)))

makeSignMatrixDWLSfromMatrix(matrix = scRNA-seq matrix, 
                             sign_gene = marker genes to use, 
                             cell_type_vector = vector of cell type for each cell in your dataset (length = ncol(matrix)))

Let me know how this goes or if you have any further questions.

Thank you for the feedback, we are working on improving our documentation to be the clearest possible and we are making this a priority.

from giotto.

Hi again,

Unfortunately installation of the Giotto suite branch, generation of a signature matrix via makeSignMatrixDWLSfromMatrix from my own single cell data, and then rerunning runDWLSDeconv still resulted in the same error Error in quadprog::solve.QP(Dmat = D, dvec = d, Amat = A, bvec = bzero) : matrix D in quadratic function is not positive definite!. This was using my larger signature matrix of 76 cell-types, but my smaller signature matrix of 9 cell-types does work as it did before. I confirmed that my previous results stayed the same using the smaller signature matrix, using the suite branch, and making my own signature matrix from the single-cell expression data.

Some minor comments:
Installation of the suite branch changed the slots of my giotto objects such that gobject@norm_expr, gobject@custom_expr, gobject@scaled_expr no longer exist and it seems that the gene expression matrices I had there in my objects are gone. I think I was able to get these back in new slots gobject@expression$rna. Had a similar issue getting the right clusters assigned to column in gobject@cell_metadata which is now gobject@cell_metadata$rna

from giotto.

Hi, thanks for trying that out! If you would like, I would be happy to take a closer look with your specific data. Perhaps you can either share a Google drive link or email me the files at [email protected].

Regarding the slots, these changes reflect the development of the suite branch to accommodate multiple modalities (such as protein expression). You can access the data using gobject@slot$rna.

from giotto.

I have sent the data via google drive and also a separate email with some information about the files just in case. Any help would be appreciated. Thank you for all of your assistance!

from giotto.