grosenberger / easypqp Goto Github PK

Hi there,

Is it possible to generate a single spectral library with easypqp from two (or, potentially more) independent MSFragger output?

For example, if I have two experiments, one digested with LysC and one digested with Trp, and run MSFragger independently on both datasets, can I somehow have a single spectral library for both?

Initially, I could generate the two libraries separately. But I want to avoid the issue of aligning RT and IM after the fact.

Thanks!

We're attempting to generate a spectral library which is compatible with OpenMS workflows (diaPASEF). We used the easyPQP method bundled in FragPipe to generate a SL from DDA-PASEF timsTOF runs which were spiked with iRT peptides (database included iRT sequences).

The task outputs three files, library.tsv, easypqp_lib_openswath.tsv, and irt.tsv. I haven't had issues converting the .tsv files to .pqp, and I've managed to include decoys as suggested. My question has to do with the correct generation of the iRT library. The OpenSWATH routine requests an iRT library, and I'm not sure how this file get generated. To my knowledge, the MSFragger library files don't include iRT data. I'm sure this is user error, as I'm still attempting to wrap my head around these various operations.

Running easypqp convert with the --enable_specific_losses parameter gives the following error on frapipe result files. Running without --enable_specific_losses is fine.

easypqp was installed via 'pip install git+https://github.com/grosenberger/easypqp.git@master' last week.

Thanks

File "/home/karlssoc/miniconda3/envs/easypqp/bin/easypqp", line 8, in
sys.exit(cli())
File "/home/karlssoc/miniconda3/envs/easypqp/lib/python3.9/site-packages/click/core.py", line 829, in call
return self.main(*args, **kwargs)
File "/home/karlssoc/miniconda3/envs/easypqp/lib/python3.9/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/home/karlssoc/miniconda3/envs/easypqp/lib/python3.9/site-packages/click/core.py", line 1289, in invoke
rv.append(sub_ctx.command.invoke(sub_ctx))
File "/home/karlssoc/miniconda3/envs/easypqp/lib/python3.9/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/karlssoc/miniconda3/envs/easypqp/lib/python3.9/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/home/karlssoc/miniconda3/envs/easypqp/lib/python3.9/site-packages/easypqp/main.py", line 56, in convert
psms, peaks = conversion(pepxmlfile, spectralfile, unimodfile, exclude_range, max_delta_unimod, max_delta_ppm, fragment_types, fragment_charges, enable_specific_losses, enable_unspecific_losses)
File "/home/karlssoc/miniconda3/envs/easypqp/lib/python3.9/site-packages/easypqp/convert.py", line 584, in conversion
theoretical[peptide['modified_peptide']][peptide['precursor_charge']] = generate_ionseries(peptide['modified_peptide'], peptide['precursor_charge'], fragment_charges, fragment_types, enable_specific_losses, enable_unspecific_losses)
File "/home/karlssoc/miniconda3/envs/easypqp/lib/python3.9/site-packages/easypqp/convert.py", line 543, in generate_ionseries
loss_type = loss.toString().decode("utf-8")

I used following command to build the library:
python -u ../fragpipe13/tools/speclib/gen_con_spec_lib.py $fastaPath ./ ${mzfiles:1} $output True unused use_easypqp c iRT 7 "--rt_lowess_fraction 0.01"
Is there any way to filter ions in the library conforming to the down criterions like spectrast2tsv?

Lower and upper mass limits of fragment ions
remove transitions with Q3 falling in the swath mass range when provided the swath/dia window.
Fragment ion charge states allowed.
Could you please check and fix this issue?

Thanks,

Dear George,

Since the last versions of EasyPQP I always get a runtime error when running the DIA-SpecLibGen workflow in Fragpipe. As the attached log tells you at the bottom, the log notes that EasyPQP cannot call on the Unimod.xml file saved within the Python folder, and suggests to rename it to the exact same name that it already has. Do you know how I can fix this?

Cheers!

log_2024-01-18_09-30-52.txt

If I set the all level fdr at 0.01, does it mean the fdr of each run is 0.01? Or the fdr of library is 0.01?

I am running EasyPQP 0.1.39 in FragPipe 20.0, which has been consistently failing with KeyError: 'run_id'. I have attached the log file from FragPipe. I have not had this issue previously when running 0.1.37. Is there something new in 0.1.39 that might be leading to this error, or something else that I should adjust in my setup?
log_2023-08-13_05-41-18.txt

I recently downloaded FragPipe 20.0 and have been having issues with the spectral library generation feature of easyPQP. The TimsTOF DDA files were successfully searched, but it looks like it hit an error as soon as it tried constructing the spectral library. Attached is the log from this run.
log_2023-08-15_19-31-37.txt

I used fragpipe to build library, but there was none decoy in library.tsv and easypqp_lib_openswath.tsv. And the header of these two library were different from tsv format of pqp (http://openswath.org/en/latest/docs/pqp.html). I am puzzled. How to use these two library to run openswath2.4 ?

Hello,

Im trying to use CommandLine easypqp to create a spectral library from msfragger output. I cant seem to figure out which files to specify too.

My msfragger has the following structure:

msfraggeroutput
|    -sample1
|      -sample1_quant.csv
|       -sample1.pepXML
|       -sample1.pep.xml
|       -sample1_ion.tsv
|       -sample1_peptide.tsv
|       -sample1_protein.fas
|       -sample1_protein.tsv
|       -sample1_psm.tsv
|   -sample2
|       -sample2_quant.csv
|       -sample2.pepXML
|       -sample2.pep.xml
|       -sample2_ion.tsv
|       -sample2_peptide.tsv
|       -sample2_protein.fas
|       -sample2_protein.tsv
|       -sample2_psm.tsv
|    -MSstats.csv
|    -combined.pep.xml
|    -combined.prot.xml
|    -combined_peptide.csv
|    -combined_protein.tsv
|    -mbr_ion.tsv
|    -reprint.int.tsv
|    -reprint.spc.tsv

Should I just specify to the pepxml of each sample?
And what about the require spectra path? Im using Timstof files but we did not change them to mzXML or mgf format.

Best,
Patrick

I built library using fragpipe. And I check the psm.tsv,protein.tsv, interact* files including "iRT" protein. But the final library library.tsv didn't have iRT in side. And , protein like "opl|OPL_HBB-G57A|P68871-G57A" also did not find in library.tsv.
temp.zip

Usage situation:
Attempting to use EasyPQP during data processing with Fragpipe/MSFragger. Added variable modification within the search which would correspond to specific crosslinks of interest (not present in UniMod). The database search would complete without error but the subsequent SL step would fail.

Testing showed that the unimod file within \pyopenms\share\OpenMS\CHEMISTRY and possibly \easypqp\data had to contain the mass descriptor of the crosslink.

Is there a way to enter a variable modification within the MSFragger GUI and have it carryover to EasyPQP without the need to edit xml files? It's not difficult to add these entries to the unimod files. I'm just wondering if there is a command line argument I'm missing which would help the situation.

Hi @grosenberger ,

I was trying to release a new version by pushing a commit (f1c6a7d) and a new tag (0.1.41). But it seems that I need your help to put the new version to PyPI. Could you please release it for us? Or if you tell me how to do that, I can push the new version to PyPI by myself.

Thanks,

Fengchao

Sage can now write matched fragments output file for downstream applications, lazear/sage#101. Any hints on how to run easypqp library generation on this output. Thanks

We find an error from EasyPQP when we use it :

we check the log of error file and find the error message below :

Can you take a look when you have time?

Hi,
I recently ran into some issues when trying to use DIA_spec_lib when using the latest EasyPQP version available (0.1.39 ). I ran similar files uisng Python 3.11 before, and it had worked, so I don't think that is the Python version causing this problem, but this version of EasyPQP.
I'd appreciate any advice on this.
Thanks,
log_2023-08-17_17-53-26.txt

I ran the following command in linux
/home/biodocker/anaconda3/bin/easypqp convert --pepxml /mnt/data/fragpipeWorkspace/interact-C20180824sunyt_shaoyk_DDA_MXM_CMZ_M4.pep.xml --spectra /mnt/data/fragpipeWorkspace/mzXML/C20180824sunyt_shaoyk_DDA_MXM_CMZ_M4.mzXML --exclude-range -1.5,3.5 --psms C20180824sunyt_shaoyk_DDA_MXM_CMZ_M4.psmpkl --peaks C20180824sunyt_shaoyk_DDA_MXM_CMZ_M4.peakpkl

I found the retention time of peptides SMAGNIIPAIATTNAVIAGLIVLEGLK and LSITGTYDLK were reversed the order in *run_peaks.tsv and pep.xml. May I know why this happend? Thanks
temp.zip
The link to mzXML file: https://westlakeu-my.sharepoint.com/:u:/g/personal/zhutiansheng_westlake_edu_cn/EZ5-T93kTdxNjT8IJCASWeIBG_LrFcqbPsXnb1YHlQZncw?e=LVlKgl

In the convert command, the parsing of files with OpenMS is multithreaded, is there a way to control the number of threads for parsing?
Nesvilab/FragPipe#567

Hi,
What is the reason behind having exclude-range parameter in easypqp convert? I have done database search with including deamidation modification, if the massdiff between [-1.5, 3.5] is not necessary, I well then remove it from the database search also. However, what is the rationale for not mapping them to UniMod?

Hi,

is there any flag to control the number of cpus in easypqp?

thanks

Hi,

I was trying to filter the unimod xml using the command below:

easypqp filter-unimod --out unimod_ids.xml --ids [1,2,4,5,7,21,26,27,28,34,35,36,40,43,121,122,137,142,259,267,299,305,307,354,621,632,1409,1506,1531,1768]

But I got a UnicodeDecodeError (see below) and no xml file was written.

I use ubuntu, python 3.9.0 & easypqp 0.1.30. I tried earlier versions of python, but with the same error. I aslo tried the docker container (grosenberger/easypqp:latest), but it lacked the filter-unimod option.

thanks,
Christofer

INFO: Loading XML data from /data/karlssoc/bin/miniconda3/envs/easypqp/lib/python3.9/site-packages/easypqp/data/unimod.xml
INFO: Appending to filtered unimod XML - title=Acetyl with record_id=1
INFO: Appending to filtered unimod XML - title=Amidated with record_id=2
INFO: Appending to filtered unimod XML - title=Carbamidomethyl with record_id=4
INFO: Appending to filtered unimod XML - title=Carbamyl with record_id=5
INFO: Appending to filtered unimod XML - title=Deamidated with record_id=7
INFO: Appending to filtered unimod XML - title=Phospho with record_id=21
INFO: Appending to filtered unimod XML - title=Pyro-carbamidomethyl with record_id=26
INFO: Appending to filtered unimod XML - title=Glu->pyro-Glu with record_id=27
INFO: Appending to filtered unimod XML - title=Gln->pyro-Glu with record_id=28
INFO: Appending to filtered unimod XML - title=Methyl with record_id=34
INFO: Appending to filtered unimod XML - title=Oxidation with record_id=35
INFO: Appending to filtered unimod XML - title=Dimethyl with record_id=36
INFO: Appending to filtered unimod XML - title=Sulfo with record_id=40
INFO: Appending to filtered unimod XML - title=HexNAc with record_id=43
INFO: Appending to filtered unimod XML - title=GG with record_id=121
INFO: Appending to filtered unimod XML - title=Formyl with record_id=122
INFO: Appending to filtered unimod XML - title=Hex(5)HexNAc(2) with record_id=137
INFO: Appending to filtered unimod XML - title=HexNAc(1)dHex(1) with record_id=142
INFO: Appending to filtered unimod XML - title=Label:13C(6)15N(2) with record_id=259
INFO: Appending to filtered unimod XML - title=Label:13C(6)15N(4) with record_id=267
INFO: Appending to filtered unimod XML - title=Carboxy with record_id=299
INFO: Appending to filtered unimod XML - title=dHex(1)Hex(3)HexNAc(4) with record_id=305
INFO: Appending to filtered unimod XML - title=dHex(1)Hex(4)HexNAc(4) with record_id=307
INFO: Appending to filtered unimod XML - title=Nitro with record_id=354
INFO: Appending to filtered unimod XML - title=Asn->Asp with record_id=621
INFO: Appending to filtered unimod XML - title=Gln->Glu with record_id=632
INFO: Appending to filtered unimod XML - title=Hex(5)HexNAc(4)NeuAc(1) with record_id=1409
INFO: Appending to filtered unimod XML - title=dHex(1)Hex(4)HexNAc(3)NeuGc(1) with record_id=1506
INFO: Appending to filtered unimod XML - title=Hex(9)HexNAc(2) with record_id=1531
INFO: Appending to filtered unimod XML - title=dHex(1)Hex(3)HexNAc(3) with record_id=1768
INFO: Writing out filtered unimod XML file to unimod_ids.xml
Traceback (most recent call last):
  File "/data/karlssoc/bin/miniconda3/envs/easypqp/bin/easypqp", line 8, in 
    sys.exit(cli())
  File "/data/karlssoc/bin/miniconda3/envs/easypqp/lib/python3.9/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/data/karlssoc/bin/miniconda3/envs/easypqp/lib/python3.9/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/data/karlssoc/bin/miniconda3/envs/easypqp/lib/python3.9/site-packages/click/core.py", line 1690, in invoke
    rv.append(sub_ctx.command.invoke(sub_ctx))
  File "/data/karlssoc/bin/miniconda3/envs/easypqp/lib/python3.9/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/data/karlssoc/bin/miniconda3/envs/easypqp/lib/python3.9/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/data/karlssoc/bin/miniconda3/envs/easypqp/lib/python3.9/site-packages/easypqp/main.py", line 185, in filter_unimod
    unimod_filter(infile, outfile, accession_ids)
  File "/data/karlssoc/bin/miniconda3/envs/easypqp/lib/python3.9/site-packages/easypqp/unimoddb.py", line 55, in unimod_filter
    lines = file_handle.readlines()
  File "/data/karlssoc/bin/miniconda3/envs/easypqp/lib/python3.9/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 3119: ordinal not in range(128)

Although rare, I found that with large libraries there can be the case where transitions can have different annotations and the same ProductMz. Currently these are listed as different transitions however I don't know if this is the ideal behaviour because DIA analysis software could assume that two different transitions have a perfect correlation with one another when in fact it is just the same transition.

Also when creating a library from fractionated runs we might have the two different annotations with the same m/z having slightly different library intensity values.

Example:

These are the transitions for one of the peptide precursors. Note that here the b5^1 ion and the b10^2 ion have the same ProductMz but different library intensities.

Looking at the corresponding .peakpkl files (a snippet is shown in the dataframe below), we can see that these specific entries come from different fractions (based on the intensity values). However there are some fractions which have both a b5^1 and a b10^2 ion (e.g. Hela_50ug_HpH_F5_S1-A5_1_4468.peakpkl).

Background On Example

This library was generated from HeLa fractionation on the timsTOF. DDA analysis was run using fragpipe and easypqp was on command line with the following command.

easypqp convert --pepxml interact-XXX.pep.xml --spectra XXX_uncalibrated.mgf --exclude-range -1.5,3.5 --psms XXX.psmpkl --peaks XXX.peakpkl

Possible Solution

I'm not sure what the best way to address this, I was thinking that if ProductMz are within a certain PPM of one another (or exactly the same) than only the ProductMz of the lowest charge state should be reported. Possibly this should be addressed here or possibly part of the pyopenms theoretical isotope generator, I'm not sure.

Hi,
I would like to use easypqp, following searches with Comet, then PeptideProphet, via TPP. Can you provide pointers to any documentation for necessary files at each step. I can convert pepXML files to psmpkl and peakpkl files, but I don't understand what role these files play in creating a library. The next step of the documentation talks about loading tsv files from Philosopher. Can EasyPQP work directly from the outputs of TPP?
thanks
Andy

Hello,

Can you help me to install easypqp in Python version 3.10?

I have already raised the issue with OpenMS "Not able to install pyopenms in Python3.10 #6199"

--
Chinmaya

Hi,

We noticed that there is a small percentage of peptides with a negative retention time, any idea what that could be / mean?

With kind regards,
Patrick

I'm having an issue with aligning some of the files I am using in the easypqp library generation. I have used an iRT reference file with the appropriate format. Most of the runs look like they have good alignment from the plots, but two of them are totally off (I'm getting normalized retention values on the order of e+16). When i manually checked for the iRT peptides in these runs, and performed a basci linear regression, it seemed like a good fit. This is the warning i'm getting when I run easypqp library on only the two files that are giving me this issue, although the output file is still successfully generated.
eg.

easypqp library --rt_reference=$irt --out=$library --peptide_fdr_threshold=$peptide_fdr_threshold \
--protein_fdr_threshold=0.01 --psm_fdr_threshold=0.01 \
--rt_lowess_fraction=0.1 --pi0_lambda=$pi0lambda --peptide_plot=$peptide_plot \
--protein_plot=$protein_plot $inputpsms $inputpeaks

Info: easypqp_rt_alignment_filename; Peptide overlap between run and reference: 56.
/home/asrini/ENV/lib/python3.6/site-packages/statsmodels/nonparametric/smoothers_lowess.py:165: RuntimeWarning: invalid value encountered in true_divide
  res = _lowess(y, x, frac=frac, it=it, delta=delta)
/home/asrini/ENV/lib/python3.6/site-packages/statsmodels/nonparametric/smoothers_lowess.py:165: RuntimeWarning: invalid value encountered in greater_equal
  res = _lowess(y, x, frac=frac, it=it, delta=delta)

Info: Library successfully generated.

Have tried to debug this issue but ran into some other issues along the way. Thanks for your help! Let me know if you need additional info.

Hello -
Is there a limit to the number of files that can be combined together in creating a library with EasyPQP?
Thanks!