parkerici / premessa Goto Github PK

Hello,
I try to concatenente FCS files, but they are not recognized as FCS files even though they are classic FCS.3 files.
I received this message:
Error in FUN(X[[i]], ...) :
'file.fcs' is not a valid file

I tried
read.FCS(in_dir,"file.fcs")
I had:
Error in if (!version %in% c("FCS2.0", "FCS3.0", "FCS3.1")) stop("This does not seem to be a valid FCS2.0, FCS3.0 or FCS3.1 file") :
argument is of length zero

Did that ever happen to someone else?

Please find below info about my session:

session_info()
Session info -----------------------------------------------------------------------------------------------
setting value
version R version 3.5.0 (2018-04-23)
system x86_64, darwin15.6.0
ui RStudio (1.1.453)
language (EN)
collate C
tz Europe/Paris
date 2018-09-22

Packages ---------------------------------------------------------------------------------------------------
package * version date source
base * 3.5.0 2018-04-24 local
Biobase 2.40.0 2018-05-01 Bioconductor
BiocGenerics 0.26.0 2018-05-01 Bioconductor
cluster 2.0.7-1 2018-04-13 CRAN (R 3.5.0)
compiler 3.5.0 2018-04-24 local
corpcor 1.6.9 2017-04-01 CRAN (R 3.5.0)
datasets * 3.5.0 2018-04-24 local
DEoptimR 1.0-8 2016-11-19 CRAN (R 3.5.0)
devtools * 1.13.6 2018-06-27 CRAN (R 3.5.0)
digest 0.6.17 2018-09-12 CRAN (R 3.5.0)
flowCore * 1.47.9 2018-09-22 Github (RGLab/flowCore@2827a9b)
graph 1.58.0 2018-05-01 Bioconductor
graphics * 3.5.0 2018-04-24 local
grDevices * 3.5.0 2018-04-24 local
grid 3.5.0 2018-04-24 local
lattice 0.20-35 2017-03-25 CRAN (R 3.5.0)
MASS 7.3-50 2018-04-30 CRAN (R 3.5.0)
matrixStats 0.54.0 2018-07-23 cran (@0.54.0)
memoise 1.1.0 2017-04-21 CRAN (R 3.5.0)
methods * 3.5.0 2018-04-24 local
mvtnorm 1.0-8 2018-05-31 CRAN (R 3.5.0)
parallel 3.5.0 2018-04-24 local
pcaPP 1.9-73 2018-01-14 CRAN (R 3.5.0)
premessa * 0.2.3 2018-09-22 Github (c47abca)
Rcpp 0.12.18 2018-07-23 cran (@0.12.18)
robustbase 0.93-0 2018-04-24 CRAN (R 3.5.0)
rrcov 1.4-4 2018-05-24 CRAN (R 3.5.0)
stats * 3.5.0 2018-04-24 local
stats4 3.5.0 2018-04-24 local
tools 3.5.0 2018-04-24 local
utils * 3.5.0 2018-04-24 local
withr 2.1.2 2018-03-15 CRAN (R 3.5.0)

beads_before_and_after.pdf

Hi,

When normalizing just ONE file (large-ish barcoded sample that would then be debarcoded into multiple smaller samples), the "beads before and after" picture gets generated, but isn't really populated.

Please see attached PDF.

I think it's an issue of R is drawing line segments from Point 1 (file 1) to Point 2 (file 2); if there's no Point 2, then there can't be a line segment.

Possible solutions:

1. Default to drawing dots at the appropriate signal intensities for each channel.
2. Switch depiction to show Before and After on the same plot (Point/file 1 = Before, Point/File 2 = After).
3. Something else?

Mike

Hello,

I viewed my files in FlowJo pre- and post-Premessa panel editing and noticed a discrepancy. Each marker's plot got altered post-panel editing. Below are the plots for the same sample, before and after panel editing. The channels are correctly renamed.

I noticed that I am getting the following warnings when I run paneleditor_GUI(). How do I fix $PnR?
Warning in readFCSdata(con, offsets, txt, transformation, which.lines, scale, :
Some data values of 'Yb171Di' channel exceed its $PnR value 1 and will be truncated!
To avoid truncation, either fix $PnR before generating FCS or set 'truncate_max_range = FALSE'
Warning in readFCSdata(con, offsets, txt, transformation, which.lines, scale, :
Some data values of 'beadDist' channel exceed its $PnR value 1 and will be truncated!
To avoid truncation, either fix $PnR before generating FCS or set 'truncate_max_range = FALSE'

With paneleditor_GUI(), there is no option to set 'truncate_max_range = FALSE'. My flowCore version is ‘2.10.0’

I recently updated to R version 4.1.0. After using the premessa panel editor GUI, I am able to see my CyTOF panel and make edits as I have done in the past. It also appears that after processing, new .fcs files are created in the "renamed" folder that I designated in the GUI. However, when I try to perform clara clustering in Scaffold (the version from the Spitzer Lab), it freezes up and R shows an error message (see first screen shot). Thinking that this was just an issue with the Scaffold program, I tried to open the re-named files in the premessa panel editor GUI; however, I am unable to open up the re-named files and receive a similar error message about a subscript out of bounds (see second screen shot). I'm not sure if I'm doing something wrong, but any help you could provide would be greatly appreciated. I've attached the info. from my R-session in Rstudio. Please let me know if there is any additional information that you need.

Thanks,

Kyle

Hi! I'm reading many fcs files at the same time. Is there anyway I can save/download the paneleditor_GUI() results as a csv?

Hi,

Got an error trying to remove beads with the normalizer GUI, I just installed the package today.

I got the following error message:

Warning: Error in remove_beads_from_file: could not find function "remove_beads_from_file"
74: isolate
73: observeEventHandler [/Library/Frameworks/R.framework/Versions/3.5/Resources/library/premessa/normalizer_shinyGUI/server.R#102]
2: shiny::runApp
1: premessa::normalizer_GUI

I think there's a typo in line 103 of normalizer_shinyGUI/server.R

remove_beads_from_file(file.path(normed.dir, input$beadremovalui_selected_fcs),

should be

premessa::remove_beads_from_file(file.path(normed.dir, input$beadremovalui_selected_fcs),

When I edited my local copy of the package it resolved the issue with the GUI.

Best,
Geoff

I had 36 files (6 files x 6 centers) that I wanted to harmonize to the same panel (some had minor issues like CD66 vs CD66b, others had more major issues).

The files were exported as new FCS from FlowJoX, and can be found here:
https://drive.google.com/drive/folders/0Bz9sd8nC272jR3VWZUtTcFRlM00?usp=sharing

I opened the Panel Editor, loaded in my files, and began to edit. I only saw 6 of the 36 files (typically Center 3). The horizontal scroll bar in either R Studio or in the "open browser" to Firefox only appeared after I scrolled all the way to the bottom. In Firefox, columns scrolled properly (both header/file name and column details), whereas in R Studio the header/file name never scrolled properly (maybe because the filenames are long?).

Anyway, I removed some channels, chose some others as the "correct" marker name, and then hit go.

In R Studio: it worked for a bit, then the GUI collapsed; I only got 12 files out of the 36 (Center 2 and Center 3) written. Looking at the R Studio window, it appeared to get hung up: I never got the ">" back.

Trying it in Firefox, it hung up as well, though Firefox grayed out rather than collapsing the window.

Here's what the R Studio window said when running in R Studio GUI:

"Listening on http://127.0.0.1:6076
[1] "Reading FCS parameters..."
[1] "Done"
Warning in flowCore::write.FCS(flowFrame, path) :
'write.FCS' is not fully tested and should be considered as experimental.
Warning in flowCore::write.FCS(flowFrame, path) :
'write.FCS' is not fully tested and should be considered as experimental.
Warning in flowCore::write.FCS(flowFrame, path) :
'write.FCS' is not fully tested and should be considered as experimental.
Warning in flowCore::write.FCS(flowFrame, path) :
'write.FCS' is not fully tested and should be considered as experimental.
Warning in flowCore::write.FCS(flowFrame, path) :
'write.FCS' is not fully tested and should be considered as experimental.
Warning in flowCore::write.FCS(flowFrame, path) :
'write.FCS' is not fully tested and should be considered as experimental.
Warning in flowCore::write.FCS(flowFrame, path) :
'write.FCS' is not fully tested and should be considered as experimental.
Warning in flowCore::write.FCS(flowFrame, path) :
'write.FCS' is not fully tested and should be considered as experimental.
Warning in flowCore::write.FCS(flowFrame, path) :
'write.FCS' is not fully tested and should be considered as experimental.
Warning in flowCore::write.FCS(flowFrame, path) :
'write.FCS' is not fully tested and should be considered as experimental.
Warning in flowCore::write.FCS(flowFrame, path) :
'write.FCS' is not fully tested and should be considered as experimental.
Warning in flowCore::write.FCS(flowFrame, path) :
'write.FCS' is not fully tested and should be considered as experimental.
Warning: Error in : all(colnames(ret$m) %in% names(names.map)) is not TRUE
Stack trace (innermost first):
67: stopifnot
66: rename_fcs_parameters_desc
65: premessa:::rename_parameters_in_files
58: isolate
57: observerFunc [/Library/Frameworks/R.framework/Versions/3.3/Resources/library/premessa/paneleditor_shinyGUI/server.R#55]
2: shiny::runApp
1: paneleditor_GUI
ERROR: [on_request_read] connection reset by peer"

Here's what the R Studio window said when running in Firefox:

Listening on http://127.0.0.1:6076
[1] "Reading FCS parameters..."
[1] "Done"
[1] "Reading FCS parameters..."
[1] "Done"
Warning in flowCore::write.FCS(flowFrame, path) :
'write.FCS' is not fully tested and should be considered as experimental.
Warning in flowCore::write.FCS(flowFrame, path) :
'write.FCS' is not fully tested and should be considered as experimental.
Warning in flowCore::write.FCS(flowFrame, path) :
'write.FCS' is not fully tested and should be considered as experimental.
Warning in flowCore::write.FCS(flowFrame, path) :
'write.FCS' is not fully tested and should be considered as experimental.
Warning in flowCore::write.FCS(flowFrame, path) :
'write.FCS' is not fully tested and should be considered as experimental.
Warning in flowCore::write.FCS(flowFrame, path) :
'write.FCS' is not fully tested and should be considered as experimental.
Warning in flowCore::write.FCS(flowFrame, path) :
'write.FCS' is not fully tested and should be considered as experimental.
Warning in flowCore::write.FCS(flowFrame, path) :
'write.FCS' is not fully tested and should be considered as experimental.
Warning in flowCore::write.FCS(flowFrame, path) :
'write.FCS' is not fully tested and should be considered as experimental.
Warning in flowCore::write.FCS(flowFrame, path) :
'write.FCS' is not fully tested and should be considered as experimental.
Warning in flowCore::write.FCS(flowFrame, path) :
'write.FCS' is not fully tested and should be considered as experimental.
Warning in flowCore::write.FCS(flowFrame, path) :
'write.FCS' is not fully tested and should be considered as experimental.
Warning: Error in : all(colnames(ret$m) %in% names(names.map)) is not TRUE
Stack trace (innermost first):
67: stopifnot
66: rename_fcs_parameters_desc
65: premessa:::rename_parameters_in_files
58: isolate
57: observerFunc [/Library/Frameworks/R.framework/Versions/3.3/Resources/library/premessa/paneleditor_shinyGUI/server.R#55]
2: shiny::runApp
1: paneleditor_GUI

The only difference I could see is that in R Studio GUI, there was the additional final line of "ERROR: [on_request_read] connection reset by peer"

As I said, I never got the ">" back indicating that it had completed....I had to hit Esc to kill the process.

I also tried running it with no deletions of parameters just in case that was messing things up, and the same thing happened: only Center 2 and Center 3 had their files rewritten.

Mike

Hi,

I hope you're well.

As per the subject, I've been testing out the different debarcoders:

• The Zunder Lab Debarcoder (ZLD) using Matlab
• The Catalyst Debarcoder from the Bodenmiller lab using R
• and your Debarcoder in Premessa using R

Using the exact same normalised dataset, I've been getting different debarcoded event yields when comparing Catalyst and Premessa to the ZLD (which is my 'control' (As I've been using this for the last couple of years)), even though the % assigned is near enough the same between the three debarcoders.

As an example, for this one experiment I'm analysing (using the exact same debarcoding settings of 0.1 BCS and 10 MD), I'm getting a debarcoded event yield of around 49% for each debarcoder (which is normal as I'm barcoding organoids in situ using the TOBis 7-c-3 barcoding system - if you want to read more, see here). But the ungated event counts I'm getting differs greatly such that the Catalyst Debarcoder results in around 15% fewer events than the ZLD and the Premessa debarcoder results in a whopping 56% fewer events than the ZLD. Please see screenshots below.

Is there a reason for this? I read that you use an R-implementation of the Nolan Lab Debarcoder (which I think it's quite similar to the ZLD, but surely there shouldn't be a 56% discrepancy between your debarcoder and t the ZLD.

Any explanation/clarification would be extremely helpful.

Many thanks,

Jahangir Sufi

Hello,
I am suddenly having trouble normalizing and debarcoding files properly using premessa. Following normalization, I lose all protein marker names associated with each metal channel, and the resulting files do not look normal when placed in FlowJo.

Can anyone suggest a troubleshooting method? I am concerned that one of the supporting packages has changed but I do not know how to diagnose it.
Thank you,
Courtney

When using the Normalizer_GUI it would be helpful to be able to scan through the bead gates on each file by clicking "next" instead of having to select each file from the drop-down menu. Thanks!

Hi I am getting the following error message when trying to normalise some of my data:

Some data values of 'Tb159Di' channel exceed its $PnR value 12000 and will be truncated!
To avoid truncation, either fix $PnR before generating FCS or set 'truncate_max_range = FALSE'
Warning in readFCSdata(con, offsets, txt, transformation, which.lines, scale,

Warning in readFCSdata(con, offsets, txt, transformation, which.lines, scale, :
Some data values of 'Time' channel exceed its $PnR value 12607 and will be truncated!
To avoid truncation, either fix $PnR before generating FCS or set 'truncate_max_range = FALSE'

some data files are working well but for the ones that are not working that is the message I am getting

I am not sure what to do can anyone help?

Hi there,

Using premessa panel editor and selecting a file in a folder containing "PATHWITHDRIVE/FILENAME.FCS" files, we had the issue that no files were found by premessa, which resulted in an error in line 35 of file paneleditor_shinyGUI/server.R:
names(panel.table)[2] <- "Most common" as panel.table is empty.

Debugging the application I found this line (line 58 paneleditor_shinyGUI/server.R ) that did not return any files, just NULL.
files.list <- list.files(working.directory, pattern = "*.fcs$", ignore.case = T)

The pattern argument in list.files as I understand is expected to be regex and not a glob expansion.
Should the correct regex pattern then not be: pattern = ".*\.fcs$"?

I don't really know why this was an issue for us and not others, but this change resulted in the application working for us, but it might be a windows specific issue.

Checked premessa versions: 0.3.2, 0.3.4/ R version 4.1.1 / operating system windows 10

Thanks and best, cwlkr

Hi,

I'm not sure this is exactly an error, per se.

I have a time series of samples using the same barcode. In certain donors, a specific timepoint might be missing. In that case, I have edited the CSV key to remove the "missing" sample.

I noticed that when I leave the blank line, the CSV uploads properly, and only after I select the FCS file and it starts working, do I then get the following error message:

"Error: Barcoding schems with a variable number of positive channels per sample are not currently supported"

If I take out the blank line, then it works normally.

So, a couple things:

1. If I have a CSV key line that doesn't exist in the sample, is that OK? Like, if my BC5 sample isn't present in the sample, can I still leave in the BC5 line without problems? Obviously, I won't get a file, but will it interfere with the debarcoding?

2. If there's a problem with the CSV key, I would suggest that it should be flagged when the CSV file is selected, rather than waiting until later.

Mike

I am using premessa-0.2.6, macOS High Sierra and RStudio version 1.2.5042.

I have the same problem as #34 . I am using the sample data from

https://github.com/nolanlab/bead-normalization/tree/master/sample_data

and I got the following error:

I typed the suggested libraries and I got the following:

Thank you for your help!

Hello, I'm trying premessa, but it seems there is something wrong when I tried to open GUI following the steps in Readme.md.
Here's my input and output:

> library(premessa)
> paneleditor_GUI()
Loading required package: shiny

Listening on http://127.0.0.1:7708
Error in utils::browseURL(appUrl) : 
  'browser' must be a non-empty character string

My OS:
Ubuntu 18.04

R version:

R version 3.6.1 (2019-07-05) -- "Action of the Toes"
Copyright (C) 2019 The R Foundation for Statistical Computing
Platform: x86_64-conda_cos6-linux-gnu (64-bit)

And I am using anaconda virtual environment, conda version:
conda 4.8.2

I finally got output daughter files from the debarcoder. Note: these had already gone through the Premessa Normalizer.

However, when I dragged them into FlowJo, there were problems:

1. They all had the same Name. If I added a column for "File Name" (one of the FJ defaults), the appropriate daughter debarcode names did appear. Therefore, they are being written to the file, but not in the place that FJ expects.

2. Some parameter naming is now a bit screwy. Anything with only one "name" in the Parent file is now doubled.
   Parent: Time
   Daughter: Time :: Time

Same thing with Event_length, Center, Offset, Residual, Width, Residual, and beadDist.

Going back and comparing the Parent to Normalized, this appears to happen at the level of the Normalization (ie, before Debarcoding). Normalized and Debarcoded are consistent, as far as I can tell.

Hi! I just ran the normalization and bead removal and ended up with nicely normalized fcs files without beads but I lost the description of each channel. It looks like Nd145Di::Nd145Di instead. Just wondering if that's supposed to happen? I'd much rather have the descriptor than the channel name at the end...The descriptor seems to be lost completely.

Fluidigm has launched new normalizer beads with 6 elements:
"Description: EQ™ Six Element Calibration Beads (EQ6 beads) is a 1X stock solution (used at 0.1X for sample acquisition) of polystyrene bead standards containing known concentrations of the natural abundance metal isotopes yttrium (89Y), indium (115In), cerium (140Ce), terbium (159Tb), lutetium (175Lu), and bismuth (209Bi)."

https://www.fluidigm.com/binaries/content/documents/fluidigm/search/hippo%3Aresultset/tds-00691---201245---eq-six-element-calibration-beads/fluidigm%3Afile

Note that these beads contain Y, In, Tb, and Bi, but do not contain the Eu and Ho from the 4-element beads.

Do you have plans to add the Six-Element beads to the Premessa normalizer, like the current 4-element beads or the original Nolan-Synthesis beads?

Hi premessa team,

I have a question related to the concatenation of files. I have run my barcoded samples in separate runs on the CyTOF, and therefore I would like to concatenate these files before normalizing.
However, I have noticed that the concatenated files generated with premessa does not have an altered time variable. Does this not lead to problems in bead normalization?

Since time intervals are used to normalize, I think it is a problem that the time variable is not corrected (the time parameter in each subfile starts with 0). CATALYST for example does a correction...

Best, Christina

Package ‘ParkerICI/premessa’ is not available (for R version 3.4.4)

Hi premessa-team,

I've got a problem trying to generate a FCS file, because most of events has an unidentifiable error once generated and opened in a flow software:

This is the code I used, maybe I'm miss-considering something...

file_q <- read.FCS("tmp.fcs")
write.FCS(as_flowFrame(as.matrix(exprs(file_q)), source.frame = file_q), "p1_asFlowFrame.fcs")

... but creating a flowFrame manually these errors (it seems) disappear.

file_q <- read.FCS("tmp.fcs")
write.FCS(flowFrame(exprs(file_q), file_q@parameters, description = file_q@description), "p1_flowFrame.fcs")

Could you explain me, if you please, if I'm doing something wrong, please? Thanks for your help.

Hi premessa team,

I am trying to preprocess my files starting with concatenation, but I keep getting this error:
Error in writeBin(as(t(exprs(x)), what), con, size = types[what, "bitwidth"], : only 2^31-1 bytes can be written in a single writeBin() call

when I am running concatenate_fcs_files. If it helps, I have the same problem with CATALYST. I tried installing the newest version of flowCore from their github after seeing this issue, but nothing changed.

Can you help me resolve this?

Best,
Christina

Hey,

I'm trying to debarcode a file that have been barcoded with a 7 choose 3 scheme.
When debarcoding, the package debarcode based on the combinations that have been described in the key, however the combination are assigned to wrong samples.

Here is an example of debarcoded file : the correct barcoding scheme for this sample(sample 38) should be 0011010 and based on the plot it's clearly 0110100.

It look like there a shift somwhere but can't figure out where.

So i have few question to try to understand the origin/fix the problem:

1. The barcodes that I'm using don't all have the same intensity can that be problematic to debarcode ?
2. It look like the normalization doesn't change anyhting (left plot and right one are the same) is that normal ?

Thanks in advance

Hi there,

Just a minor enhancement recommendation with how keywords are assigned to make it more compatible with immediately viewing in FlowJo.

Right now, when running concatenate_fcs_files, if the $PnS short name is missing in the fcs files being concatenated, the function automatically copies the $PnN name to the $PnS short name of the output file. When you load the newly concatenated fcs files into FlowJo alongside other original fcs files, this causes flowjo to recognize them as alternative parameters (since some now have a $PnS short name, and some do not). Because of this, modifications to the axis (changing to biex, log, etc) are not properly copied across a group, as shown in the attached screenshot. If a $PnS short name existed for a parameter in the input fcs file(s), these compatibility issues do not arise.

While I'm sure I can remove the $PnS names from the concatenated files, having the option to not copy the $PnN name to blank $PnS spaces in the first place would be nice too!

Hi,

Is there an explicit order of operations for the steps of Premessa?

From the order in the Wiki, my assumption is that concatenation should happen before normalization. And that's consistent with old how-to documents like the attached PDF (see page 7).

However, since that's the opposite of how the Fluidigm software workflow operates, it would be nice to have an explicit statement in the Premessa wiki on this......

Mike

Normalization_EQ-4-Element-Beads_ug.pdf

Hi,

I've been using Premessa now for several months, with no real problems.

However, this week I ran into something: the plots no longer draw themselves in Premessa. For example, when I call the normalizerGUI in RStudio, the window opens, I can select my FCS file directory, and then I get the dropdown menus. The file menu lists all the relevant FCS files, but when I select one, the Ir vs bead channel plots never materialize......not even after 5-10min for a single 250K event FCS file.

If I have it open in the browser, then it does work, and I get the expected daughter normalized files.

I'm also able to run the Debarcoder through the web browser interface.

I can't even get old files I've previously normalized to work, now. I've tried updating RStudio, uninstalling Shiny and Premessa and reinstalling, etc. Nothing I do makes a difference.

One thing that's a little weird: in the Normalizer, it's like there are placeholders for the graphs. If I double-click in the white space under the drop-downs and buttons, I get blue rectangles, but I never get any plots.....maybe the plotting function is broken somehow?

Please advise: as you can understand, this makes Premessa unusable.

I'm on a MacOSX, running RStudio 1.1.456, R 3.3.3.

Mike

Hello,

I'm new to Premessa, and I've been trying to use the normalizer_GUI() to do bead normalization but running into repeated issues with this error:

I've triple checked that the fcs files I'm feeding in have the same number of channels and channel names, but I keep getting this error no matter which fcs files I feed in (even two fcs files from the same experiment).

Would you have any insight on this? This also seems to happen regardless of if I choose "Fluidigm" or "beta beads" as an option. Thank you so much!

When changing parameters for the debarcoder, the plots update when there's even a pause in typing. I think shiny's reactive values are a little bit too reactive, especially since making plots can get very computationally expensive with larger barcoded runs. It would run much more smoothly if plot updating could be tied to an update button rather than just reactive values.

Hi premessa team,

I have detected a problem in the normalization process. Files are read into the program without using the truncate_max_range = F option for read.FCS, and this results in times being truncated for my concatenated files. I also think it confuses the bead normalization a lot.

Perhaps you would want to look into this?

Best, Christina

Hi,

I have 20 .FCS files, when I use "normalizer_GUI" to normalize them one by one, 10 files can have output "xxx_normalized.fcs", but the other 10 got an error messeage:

Warning: Error in solve.default: system is computationally singular: reciprocal condition number = 1.00251e-19
93: solve.default
91: mahalanobis
90: get_mahalanobis_distance_from_beads
88: FUN
87: lapply
86: premessa::normalize_folder
73: observeEventHandler [C:\Users\ywang535\Documents\R\R-3.4.4\library\premessa\normalizer_shinyGUI/server.R#266]
2: shiny::runApp
1: normalizer_GUI

Setting is just default, as this:

Screenshot of error in R is this:

Can you help me with this? Thank you!

Thanks for sharing premessa with the world! I tried it on a 350MB file from Rachel's normalizer and it worked fine. I then tried it on a 3.07GB file from the Fluidigm CyTOF2 normalizer and got a "subscript out of bounds" error. I'll send you a link via email to download the 3.07GB file if you'd like to test it yourself.

Erin

lazy-load database '/Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/library/premessa/R/premessa.rdb' is corrupt

sessionInfo()
R version 4.2.2 (2022-10-31)
Platform: aarch64-apple-darwin20 (64-bit)
Running under: macOS Ventura 13.2.1

Matrix products: default
LAPACK: /Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] grid stats4 stats graphics grDevices utils datasets methods base

other attached packages:
[1] shiny_1.7.4 premessa_0.3.4 BiocManager_1.30.19 RColorBrewer_1.1-3 viridis_0.6.2
[6] viridisLite_0.4.1 ggbeeswarm_0.7.1 ggridges_0.5.4 readxl_1.4.2 uwot_0.1.14
[11] Matrix_1.5-3 boot_1.3-28.1 MASS_7.3-58.2 magrittr_2.0.3 scico_1.3.1
[16] devtools_2.4.5 usethis_2.1.6 reshape2_1.4.4 doBy_4.6.16 FlowSOM_2.6.0
[21] igraph_1.4.0 gdata_2.18.0.1 pheatmap_1.0.12 paletteer_1.5.0 effsize_0.8.1
[26] patchwork_1.1.2.9000 lubridate_1.9.2 forcats_1.0.0 stringr_1.5.0 purrr_1.0.1
[31] readr_2.1.4 tidyr_1.3.0 tibble_3.1.8 tidyverse_2.0.0 ggrepel_0.9.3
[36] ggrastr_1.0.1 broom_1.0.3 dplyr_1.1.0 easypackages_0.1.0 ggcyto_1.26.4
[41] flowWorkspace_4.10.1 ncdfFlow_2.44.0 BH_1.81.0-1 ggplot2_3.4.1 flowCore_2.10.0
[46] cowplot_1.1.1 CATALYST_1.22.0 SingleCellExperiment_1.20.0 SummarizedExperiment_1.28.0 Biobase_2.58.0
[51] GenomicRanges_1.50.2 GenomeInfoDb_1.34.9 IRanges_2.32.0 S4Vectors_0.36.1 BiocGenerics_0.44.0
[56] MatrixGenerics_1.10.0 matrixStats_0.63.0

loaded via a namespace (and not attached):
[1] scattermore_0.8 ragg_1.2.5 SeuratObject_4.1.3 knitr_1.42 irlba_2.3.5.1
[6] multcomp_1.4-22 DelayedArray_0.24.0 data.table_1.14.8 RCurl_1.98-1.10 doParallel_1.0.17
[11] generics_0.1.3 ScaledMatrix_1.6.0 callr_3.7.3 TH.data_1.1-1 RANN_2.6.1
[16] future_1.31.0 tzdb_0.3.0 spatstat.data_3.0-0 httpuv_1.6.9 assertthat_0.2.1
[21] xfun_0.37 jquerylib_0.1.4 hms_1.1.2 evaluate_0.20 promises_1.2.0.1
[26] fansi_1.0.4 Rgraphviz_2.42.0 DBI_1.1.3 htmlwidgets_1.6.1 reshape_0.8.9
[31] spatstat.geom_3.0-6 ellipsis_0.3.2 ggnewscale_0.4.8 ggpubr_0.6.0 backports_1.4.1
[36] cytolib_2.10.0 deldir_1.0-6 sparseMatrixStats_1.10.0 vctrs_0.5.2 remotes_2.4.2
[41] ROCR_1.0-11 abind_1.4-5 cachem_1.0.6 withr_2.5.0 ggforce_0.4.1
[46] progressr_0.13.0 sctransform_0.3.5 prettyunits_1.1.1 goftest_1.2-3 cluster_2.1.4
[51] lazyeval_0.2.2 crayon_1.5.2 drc_3.0-1 spatstat.explore_3.0-6 pkgconfig_2.0.3
[56] labeling_0.4.2 tweenr_2.0.2 nlme_3.1-162 vipor_0.4.5 pkgload_1.3.2
[61] rlang_1.0.6 globals_0.16.2 lifecycle_1.0.3 miniUI_0.1.1.1 sandwich_3.0-2
[66] rsvd_1.0.5 rprojroot_2.0.3 cellranger_1.1.0 polyclip_1.10-4 lmtest_0.9-40
[71] graph_1.76.0 carData_3.0-5 zoo_1.8-11 beeswarm_0.4.0 GlobalOptions_0.1.2
[76] processx_3.8.0 png_0.1-8 rjson_0.2.21 bitops_1.0-7 ConsensusClusterPlus_1.62.0
[81] KernSmooth_2.23-20 DelayedMatrixStats_1.20.0 shape_1.4.6 parallelly_1.34.0 spatstat.random_3.1-3
[86] shinyjqui_0.4.1 rstatix_0.7.2 ggsignif_0.6.4 beachmat_2.14.0 scales_1.2.1
[91] memoise_2.0.1 plyr_1.8.8 hexbin_1.28.2 ica_1.0-3 zlibbioc_1.44.0
[96] compiler_4.2.2 plotrix_3.8-2 clue_0.3-64 fitdistrplus_1.1-8 cli_3.6.0
[101] urlchecker_1.0.1 XVector_0.38.0 listenv_0.9.0 pbapply_1.7-0 ps_1.7.2
[106] tidyselect_1.2.0 stringi_1.7.12 RProtoBufLib_2.10.0 textshaping_0.3.6 yaml_2.3.7
[111] BiocSingular_1.14.0 sass_0.4.5 timechange_0.2.0 tools_4.2.2 future.apply_1.10.0
[116] parallel_4.2.2 circlize_0.4.15 rstudioapi_0.14 foreach_1.5.2 gridExtra_2.3
[121] farver_2.1.1 Rtsne_0.16 digest_0.6.31 Rcpp_1.0.10 microbenchmark_1.4.9
[126] car_3.1-1 scuttle_1.8.4 later_1.3.0 RcppAnnoy_0.0.20 httr_1.4.4
[131] ComplexHeatmap_2.14.0 Deriv_4.1.3 colorspace_2.1-0 XML_3.99-0.13 fs_1.6.1
[136] tensor_1.5 reticulate_1.28 splines_4.2.2 rematch2_2.1.2 spatstat.utils_3.0-1
[141] scater_1.26.1 sp_1.6-0 systemfonts_1.0.4 plotly_4.10.1 sessioninfo_1.2.2
[146] xtable_1.8-4 jsonlite_1.8.4 R6_2.5.1 profvis_0.3.7 pillar_1.8.1
[151] htmltools_0.5.4 mime_0.12 nnls_1.4 glue_1.6.2 fastmap_1.1.0
[156] BiocParallel_1.32.5 BiocNeighbors_1.16.0 codetools_0.2-19 pkgbuild_1.4.0 mvtnorm_1.1-3
[161] utf8_1.2.3 bslib_0.4.2 lattice_0.20-45 spatstat.sparse_3.0-0 curl_5.0.0
[166] leiden_0.4.3 colorRamps_2.3.1 gtools_3.9.4 survival_3.5-3 rmarkdown_2.20
[171] desc_1.4.2 munsell_0.5.0 GetoptLong_1.0.5 GenomeInfoDbData_1.2.9 iterators_1.0.14
[176] gtable_0.3.1 Seurat_4.3.0

Hi

I´m getting this error when using files from FCS Express v7:

Warning: Error in rename_fcs_parameters_name: is.matrix(m) is not TRUE
62:

Any idea of where the problem could be?

This is a copy of the structure of one of the fcs files:
Formal class 'flowFrame' [package "flowCore"] with 3 slots
..@ exprs : num [1:41381, 1:24] 3.35 3.85 5.01 5.49 5.84 ...
.. ..- attr(, "dimnames")=List of 2
.. .. ..$ : NULL
.. .. ..$ : Named chr [1:24] "Time" "FSC-A" "FSC-H" "FSC-W" ...
.. .. .. ..- attr(, "names")= chr [1:24] "$P1N" "$P2N" "$P3N" "$P4N" ...
.. ..- attr(, "ranges")= num [1:24] 262143 262143 382397 262143 262143 ...
..@ parameters :Formal class 'AnnotatedDataFrame' [package "Biobase"] with 4 slots
.. .. ..@ varMetadata :'data.frame': 5 obs. of 1 variable:
.. .. .. ..$ labelDescription: chr [1:5] "Name of Parameter" "Description of Parameter" "Range of Parameter" "Minimum Parameter Value after Transforamtion" ...
.. .. ..@ data :'data.frame': 24 obs. of 5 variables:
.. .. .. ..$ name : 'AsIs' Named chr [1:24] "Time" "FSC-A" "FSC-H" "FSC-W" ...
.. .. .. .. ..- attr(, "names")= chr [1:24] "$P1N" "$P2N" "$P3N" "$P4N" ...
.. .. .. ..$ desc : 'AsIs' Named chr [1:24] NA NA NA NA ...
.. .. .. .. ..- attr(*, "names")= chr [1:24] NA NA NA NA ...
.. .. .. ..$ range : num [1:24] 262144 262144 382398 262144 262144 ...
.. .. .. ..$ minRange: num [1:24] 0 0 0 0 0 0 0 -111 -111 -111 ...
.. .. .. ..$ maxRange: num [1:24] 262143 262143 382397 262143 262143 ...
.. .. ..@ dimLabels : chr [1:2] "rowNames" "columnNames"
.. .. ..@ .classVersion:Formal class 'Versions' [package "Biobase"] with 1 slot
.. .. .. .. ..@ .Data:List of 1
.. .. .. .. .. ..$ : int [1:3] 1 1 0
..@ description:List of 319
.. ..$ FCSversion : chr "3"
.. ..$ $BTIM : chr "12:55:01"
.. ..$ $BYTEORD : chr "1,2,3,4"
.. ..$ $DATATYPE : chr "F"
.. ..$ $DATE : chr "29-NOV-2019"
.. ..$ $ETIM : chr "12:58:16"
.. ..$ $FIL : chr "398160.fcs"
.. ..$ $INST : chr " "
.. ..$ $MODE : chr "L"
.. ..$ $NEXTDATA : chr "0"
.. ..$ $P10B : chr "32"
.. ..$ $P10E : chr "0,0"
.. ..$ $P10G : chr "1.0"
.. ..$ $P10N : chr "640 730/45-A"
.. ..$ $P10R : chr "2113174"
.. ..$ $P10S : chr "PD1"
.. ..$ $P10V : chr "555"
.. ..$ $P11B : chr "32"
.. ..$ $P11E : chr "0,0"
.. ..$ $P11G : chr "1.0"
.. ..$ $P11N : chr "405 450/50-A"
.. ..$ $P11R : chr "262144"
.. ..$ $P11S : chr "CXCR3"
.. ..$ $P11V : chr "471"
.. ..$ $P12B : chr "32"
.. ..$ $P12E : chr "0,0"
.. ..$ $P12G : chr "1.0"
.. ..$ $P12N : chr "405 610/20-A"
.. ..$ $P12R : chr "1248827"
.. ..$ $P12S : chr "CD57"
.. ..$ $P12V : chr "585"
.. ..$ $P13B : chr "32"
.. ..$ $P13E : chr "0,0"
.. ..$ $P13G : chr "1.0"
.. ..$ $P13N : chr "405 780/60-A"
.. ..$ $P13R : chr "6428500"
.. ..$ $P13S : chr "CD4"
.. ..$ $P13V : chr "610"
.. ..$ $P14B : chr "32"
.. ..$ $P14E : chr "0,0"
.. ..$ $P14G : chr "1.0"
.. ..$ $P14N : chr "355 379/28-A"
.. ..$ $P14R : chr "262144"
.. ..$ $P14S : chr "CD3"
.. ..$ $P14V : chr "565"
.. ..$ $P15B : chr "32"
.. ..$ $P15E : chr "0,0"
.. ..$ $P15G : chr "1.0"
.. ..$ $P15N : chr "355 515/30-A"
.. ..$ $P15R : chr "415039"
.. ..$ $P15S : chr "CD8"
.. ..$ $P15V : chr "500"
.. ..$ $P16B : chr "32"
.. ..$ $P16E : chr "0,0"
.. ..$ $P16G : chr "1.0"
.. ..$ $P16N : chr "355 740/35-A"
.. ..$ $P16R : chr "1470702"
.. ..$ $P16S : chr "CD25"
.. ..$ $P16V : chr "748"
.. ..$ $P17B : chr "32"
.. ..$ $P17E : chr "0,0"
.. ..$ $P17G : chr "1.0"
.. ..$ $P17N : chr "561 586/15-A"
.. ..$ $P17R : chr "13737799"
.. ..$ $P17S : chr "CCR6"
.. ..$ $P17V : chr "556"
.. ..$ $P18B : chr "32"
.. ..$ $P18E : chr "0,0"
.. ..$ $P18G : chr "1.0"
.. ..$ $P18N : chr "561 610/20-A"
.. ..$ $P18R : chr "3086331"
.. ..$ $P18S : chr "CD56"
.. ..$ $P18V : chr "516"
.. ..$ $P19B : chr "32"
.. ..$ $P19E : chr "0,0"
.. ..$ $P19G : chr "1.0"
.. ..$ $P19N : chr "561 780/60-A"
.. ..$ $P19R : chr "262144"
.. ..$ $P19S : chr "CCR7"
.. ..$ $P19V : chr "516"
.. ..$ $P1B : chr "32"
.. ..$ $P1E : chr "0,0"
.. ..$ $P1G : chr "0.01"
.. ..$ $P1N : chr "Time"
.. ..$ $P1R : chr "262144"
.. ..$ $P20B : chr "32"
.. ..$ $P20E : chr "0,0"
.. ..$ $P20G : chr "1.0"
.. ..$ $P20N : chr "561 710/50-A"
.. ..$ $P20R : chr "262144"
.. ..$ $P20S : chr "7AAD"
.. ..$ $P20V : chr "533"
.. ..$ $P21B : chr "32"
.. ..$ $P21E : chr "0,0"
.. ..$ $P21G : chr "1.0"
.. ..$ $P21N : chr "488 695/40-A"
.. ..$ $P21R : chr "2727842"
.. ..$ $P21S : chr "CD127"
.. ..$ $P21V : chr "590"
.. .. [list output truncated]

Regards
Juan

The file selection window is not showing upon starting the normalizer_GUI().

Hi,

I am able to initiate the panel editor just fine and select the FCS file I want to alter, but when I click the button in the UI to process files I always get the following error:

Listening on http://127.0.0.1:3114
[1] "Reading FCS parameters..."
[1] "Done"
Warning: Error in do.call: second argument must be a list
Stack trace (innermost first):
66: do.call
65: rhandsontable::hot_to_r
58: isolate
57: observerFunc [/Users/knmorro/Library/R/3.4/library/premessa/paneleditor_shinyGUI/server.R#54]
2: shiny::runApp
1: paneleditor_GUI
ERROR: [on_request_read] connection reset by peer

I suddenly started getting error during debarcoding. I can loads the CSV and the FCS and see the plots but it crashes during the attempt to save the daughter files.

R, Rstudio and all relevant packages are updated.
Any ideas?
Thanks!
Natalia

I realized today that the scales for all the scatterplots are different when looking at the "plot all barcode biaxials" view, which makes it challenging to assess the strength of the signal on the positive channels in relation to the negative channels. I think it would be a much more useful visualization if they all had fixed scales. I wrote a small modification to the plot_all_barcode_biaxials() function in the package that fixes the scales on all the plots to be between 0-6 in arcsinh space, and it works well I assign it in my namespace before running the GUI. If you agree that it's a worthwhile feature, feel free to use the code below:

fixed_scales_plot <- function (m, bc.channels) {
    plotlist <- list()
    mahal.dist <- m[, "mahal.dist"]
    m <- m[, bc.channels]
    m <- cbind(m, mahal.dist = mahal.dist)
    m <- data.frame(m)
    for (i in 1:length(bc.channels)) for (j in 1:length(bc.channels)) {
        if (i == j) {
            plot <- (ggplot2::ggplot(ggplot2::aes_string(x = names(m)[i]), 
                                     data = m) + ggplot2::geom_histogram() + ggplot2::scale_x_continuous("") + 
                         ggplot2::scale_y_continuous("") +
                         ggplot2::expand_limits(x = c(0, 6)))
            if (i == 1) 
                plot <- plot + ggplot2::scale_y_continuous(names(m)[i])
            if (j == length(bc.channels)) 
                plot <- plot + ggplot2::scale_x_continuous(names(m)[i])
        }
        else {
            plot <- ggplot2::ggplot(ggplot2::aes_string(x = names(m)[j], 
                                                        y = names(m)[i], colour = "mahal.dist"), data = m)
            if (i > j) {
                plot <- plot + ggplot2::geom_point()
                plot <- plot + ggplot2::scale_colour_gradientn("", 
                                                               breaks = 0:30, colours = rainbow(3))
                plot <- plot + ggplot2::expand_limits(x = c(0, 6), y = c(0, 6))
            }
            else plot <- (plot + ggplot2::geom_blank() + ggplot2::theme(panel.grid.major = ggplot2::element_blank(), 
                                                                        panel.grid.minor = ggplot2::element_blank(), 
                                                                        panel.border = ggplot2::element_blank(), panel.background = ggplot2::element_blank()))
            if (j != 1) 
                plot <- plot + ggplot2::scale_y_continuous("")
            if (i != length(bc.channels)) 
                plot <- plot + ggplot2::scale_x_continuous("")
        }
        plot <- plot + ggplot2::theme(axis.line = ggplot2::element_blank(), 
                                      axis.ticks = ggplot2::element_blank(), axis.text = ggplot2::element_blank(), 
                                      legend.position = "none")
        plotlist <- c(plotlist, list(plot))
    }
    plotlist <- c(plotlist, list(nrow = length(bc.channels)))
    fun <- get("grid.arrange", asNamespace("gridExtra"))
    return(do.call(fun, plotlist))
}

assignInNamespace("plot_all_barcode_biaxials", fixed_scales_plot, "premessa")

• My configuration:

  • OS X Sierra or High Sierra (tried both)
  • Premessa version 0.1.8
  • RStudio version 1.1.383 or 1.1.453 (tried both)

• Symptom:

  • Dot plots in normalizer_GUI() are overlapping in Rstudio's built-in Shiny viewer, making it difficult to see and gate beads (see screenshots)

• Workaround, but it's kind of annoying:

  • Click "Open in Browser" in Rstudio's Shiny Viewer. This opens a new session of normalizer_GUI() in Google Chrome. With the zoom at 100% they are still somewhat overlapping. When I zoom out to 80%, it looks good. This gets pretty confusing though because they Shiny Viewer is still open in the background, and I need to choose the FCS file folder twice.

Thanks!

Erin

See RGLab/flowCore#169
I think

keyval[[r]] <- max(exprs.m[,i])

At least this is how flowCore interpret PnR

It would be nice if there were an option to select Custom channels for beads. This was a feature in the original Finck MATLAB normalizer ("Custom").

This would help address any future Norm bead developments before they got added to the default dropdowns (like how it was a few weeks before XT beads got added), and would also help cases where a bead channel got left out of a panel (and therefore FCS file) by mistake.

See here for one example: http://cytoforum.stanford.edu/viewtopic.php?f=4&t=3705&p=7957

The panel editor is a very interesting tool. Thanks for delivering it.

Whereas the orange color is dedicated to highlight missing parameters, missing parameters are currently colored in pink. This seems to be due to that the 'NA' keyword is not present in such cells of the table. It may be due to a change in handsometable.

Anyway, I think the code could force the resulting from the merge function to be coded as 'NA'. Alternatively, I would suggest that NA should be replaced with a more informative word such as 'missing' or 'absent'.

See the discussion in #5

Hi,

I'm trying to clean multiple FCS data files for use with cytofkit. I've got samples analysed with the same markers but with slightly different channel numbers and names between fcs files. I've successfully used Premessa to delete additional channels but when I've edited channel names to match between panels, the result was that when I loaded the edited files into Premessa I've got two channels with the same name but no merging between the sample data, which means I can't use the FCS files together for downstream analysis.

I've included two images which show my problem, the first is before changing channel names, the second is after, showing the identical channel names but no merging between the panels.

Is it possible to use your program to merge these data files?

Thank you!

Barcoding schems with a variable number of positive channels per sample are not currently supported.

Hi! I have version R 4.0.0 and I tried to download the install_github("ParkerICI/premessa") but it doesn't work. I tried to download the vctrs package as well. Can you help me, please? Thanks

Hi,

1. For normalization: The Wiki states:
   "Identify beads: clicking this button will color in red the events that are recognized as beads events in the gating plots.

Apply current gates to all files: applies the current gates to all the files.

Normalize: starts the normalization routine. When the process is completed a confirmation dialog will appear

The workflow involves cycling through all the files and adjusting the beads gates in the plot, in order to identify the beads. Only events that are included in all the beads gates are identified as beads. As detailed in the dialog box that is above the row of buttons, only files for which the gates have been defined will be used as input for normalization.

You can cycle back and forth between different files, as the GUI will remember the gates you have selected for each file."

My question is about:
"applies the current gates to all the files"
"Only events that are included in all the beads gates are identified as beads."

By "all the beads gates", I assume you mean within each file, separately from the other files, right?
For example, I had a bit of a shift in my Eu151 signal between my two files (A and B), and therefore had to nudge that. The position of that gate in A does not affect the position of that gate in B (aside from the later normalization calculation, of course).

2. When the GUI first opened, the x-dimension was too small: the plots piled on top of each other, to the point that the next higher mass plot actually hid the beads (151 hid 140, 153 hid 151, etc), to where I could only see 175. It was a simple fix to resize the Gui window, but it took me a second to realize what had happened.

Hello, I hope you're well. Lately, when I have been using the premessa Debarcoder, the cell counts I get on the y-axis of the bar chart and in the separation curve has been in the 1000s, but each debarcoded sample actually has cell/event counts in 100,000s (which I can see when i upload the debarcoded FCS file into FlowJo). I've attached some screenshots to demonstrate.

When I started using the debarcoder many months ago, it didn't seem to have this problem previously. I'm not sure if this is affecting the actual debarcoding of the sample.

Thanks in advance for your help/reply.
Debarcoder screenshot with cell counts.pdf
Debarcoder separation screenshot with cell counts.pdf
FlowJo screenshot with event counts.pdf

Best,
Rebecca

Hi,

I had some problems with my BC key CSV being allowed by the Debarcoder GUI.

I created the BC key in Excel, then saved as CSV. I then got:
"Error: duplicate 'row.names' are not allowed"

My row (BC) names were in the format "Donor_Cells", so "Pat03_2M". I have 2M, 1M, and 05M samples, for each of 3 donors.

• see BC key.csv and .xlsx

I have been unable to attach CSV files to this post by Dropping them (only xlsx come over), so all the relevant CSV files that I tried can be found here:
https://drive.google.com/open?id=0Bz9sd8nC272ja1ZCS25FY19aNDg

I tried removing the "_", and it wouldn't work (BC key-B.csv and .xlsx files)

I tried the copy and paste-as-text trick mentioned in the MATLAB normalizer Wiki, and it also didn't work (BC key-1.csv)

So, I thought maybe the names were too long, and I started paring them down. When I got one to work, I extended the name a bit longer and tried again. Almost all of them worked (only E, a true duplicate, failed) (files through K).

I then started replacing individual lines with my Donor names, again (starting in L), and each of them worked.....all the way out to S, which to my non-computer eyes is identicalto* the original BC key.csv that failed.

In short: I don't know why my original CSV file was disallowed by the Debarcoder, nor why BC key-S.csv" is allowed, even though it looks identical to "BC key.csv"

Please advise.....

Mike