bsungwoo / spseudomap Goto Github PK

GPU is available and will be used
Shape of the provided spatial data is (2702, 32285)
normalizing counts per cell
    finished (0:00:00)
Using TensorFlow backend.
Shape of the provided single-cell data is (13309, 12819)
normalizing counts per cell
    finished (0:00:00)
ranking genes
    finished: added to `.uns['rank_genes_groups']`
    'names', sorted np.recarray to be indexed by group ids
    'scores', sorted np.recarray to be indexed by group ids
    'logfoldchanges', sorted np.recarray to be indexed by group ids
    'pvals', sorted np.recarray to be indexed by group ids
    'pvals_adj', sorted np.recarray to be indexed by group ids (0:00:10)
Total number of cell type marker genes is 524
Dataframe summarizing counts of pseudobulk transcriptomes and log fold change
               sp_count  sc_count      sp_ratio      sc_ratio      logfc
mt-Nd3         498123.0     616.0  6.642646e-03  5.731767e-06  10.178562
Hba-a1         122435.0     175.0  1.632714e-03  1.628343e-06   9.969652
Hba-a2         100737.0     148.0  1.343363e-03  1.377113e-06   9.929987
Msx1              652.0       1.0  8.694650e-06  9.304817e-09   9.867933
Hbb-bt          55835.0      94.0  7.445794e-04  8.746528e-07   9.733500
...                 ...       ...           ...           ...        ...
Hs6st3              3.0  123132.0  4.000606e-08  1.145721e-03 -14.805677
4930419G24Rik       1.0   44670.0  1.333535e-08  4.156462e-04 -14.927814
Gm37240             1.0   51744.0  1.333535e-08  4.814685e-04 -15.139898
Sorbs2os            1.0   65776.0  1.333535e-08  6.120336e-04 -15.486068
Xist                1.0   67598.0  1.333535e-08  6.289870e-04 -15.525488

[11939 rows x 5 columns]
Total number of genes for training: 655
computing score 'pseudotype'
    finished: added
    'pseudotype', score of gene set (adata.obs).
    300 total control genes are used. (0:00:00)
Genes used for calculation of pseudotype fraction
['mt-Nd3', 'Hba-a1', 'Hba-a2', 'Msx1', 'Hbb-bt', 'Hbb-bs', 'mt-Atp6', 'Cldn5', 'Ly6a', 'mt-Cytb', 'mt-Co2', 'Sparc', 'mt-Nd1', 'Myoc', 'Vtn', 'mt-Co1', 'Pmch', 'Otp', 'Mlc1', 'Ttr']
Size of spatial, single-cell, pseudospot, and cell fraction data: (2702, 655) (13309, 655) (13510, 655) (13510, 24)
Epoch 1/10
27/27 [==============================] - 1s 2ms/step - loss: 1.9174 - mae: 0.0557
Epoch 2/10
27/27 [==============================] - 0s 2ms/step - loss: 1.2118 - mae: 0.0478
Epoch 3/10
27/27 [==============================] - 0s 2ms/step - loss: 0.7561 - mae: 0.0397
Epoch 4/10
27/27 [==============================] - 0s 2ms/step - loss: 0.4277 - mae: 0.0283
Epoch 5/10
27/27 [==============================] - 0s 2ms/step - loss: 0.2626 - mae: 0.0188
Epoch 6/10
27/27 [==============================] - 0s 2ms/step - loss: 0.2002 - mae: 0.0142
Epoch 7/10
27/27 [==============================] - 0s 2ms/step - loss: 0.1728 - mae: 0.0123
Epoch 8/10
27/27 [==============================] - 0s 2ms/step - loss: 0.1558 - mae: 0.0113
Epoch 9/10
27/27 [==============================] - 0s 2ms/step - loss: 0.1428 - mae: 0.0105
Epoch 10/10
27/27 [==============================] - 0s 2ms/step - loss: 0.1338 - mae: 0.0099
initial_train_done
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Input In [18], in <cell line: 3>()
      1 # from spSeudoMap.pred_cellf_spSeudoMap import pred_cellf_spSeudoMap
      2 from spSeudoMap.pred_cellf_spSeudoMap import pred_cellf_spSeudoMap
----> 3 adata_spatial_cellf = pred_cellf_spSeudoMap(adata_sp=adata_spatial, adata_sc=adata_sc_sub, count_from_raw=False, 
      4                                             gpu=True, celltype=celltype, num_markers=num_markers,
      5                                             mixture_mode='pseudotype', seed_num=0, 
      6                                             mk_ratio_fix=True, mk_ratio=mk_ratio,
      7                                             pseudo_frac_m=pseudo_frac_m, pseudo_frac_std=pseudo_frac_std,
      8                                             nmix=nmix, npseudo=npseudo, alpha=0.6, alpha_lr=5, emb_dim=64, 
      9                                             batch_size=512, n_iterations=3000, init_train_epoch=10, 
     10                                             outdir=out_dir, return_format='anndata')

File ~/CMRI Dropbox/Chunlei liu/project_integration_benchmark/Chunlei/benchmark_method/diagonal integration/python/spSeudoMap/spSeudoMap/spSeudoMap/pred_cellf_spSeudoMap.py:243, in pred_cellf_spSeudoMap(adata_sp, adata_sc, count_from_raw, gpu, celltype, num_markers, mixture_mode, seed_num, mk_ratio_fix, mk_ratio, pseudo_num_genes, pseudo_frac_m, pseudo_frac_std, num_top_genes, nmix, npseudo, alpha, alpha_lr, emb_dim, batch_size, n_iterations, init_train_epoch, outdir, return_format)
    238 print('Size of spatial, single-cell, pseudospot, and cell fraction data:',
    239         mat_sp_s.shape, mat_sc_s.shape, sc_mix_s.shape, lab_mix.shape)
    242 # Train the domain adaptation model
--> 243 embs, clssmodel = da_cellfraction.train(sc_mix_s, lab_mix, mat_sp_s, 
    244                                         emb_dim = emb_dim, batch_size = batch_size,
    245                                         n_iterations = n_iterations,
    246                                         enable_dann = True,
    247                                         alpha = alpha, alpha_lr = alpha_lr,
    248                                         initial_train = True,
    249                                         initial_train_epochs = init_train_epoch)
    250 # Prediction of cell fraction in each spot
    251 pred_sp = pd.DataFrame(clssmodel.predict(mat_sp_s))

File ~/CMRI Dropbox/Chunlei liu/project_integration_benchmark/Chunlei/benchmark_method/diagonal integration/python/spSeudoMap/spSeudoMap/spSeudoMap/da_cellfraction.py:164, in train(Xs, ys, Xt, yt, emb_dim, batch_size, enable_dann, n_iterations, alpha, alpha_lr, initial_train, initial_train_epochs)
    162 print(y_adversarial_2.shape)
    163 print("!!!!")
--> 164 domain_classification_model.train_on_batch(X_adv, [y_adversarial_2])
    165 # retrain the domain classification model with inverted labels for pseudospot + spots
    167 k = 0

File ~/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py:1722, in Model.train_on_batch(self, x, y, sample_weight, class_weight, reset_metrics, return_dict)
   1718   iterator = data_adapter.single_batch_iterator(self.distribute_strategy, x,
   1719                                                 y, sample_weight,
   1720                                                 class_weight)
   1721   self.train_function = self.make_train_function()
-> 1722   logs = self.train_function(iterator)
   1724 if reset_metrics:
   1725   self.reset_metrics()

File ~/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py:828, in Function.__call__(self, *args, **kwds)
    826 tracing_count = self.experimental_get_tracing_count()
    827 with trace.Trace(self._name) as tm:
--> 828   result = self._call(*args, **kwds)
    829   compiler = "xla" if self._experimental_compile else "nonXla"
    830   new_tracing_count = self.experimental_get_tracing_count()

File ~/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py:871, in Function._call(self, *args, **kwds)
    868 try:
    869   # This is the first call of __call__, so we have to initialize.
    870   initializers = []
--> 871   self._initialize(args, kwds, add_initializers_to=initializers)
    872 finally:
    873   # At this point we know that the initialization is complete (or less
    874   # interestingly an exception was raised) so we no longer need a lock.
    875   self._lock.release()

File ~/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py:725, in Function._initialize(self, args, kwds, add_initializers_to)
    722 self._lifted_initializer_graph = lifted_initializer_graph
    723 self._graph_deleter = FunctionDeleter(self._lifted_initializer_graph)
    724 self._concrete_stateful_fn = (
--> 725     self._stateful_fn._get_concrete_function_internal_garbage_collected(  # pylint: disable=protected-access
    726         *args, **kwds))
    728 def invalid_creator_scope(*unused_args, **unused_kwds):
    729   """Disables variable creation."""

File ~/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/eager/function.py:2969, in Function._get_concrete_function_internal_garbage_collected(self, *args, **kwargs)
   2967   args, kwargs = None, None
   2968 with self._lock:
-> 2969   graph_function, _ = self._maybe_define_function(args, kwargs)
   2970 return graph_function

File ~/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/eager/function.py:3361, in Function._maybe_define_function(self, args, kwargs)
   3357   return self._define_function_with_shape_relaxation(
   3358       args, kwargs, flat_args, filtered_flat_args, cache_key_context)
   3360 self._function_cache.missed.add(call_context_key)
-> 3361 graph_function = self._create_graph_function(args, kwargs)
   3362 self._function_cache.primary[cache_key] = graph_function
   3364 return graph_function, filtered_flat_args

File ~/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/eager/function.py:3196, in Function._create_graph_function(self, args, kwargs, override_flat_arg_shapes)
   3191 missing_arg_names = [
   3192     "%s_%d" % (arg, i) for i, arg in enumerate(missing_arg_names)
   3193 ]
   3194 arg_names = base_arg_names + missing_arg_names
   3195 graph_function = ConcreteFunction(
-> 3196     func_graph_module.func_graph_from_py_func(
   3197         self._name,
   3198         self._python_function,
   3199         args,
   3200         kwargs,
   3201         self.input_signature,
   3202         autograph=self._autograph,
   3203         autograph_options=self._autograph_options,
   3204         arg_names=arg_names,
   3205         override_flat_arg_shapes=override_flat_arg_shapes,
   3206         capture_by_value=self._capture_by_value),
   3207     self._function_attributes,
   3208     function_spec=self.function_spec,
   3209     # Tell the ConcreteFunction to clean up its graph once it goes out of
   3210     # scope. This is not the default behavior since it gets used in some
   3211     # places (like Keras) where the FuncGraph lives longer than the
   3212     # ConcreteFunction.
   3213     shared_func_graph=False)
   3214 return graph_function

File ~/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/framework/func_graph.py:990, in func_graph_from_py_func(name, python_func, args, kwargs, signature, func_graph, autograph, autograph_options, add_control_dependencies, arg_names, op_return_value, collections, capture_by_value, override_flat_arg_shapes)
    987 else:
    988   _, original_func = tf_decorator.unwrap(python_func)
--> 990 func_outputs = python_func(*func_args, **func_kwargs)
    992 # invariant: `func_outputs` contains only Tensors, CompositeTensors,
    993 # TensorArrays and `None`s.
    994 func_outputs = nest.map_structure(convert, func_outputs,
    995                                   expand_composites=True)

File ~/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py:634, in Function._defun_with_scope.<locals>.wrapped_fn(*args, **kwds)
    632     xla_context.Exit()
    633 else:
--> 634   out = weak_wrapped_fn().__wrapped__(*args, **kwds)
    635 return out

File ~/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/framework/func_graph.py:977, in func_graph_from_py_func.<locals>.wrapper(*args, **kwargs)
    975 except Exception as e:  # pylint:disable=broad-except
    976   if hasattr(e, "ag_error_metadata"):
--> 977     raise e.ag_error_metadata.to_exception(e)
    978   else:
    979     raise

AttributeError: in user code:

    /home/lcl/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py:800 train_function  *
        return step_function(self, iterator)
    /home/lcl/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py:790 step_function  **
        outputs = model.distribute_strategy.run(run_step, args=(data,))
    /home/lcl/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/distribute/distribute_lib.py:1259 run
        return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
    /home/lcl/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/distribute/distribute_lib.py:2730 call_for_each_replica
        return self._call_for_each_replica(fn, args, kwargs)
    /home/lcl/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/distribute/distribute_lib.py:3417 _call_for_each_replica
        return fn(*args, **kwargs)
    /home/lcl/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py:783 run_step  **
        outputs = model.train_step(data)
    /home/lcl/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py:753 train_step
        self.compiled_metrics.update_state(y, y_pred, sample_weight)
    /home/lcl/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/keras/engine/compile_utils.py:387 update_state
        self.build(y_pred, y_true)
    /home/lcl/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/keras/engine/compile_utils.py:317 build
        self._metrics = nest.map_structure_up_to(y_pred, self._get_metric_objects,
    /home/lcl/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/util/nest.py:1159 map_structure_up_to
        return map_structure_with_tuple_paths_up_to(
    /home/lcl/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/util/nest.py:1257 map_structure_with_tuple_paths_up_to
        results = [
    /home/lcl/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/util/nest.py:1258 <listcomp>
        func(*args, **kwargs) for args in zip(flat_path_gen, *flat_value_gen)
    /home/lcl/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/util/nest.py:1161 <lambda>
        lambda _, *values: func(*values),  # Discards the path arg.
    /home/lcl/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/keras/engine/compile_utils.py:418 _get_metric_objects
        return [self._get_metric_object(m, y_t, y_p) for m in metrics]
    /home/lcl/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/keras/engine/compile_utils.py:418 <listcomp>
        return [self._get_metric_object(m, y_t, y_p) for m in metrics]
    /home/lcl/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/keras/engine/compile_utils.py:439 _get_metric_object
        y_t_rank = len(y_t.shape.as_list())

    AttributeError: 'tuple' object has no attribute 'shape'