pyt-team / topobenchmarkx Goto Github PK

Some conflicts in the installation appear when running the tests

When the best score is achieved, something eats all the CPU memory; hence, at some point, the process has to be stopped. Look closely at what happens when the best score is achieved. It happens only with Topological models.

Random guess: Maybe in PyTorch lightning, there is something special about saving model/data after the best loss is achieved

Some TopoModelX models do not have normalization of outputs, which results in an inability to stack multiple layers. For example, now we use external implementation of cwn, can be found in custom/ folder.

The same also happens with other models in the topomodelx library.

1: topobenchmarkx/io/load/utils.py
2: topobenchmarkx/io/load/utils.py

Temporally added "PERF203" to Ruff ignores

I have found that all the wrappers in the doc have exactly the same row, see below

Please update it with the appropriate documentation

While using Hydra config instantiation I have faced the issue that it is not trivial to pass torch modules as an argument, I guess that it is somehow possible to do.

Example:

backbone:
  _target_: topomodelx.nn.hypergraph.allset.AllSet
  in_channels: ${data.num_features}
  hidden_channels: 64
  mlp_activation_layer: torch.nn.ReLU

How to make it possible to pass torch.nn.ReLU? Is it possible?

The classes Graph2SimplicialLifting, Graph2HypergraphLifting, Graph2CellLifting share most of the interface. Unify through the class Graph2Domain will define general logic for all of the domains.

I have partially implemented this logic for Graph2SimplicialLifting

General Evaluator

It is essential to add a broader number of metrics before starting any experiments. Which metrics to add for the experiments for Neurips?

Metrics:

Classification task:

• Accuracy
• F1
  -...
  Regression task:
• MSE
• MAE
• ...

Define BaseData class logic and classes for every corresponding domain.

It seems that there is a memory leak somewhere, because allsettransformer with 2 layers cannot be allocated.

backbone:
target: topomodelx.nn.hypergraph.allset_transformer.AllSetTransformer
in_channels: ${data.num_features}
hidden_channels: 256
n_layers: 1
heads: 4
dropout: 0.2
mlp_num_layers: 1
mlp_dropout: 0.2

For some datasets it is too slow

We need to show that our lib can easily process datasets that come from any source; till now, we have everything from torch_geometric.

Maybe we need to tidy up io/load

The class is first imported: from topomodelx.nn.cell.cwn_layer import CWNLayer but, then, it is redefined in the same file

class CWNLayer(nn.Module):
    r"""Layer of a CW Network (CWN).

Make the abstract ReadOut class, and create a set of different ReadOut classes that inherit from the general ReadOut class.

Till now we had graph regression. We need to make sure / update the pipeline to be able to do node regression

Create a config logic:

Load config
Validate config
Save config

Make general utils similar to pyg

Define which tutorial to do and make them

Fix the issues with the docker file:

• Very long launching (40 min)
• Need always to install jupyter
• Existence of two different Python paths

Some tricks might be taken from Graphworld repo

SAN layer has to be mapped to torch.nn.ModuleList

Make sure that two consecutive runs with the same configuration will produce the same train/val/test outputs

How to make it through CLI

It is not possible to change mlp_activation argument in AllSetTransformerLayer while initializing AllSetTransformer

1. torch_geometric.data.lightning.LightningDataset do not work with Data objects that have sparse representations.
2. TopomodelX works only with sparse matrix multiplications

Do you have any ideas on how to allow mini-batching?

Define BaseData class logic and classes for every corresponding domains.

Add more test for the library

I suppose there was a bug on lines 84 and 94; the variable for cell_i was always current_number_of_nodes but not current_number_of_cells. Doublecheck that it should be current_number_of_cells.

Make a class that infers statistics from the initial and lifted dataset.
Collect the statistics and add to overleaf into table

We must resolve the module's inconsistency; all modules must follow the same logic.

There are two options:

1. We define the abstract class and call methods, then inherit and write the corresponding forward. In this way, we will need to additionally always specify the module's name executed, and the module name, along with the import, must be added to the file with the abstract class. (An example of this implementation is the readout module)

2. Every defines separate classes and imports them through the path to the config file (Example is feature encoder)

I believe we should go with strategy 1. It allows the definition of a stable pipeline while adding modularity by adding the classes with forward. The problem is that it is unclear if this approach is best suited for some modules like Evaluator.

There are 3 additional datasets I have found:

1. https://www.cs.cornell.edu/~arb/data/US-county-demos/
   Note: I already have created the loading pipeline for this one.
   The cool part of this dataset is that it is a node regression task where in total 6 variables. In this paper, they used a couple of them as targets. Hence, we can sell one dataset as 1-6.

These 2) and 3) datasets can be uploaded mostly with the same pipeline. Hence, we can also add them, but I didn't go into details.
2) https://www.cs.cornell.edu/~arb/data/CDC-climate/
3) https://www.cs.cornell.edu/~arb/data/US-county-fb/

Define the base transform class with will be responsible for any transforms

There are some functionalities provided by hydra template that we didn't use during development.

For example:

Incorporate CE integration