Comments (1)
I have a couple of clarifying questions about cellular attention networks and the code for the attention mechanism in the conv and message passing files.
- I may be mistaken, but it seems like there is no normalization being done for the attention coefficients in the current implementation of attention (the reference paper uses softmax for this purpose). Should we leave the current attention mechanism as is or is it worth rewriting the code to implement the normalization?
- With regards to the tensor diagram for CAN's when it comes to the neighborhood aggregation the tensor diagram here calls for applying a non-linearity to each within neighborhood aggregation and then performing the inter-neighborhood aggregation, whereas the referenced CAN paper performs the inter-neighborhood aggregation and then applies the non-linearity. Would it be ok to go with the formula given by the paper? This would also make it so our implementation can reduce to the Hodge Laplacian layer in the referenced Rodenberry et al. paper when the option to use attention is set to false.
(1) I would suggest you normalize as done by the original.
(2) Also the same, please go by what the original paper suggests. We will update the tensor diagram accordingly later.
@mathildepapillon
from topomodelx.
Related Issues (20)
- Check docstrings everywhere HOT 5
- Make unique doc website for the three packages HOT 8
- RED LIST: Check before submitting
- Write coverletter
- Diagnose & Speed-up Hypergraph tutorials HOT 10
- `Dist2Cycle` never calls `Dist2CycleLayer`s HOT 5
- Extends TopoEmbedX to ColoredHypergraphs and Path complexes
- Possible bug in scatter_sum
- Best Practices for batching?
- Add path complex neural network in TopoModelX
- can_train tutorial fails with new sparse casting
- Review nn/hypergraph models HOT 1
- Review nn/simplicial models
- Review nn/cell and nn/combinatorial models HOT 1
- Fix bug in CAN tutorial
- Create Dataloaders HOT 1
- Add notebooks from TDL paper's experiments HOT 1
- Review and rewrite models that are not using topomodelx.base.conv primitives
- `SCConv` needs a full revision HOT 2
- Add `numpydoc` validation to CI
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from topomodelx.