Annotated implementations of equivariant graph neural networks in Jax: EGNN, SEGNN, NequIP.
License: Creative Commons Attribution 4.0 International
I'm a physicist and machine learning researcher, currently at the The NSF AI Institute for Artificial Intelligence and Fundamental Interactions (IAIFI) at MIT.
I work at the intersection of differentiable programming, machine learning, particle physics, and astrophysics and am especially interested in developing and using novel statistical techniques to accelerate the search for new physics. A list of my recent publications may be found on INSPIRE, ADS, or Google Scholar.
Was interested in adding TensorProductLinearGate to the profiling bank that I am building here to document JAX fusion behavior:
Is this the correct code snippet to tinker around with ?
Are there any sharp bits or assumptions that would be good to know as I am comparing the kernel trace with TensorProduct and TensorProductLinear ?
Apologies for being vague. Just wanted to be extra cautious to make sure the profiles are helpful.
Thanks !
Based on a talk by Dan Levy (Mila) at the Cosmic Connections conference.
Sketch of code:
from einops import repeat, rearrange
import jax.numpy as jnp
n_channels = 5
X_i = repeat(x_i, "n d -> n d c", c=n_channels)
X_j = repeat(x_j, "n d -> n d c", c=n_channels)
D_ij2 = jnp.sum((X_i - X_j) ** 2, axis=1, keepdims=True)
Phi_e = MLP([d_hidden] * (n_layers), activation=activation, activate_final=True)
Phi_x_last_layer = nn.Dense(n_channels * n_channels, use_bias=False, kernel_init=jax.nn.initializers.variance_scaling(scale=1e-3, mode="fan_in", distribution="truncated_normal"))
D_ij2 = jax.vmap(fourier_features, in_axes=(2), out_axes=(2))(D_ij2) if use_fourier_features else D_ij2 # FF
D_ij2 = D_ij2.reshape(D_ij2.shape[0], -1) # Flatten
message_scalars = jnp.concatenate([D_ij2, concats], axis=-1)
m_ij = Phi_e(message_scalars)
trans = Phi_x_last_layer(phi_x(m_ij))
trans = rearrange(trans, "n (c1 c2) -> n c1 c2", c1=n_channels, c2=n_channels)
out = jnp.matmul((X_i - X_j), trans) # n_dim x n_channels
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.