Giter Club home page Giter Club logo

www's People

Contributors

berab avatar carlocab avatar cookieli avatar girish946 avatar lucaferranti avatar ludgerpaehler avatar proteneer avatar samuelpmishllnl avatar tgymnich avatar wsmoses avatar zusez4 avatar

Watchers

 avatar  avatar  avatar  avatar

www's Issues

Document Split Mode

Some examples / text from a conversation with @harshithamenon I'm copying here so we don't lose/forget about, and we can reincorporate into docs: https://fwd.gymni.ch/WfDU2f

#include <math.h>
#include <stdio.h>

#define nullptr ((void*)0)

// f(x, y) -> { sin(x), sin(x)^2 + y }
void mysimulation(double *data) {
    data[0] = sin(data[0]);
    data[1] += data[0] * data[0];
}

void __enzyme_autodiff(void (*)(double *), ...);
void* __enzyme_augmentfwd(void (*)(double *), ...);
void __enzyme_reverse(void (*)(double *), ...);

int enzyme_nofree;
int enzyme_tape;

int main()
{
    double data[2] = {2.0, 3.0};
    mysimulation(data);
    printf("vanilla run %f %f\n", data[0], data[1]);

    data[0] = 2.0; data[1] = 3.0;

    // Take the derivative wrt x, on fresh data.
    double d_data[2] = {1.0, 0.0};
    __enzyme_autodiff(mysimulation, data, d_data);
    printf("result run %f %f | derivative %f %f\n", data[0], data[1], d_data[0], d_data[1]);

    d_data[0] = 1.0; d_data[1] = 0.0;
    __enzyme_autodiff(mysimulation, data, d_data);
    printf("rerun derivative without resetting data result %f %f | derivative %f %f\n", data[0], data[1], d_data[0], d_data[1]);


    data[0] = 2.0; data[1] = 3.0;

    // Compute the original result, and also save any information
    // needed for the derivative. We don't need the shadow (derivative)
    // array here so we can pass nullptr.
    void* tape = __enzyme_augmentfwd(mysimulation, data, nullptr);
    printf("augmented run %f %f\n", data[0], data[1]);

    // Just for extra fun, overwrite the data
    data[0] = 0.0/0.0;
    data[1] = 0.0/0.0;

    // Compute the original result, and also save any information
    // needed for the derivative. We don't need the primal (original)
    // array here so we can pass nullptr.
    d_data[0] = 1.0; d_data[1] = 0.0;
    __enzyme_reverse(mysimulation, enzyme_nofree, nullptr, d_data, tape);
    printf("rerun derivative without resetting data result %f %f | derivative %f %f\n", data[0], data[1], d_data[0], d_data[1]);

    // We can in fact compute the derivative now a second time, using the same cached
    // data and again get the same (correct) result. Now without enzyme_nofree, we'll
    // free the preserved data.
    d_data[0] = 1.0; d_data[1] = 0.0;
    __enzyme_reverse(mysimulation, nullptr, d_data, tape);
    printf("second run derivative without resetting data result %f %f | derivative %f %f\n", data[0], data[1], d_data[0], d_data[1]);

    return 0;
}
Iā€™m not sure this is exactly what youā€™re thinking, but basically what happens in that snippet is we have a ā€œsimulationā€ (aka overwrite data as f(x, y) -> { sin(x), sin(x)^2 + y }).
If you first run the usual mode it will compute the original result and the derivative. Thus if you ask for the derivative again, youā€™d get the derivative at the second time step.
If instead you used the __enzyme_augmentfwd function (aka preserve values you need to compute the derivative at a later time), you can arbitrarily overwrite the original data (e.g. here setting it to nan), and successfully get the correct derivative. In fact you can use that same ā€œtapeā€ or ā€œcacheā€ a second time, again again get that same correct value.
Iā€™m not sure this is 100% the same thing as youā€™re thinking since we explicitly only preserve the values that are needed to compute the derivative rather than entirely restart the simulation (its desirable to preserve less)
What would happen if the application exits and restarts from some state stored in a file. Wouldn't you need some hook into the application to specify how to reconstruct the tape?

Itā€™s just data you could save and load the X bytes it needs to/from a file (thereā€™s a separate get size of tape function). You can also specify Enzyme to store and/or load the tape data at a specific pointer
Thereā€™s a separate get size of tape function. You can also specify Enzyme to store and/or load the tape data at a specific pointer

Hereā€™s the libCEED (forward mode version of this) code which just uses some random memory location provided by libCEED: https://github.com/CEED/libCEED/blob/99e8d5bed2f93906167219e46a292a0381310e8b/examples/solids/qfunctions/finite-strain-neo-hookean-initial-ad.h#L175
CEED_QFUNCTION_HELPER void S_fwd(double *S, double *E, const double lambda,
                                 const double mu, double *tape) {
  int tape_bytes = __enzyme_augmentsize((void *)computeS, enzyme_dup, enzyme_dup,
                                        enzyme_const, enzyme_const);
  __enzyme_augmentfwd((void *)computeS, enzyme_allocated, tape_bytes,
                      enzyme_tape, tape, enzyme_nofree, S, (double *)NULL, E, (double *)NULL,
                      enzyme_const, lambda, enzyme_const, mu);
}

CEED_QFUNCTION_HELPER void grad_S(double *dS, double *dE, const double lambda,
                                  const double mu, const double *tape) {
  int tape_bytes = __enzyme_augmentsize((void *)computeS, enzyme_dup, enzyme_dup,
                                        enzyme_const, enzyme_const);
  __enzyme_fwdsplit((void *)computeS, enzyme_allocated, tape_bytes,
                    enzyme_tape, tape, (double *)NULL, dS, (double *)NULL, dE,
                    enzyme_const, lambda, enzyme_const, mu);
}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    šŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. šŸ“ŠšŸ“ˆšŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ā¤ļø Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.