nicktfranklin / sem_paper_simulations Goto Github PK
View Code? Open in Web Editor NEWLicense: MIT License
License: MIT License
Could do something like particle filtering, where each particle = hypothesis about the event sequence (rather than considering the MAP only).
Could include pruning of unlikely hypothesis (e..g as you accumulate more scenes of the same event type) and resampling at moment of high uncertainty (e.g. at event boundaries when there's lots of potential candidate events). Would address the problem of how do you determine the event type given a single scene after an event boundary. Would also make distinct predictions about order effects (e.g. events that start off similarly but evolve differently -> do people re-assign the initial scenes to the eventually inferred event?).
Alternatively, could do something more naive like only considering the last K scenes (e.g. last 3 scenes) for resegmentation, and only testing the hypothesis that they all belong to the currently active event type vs. that they were segmented correctly the first time. Would again address the issue of segmenting at event boundaries.
Notice this makes a difference both for prediction (it could turn out that we're more likely to be in a bar fight but only if we consider the full history of scenes from the beginning, which we may have assigned to a different event type already, e.g. an academic debate) as well as for training (we should train the bar fight model on those scenes too; conversely, even though we may not be able to "untrain" the academic debate model, we could remove them from its "episodic memory" if we're doing batch gradient descent later on).
It would be nice to have a way to very quickly compare performance of different event models (e.g. RNN vs. stateful RNN vs. LDS vs. LSTM ... etc) on different examples. Ideally, we could have something that automatically (deterministically -- for reproducibility) generates test cases based on different templates (e.g. simple LDS, coffee shop world, the dancing people study, etc) and evaluates different versions of SEM on:
Potentially we could parametrize all of the SEM code to make the process easier -- it will make the code kind of ugly but we will avoid the need to maintain a thousand different files / branches with each separate version.
The prediction appear to be sensitive to the optimization algorithm of the event model. Add these to a test bed.
Requires some investigation.
Say we have the following "true" event sequence: AAAAABBBBBCCCCC, so e1..e5 = A, e6..e10 = B, and e11..e15 = C.
Say we've segmented s1..s10 correctly and now we're at s11.
We have three options for the event type of s11:
How do we compute the likelihood of s11 in each of these?
Case 1 is straightforward -- we feed the previous scene s10 to the current event model B to get the predicted next scene s11.
Case 2 is not so straightforward, because of the implied event boundary. Several options:
Case 3 is even less straightforward, because we don't have an initialized event model. Options:
The Jacobian (technically df/dW, not dX/dW) used to update the LDS seems incorrect.
the predicted next state is:
f = b + x * W
for each component i of the next state, we have:
f_i = b_i + sum over j of x_j * W_ji
so the partials are:
df_i / dW_ji = x_j
...and 0 for all others (i.e. df_i / dW_jk = 0 for all k != i).
More intuitively, it cannot be that e.g. W_11 contributes to f_2; so that partial should be 0. Right now, there are no 0's in the Jacobian because it's something like:
df_i / dW_jk = x_j
for all i,j,k, which is incorrect.
I think the line in update
should be more like this:
dXdW = np.tile((np.tile(X, (1, self.D)) * np.eye(self.D)), (self.D, 1))
It probably doesn't matter because we're only using the LDS as a toy example, but would be nice to have it working for sanity checks (and if we're publishing this code)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.