dollabs / pamela Goto Github PK
View Code? Open in Web Editor NEWProbabalistic Advanced Modeling and Execution Learning Architecture
License: Apache License 2.0
Probabalistic Advanced Modeling and Execution Learning Architecture
License: Apache License 2.0
With instaparse performing syntax checks and semantic check work in progress, pamela needs a 'check' action that will perform syntax and semantic checks of the given input file.
pamela -i biased-coin.pamela check # Check syntax and semantics of the input file
Calling a method with the wrong number of arguments fails silently when generating HTN and TPN. Both name and arity are needed for a method to be applicable for method expansion, but the user isn't informed of any failure.
In contrast, changing (two 1 2 3)
to (dos 1 2 3)
(an undefined method name) results in an error at parse-time. Although it's not important that both cases generate errors in the same way, an arity mismatch should result in some kind of warning at HTN generation time.
Note: I'll probably create an issue suggesting that multiple arities be supported for the same method (which will obviously impact the resolution to this issue).
(defpclass arg-test2 []
:methods [(defpmethod one []
(sequence
(two 1 2 3))) ;;wrong arity
(defpmethod two [a1 a2]
(three a1 a2))
(defpmethod three [b1 b2])
])
There is a quoting/output problem.
When using the htn
action without specifying a root task (-t
), if the Pamela file contains a single 0-arg main
pmethod, that pmethod should be used as the root task. It currently results in a cryptic error.
dcerys@nebbiolo:~/DOLL/github/pamela/test/pamela$ pamela -v -i statements.pamela -o ../../dan/statements-nrt htn
verbosity level: 1
log level: :warn
construct-tpn: nil
file-format: json
input: [#object[java.io.File 0x2bb62414 /Users/dcerys/DOLL/github/pamela/test/pamela/statements.pamela]]
output: ../../dan/statements-nrt
magic: nil
output-magic: nil
root-task: nil
cmd: htn (valid)
17-05-26 14:51:05 nebbiolo WARN [pamela.parser:0] - unable to determine if this keyword value is valid in pclass main :illuminated
17-05-26 14:51:05 nebbiolo WARN [pamela.parser:0] - unable to determine if this keyword value is valid in pclass main :dead
17-05-26 14:51:05 nebbiolo WARN [pamela.parser:0] - unable to determine if this keyword value is valid in pclass main :open
17-05-26 14:51:05 nebbiolo ERROR [pamela.cli:0] -
ERROR caught exception:
cannot create arg-mappings with static-task-args: [] and dynamic-task-args:
pamela script needs to use uberjar when available so that it does not incur compile penalty every time is it used.
lein clean; lein uberjar
...
Created /xxx/github/pamela/target/uberjar/pamela-0.3.0.jar
Created /xxx/github/pamela/target/uberjar/pamela.jar
~/p/g/pamela> pamela -h
Compiling pamela.core
Compiling ClojureScript...
Compiling "resources/public/js/compiled/app.js" from ["src/main/cljs"]...
Successfully compiled "resources/public/js/compiled/app.js" in 13.46 seconds.
Probabalistic Advanced Modeling and Execution Learning Architecture (PAMELA)
HTN methods in PAMELA must have exactly one of {sequence parallel choose}
The place to do this is in src/pamela/htn.clj #'make-htn-methods because that's where we convert the PAMELA IR into the Clojure HTN data structure.
We need to keep track of the number of times we've seen (#{:sequence :parallel :choose} type)
Which means "is the type of this subtask in the set of :sequence :parallel :choose ?
At the end of one run through the function we need to verify that the counter is exactly 1 or error out.
When using bin/pamela script, errors are logged to logs/pamela-service.log file. Errors in pamela log files are perfectly fine but it would be very helpful if they are also logged to stdout.
Ex:
pamela -v -v -i biased-coin.pamela -f json -o biased-coin.tpn.json tpn
16-10-03 20:59:24 xxxxxx----yyyy ERROR [pamela.tpn:1] - unable to determine tpn plant, more than one pclass takes zero args
16-10-03 20:59:24 xxxxxx----yyyy ERROR [pamela.tpn:1] - unable to find the default TPN method: please specify --construct-tpn pclass:field:method
The Pamela parser doesn't do enough validation of the arguments within field constructors.
In the following case, the parser should warn (or error) about the field definitions of :field3
and :field4
since arg3
and arg4
aren't defined (e.g., by the pclass constructor arglist).
(defpclass plant [arg1 arg2]
:meta {:doc "Plant with 2-arg constructor"}
:fields {:field1 arg1
:field2 {:observable true :initial arg2}
:field3 arg3
:field4 {:observable true :initial arg4}}
:methods [(defpmethod do-it-now [a1 a2 a3 a4]
)
(defpmethod do-it []
(do-it-now :field1 :field2 :field3 :field4))])
(defpclass constructor-arg-test []
:meta {:doc "Test of constructor args"}
:fields {:plant1 (plant 1 2)}
:methods [(defpmethod main []
(:plant1.do-it)
)]
)
This single-pclass Pamela file displays a spurious ERROR:
ERROR, not currently supported plant-fn type= nil
Using (arg-test1.one)
as the root task
However, the generated HTN and TPN seem correct
(defpclass arg-test1 []
:methods [(defpmethod one []
(sequence
(two 1 2)))
(defpmethod two [a1 a2]
(three a1 a2))
(defpmethod three [b1 b2])
])
A few related modifications are needed in the HTN (HEM) args that we just added:
[]
instead of nil
Example of IR-expanded references:
:args
({:type :pclass-ctor,
:pclass qc,
:args [],
:id "qc3",
:plant-part "pp3",
:param qc3})
Instead, we should generate
:args [qc3]
Note: This assumes that all current consumers of the :args
are interested in the non-IR-digested form of the argument list. In the future, there will be additional IR-digestion of references in the argument lists (see #81), and there likely will be users of that IR representation of those arguments (e.g., Dispatcher and BSM). When this happens, we may need to modify the schema such that we have 2 versions of the args: one for display, and one for computation. However, this isn't yet needed.
Reorganize the tests under the the test/pamela
directory to include the following subdirectories:
examples
: This will include all of the current test/pamela/*.pamela
files (moved now to a subdirectory). Automated testing will process this directory.regression
(or micro-examples
): These are small examples testing a single Pamela language feature. Automated testing will process this directory.errors
: Expected errors for IR and HTN. Automated testing will process this directory.pending
: Pamela files that may be promoted to examples
or errors
, as the implementation changes. Automated testing will ignore this directory.Note that the subdirectory structure of examples
and regression
will be the same. (Although errors
will also have the same structure, the interpretation of those files is different).
Currently, values of the arguments to plant functions are defined as:
argval ::= symbol
| boolean
| string
| number
| safe-keyword
Extend this to also include vectors and maps.
Calling (foo [arg1]) as (foo "a") should result in an HEM of (foo "a"). Instead, we currently leave it unexpanded as (foo "arg1").
Planviz is complaining about end-nodes.
In pamela/test/simple-choice-parallel
pamela -i parallel.feasible.pamela -t "(parallel-feasible.main)" -o parallel.feasible htn
planviz -v -v -l debug -i parallel.feasible.tpn.json -i parallel.feasible.htn.json
# snip
Reading input from: parallel.feasible.tpn.json
17-Mar-31 15:56:32.924 pablo DEBUG [planviz.server:0] - END NODE :parallel.feasible.tpn:node-20 NOT FOUND?
17-Mar-31 15:56:32.925 pablo DEBUG [planviz.server:0] - END NODE :parallel.feasible.tpn:node-5 NOT FOUND?
17-Mar-31 15:56:32.926 pablo DEBUG [planviz.server:0] - END NODE :parallel.feasible.tpn:node-14 NOT FOUND?
In the generated tpn file, state nodes have an end node. Only the state nodes that represent beginning of a actual sequence
should have end node slot. i.e (sequence (act1) (act2)) and not otherwise
Currently we have two slots for plant methods
However argsmap drops the position of the args.
If, instead, we had a vector of vectors then we would have all the data:
argsmap: [[param0 val0] [param1 val1] ....]
for example
argsmap: [[loc1 "e1h1"] [loc2 "e2h1"] [loc3 "e3h1"] [loc4 "e4h1"]]
In fact.. with this argsmap we really wouldn't need the :args slot
because we could derive it with: (mapv second argsmap)
The goal of this ticket is to decide which format to use for argsmap.
We use :id in pamela class to route messages for a instance of the plant however we don't have a mechanism to uniquely identify observations from various parts of the plant. Currently, all observations are assumed to be coming from the plant.
So we need to add a :plant-part attribute to the language as below:
# Before
- :bulb1 (bulb :source :drain :id "Bulb1")
- :switchedpower (psw :source :drain :id "PSW1" :interface "RMQ")})
# After
+ :bulb1 (bulb :source :drain :id "switchedcircuit" :plant-part "Bulb1" :interface "RMQ")
+ :switchedpower (psw :source :drain :id "switchedcircuit" :plant-part "PSW1" :interface "RMQ")})
## In github/dollabs/pamela
pamela -v -v -i sequence.feasible.pamela -o sequence.feasible.cfm.tpn.json -c sequence-feasible:main:main -f json tpn
Generated TPN does not has any constraints.
HTN generation attempts to expand non-empty methods, despite being marked :primitive.
Instead it should simply treat the method body as if it were empty.
Prior to IR days, generated TPN state
nodes had :sequence-end node pointing to the end node of the sequence.
Current TPN state
nodes have :end-node pointing to the end-node of the activity instead of the end-node of the sequence. This is the case for TPNs generated by both htn and tpn
task.
Also, the :end-node is not always present.
from first.tpn.json
"node-28": {
"tpn-type": "state",
"uid": "node-28",
"constraints": [],
"activities": [
"na-34"
],
"incidence-set": [
"act-46"
]
},
prakash@pablo ~/p/b/p/demo-nov-2016-pamela> pwd
/Users/prakash/projects/bitbucket/pamela/demo-nov-2016-pamela
prakash@pablo ~/p/b/p/demo-nov-2016-pamela> pamela -i biased-coin.pamela -o biased-coin.1000-flips -f json -t "(coin.flip-sequence)" htn
17-03-08 23:31:06 pablo ERROR [pamela.cli:288] - unable to parse: [#object[java.io.File 0x253f8e97 "/Users/prakash/projects/bitbucket/pamela/demo-nov-2016-pamela/biased-coin.pamela"]]
error: Keyword argument to pclass constructor :observable is not a field in the pclass coin
See #101 for details. I am blocked on resolution of this issue
[Once agreed upon, this information is a candidate for inclusion in the PAMELA language manual]
When calling a method (non-primitive or primitive), each of the arguments can be a literal (i.e., boolean, string, number, map, vector, (and maybe a safe-keyword)) or a reference.
There are 1+ types of reference: symbols and (maybe) safe-keywords:
Symbols must have a value (through argument unification via args list) that is one of the supported literals (see above). In addition, the value of a symbol could be a pclass instance.
Question: Although it makes sense to pass a pclass instance to a non-primitive method, does it make sense for a primitive method? [Doing so would somehow be a way of passing a plant-id and plant-part to a primitive method]
Based on some recent clarifications, it may be the case that a keyword, when used as a reference is only used as a reference to a mode.
Question: Do we support modes (designated by keywords) as arguments (for both non-primitive and primitive methods)? [I think NO]
Question: Is any non-mode keyword a literal? Do we support keyword literals as arguments (for both non-primitive and primitive methods)?
When specifying a complex literal as an argument value (map or vector), any of the components of that complex literal can be a literal or a reference.
See issue #101 and use first.pamela for details. I am blocked on this issue
Currently, when -f option is not specified, pamela generates output in edn format. Whereas edn format is good debugging, our preferred output format is json and it should be the default.
-- JSON for activities should have argsmap filled. Note args are correctly filled.
-- add plantid slot where applicable
"act-92": {
"args": [
5,
0.5
],
"constraints": [],
"uid": "act-92",
"name": "move_takeoff({})",
"tpn-type": "activity",
"argsmap": {},
"command": "move_takeoff",
"htn-node": "hem-78",
"controllable": false,
"end-node": "node-98",
"label": "TEMP-move_takeoff",
"display-name": "moveTakeoff"
},
Resolve compiler warnings and errors
pamela -i test/pamela/cannon.pamela -t "(game.main)" htn
17-01-26 19:45:40 pablo WARN [pamela.parser:0] - unable to determine if this keyword value is valid in pclass game :open
17-01-26 19:45:40 pablo WARN [pamela.parser:0] - unable to determine if this keyword value is valid in pclass game :open
17-01-26 19:45:40 pablo ERROR [pamela.htn:0] - unhandled function type in HTN: :whenever
17-01-26 19:45:40 pablo ERROR [pamela.htn:0] - unhandled function type in HTN: :whenever
pamela -i ir-test.pamela -t "(game.main)" htn
17-01-30 20:43:09 pablo WARN [pamela.parser:0] - unable to determine if this keyword value is valid in pclass box :high
17-01-30 20:43:09 pablo ERROR [pamela.cli:0] - unable to parse: [#object[java.io.File 0x3cc6d144 "/Users/prakash/projects/github/dollabs/pamela/test/pamela/ir-test.pamela"]]
error: Symbol argument to pclass constructor bob is neither a formal argument to, nor a field of the pclass game
Appears to be a bug as the error suggests. However, since there is a valid IR, I might be missing something.
time pamela -h
gives
5.70 real 21.29 user 0.77 sys
time java -jar target/pamela.jar -h
2.42 real 7.90 user 0.29 sys
Why? What changed?
Pamela compiles a model perfectly fine but planviz refuses to load it.
TPN generated by following command does not load in planviz.
pamela -v -v -i biased-coin.pamela -f json -o biased-coin.tpn.json --construct-tpn main:necessary-evil:flip tpn
16-Oct-03 17:16:04.787 xxx-yyy ERROR [planviz.server:1014] - error parsing TPN plan: ../biased-coin.tpn.json
16-Oct-03 17:16:04.788 xxx-yyy ERROR [planviz.server:1015] - {:na-104 {:probability (not (instance? java.lang.Number a-clojure.lang.PersistentArrayMap))}, :na-95 {:probability (not (instance? java.lang.Number a-clojure.lang.PersistentArrayMap))}, :na-121 {:probability (not (instance? java.lang.Number a-clojure.lang.PersistentArrayMap))}, :na-130 {:probability (not (instance? java.lang.Number a-clojure.lang.PersistentArrayMap))}}
For first.pamela, When using htn
task,
first-tpn
instead of first-tpn-htn
in between
constraints are not createdcfm
methodhtn
taskFollowing command line used to work but not anymore.
ls -l biased-coin.pamela
-rw-r--r-- 1 ... staff 2286 Nov 9 14:00 biased-coin.pamela
~/p/p/demo-nov-2016-pamela> pamela -i biased-coin.pamela -o biased-coin.1000-flips.tpn.json --construct-tpn main:flip-sequence:flip-1000 -f json tpn
16-11-09 19:03:58 prakash-retina ERROR [pamela.cli:0] -
Failed to validate "-i biased-coin.pamela": INPUT file does not exist
Probabalistic Advanced Modeling and Execution Learning Architecture (PAMELA)
Usage: pamela [options] action
Generated TPN's have empty incidence set. See tpn below.
{
"node-85": {
"tpn-type": "state",
"uid": "node-85",
"constraints": [],
"activities": [
"act-84"
],
"incidence-set": []
},
"net-74": {
"tpn-type": "network",
"uid": "net-74",
"begin-node": "node-85",
"end-node": "node-72"
},
"network-id": "net-74",
"act-100": {
"tpn-type": "activity",
"uid": "act-100",
"constraints": [],
"end-node": "node-72",
"controllable": false,
"plant": "this",
"command": "flip",
"args": [],
"argsmap": {}
},
"node-93": {
"tpn-type": "state",
"uid": "node-93",
"constraints": [],
"activities": [
"act-92"
],
"incidence-set": []
},
"act-92": {
"tpn-type": "activity",
"uid": "act-92",
"constraints": [],
"end-node": "node-101",
"controllable": false,
"plant": "this",
"command": "flip",
"args": [],
"argsmap": {}
},
"act-84": {
"tpn-type": "activity",
"uid": "act-84",
"constraints": [],
"end-node": "node-93",
"controllable": false,
"plant": "this",
"command": "flip",
"args": [],
"argsmap": {}
},
"node-72": {
"tpn-type": "state",
"uid": "node-72",
"constraints": [],
"activities": [],
"incidence-set": []
},
"node-101": {
"tpn-type": "state",
"uid": "node-101",
"constraints": [],
"activities": [
"act-100"
],
"incidence-set": []
}
}
Currently the "tpn" action in PAMELA will generate a TPN directly from PAMELA source (1). Instead we want to, in the medium term, use the existing HTN+TPN generation algorithm (2). In the future we'll use the new HTN->TPN algorithm (#41).
pamela fails to generate tpn for big-sequence
pamela -i big-sequence.pamela -o big-sequence.tpn.json -f json --construct-tpn main:flip-sequence:flip-sequence tpn
Compiling ClojureScript...
ERROR caught exception: nil
analyze reports success for a pamela file but pamela fails to generate json.
analyze sequence.infeasible.pamela
Analyzing PAMELA file: /Users/prakash/projects/github/pamela/src/test/pamela/sequence.infeasible.pamela
Checking grammar for: sequence.infeasible.pamela
Success! parse tree in: /Users/prakash/projects/github/pamela/src/test/pamela/sequence.infeasible.txt
parse diagram in: /Users/prakash/projects/github/pamela/src/test/pamela/sequence.infeasible.png
pamela -f json -i sequence.infeasible.pamela -o sequence.infeasible.json tpn
Compiling ClojureScript...
16-05-27 21:28:49 prakash-retina DEBUG [io.netty.util.internal.logging.InternalLoggerFactory:71] - Using SLF4J as the default logging framework
16-05-27 21:28:49 prakash-retina DEBUG [io.netty.util.internal.PlatformDependent0:76] - java.nio.Buffer.address: available
16-05-27 21:28:49 prakash-retina DEBUG [io.netty.util.internal.PlatformDependent0:76] - sun.misc.Unsafe.theUnsafe: available
16-05-27 21:28:49 prakash-retina DEBUG [io.netty.util.internal.PlatformDependent0:71] - sun.misc.Unsafe.copyMemory: available
16-05-27 21:28:49 prakash-retina DEBUG [io.netty.util.internal.PlatformDependent:76] - sun.misc.Unsafe: available
16-05-27 21:28:49 prakash-retina DEBUG [io.netty.util.internal.PlatformDependent:71] - Javassist: available
ERROR caught exception: clojure.lang.ArityException: Wrong number of args (0) passed to: pclass/defpmethod, compiling:(null:1:1)
Argument values in method calls (i.e., argvals of plant-fn calls) that are symbols/keywords are kept as literal symbols, in contrast to other symbols (like the method name) which are parsed into context-ful structures (maps). This is true for: the generated IR, HTN, and TPN.
It might be useful (perhaps essential?) to do the same as for arguments to a Pamela conditional.
E.g.,
The method invocation:
(generator.generate-candidate-positions :gu1.next-lat :gu1.next-lon
:gu2.next-lat :gu2.next-lon :gu3.next-lat :gu3.next-lon)
generates the following in the IR:
{:type :plant-fn-field,
:method generate-candidate-positions,
:args
[:gu1.next-lat
:gu1.next-lon
:gu2.next-lat
:gu2.next-lon
:gu3.next-lat
:gu3.next-lon],
:field :generator}
These literals carry through from the IR to the generated HTN and TPN. So, it's up to the user of the activities (e.g., the Dispatcher) to (re)construct the context and perform the correct dereferencing. The problem is that the complete context won't be available (unless we pull the IR along all the way). In the above example, there is no way that the field variable :gu1
corresponds to the ground-unit
pclass instance with the :plant-id
of "ground-unit1"
.
In contrast, the conditional (= :cannon-f.:ammunitions 0)
is expanded into
{:type :equal,
:args [{:type :field-reference-field,
:pclass this,
:field :cannon-f,
:value :ammunitions}
{:type :literal, :value 0}]}
Still, this assumes some context that might not be available downstream, but it's a better situation. We would still need to be able to determine the appropriate pclass-ctor (in order to obtain the correct :plant-id and :plant-part). Of course, expanding all method args in this manner would require changes throughout htn.clj
and in the Dispatcher.
Similar to Clojure's dotimes this should expand
(dotimes 3 (flip))
into
(sequence (flip) (flip) (flip))
This is a 'macro' in that dotimes will never appear in the IR, instead the expanded sequence will be put in it's place.
Currently there are certainly legacy "slots" in the TPN schema (e.g. name) and new slots in the HTN schema (e.g. label) that are either obsolete or need to be renamed for consistency. This issue will track this harmonization effort.
Add optional :primtive true meta data to defmethod such that
the method body is saved in the IR (but still treated like a primitive
method in generating the TPN).
Low Priority
At some point, it will be useful to have multiple methods of the same name, with different arities:
E.g., (foo 1 2 3)
and (foo 1 2)
For now, this is Low Priority
This will allow us to do regression testing of the HTN/TPN generation process
As a first pass at supporting ask/tell/assert, we will treat them like 1-arg plant functions. Initially, the plant-id for these will refer to the Belief State Manager.
In the future, to handle all of Pamela constructs (e.g., unless, when), we’ll need to make some structural changes to the HTN and TPN representation.
In the following example, the :bounds
declared for the primitive methods (e.g., do-a
) aren't included in the generated TPN (generated using the htn
action)
(defpclass plant []
:meta {:doc "The Plant API"}
:methods [(defpmethod do-a {:bounds [2 4]} [])
(defpmethod do-b {:bounds [3 6]} [])
(defpmethod do-c {:bounds [4 8]} [])
(defpmethod do-d [])
(defpmethod do-e {:bounds [5 10]} [])])
(defpclass infeasible-sequence [plnt]
:meta {:doc "An example of infeasible sequence of activties"}
:methods [(defpmethod start
{:doc "Simple TPN with constraints"}
[]
(sequence :bounds [10 30]
(plnt.do-a)
(plnt.do-b)
(plnt.do-c)
(plnt.do-d)
(plnt.do-e)))])
PAMELA should be able to convert a well formed HTN with a root-task into the corresponding TPN. (This has been called TPN generation algorithm 3).
Recently the delay-activity TPN-TYPE was defined in plan-schema.
The TPN generation function in PAMELA should translate 'delay' functions into delay-activities.
For pamela/test/simple-choice/choice.feasible.pamela
TPN in planviz does not render correctly.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.