cloudslang / cloud-slang Goto Github PK

i created a flow like this with these tasks :
get_used_images:
do:
ops.get_used_images:
- all_containers_ids
get_used_images2:
do:
ops.get_used_images2:
- all_containers_ids
publish:
- usedImages: imageList
get_used_images:
do:
ops.get_used_images:
- all_containers_ids: usedImages

meanning same task name twice, result is that last task is not executed at all..
should this fail in compile time? or be supported and is a bug??

When running via the slang CLI in async mode, can the execution id be included with the output? As you can see below, it's impossible to tell which output belongs to which run at the present time.

slang>env --setAsync true
flow execution ASYNC execution was changed to : true
slang>run hello_world_sleep.sl
Triggered flow : hello_world_sleep , with execution id : 101600057
slang>Hello, World
slang>run hello_world_sleep.sl
Triggered flow : hello_world_sleep , with execution id : 101600062
slang>run hello_world_sleep.sl
Triggered flow : hello_world_sleep , with execution id : 101600067
slang>run hello_world_sleep.sl
Triggered flow : hello_world_sleep , with execution id : 101600072
slang>Hello, World
Hello, World
Hello, World

currently the batch file is under /appassembler/bin

need to change to maybe:
/slang/bin

They should appear in normal Java block comments first thing before package declaration

slang-cli returns an exit code of 0, independent of the success or failure of the flow that has been invoked. This makes it very annoying to try and include calls to run a flow inside of other scripting, since custom logic is then needed to determine if the flow succeeded or failed.

The flow runs successfully:

$ ../score-lang-cli/slang/bin/slang run division.sl --i input1=10,input2=-1
Database name is: H2 databaseMajorVersion is: 1
- divider
- printer
10/-1 = -10.0
Flow : division finished with result : SUCCESS
Jan 28, 2015 6:00:22 PM org.springframework.shell.core.AbstractShell handleExecutionResult
INFO: Flow execution time took  0:00:03.158 , with execution id : 101600001
$ echo $?
0

The flow fails:

$ ../score-lang-cli/slang/bin/slang run division.sl --i input1=10,input2=a 
Database name is: H2 databaseMajorVersion is: 1
- divider
Slang Error : Error executing python script: Traceback (most recent call last):
  File "<string>", line 4, in <module>
ValueError: invalid literal for __float__: a

Jan 28, 2015 6:10:00 PM org.springframework.shell.core.AbstractShell handleExecutionResult
INFO: Flow execution time took  0:00:02.913 , with execution id : 101600001
$ echo $?
0

Opening the CLI, writing only 'run' gives a not-very-clear message.

Why not showing the help menu?

look at class SlangCLI param SCORE_VERSION...

The slang-cli documentation doesn't mention that slang is left hand side whitespace indentation sensitive, or that tabs aren't considered to be valid whitespace. This is both important to know, and potentially frustrating, as the error messages aren't always helpful:

(1) The error message below comes from breaking the indentation model, and the error actually points to the line after the problem indentation:

namespace: user.examples.hello_world-s

imports:
  ops: user.examples.hello_world-s

flow:
  name: hello_world-s
  workflow:
    sayHi:
      do:
ops.print:
          - text: "'Hello, World'"

slang>run hello_world-s.sl
Command failed java.lang.RuntimeException: java.lang.RuntimeException: There was a problem parsing the YAML source: hello_world-s.sl.
Cannot create property=ops.print for JavaBean=org.openscore.lang.compiler.model.ParsedSlang@51a9d2a5
 in 'string', line 1, column 1:
    namespace: user.examples.hello_w ... 
    ^
Unable to find property 'ops.print' on class: org.openscore.lang.compiler.model.ParsedSlang
 in 'string', line 12, column 11:
              - text: "'Hello, World'"

(2) This error is the result of failing to indent between do: and ops.print -- and the error doesn't provide the line number where things went sideways, or indicate that there's an indentation error, vs an issue with the property.

namespace: user.examples.hello_world-s

imports:
  ops: user.examples.hello_world-s

flow:
  name: hello_world-s
  workflow:
    sayHi:
      do:
      ops.print:
          - text: "'Hello, World'"

slang>run hello_world-s.sl
Command failed java.lang.RuntimeException: java.lang.RuntimeException: Error compiling source: hello_world-s.sl. Property: ops.print at: sayHi is illegal

Note that (1) and (2) are both indentation erorrs, but produce notably different error messages.

(3) The following error comes from using tabs for whitespace, and while surprising, at least allows for easy correction of the error.

slang>run hello_world-s.sl
Command failed java.lang.RuntimeException: java.lang.RuntimeException: There was a problem parsing the YAML source: hello_world-s.sl.
while scanning for the next token
found character      '\t(TAB)' that cannot start any token. (Do not use \t(TAB) for indentation)
 in 'string', line 11, column 1:
            ops.print:
    ^

For example, if you have 2 operations within the dependencies with same fully qualified name, currently they will overwrite each other silently. Should the compiler prevent that?

When sending a system_properties slang file to the compiler for compilation or pre-compilation, the compiler throws an exception, since it only knows how to compile operations & flows

I am getting this error message on a file that exist in the given path (report_container_metrics_cAdvisor.sl), when I rename the file to hello_word it works fine.
CloudSlang/cs-content#26

Update to new CLI version including the slang operation new structure

@orius123

In the outputs/publish values/ results of a flow or operation, expressions which are not String (Integer values, Boolean...) are discarded.
For example, if we have output such as:

• output1: 30
  It is discarded without any error, so we will have a successful compilation, but the output will be missing from the compiled object.

Do we want to support other types?
If not, we should at least throw an exception from the compiler.

Currently, there is also usually a duplication of the exception type printed there...
I don't think it should be there at all since in almost all of the cases we have a RuntimeException, and the type adds no incentive for the problem.

In the following screenshot you can see two executions which the only difference between them is the space before the second parameter.

No problem with requiring to not put a space, but the message is super-unclear.

'run' currently prints out volumes of extra information that would be very awkward to use as part of a script, or anything else that parses results, eg:

$ ../score-lang-cli/slang/bin/slang run division.sl --i input1=10,input2=-1
Database name is: H2 databaseMajorVersion is: 1
- divider
- printer
10/-1 = -10.0
Flow : division finished with result : SUCCESS
Jan 28, 2015 6:00:22 PM org.springframework.shell.core.AbstractShell handleExecutionResult
INFO: Flow execution time took  0:00:03.158 , with execution id : 101600001

A quiet mode, that only returns what is specified as output by the flow would be very useful. In this case, that would be:

10/-1 = -10.0

Additionally (and I'll file a separate issue on this), it would be a major plus if the exit code reflected the success or failure of the flow.

When compiling a task, we should verify that we have navigations for all the possible responses of the flow/operation the task calls to

Some of the error messages produced by slang-cli are unclear or unhelpful in resolving issues. For example, the following actually means "your indentation is wrong"

In this case, "File not found" would be more helpful than "filePath must lead to a file" -- the first tells me exactly what's wrong; the current message had me checking import statements first.

Here, everything is in the same directory that slang was started from, so I'm not quite sure what part it can't find (probably an include, but in theory, it should 'just work', if it's all in the same directory?)

In order to use external Python libraries in your Slang operations' python scripts, you must edit slang.bat, and add the argument -Dpython.path= to the line invoking java.exe, pointing to a path where your (extracted) Python libraries are located at. Figuring this out yourself and "hacking" slang.bat is fine for a hackaton, but not a very nice end-user experience.

I would suggest adding a documented optional argument to slang.bat to set this property, so the user wouldn't need to edit the file.

The following python action:

      action:
        python_script: |
          import time
          print text
          time.sleep(10)

results in the following (non-intuitive) error message:

slang>run hello_world_sleep.sl
Command failed java.lang.RuntimeException: java.lang.RuntimeException: Reference: user.examples.hello_world_sleep.print_sleep in executable: hello_world_sleep, wasn't found in path

Is it possible to include python modules, and if not, why not?

score-language/score-language-tests/src/test/resources/yaml/docker-demo/app_test_operations.yaml
in line #31 - in case the call to the application URL did not fail, the count is set to max_count even though this is not the case.
in addition, 'returnResult' is misleading since the code actually means 'returnResultOK'

python actions appear to be parsed as:
(1) Single line of python:

      action:
        python_script:  print text

(2) Multi-line python:

      action:
        python_script: |
            print text
            print text

Unfortunately, it seems that:
(1) The documentation doesn't call out that the use of | is REQUIRED for multi-line python scripts
(2) The documentation doesn't call out that the use of | is FORBIDDEN for single-line python scripts

The behaviour is inconsistent and also confusing -- it would be infinitely better to simply have single-line and multi-line python scripts behave in the same way.

If the result of a flow is not SUCCESS or FAILURE the CLI does not display the line "Flow: flow_x finished with result :".

Add error handling and validation to the syntax below the action proerty

Ok, so I want ti start with defining how loops will look in our slang.

Lets take this basic flow as an example

flow:
  inputs: 
    - list_input: [1,2,3]
    - normal_input
  workflow:
    iter_task:
      do:
        ops.print:
          - list_input
          - normal_input

I was thinking of few ways we can go:
Currently those two ways don't give a way to aggregate the result,
and therefore they are not so useful because you can't publish to next tasks.
(you can still make void action for each though)

For..do

iter_task:
  # We can use another word of course like foreach..
  for: input in list_input
  do: 
    ops.print:
      - input
      - normal_input

Pros:

• verbose
• clean

Cons:

• verbose

For inline

iter_task:
  do: 
    ops.print:
      - input in list_input
      - normal_input

Pros:

• less to write
  Cons:
• theoretically i can iterate over more than one input (matrix)

Lets here you.
@meshipeer @oritstone @meirwah @avimora @TudorLesanHP @Bonczidai @lrevnic @avigail-oron

which one did you like?
do you have another suggestion?
do you think we need to give an option to stop the iteration according to a condition? (while style..)
do you think we need to aggregate the result? and how?

When using flows and operations outside the current package, I need to use the --cp argument to let slang know where my .sl files are located at.

For example, I wanted to use the operation 'base_mail' from 'org.openscore.slang.base.mail', so I needed to use the following:

--cp c:/Work/hackaton/slang-content/org/openscore/slang/base/mail

Since it's a best practice to orginaize the .sl files into folders corresponding to their namespaces, I would like to be able to point to the root of these packages, like this:

--cp c:/Work/hackaton/slang-content

When looking in the execution.log file, the nextPosition key of the EVENT_OUTPUT_END event maps to a number which is meaningless to the user. It should map to something more useful like the name of the next task to be run.

Example: 2015-01-22 09:50:55:344 53572 [WorkerExecutionThread-0_101600001] INFO org.openscore.lang.cli.SlangCLI - Event received: EVENT_OUTPUT_END Data is: {PATH=0/1, DESCRIPTION=Output binding finished, EXECUTIONID=101600001, TASK_NAME=division1, TIMESTAMP=Thu Jan 22 09:50:55 IST 2015, nextPosition=6, TYPE=EVENT_OUTPUT_END, RESULT=SUCCESS, OUTPUTS={ans=1.5}}

May be value? expression?

@meshipeer @meirwah @orius123 @avimora @avigail-oron - please review and comment

I'm working now on improving the compiler error handling.
Our error handling is based on Runtime Exceptions thrown from the relevant places in the code, containing the relevant error message.

I basically do the following:

• Simulate corrupted slang files, problematic executables, and see if we receive the error message.
• Make the error messages more comprehensive
• Add missing error messages
• Add missing more advanced validations, such as checking that all results of an executables have matching navigation in the embedding task.
• Consider improving the YAML parser error messages

1. Do you think anything is missing?
2. Currently we throw runtime exception for each compilation error. Do you think we should throw dedicated Compilation exceptions instead, so that the CLI for example show them in a different manner than the runtime exceptions, or is the message enough?

it should be score-lang-all .. or something else since it is not API it is package...

This will allow users to have a customeized script that launches the CLI with the system props built in.
Having that will make the system properties really global and reusable in the CLI, as they are potentially in the embedded lib mode.

May be force? or invert it's meaning?

After the change,
we need to upload the artifact to maven central @hararzafrir , and then update the website @orius123

Error messages in the Slang console are difficult to read because they are colored dark red on a black background, having a weak contrast. We have found this during the Hackaton pretty annoying when doing pair programming, sitting in the front of laptop screens.

I would suggest using a brighter color for error messages.

Chmod issue?

when writing a flow I cannot access a flow var in the task publish statement.

for example:

flow:
  inputs:
    - input: 1
  workflow:
    some_task:
      ...
      publish:
        - new_var: input + 1

new output will be empty instead of 2

Currently there is no decent way to know the slang event type (exec or task) is by the ExecutionPath inner enum.
I think we should give a better way.
@avimora WDYT?

Got a compilation error, but instead of pointing me to the problematic file, it pointed to 'reader'.

Let's say we have an operation which has as action a python_script. Inside we have variables a = 100, b = 2
If I use
result = a < b it gets correctly evaluated to false.

But if I use
results:
- SUCCESS: a < b
Then the operation succeeds since it's doing a string comparison.

Submitted by @shajyhia

We wrote our flow as the the following (it's just a part of it) :

Our first 'publish' and 'navigate' tags were not indented correctly and we got an unclear error message without knowing in which step it failed.
The error message is the following : 'Slang Error : Input with name : text is Required, but value is empty'

I wanted to run the http_client_action operation in the CLI. The body input was JSON and should have contained commas, but that is the delimiter for the inputs.

Slang compiler should populate ExecutionPlan subflowUUID Set for dependencies resolving later.
I think the the member subflowUUID should be renamed to dependenciesUUIDs

I recently moved most of the Openstack/Docker slang content from score-language to slang-content and aligned it to the guidelines. Should I delete the files from score-language WDYT?

The Sublime snippet has a spot for flow results:

flow:
      name: flow name - {sub.sl}
      # inputs:
      workflow:
      # outputs:
      # results:

Attempting to use results the CLI gives you this: Command failed java.lang.RuntimeException: java.lang.NullPointerException.

Currently, a subflow can pass it's results to its superflow in the usual manner: by the results/navigation of the final task (this itself only works for SUCCESS and FAILURE, but not other results).

It seems like input assignments on steps are ignored when there a flow input with the same name as the step input. Check the following modified hello-world application:

namespace: user.examples.hello_world

imports:
  ops: user.examples.hello_world

flow:
  name: hello_world
  inputs:
    - text1
    - text2
    - text3
  workflow:
    sayHi:
      do:
        ops.print:
          - text1: text3
          - text2: text3

with the corresponding 'print' operation:

namespace: user.examples.hello_world

operations:
  - print:
      inputs:
        - text1
        - text2
      action:
        python_script: print text1, text2
      results:
        - SUCCESS

When running with the arguments --i text1=Hello,text2=World,text3=Unrelated, I would expect it to print "Unrelated Unrelated", but instead, it seems to print "Hello World":

slang>run --f c:\Work\hackaton\stuff\hello_world.sl --i text1=Hello,text2=World,text3=Unrelated
- sayHi
Hello World
Flow : hello_world finished with result : SUCCESS
Flow execution time took  0:00:00.411 , with execution id : 101600729

Maybe we can add prints of flow outputs to the CLI prints (like in task name).
so it will be easy to see flow result...