Unable to format data from store as JSON in javascript script handler about imposter HOT 11 OPEN

To simplify things here, there's a new loadAsJson() function for stores in v3.39.0.

Using that, the example is now:

var personsStore = stores.open('persons');
var persons = personsStore.loadAsJson('persons');

respond().withContent(persons);

You can also use JSON.parse(persons) here if you need to manipulate the data in JavaScript.

from imposter.

Maybe oracle/graaljs#478 is related?

from imposter.

According to oracle/graaljs#766 and oracle/graaljs#757 I should be able to use JSON.stringify(Object.fromEntries(Array.from(dataMap))), but this results in an exception:

...
Caused by: org.graalvm.polyglot.PolyglotException: TypeError: (intermediate value).fromEntries is not a function
...

According to oracle/graaljs#766 this is related to "enabled nashorn compability".

Edit: Looking at

nashorn compatibility seems enabled indeed. Is there a way to disable this?

from imposter.

@outofcoffee I think one solution would be to get rid of the nashorn JS altogether and only rely on graaljs without the nashorn-compat property set to get good support for newer JS features. What do you think of this?

from imposter.

Hi @rnestler, thanks for raising this.

var ObjectMapper = Java.type('com.fasterxml.jackson.databind.ObjectMapper');
var serialiser = new ObjectMapper();

var personsStore = stores.open('persons');
var persons = personsStore.load('persons');

var json = serialiser.writeValueAsString(persons);

respond().withContent(json);

I created a branch (https://github.com/outofcoffee/imposter/tree/spike/graal-js-modern) with a new Graal script engine implementation, without Nashorn compatibility. Unfortunately the results were the same.

So, digging deeper, the loaded data is deserialised by Java in an ArrayList of LinkedHashMap objects. These don't seem to be understood by Graal (or Nashorn)'s JSON.stringify function.

A work-around is to make use of a Java serialiser - in this case, Jackson:

var ObjectMapper = Java.type('com.fasterxml.jackson.databind.ObjectMapper');
var serialiser = new ObjectMapper();

var personsStore = stores.open('persons');
var persons = personsStore.load('persons');

var json = serialiser.writeValueAsString(persons);

respond().withContent(json);

This produces the following output:

$ curl http://localhost:8080/persons

[{"id":1,"firstName":"Foo","lastName":"Bar"},{"id":2,"firstName":"Test","lastName":"Ing"}]

It works with Nashorn as well as Graal.

from imposter.

I created a branch (https://github.com/outofcoffee/imposter/tree/spike/graal-js-modern) with a new Graal script engine implementation, without Nashorn compatibility. Unfortunately the results were the same

But did the JSON.stringify(Object.fromEntries(Array.from(dataMap))) snipped also still raise the fromEntries is not a function error?

from imposter.

The new loadAsJson() works quite nice! See rnestler/imposter-state-minimum-example@962522a

Thanks for being so super responsive on this issue!

I saw that loadAsJson() is already documented in https://docs.imposter.sh/stores/. But maybe it would make sense to document that without it, the returned data may not behave as expected due to it being ArrayList / LinkedHashMap.

from imposter.

Hi @rnestler, you're most welcome :) Thank you for your contributions.

We've released v3.40.0, which has an improved js-graal plugin. This one has Nashorn compatibility turned off and new plumbing for Graal to understand how to parse objects returned from stores.

You can enable this behaviour with the following environment variable:

IMPOSTER_GRAAL_STORE_PROXY=true

Would you mind seeing if this helps with the original issue?

from imposter.

@outofcoffee I tested again with 3.40.0:

• With loadAsJson everything works as before (https://github.com/rnestler/imposter-state-minimum-example/tree/eef11675c91f0d156ed4f87db3dbc4123c5d0e14)
• With just using load I get an error

  imposter  | Caused by: org.graalvm.polyglot.PolyglotException: TypeError: invokeMember (withContent) on io.gatehill.imposter.script.ReadWriteResponseBehaviourImpl failed due to: Cannot convert '[{id=1, firstName=Foo, lastName=Bar}, {id=2, firstName=Test, lastName=Ing}]'(language: Java, type: java.util.ArrayList) to Java type 'java.lang.String': Invalid or lossy primitive coercion.
  

  (https://github.com/rnestler/imposter-state-minimum-example/tree/c2328872814b664c8bcc38480053e997d96a9b3b)
• With load and JSON.stringify I again get the [{},{}] response. (https://github.com/rnestler/imposter-state-minimum-example/tree/b82d6e68d2685139fa49ff9ee5133adde9b35a3e)
• When I set IMPOSTER_GRAAL_STORE_PROXY=true the docker container doesn't start but errors with:

imposter  | 15:40:05 DEBUG i.g.i.s.g.s.GraalvmScriptServiceImpl - Enabling Graal store proxy
imposter  | 15:40:05 ERROR i.v.c.i.l.c.VertxIsolatedDeployer - Failed in deploying verticle
imposter  | java.lang.RuntimeException: Error registering plugin: class io.gatehill.imposter.scripting.graalvm.service.GraalvmScriptServiceImpl
imposter  | 	at io.gatehill.imposter.plugin.PluginManagerImpl.createPlugins(PluginManagerImpl.kt:105) ~[imposter-config-3.40.0.jar:?]
imposter  | 	at io.gatehill.imposter.plugin.PluginManagerImpl.startPlugins(PluginManagerImpl.kt:90) ~[imposter-config-3.40.0.jar:?]
imposter  | 	at io.gatehill.imposter.Imposter$start$1.invokeSuspend(Imposter.kt:132) ~[imposter-engine-3.40.0.jar:?]
imposter  | 	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) ~[kotlin-stdlib-1.9.10.jar:1.9.10-release-459]
imposter  | 	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108) ~[kotlinx-coroutines-core-jvm-1.7.3.jar:?]
imposter  | 	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584) ~[kotlinx-coroutines-core-jvm-1.7.3.jar:?]
imposter  | 	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793) ~[kotlinx-coroutines-core-jvm-1.7.3.jar:?]
imposter  | 	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697) ~[kotlinx-coroutines-core-jvm-1.7.3.jar:?]
imposter  | 	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684) ~[kotlinx-coroutines-core-jvm-1.7.3.jar:?]
imposter  | Caused by: com.google.inject.ProvisionException: Unable to provision, see the following errors:
imposter  |
imposter  | 1) [Guice/ErrorInjectingMethod]: RuntimeException: Unable to load store driver: store-inmem. Must be an installed plugin implementing StoreFactory
imposter  |   at GraalvmScriptServiceImpl.onPostInject(GraalvmScriptServiceImpl.kt:92)
imposter  |   at GraalvmScriptingModule.configure(GraalvmScriptingModule.kt:54)
imposter  |   while locating GraalvmScriptServiceImpl

https://github.com/rnestler/imposter-state-minimum-example/tree/cc3109a223616f3035282ffbe85f9c027118fbb0

from imposter.

But it seems that with the improved js-graal plugin Object.fromEntries(Array.from(persons[0])) works.

from imposter.