Python functions called from Lua do not unpack returned tuples. about lupa HOT 9 CLOSED

But what if the Lua code was this instead:

py_tuple = pyfun()

I don't see a way to distinguish between the two use cases at the point where the Python function is being called.

from lupa.

I don't understand. Do you actually have to distinguish these things? Since

a, b, c = lua.eval("1, 2, 3")
d = lua.eval("1,2,3")

both work fine and the way one would expect. So if pyfun() returns an unpacked result into Lua instead of a table you get the same result translated back into Python in the same way.

What I am suggesting is, when a Python function is called from Lua and returns a tuple, it gets unpacked. That unpacked result, if it is returned back to Python gets handled the same way as the above code and "just works". And it also works in Lua. It seems problematic to me that:

lua.execute("a,b,c = pyfun()")
a, b, c = lua.eval("a, b, c")
print a, b, c
a, b, c = pyfun()
print a, b, c

gives you:

('one', 'two', 'three'), None, None
'one' 'two' 'three'

I've committed a change to a fork I've made and it seems to work. Here's the diff.

from lupa.

I understand your request. My question was: what if you actually wanted a Python tuple as a return value? The wrapper code that returns it cannot know what the Lua code will do with that result. Always exploding the tuple will come as the same kind of surprise in some cases as not exploding the tuple. Which is better isn't obvious to me.

from lupa.

Ah. I think I get it. You mean:

lua.execute("a = pyfun()")
print lua.eval("a")

just giving you "one" and not "one", "two", "three"?

I don't see a problem with that. Lua throws stuff like that away all the time. There is definitely a mismatch in how Python and Lua deal with these things. If you really want the behavior to be returning set of things, pyfun can return a list, which would not get exploded.

With out the exploding, I don't see how to write Python functions that work in the Lua way. eg

result, message = python_lib_fun()

If everything gets packed into result and message is always nil, all I have is a userdata blob in result that I can't really look into without risking an exception. So you end up always returning some kind of consistent structure that looks unnatural on both they Python and the Lua side and we lose a lot of transparency.

I think losing the ability to unpack return values is far worse than possibly dropping tuple elements. Lua functions that want to pack multiple values into one unexplodable thing have to do that explicitly. I don't see why Python functions should be any different.

from lupa.

what about calling a function to explode the tuple, e.g. "a,b,c = python.unpack(py_fun())" ?

from lupa.

I tried playing with that, but it only works on the Lua side. There doesn't seem to be anything you can do on the Python side to hide the fact that Lua is calling a Python function. It seems clunky to me that the Lua code needs to know that it's calling a Python function. What happens when you change the implementation of py_fun to something written in pure Lua? You have to change all the calling code. And all the Python functions have to be written to always return tuples that may or may not include error information (or some similar workaround), which makes those Python functions look weird if you want to use them in Python. So I end up with code that looks funny on both sides.

Given the language mismatch, there is going to be some goofiness. It seems to me that simply exploding tuples returned from Python function calls reduces the clunkiness considerably.

It seems asymmetric to me that a, b, c = f() unpacks the tuple in all cases except when f() is a Python function and the call and assignment happen inside Lua.

a, b, c = f()                 # unpacks
a, b, c = lua.eval("f()")     # unpacks
lua.execute("a, b, c = f()")  # doesn't unpack.  unless f() is a Lua function

The seam between Lua and Python here is exposed. I suppose there is going to be some exposure somewhere and we are left with a trade-off -- either possibly losing some tuple values (which is proper Lua behavior anyway) or by having to know on the Python side who is using your function, and on the Lua side, who is implementing your function.

from lupa.

It makes sense to me to explode tuple return values by default. It's certainly easier to deal with the separate items on Lua side than to have to deal with the tuple and unpack it yourself.

The problem is that it breaks existing code. Not sure if there are any cases in Lua where this would be hard to detect or debug, but given that return values just vanish if you don't collect them, it seems likely that there are such cases, so this change can become annoying for users.

I think the safest would be an option in the LuaRuntime for now. Something like "unpack_returned_tuples=False", which users can set to True if they want. I would add a warning to the docs that the default might change in the future, so users should always pass the option the way they want it to behave.

from lupa.

That sounds perfect. I'll work this up if you like and get back to you.

Thanks so much for this awesome tool!

from lupa.

Fix released in Lupa 0.21.

from lupa.