Add support for operations on List types to cls_tabular class about skyhookdm-ceph-cls HOT 4 OPEN

Hi Jeff, Can you provide a bit more info on why instead of what? Who wants this functionality? I also like to see a plan of how this work can be isolated from any effects due to awkward rewrites (one already happened since the original release). Also, it would be so nice if the debugging work that Jim and others already performed on equivalent functionality withinawkward doesn't have to be repeated here. Perhaps this is a good use case for thinking about how to run libraries within Ceph/CLS using outcomes of Saloni's work on WebAssembly isolates.

from skyhookdm-ceph-cls.

Good points Carlos, thanks. I totally agree and updated the beginning of the comment to add motivation. We should definitely not duplicate awkward functionality, and get in the business of closely tracking awkward lib. It was really meant to use as a motivation of the types of list ops scientific analysts use. Probably the simple ops can apply to lists in general so I think this will be very useful. Regarding WebAssembly, yes I think once we understand how to express and offload list predicates, we should definitely consider the feasibility of a WebAssembly approach for awkward.

from skyhookdm-ceph-cls.

Now that I know a bit more about Arrow, doesn't this duplicate Arrow compute and function functionalities?

from skyhookdm-ceph-cls.

Yes we can use the array ops available in Arrow. Xiongfeng is working on this.

from skyhookdm-ceph-cls.