I just realized I didn't make it very clear my stance here ... here's another try:
I think it's a no brainer to have a good columnar UDF interface. This would facilitate a lot of high performance applications, e.g. GPU-based accelerations for machine learning algorithms.
On rewriting the entire internals of Spark SQL to leverage columnar processing, I don't see enough evidence to suggest that's a good idea yet.
On Wed, Mar 27, 2019 at 8:10 AM, Bobby Evans < [EMAIL PROTECTED] > wrote: