How to unwrap nested Struct column into multiple columns?

You can select

data.bar.baz

bar.baz

df.show()+-------+|   data|+-------+|[3,[2]]|+-------+df.printSchema()root |-- data: struct (nullable = false) |    |-- foo: long (nullable = true) |    |-- bar: struct (nullable = false) |    |    |-- baz: long (nullable = true)

In pyspark:

import pyspark.sql.functions as Fdf.select(F.col("data.foo").alias("foo"), F.col("data.bar.baz").alias("bar.baz")).show()+---+-------+|foo|bar.baz|+---+-------+|  3|      2|+---+-------+