栏目分类:
子分类:
返回
名师互学网用户登录
快速导航关闭
当前搜索
当前分类
子分类
实用工具
热门搜索
名师互学网 > IT > 前沿技术 > 大数据 > 大数据系统

SparkSQL中posexplode高阶函数设置别名

SparkSQL中posexplode高阶函数设置别名

  • posexplode(t.names) as (`arr_pos`, `arr_value`)
  • posexplode(t.names) as (arr_pos, arr_value)
spark.sql(
      """
        |with t_emp as(
        |select t.deptno,
        |       collect_list(t.ename) as names
        |  from emp t
        | group by t.deptno
        | )
        |select t.deptno, t.names, posexplode(t.names) as (`arr_pos`, `arr_value`)
        |  from t_emp t
        |""".stripMargin)
        .show(false)
+------+-------------------------------------------+-------+---------+
|deptno|names                                      |arr_pos|arr_value|
+------+-------------------------------------------+-------+---------+
|10    |[CLARK, KING, MILLER, MILLER]              |0      |CLARK    |
|10    |[CLARK, KING, MILLER, MILLER]              |1      |KING     |
|10    |[CLARK, KING, MILLER, MILLER]              |2      |MILLER   |
|10    |[CLARK, KING, MILLER, MILLER]              |3      |MILLER   |
|30    |[ALLEN, WARD, MARTIN, BLAKE, TURNER, JAMES]|0      |ALLEN    |
|30    |[ALLEN, WARD, MARTIN, BLAKE, TURNER, JAMES]|1      |WARD     |
|30    |[ALLEN, WARD, MARTIN, BLAKE, TURNER, JAMES]|2      |MARTIN   |
|30    |[ALLEN, WARD, MARTIN, BLAKE, TURNER, JAMES]|3      |BLAKE    |
|30    |[ALLEN, WARD, MARTIN, BLAKE, TURNER, JAMES]|4      |TURNER   |
|30    |[ALLEN, WARD, MARTIN, BLAKE, TURNER, JAMES]|5      |JAMES    |
|20    |[SMITH, JONES, SCOTT, ADAMS, FORD]         |0      |SMITH    |
|20    |[SMITH, JONES, SCOTT, ADAMS, FORD]         |1      |JonES    |
|20    |[SMITH, JONES, SCOTT, ADAMS, FORD]         |2      |SCOTT    |
|20    |[SMITH, JONES, SCOTT, ADAMS, FORD]         |3      |ADAMS    |
|20    |[SMITH, JONES, SCOTT, ADAMS, FORD]         |4      |FORD     |
+------+-------------------------------------------+-------+---------+

转载请注明:文章转载自 www.mshxw.com
本文地址:https://www.mshxw.com/it/654489.html
我们一直用心在做
关于我们 文章归档 网站地图 联系我们

版权所有 (c)2021-2022 MSHXW.COM

ICP备案号:晋ICP备2021003244-6号