spark报错总结
- spark sql
- 1、spark sql 删除分区报错 mismatched input '<=' expecting
spark sql
1、spark sql 删除分区报错 mismatched input ‘<=’ expecting
- 记录时间
2021年11月23日20:11:07 - spark版本 2.1.0
- 报错详细信息
Traceback (most recent call last):
File "/tmp/voldemort/0000003358711/resource.test.py", line 28, in
spark.sql(sql_drop_part)
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera4-1.cdh5.13.3.p0.818552/lib/spark2/python/lib/pyspark.zip/pyspark/sql/session.py", line 545, in sql
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera4-1.cdh5.13.3.p0.818552/lib/spark2/python/lib/py4j-0.10.7- src.zip/py4j/java_gateway.py", line 1257, in __call__
File "/opt/cloudera/parcels/SPARK2-2.1.0.cloudera4-1.cdh5.13.3.p0.818552/lib/spark2/python/lib/pyspark.zip/pyspark/sql/utils.py", line 73, in deco
pyspark.sql.utils.ParseException: u"nmismatched input '<=' expecting {')', ','}(line 1, pos 85)nn== SQL ==nalter table ods.test_table_name drop if exists partition(dt <= '20211102')n-------------------------------------------------------------------------------------^^^n"
spark版本问题，删除分区，无法匹配 '>' 或者 ‘<’，详情见 url - 目前处理方式
# 使用 =
sql = """
alter table ods.test_table_name drop if exists partition(dt = '20211102')
"""
