pyspark 可以通过pip或pycharm 直接安装,在用pyspark连mysql的时候发现报错
: java.lang.ClassNotFoundException: com.mysql.jdbc.Driver
这个时候要想办法把mysql jar包加载进来,直接看代码
from pyspark.sql import SparkSession
from pyspark import SparkConf, SparkContext, SQLContext
from pyspark.sql.functions import *
sparkconf = SparkConf().set("spark.driver.extraClassPath",
"/Users/duzhixin/Downloads/baiwei/mysql-connector-java-5.1.17.jar")
spark = SparkSession.builder.config(conf=sparkconf).appName("SimpleApp").getOrCreate()
# spark.conf().set("spark.driver.extraClassPath", "/Users/duzhixin/Downloads/mysql-connector-java-5.1.17.jar")
df = spark.read.format("jdbc").options(
url="jdbc:mysql://***:3306/abi-cloud-middle-task-center-dev",
driver="com.mysql.jdbc.Driver",
dbtable="**", user="**", password="**").load()
df.show()



