有,
SchemaConverters.createConverterToSQL但是不幸的是私人的。有一些PR可以将其公开,但是它们从未合并:
- https://github.com/databricks/spark-avro/pull/89
- https://github.com/databricks/spark-avro/pull/132
尽管我们使用了一种解决方法。
您可以通过在
com.databricks.spark.avro包中创建一个类来公开它:
package com.databricks.spark.avroimport org.apache.avro.Schemaimport org.apache.avro.generic.GenericRecordimport org.apache.spark.sql.Rowimport org.apache.spark.sql.types.DataTypeobject MySchemaConversions { def createConverterToSQL(avroSchema: Schema, sparkSchema: DataType): (GenericRecord) => Row = SchemaConverters.createConverterToSQL(avroSchema, sparkSchema).asInstanceOf[(GenericRecord) => Row]}然后,您可以在代码中使用它,如下所示:
final DataType myAvroType = SchemaConverters.toSqlType(MyAvroRecord.getClassSchema()).dataType();final Function1<GenericRecord, Row> myAvroRecordConverter = MySchemaConversions.createConverterToSQL(MyAvroRecord.getClassSchema(), myAvroType);Row[] convertAvroRecordsToRows(List<GenericRecord> records) { return records.stream().map(myAvroRecordConverter::apply).toArray(Row[]::new);}对于一条记录,您可以这样称呼它:
final Row row = myAvroRecordConverter.apply(record);



