您可以对BigQuery加载API使用自动检测,即使用bq cli工具的示例如下所示:
~$ cat /tmp/x.json{"name":"xyz","mobile":"xxx","location":"abc"}{"name":"xyz","mobile":"xxx","age":"22"}~$ bq load --autodetect --source_format=newline_DELIMITED_JSON tmp.x /tmp/x.jsonUpload complete.~$ bq show tmp.xTable tmp.x Last modified Schema Total Rows Total Bytes Expiration ----------------- --------------------- ------------ ------------- ------------ 16 Aug 08:23:35 |- age: integer 2 33|- location: string |- mobile: string |- name: string~$ bq query "select * from tmp.x"+------+----------+--------+------+| age | location | mobile | name |+------+----------+--------+------+| NULL | abc | xxx | xyz || 22 | NULL | xxx | xyz |+------+----------+--------+------+更新:
如果以后需要添加其他字段,则可以使用schema_update_option允许新字段。遗憾的是,它还不能与自动检测一起使用,因此您需要为负载API明确提供新的架构:
~$ cat /tmp/x1.json {"name":"abc","mobile":"yyy","age":"25","gender":"male"}~$ bq load --schema=name:STRING,age:INTEGER,location:STRING,mobile:STRING,gender:STRING --schema_update_option=ALLOW_FIELD_ADDITION --source_format=newline_DELIMITED_JSON tmp.x /tmp/x1.jsonUpload complete.~$ bq show tmp.xTable tmp.x Last modified Schema Total Rows Total Bytes Expiration ----------------- --------------------- ------------ ------------- ----------- 19 Aug 10:43:09 |- name: string 3 57|- age: integer|- location: string |- mobile: string |- gender: string~$ bq query "select * from tmp.x"status: DONE +------+------+----------+--------+--------+| name | age | location | mobile | gender |+------+------+----------+--------+--------+| abc | 25 | NULL | yyy | male || xyz | NULL | abc | xxx | NULL || xyz | 22 | NULL | xxx | NULL |+------+------+----------+--------+--------+


