场景报错信息排查过程处理方式字段记录(未完待续)
场景业务要求,需要按照租户进行分表,并需要对敏感信息进行脱敏存储决定使用业界成熟的数据库中间件shardingshere-proxy使用的是PostgreSQL数据库,版本13.5项目使用的hibernate 版本 5.4,jdk版本11部署使用中间件版本shardingsphere 5.0.0,jdk 8(无法使用11)使用JPA查询时报错,但是不使用shardingphere-proxy时,JPA可以正常查询报错的代码
// 报错片段1,get报错
Map params = new HashMap<>();
params.put("villageId", villageId);
params.put("customerId", customerId);
params.put("relaId", id);
List attachments = this.getEntities(VillageAttachmentEO.class, params);
// 报错片段2,JPA的deleteBy删除方法报错
void deleteByRelaId(Long relaId);
报错信息
[ERROR] 2022-02-25 17:54:37.073 [Connection-1-ThreadExecutor] o.a.s.p.f.c.CommandExecutorTask - Exception occur: com.google.common.util.concurrent.UncheckedExecutionException: org.apache.shardingsphere.sql.parser.exception.SQLParsingException: You have an error in your SQL syntax. at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2051) at com.google.common.cache.LocalCache.get(LocalCache.java:3951) at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974) at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4958) at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4964) at org.apache.shardingsphere.infra.parser.sql.SQLStatementParserEngine.parse(SQLStatementParserEngine.java:48) at org.apache.shardingsphere.infra.parser.ShardingSphereSQLParserEngine.parse0(ShardingSphereSQLParserEngine.java:71) at org.apache.shardingsphere.infra.parser.ShardingSphereSQLParserEngine.parse(ShardingSphereSQLParserEngine.java:60) at org.apache.shardingsphere.proxy.frontend.postgresql.command.query.binary.parse.PostgreSQLComParseExecutor.parseSql(PostgreSQLComParseExecutor.java:54) at org.apache.shardingsphere.proxy.frontend.postgresql.command.query.binary.parse.PostgreSQLComParseExecutor.排查过程(PostgreSQLComParseExecutor.java:42) at org.apache.shardingsphere.proxy.frontend.postgresql.command.PostgreSQLCommandExecutorFactory.newInstance(PostgreSQLCommandExecutorFactory.java:70) at org.apache.shardingsphere.proxy.frontend.postgresql.command.PostgreSQLCommandExecuteEngine.getCommandExecutor(PostgreSQLCommandExecuteEngine.java:72) at org.apache.shardingsphere.proxy.frontend.command.CommandExecutorTask.executeCommand(CommandExecutorTask.java:97) at org.apache.shardingsphere.proxy.frontend.command.CommandExecutorTask.run(CommandExecutorTask.java:72) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.shardingsphere.sql.parser.exception.SQLParsingException: You have an error in your SQL syntax. at org.apache.shardingsphere.sql.parser.core.database.parser.SQLParserExecutor.twoPhaseParse(SQLParserExecutor.java:68) at org.apache.shardingsphere.sql.parser.core.database.parser.SQLParserExecutor.parse(SQLParserExecutor.java:49) at org.apache.shardingsphere.sql.parser.api.SQLParserEngine.parse(SQLParserEngine.java:51) at org.apache.shardingsphere.infra.parser.sql.SQLStatementParserExecutor.parse(SQLStatementParserExecutor.java:47) at org.apache.shardingsphere.infra.parser.cache.SQLStatementCacheLoader.load(SQLStatementCacheLoader.java:40) at org.apache.shardingsphere.infra.parser.cache.SQLStatementCacheLoader.load(SQLStatementCacheLoader.java:29) at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529) at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278) at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155) at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045) ... 16 common frames omitted
未使用sharingphere-proxy前不报错,使用后报错,说明是中间件的问题报错信息很明显,就是SQL执行报错,需要把SQL打印出来修改源码,把报错SQL打印出来,根据报错行数,修改类SQLParserExecutor,增加报错的SQL打印,输出后如下:
select villageatt0_.id as id1_8_, villageatt0_.create_date as create_d2_8_, villageatt0_.create_organ_id as create_o3_8_, villageatt0_.create_user_id as create_u4_8_, villageatt0_.update_date as update_d5_8_, villageatt0_.update_user_id as update_u6_8_, villageatt0_.customer_id as customer7_8_, villageatt0_.file_name as file_nam8_8_, villageatt0_.file_type as file_typ9_8_, villageatt0_.path as path10_8_, villageatt0_.rela_id as rela_id11_8_, villageatt0_.size as size12_8_, villageatt0_.type as type13_8_, villageatt0_.village_id as village14_8_ from village_attachment villageatt0_ where villageatt0_.rela_id=?
没发现SQL有任何问题,拿到原有pgsql建立连接,去执行SQL,没问题使用sharding的ip端口建立连接,执行SQL,发现确实报错把SQL拿到navcat,使用美化SQL功能,发现有几个字段转化为了大写,疑似为关键字或函数名称,ID、PATH、SIZE、TYPE对这4个字段逐个测试,发现会出现大小写异常,使用小写的继续测试最终发现是path字段影响的,去除该字段就不会报错,即使大写也不会报大小写异常 处理方式
既然确定了是字段path的影响,修改该字段即可,修改为path_,问题解决
@Column(name = "path_")
private String path;
其实也可以通过修改源码解决,SQL解析时,对字段名称拼接上双引号。path字段加上双引号,就可以执行成功,即"path"但是考虑到以后中间件的版本升级替换问题,暂时不采用此方法 字段记录(未完待续)
对使用中遇到的字段进行汇总,后续开发使用时,作为参考path



