栏目分类:
子分类:
返回
名师互学网用户登录
快速导航关闭
当前搜索
当前分类
子分类
实用工具
热门搜索
名师互学网 > IT > 前沿技术 > 大数据 > 大数据系统

flink 压测一些参数调优

flink 压测一些参数调优

####### jobmanager config options #######

jobmanager.heap.size: 20000m
cluster.evenly-spread-out-slots: true

####### timeout config options #######

akka.ask.timeout: 300 s
akka.lookup.timeout: 300 s
web.timeout: 300000
heartbeat.timeout: 600000
resourcemanager.taskmanager-timeout: 900000
jobstore.expiration-time: 36000

####### restart config options ######
jobmanager.execution.failover-strategy: region
restart-strategy.fixed-delay.delay: 10 s
restart-strategy: fixed-delay
restart-strategy.fixed-delay.attempts: 50

####### taskmanager config options #######

# totally 10 taskmanagers in one machine.
taskmanager.memory.process.size: 22000m
# configure heap-cutoff-ratio to leave memory to system page cache
containerized.heap-cutoff-ratio: 0.3
taskmanager.memory.managed.size: 8000m
taskmanager.numberOfTaskSlots: 10
# not limit by yarn vcores
yarn.containers.vcores: 1

taskmanager.network.memory.fraction: 0.99
taskmanager.network.memory.min: 64mb
taskmanager.network.memory.max: 2200mb
taskmanager.network.memory.exclusive-buffers-request-timeout-ms: 1000000
taskmanager.network.blocking-shuffle.type: mmap
taskmanager.network.blocking-shuffle.compression.enabled: true

参考地址:

https://github.com/ververica/flink-sql-benchmark/blob/master/flink-tpcds/flink-conf.yaml

flinksql客户端一些配置信息文档:

Configuration | Apache Flink

group by 的状态优化

table.exec.state.ttl=180 s

转载请注明:文章转载自 www.mshxw.com
本文地址:https://www.mshxw.com/it/742738.html
我们一直用心在做
关于我们 文章归档 网站地图 联系我们

版权所有 (c)2021-2022 MSHXW.COM

ICP备案号:晋ICP备2021003244-6号