[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Flink 1.5 + resource elasticity resulting in overloaded workers


On Thu, Dec 20, 2018, 12:56 AM jelmer <jkuperus@xxxxxxxxx wrote:
Hi, We recently upgraded to flink 1.6 and seem to be suffering from the issue described in this email

Our workers have 8 slots and some workers are fully loaded and as a consequence get to cope with heavy load during peak times. while other workers sit completely idle. and have 0 jobs assigned to its slots

Is there any workaround for this . short of reducing the number of slots on a worker ? 

We need to have double the slots we need available in order to cope with availability zone maintenance . So if we where to reduce the number of slots we'd have to add new nodes that would then mostly sit idle