HA mode not activated java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.security.SecurityUtil

[Vince-Cercury]

I've setup a zookeeper:

helm --namespace vince2 install --name vince2-zookeeper incubator/zookeeper

created a values.yaml

cat > values.yaml <<EOF
flink:
num_taskmanagers: 3
highavailability:
enabled: true
zookeeper_quorum: vince2-zookeeper-zookeeper
state_s3_bucket: s3://my-bucket/vince2/ha
EOF

( don't need AWS keys, I have instanceProfile allowing any pod to read/write into that S3. Testing with another component)

Deploy FLink:

helm --namespace vince2 install --name=vince2-flink-ha --values=values.yaml flink-1.4.0.tgz

But the job manager logs says it is not starting with HA

2018-04-16 04:59:11,606 INFO org.apache.flink.runtime.jobmanager.JobManager - Starting JobManager without high-availability
2018-04-16 04:59:11,609 INFO org.apache.flink.runtime.jobmanager.JobManager - Starting JobManager on vince2-flink-ha-flink-jobmanager:6123 with execution mode CLUSTER

It seems the HA settings didn't make it to the configmap:

flink-conf.yaml: | blob.server.port: 6124 jobmanager.rpc.address: vince2-flink-ha-flink-jobmanager jobmanager.rpc.port: 6123 jobmanager.heap.mb: 1024 taskmanager.heap.mb: 1024 taskmanager.numberOfTaskSlots: 1

[Vince-Cercury]

I could load the right settings in configmap.

However with HA, it's missing a SecurityUtil

Exception in thread "main" java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.security.SecurityUtil

Has someone got it working with HA?

[patricklucas]

Hmm. Usually, I would say that this is a consequence of using Hadoop-less Flink but using an s3:// filesystem URI without copying the Presto S3 filesystem implementation from opt to lib, but if you're using the Flink images hosted on Docker Hub, they call include Hadoop.

Have you searched the Flink JIRA issues and mailing list? Someone might have run into this before.