Code of Conduct
Search before asking
Describe the feature
AuthzConfigurationChecker matches the shape of the plan — SetCommand and ResetCommand — and throws when a SET carries spark.sql.optimizer.excludedRules with a value from org.apache.kyuubi.plugin.spark.authz.ranger.* (AuthzConfigurationChecker.scala:42-53). Every path that writes the same key without producing a plan is unaffected, because RuleAuthorization is an injected optimizer rule (RangerSparkExtension.scala:65) and extension rules are not part of SparkOptimizer.nonExcludableRules. I would like the check rule to read the effective value on every plan instead, so that enforcement no longer depends on which API set the config.
Motivation
Kyuubi 1.13.0-SNAPSHOT built for Spark 4.0.3, engine in local[*] with RangerSparkExtension and no policies, share level USER, stock server configuration. Two JDBC connections of the same user, so the second one reuses the engine started by the first:
A1 create table, plain session -> Permission denied: user [probeuser] does not have [create] privilege on [default/jdbc_a51791]
A2 SET excludedRules via SQL -> Excluding Authz security rules is not allowed
B1 create table, conf in JDBC url -> OK
B differs from A only by ;#spark.sql.optimizer.excludedRules=org.apache.kyuubi.plugin.spark.authz.ranger.RuleAuthorization in the JDBC URL. Session confs are applied through SparkSessionImpl.open() → spark.conf.set (SparkSessionImpl.scala:55-61, :101-103), which never builds a plan.
This is not an undocumented hole. docs/security/authorization/spark/overview.rst:85-93 prescribes kyuubi.session.conf.restrict.list = spark.driver.memory,spark.sql.optimizer.excludedRules, and with that set the connection is rejected at open — I verified that as well. The gap is that enforcement lives in two places that have to agree, the safe one is off by default, and it only covers the channels that pass through the server's SessionManager.
Describe the solution
Have AuthzConfigurationChecker read spark.sql.optimizer.excludedRules from the session config on every plan and throw if it names an authz rule, in addition to the existing SetCommand and ResetCommand cases. Check rules are not subject to excludedRules — that config filters optimizer batches only — so the check itself cannot be removed by the same trick. The restrict list then becomes defence in depth rather than the only line, and any client writing the key through spark.conf.set is covered too.
Additional context
Related to #7454, and to the fail-open theme in #7593.
Are you willing to submit PR?
Code of Conduct
Search before asking
Describe the feature
AuthzConfigurationCheckermatches the shape of the plan —SetCommandandResetCommand— and throws when aSETcarriesspark.sql.optimizer.excludedRuleswith a value fromorg.apache.kyuubi.plugin.spark.authz.ranger.*(AuthzConfigurationChecker.scala:42-53). Every path that writes the same key without producing a plan is unaffected, becauseRuleAuthorizationis an injected optimizer rule (RangerSparkExtension.scala:65) and extension rules are not part ofSparkOptimizer.nonExcludableRules. I would like the check rule to read the effective value on every plan instead, so that enforcement no longer depends on which API set the config.Motivation
Kyuubi 1.13.0-SNAPSHOT built for Spark 4.0.3, engine in
local[*]withRangerSparkExtensionand no policies, share level USER, stock server configuration. Two JDBC connections of the same user, so the second one reuses the engine started by the first:B differs from A only by
;#spark.sql.optimizer.excludedRules=org.apache.kyuubi.plugin.spark.authz.ranger.RuleAuthorizationin the JDBC URL. Session confs are applied throughSparkSessionImpl.open()→spark.conf.set(SparkSessionImpl.scala:55-61,:101-103), which never builds a plan.This is not an undocumented hole.
docs/security/authorization/spark/overview.rst:85-93prescribeskyuubi.session.conf.restrict.list = spark.driver.memory,spark.sql.optimizer.excludedRules, and with that set the connection is rejected at open — I verified that as well. The gap is that enforcement lives in two places that have to agree, the safe one is off by default, and it only covers the channels that pass through the server'sSessionManager.Describe the solution
Have
AuthzConfigurationCheckerreadspark.sql.optimizer.excludedRulesfrom the session config on every plan and throw if it names an authz rule, in addition to the existingSetCommandandResetCommandcases. Check rules are not subject toexcludedRules— that config filters optimizer batches only — so the check itself cannot be removed by the same trick. The restrict list then becomes defence in depth rather than the only line, and any client writing the key throughspark.conf.setis covered too.Additional context
Related to #7454, and to the fail-open theme in #7593.
Are you willing to submit PR?