Skip to content

Add Spark 3.5 Scala 2.13 DBF datasource#1

Merged
DAChernikov merged 5 commits into
masterfrom
feature/spark-3.5-scala-2.13
Jul 14, 2026
Merged

Add Spark 3.5 Scala 2.13 DBF datasource#1
DAChernikov merged 5 commits into
masterfrom
feature/spark-3.5-scala-2.13

Conversation

@DAChernikov

Copy link
Copy Markdown
Owner

Summary

  • migrate build to Spark 3.5 / Scala 2.13 with provided Spark, Scala and Hadoop dependencies
  • implement read-only DBF DataSource V1 with Hadoop FileSystem access and executor-side reads
  • add Scala Spark tests, PySpark assembly smoke checks, jar inspection, docs and release workflows

Local verification

  • ./mvnw clean verify
  • ./mvnw clean verify -Dspark.version=3.5.2
  • ./mvnw clean verify -Dspark.version=3.5.3
  • ./mvnw clean package -DskipTests
  • scripts/inspect-jar.sh target/spark-dbf_2.13-0.1.0-assembly.jar
  • Spark 3.5.4 Scala 2.13 spark-submit PySpark smoke test

@DAChernikov
DAChernikov merged commit 64e3ce7 into master Jul 14, 2026
8 checks passed
@DAChernikov
DAChernikov deleted the feature/spark-3.5-scala-2.13 branch July 14, 2026 15:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant