Skip to content

v0.14.0: Spark DataFrameReader Fluent API support
Compare
Choose a tag to compare
@colindean colindean released this 15 Jun 21:47
v0.14.0
ae98cb0
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Verified
Learn about vigilant mode.

Added

  • ⭐ Spark DataFrameReader Fluent API support - you can define a "table" that will use spark.read.format(something).options(optionsMap).load(path) to load a DataFrame
  • Documentation for Config Check functionality
  • devex: You can generate test data with sbt generateTestData
  • devex: Environment Setup instructions
  • devex: Makefile with common tasks
  • devex: We're actually using scalafmt now
  • devex: We provide a Brewfile for consistent OS packages, Homebrew is the official source for OS packages for DV
  • repo: We've got templates for feature requests and bug reports now

Changed

  • Improved option parser
  • Dependencies updated: Spark 2.3.1 -> 2.3.4, Circe YAML 0.9.x -> 0.10.x
  • dev: We're now fully moved to GitHub Actions for build and release CI
    • dev: Dependabot will update our GitHub Actions
    • dev: Scala Steward will now ~automate our dependency updates
Assets 2
Loading