Skip to content

hugeshi/spline

 
 

Repository files navigation

Spline (from Spark lineage) project helps people get insight into data processing performed by Apache Spark ™

Maven Central TeamCity build (develop) Codacy Badge Sonarcloud Status SonarCloud Maintainability SonarCloud Reliability SonarCloud Security

The project consists of three main parts:

  • Spark Agent that sits on drivers, capturing the data lineage from Spark jobs being executed by analyzing the execution plans

  • Rest Gateway, that receive the lineage data from agent and stores it in the database

  • Web UI application that visualizes the stored data lineages

Spline diagram

Spline is aimed to be used with Spark 2.3+ but also provides limited support for Spark 2.2.

For documentation and examples please visit Spline GitHub Pages.


Copyright 2019 ABSA Group Limited

you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

About

Data Lineage Tracking And Visualization Solution

Topics

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Scala 53.2%
  • TypeScript 32.8%
  • HTML 7.7%
  • CSS 4.0%
  • JavaScript 1.4%
  • Shell 0.4%
  • Other 0.5%