sportscrape

A Go package for collecting and transforming sports statistics from various sources into standardized formats.

Installation

go get github.com/lightning-dabbler/sportscrape

Quick start

Retrieve and output 2025-02-20 NBA matchups from https://foxsports.com

package main

import (
	"encoding/json"
	"fmt"
	"log"

	"github.com/lightning-dabbler/sportscrape/dataprovider/foxsports"
	"github.com/lightning-dabbler/sportscrape/runner"
)

func main() {
	date := "2025-02-20"
	// define matchup scraper
	matchupScraper := foxsports.NewMatchupScraper(
		foxsports.MatchupScraperLeague(foxsports.NBA),
		foxsports.MatchupScraperSegmenter(&foxsports.GeneralSegmenter{Date: date}),
	)
	// define matchup runner
	matchuprunner := runner.NewMatchupRunner(
		runner.MatchupRunnerScraper(matchupScraper),
	)
	// Retrieve matchups
	matchups, err := matchuprunner.Run()
	if err != nil {
		panic(err)
	}
	// Output each matchup as pretty json
	for _, matchup := range matchups {
		jsonBytes, err := json.MarshalIndent(matchup, "", "  ")
		if err != nil {
			log.Fatalf("Error marshaling to JSON: %v\n", err)
		}
		fmt.Println(string(jsonBytes))
	}
}

Usage

• basketball-reference.com NBA scrape examples
• baseball-reference.com MLB scrape examples
• foxsports.com scraping examples
• baseballsavant.mlb.com scraping examples

Data providers

Source	League	Feed	Periods Available	Data Model	Deprecated	Point-in-time
https://basketball-reference.com	NBA	Matchup	Full	model		✅
https://basketball-reference.com	NBA	Basic box score stats	H1, H2, Q1, Q2, Q3, Q4, Full	model		✅
https://basketball-reference.com	NBA	Advanced box score stats	Full	model		✅
https://baseball-reference.com	MLB	Matchup	Full	model		✅
https://baseball-reference.com	MLB	Batting box score stats	Full	model		✅
https://baseball-reference.com	MLB	Pitching box score stats	Full	model		✅
https://www.foxsports.com	NBA	Matchup	Live, Full	model		✅
https://www.foxsports.com	NBA	Box score stats	Live, Full	model		✅
https://www.foxsports.com	WNBA	Matchup	Live, Full	model		✅
https://www.foxsports.com	WNBA	Box score stats	Live, Full	model		✅
https://www.foxsports.com	MLB	Matchup	Live, Full	model		✅
https://www.foxsports.com	MLB	Batting Box score stats	Live, Full	model		✅
https://www.foxsports.com	MLB	Pitching Box score stats	Live, Full	model		✅
https://www.foxsports.com	MLB	Probable starting pitcher	Full	model		✅
https://www.foxsports.com	MLB	Betting Odds Money line	Full	model		✅
https://www.foxsports.com	MLB	Betting Odds Total	Full	model		✅
https://www.foxsports.com	NCAAB	Matchup	Live, Full	model		✅
https://www.foxsports.com	NFL	Matchup	Live, Full	model		✅
https://baseballsavant.mlb.com	MLB	Matchup	Live, Full	model		✅
https://baseballsavant.mlb.com	MLB	Batting box score stats	Live, Full	model		✅
https://baseballsavant.mlb.com	MLB	Pitching box score stats	Live, Full	model		✅
https://baseballsavant.mlb.com	MLB	Fielding box score stats	Live, Full	model		✅
https://baseballsavant.mlb.com	MLB	Play by play	Live, Full	model		✅

Supported Formats

File formats the constructed data models support on export and import.

Format	Export	Import	Go Package
Parquet	✅	✅	xitongsys/parquet-go
JSON	✅	✅	encoding/json

Development

Prerequisites

Go 1.24 or higher

Testing

This project is using mockery v3.5.0 to mock interfaces.

To run unit tests:

make unit-tests

To run unit and integration tests:

make all-tests

Tests are also being ran as CI workflows on Github Actions.

License

MIT