NSIDC-Data-Cookbook/tools-we-use.qmd at main · saberbrasher/NSIDC-Data-Cookbook · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
---
title: "Tools we use"
#number-sections: true
---

Currently, this cookbook features [Python Packages](#sec-pythonpkg) for working with data supported by NSIDC DAAC.  It also features some [applications](#sec-applications) that are accessible through a web browser or as stand alone packages that need to be installed on your local machine.

The focus on Python not only reflects the expertise of NSIDC DAAC but also reflects the popularity of Python within the Earth and atmospheric science communities.  However, we recognize that many of our users are more familiar with other programming langauges such as R and Matlab.  We hope that we will be able to include these langauges as the Cookbook develops.

Using web or locally-installed applications is a good way to start to discover and learn about data.  We often use these applications to explore datasets, find what data is available, and quickly visualize data.  However, many investigations require large numbers of files to be accessed and processed.  It is often more efficient to write scripts in Python or some other language to do this.  In the cloud, scripts are often the only way to search for and access data.  Scripts are also a way to make workflows reproducible, something that is difficult to do with a GUI application.

## Python Packages {#sec-pythonpkg}

There are many Python packages available for working with Earth science data.  The packages we use in this Cookbook are an unashamedly opinionated selection; they are the tools we like to use.  We also think that these tools are the easiest to use for the types of data mananaged by NSIDC DAAC.  Most of the tools have been developed so that researchers do not have to worry about the low-level details of accessing and working with often complicated data used in Earth science.  This reduces the amount of code you have to write and also reduces the number of mistakes you will inevitably make.

#### earthaccess

[`earthaccess`](https://earthaccess.readthedocs.io/en/latest/) — search for and access NASA Earth science data.

#### xarray

[`xarray`](https://xarray.dev/) — work with N-dimensional labeled data (e.g. `(time, x, y, z)`).

#### rioxarray

[`rioxarray`](https://corteva.github.io/rioxarray/stable/index.html) — add geospatial awareness to `xarray`.

#### rasterio

[`rasterio`](https://rasterio.readthedocs.io/en/stable/intro.html) — read, write, and process raster datasets.

#### pandas

[`pandas`](https://pandas.pydata.org/docs/index.html) — handle tabular data (spreadsheets, databases, CSV).

#### geopandas

[`geopandas`](https://geopandas.org/en/stable/) — extend `pandas` to work with geospatial vector data.

#### cartopy

[`cartopy`](https://scitools.org.uk/cartopy/docs/latest/) — map projections, transformations, and plotting for geospatial data.

#### SlideRule

[`SlideRule`](https://github.com/ICESat2-SlideRule/sliderule-python) — on-demand processing and access to ICESat-2 data.

#### icepyx

[`icepyx`](https://icepyx.readthedocs.io/en/latest/) — search, subset, and download ICESat-2 data with built-in metadata handling.

#### satpy

[`satpy`](https://satpy.readthedocs.io/en/stable/) — read, composite, and visualize meteorological satellite data.

#### dask

[`dask`](https://www.dask.org/) — scale computations from laptops to clusters with parallel, out-of-core workflows.

### Installing Python Packages

Most Python packages can be installed from the [Python Package Index (PyPI)](https://pypi.org/) using [pip](https://pip.pypa.io/en/stable/). Some scientific and geospatial packages are easier to install with [conda](https://docs.conda.io/), particularly through the [conda-forge community channel](https://conda-forge.org/).

For more details, see the [official Python documentation on installing packages](https://docs.python.org/3/installing/index.html), the [pip documentation](https://pip.pypa.io/en/stable/getting-started/), and the [conda user guide](https://docs.conda.io/projects/conda/en/latest/user-guide/index.html).

## Applications {#sec-applications}

Applications provide graphical interfaces for exploring, visualizing, and working with Earth science data. Some run directly in a web browser, while others require installation on your local machine. They are useful for quickly examining datasets, generating plots or maps, and learning about data structures before moving on to scripted workflows.

#### Earthdata Search

[`Earthdata Search`](https://search.earthdata.nasa.gov/) — NASA’s primary web application for searching, filtering, and downloading Earth science data.

#### Open Altimetry

[`Open Altimetry`](https://openaltimetry.org/) — a browser-based tool for visualizing and accessing ICESat and ICESat-2 altimetry data.

#### HDFView

[`HDFView`](https://portal.hdfgroup.org/display/support/HDFView) — a desktop application for viewing and editing HDF4 and HDF5 files.

#### Panoply

[`Panoply`](https://www.giss.nasa.gov/tools/panoply/) — a NASA tool for visualizing netCDF, HDF, and GRIB files with quick map and plot options.

#### NCView

[`NCView`](http://meteora.ucsd.edu/~pierce/ncview_home_page.html) — a lightweight utility for quickly viewing netCDF files, especially time series of 2D variables.

#### QGIS

[`QGIS`](https://qgis.org/) — an open-source geographic information system for analyzing and visualizing spatial data.

#### ArcGIS

[`ArcGIS`](https://www.esri.com/en-us/arcgis/about-arcgis/overview) — a commercial GIS platform for advanced mapping, spatial analysis, and data management.