Skip to content

Commit

Permalink
Merge branch 'main' into chunking
Browse files Browse the repository at this point in the history
  • Loading branch information
clausmichele authored Feb 10, 2025
2 parents e30c0e4 + 042c6a2 commit 7d317f9
Show file tree
Hide file tree
Showing 33 changed files with 43,649 additions and 522 deletions.
17 changes: 16 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ This is a online course about Open Science on EO Cloud Platforms. By it's nature
## Types of contributions
We're happy to receive contributions to improve the quality of the course. Here are some potential areas where help would be greatly appreciated
### Review
- Read through the material.
- Read through the material through the [rendered web pages](http://eo-college.github.io/cubes-and-clouds). However, bear in mind that the official content of the course is available on the EO-College platform at [https://eo-college.org/courses/cubes-and-clouds/](https://eo-college.org/courses/cubes-and-clouds/) and you may find slightly differences between the two material. We strongly encourage you to check both content before suggesting changes.
- If you want to **fix something directly**: Open a pull request to fix it. Don't add too much new information though. The lessons have a certain length.
- If you have **comments**: Open an issue for your review. Name it for example "Review Section 1.3 Open Science". Add all comments you have in that issue with links to the file where they apply.

Expand All @@ -13,6 +13,21 @@ We're happy to receive contributions to improve the quality of the course. Here
- **Report Bugs:** If any of the exercises are not working. Please report a bug here by opening an issue.
- **Fix Bugs:** Make a pull request with your fix.

### Generate Rendered Web Pages Locally

We recommend you render the web pages locally to check your changes. Below is a short guide on how to generate the rendered web pages locally:

1. If conda is installed on your platform, you can skip this first step. Otherwise, we recommend you install [miniconda](https://docs.anaconda.com/miniconda/install/).
2. Create the `cubes-and-clouds` conda environment (`environment.yml` file is located in the root folder of the **cubes-and-clouds** github repository:
```
conda env create -f environment.yml
```
3. Build the rendered web pages using [Jupyter Book](https://jupyterbook.org):
```
jupyter-book build lectures
```
4. To visualize the rendered web pages, open `lectures/_build/html/index.html` in your preferred web browser from the root folder of the **cubes-and-clouds** GitHub repository.

## Acknowledgement
We want every conrtibution to be acknwoledged. That's why we have the [all-contributors](https://allcontributors.org/) bot installed. It allows you to acknowledge your own contributions and to appear in the [CONTRIBUTORS.md](CONTRIBUTORS.md). If you want your contribution to be acknowledged **Comment on your Issue or Pull Request, asking the @all-contributors bot to add a contributor (likely yourself)**:
```
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ We're happy to receive your contributions to lessons, exercises, bug reports etc
- The course is openly available on **EOCollege**, a cloud learning platform for EO content. EOCollege is your one-stop-shop for this course. You will have an integrated experience of all the components: lectures, exercises, quizzes and get a certificate upon successful completion: https://eo-college.org/courses/cubes-and-clouds
- The course material is available on **zenodo**. Each lecture can be accessed individually: [Cubes and Clouds Zenodo Community](https://zenodo.org/communities/cubes_and_clouds?q=&l=list&p=1&s=10&sort=newest)
- Alternative Rendering of the Course
- [Jupyter Book](https://eo-college.github.io/cubes-and-clouds/): web page of the course deployed using GitHub pages and Jupyter Books, updated at every merge of PR via GitHub actions.
- [LIAScript](https://liascript.github.io/): Use the link to the .md files and copy them into LIAscript. It will give you an online lecture directly.
- [Obsidian](https://obsidian.md/): Obsidian is a knowledge management system. Add obsidian file toplevel to create your void, or add the course to an existing one.
- [Bookdown](https://bookdown.org/): You can render the course using bookdown or similar to have a rendered version of the markdown files in the repo. Some work needed to do that.
Expand Down
3 changes: 3 additions & 0 deletions lectures/1.1_intro_platform/1.1_intro_platform.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,9 @@ On top of providing the infrastructure which allows users to do the computations
3. [Microsoft Planetary Computer](https://planetarycomputer.microsoft.com) is a platform that combines geospatial data and AI capabilities for Earth observation. It provides access to various global datasets, including satellite imagery, climate data, and environmental data. The platform aims to facilitate large-scale data analysis and support sustainable development and conservation efforts.
4. [Euro Data Cube](https://eurodatacube.com) is a platform on top of various cloud infrastructures to provide an interactive development environment with a standardized access to various EO data. It provides a JupyterLab environment for data exploration and analysis, as well as capabilities to run processing jobs.
5. [OpenEO Platform](https://openeo.cloud) is a platform based on OpenEO, which aims to standardize and simplify the access and processing of EO data. It provides a unified API and common data model, enabling interoperability across multiple EO data providers and processing backends. The platform allows users to run EO workflows on various cloud-based infrastructure providers.
6. [Pangeo](https://pangeo.io) is a community platform for big data geoscience built on the Pangeo ecosystem. It aims to foster collaboration among researchers, engineers, research software engineers, and innovators within the open-source scientific Python ecosystem, focusing on Ocean, Atmosphere, Land, and Climate science. There is a strong focus on portability and interoperability, enabling deployments of the Pangeo platform on various infrastructure (laptops, cloud providers, etc.) and providing APIs that allow users to prototype on their laptops and easily scale to the cloud or High-Performance Computers with minimal changes to their code. The current deployment of the Pangeo platform in Europe, [Pangeo@EOSC](https://pangeo-eosc.vm.fedcloud.eu/) is hosted on the [European Open Science Cloud](https://eosc.eu) (EOSC), providing a scalable and collaborative environment for big data analysis and research for all European researchers and their collaborators.

Different EO platform providers may offer end-users access to specific tools and packages that can be highly beneficial for particular workflows. However, users should carefully evaluate whether the convenience of these tools justifies the potential trade-off of being locked into a particular platform. In some cases, opting for platforms that prioritize portability and openness, like OpenEO or Pangeo, might be more advantageous, especially for those who value flexibility and long-term interoperability across various environments and infrastructures.

### Summary

Expand Down
4 changes: 3 additions & 1 deletion lectures/1.2_data_cube/1.2_data_cube.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,9 @@ One important feature of an EO cloud platform is to host satellite data. Usually
### Further Reading

- [David Montero Loaiza et al (2023). Data Cubes for Earth System Research: Challenges Ahead. Earth ArXiv (Preprint). https://doi.org/10.31223/X58M2V](https://doi.org/10.31223/X58M2V)
- [Montero, D., Kraemer, G., Anghelea, A., Aybar, C., Brandt, G., Camps-Valls, G., … Mahecha, M. D. (2024). Earth System Data Cubes: Avenues for advancing Earth system research. Environmental Data Science, 3, e27. doi:10.1017/eds.2024.22
](https://doi.org/10.1017/eds.2024.22)

- Popular Data Cube Management Systems:
- [Open Data Cube](https://www.opendatacube.org/)
- [Rasdaman](https://www.rasdaman.com/):
Expand Down
14 changes: 12 additions & 2 deletions lectures/2.1_data_discovery/2.1_data_discovery.md
Original file line number Diff line number Diff line change
Expand Up @@ -314,7 +314,7 @@ A STAC Catalog is an entity that logically groups other Catalogs, Collections, a

There are no restrictions on the way STAC Catalogs are organized. Therefore, the combination of STAC components within a STAC Catalog is quite variable and flexible. Many implementations use a set of 'sub-catalog(s)' that group the items in some sensible way, e.g. by years as a first level and months as a second level. It can be easily extended, for example, to include additional metadata to further describe its holdings, as the STAC Collection does.

#### STAC Colection
#### STAC Collection

A STAC Collection is similar to a STAC Catalog, but includes and partially requires additional metadata about a set of items that exist as part of the collection. It adds additional fields to enable the description of information like the spatial and temporal extent of the data, the license, keywords, providers, etc. Therefore, it can easily be extended with additional collection-level metadata that is common across all children. For example, it could summarize that all Items underneath hold data in either 10m or 30m spatial resolution.

Expand Down Expand Up @@ -523,7 +523,17 @@ Earth observation data access is not limited to a single platform or a single en
- [STAC Index](https://stacindex.org/catalogs): A list of publicly available STAC APIs and Static Catalogs.


<iframe width="1000" height="600" src="https://openeo.cloud/data-collections/" title="openEO platform collections" frameborder="0" allowfullscreen></iframe>
<iframe
width="1000"
height="600"
src="https://documentation.dataspace.copernicus.eu/APIs/openEO/Collections.html"
title="openEO platform collections"
frameborder="0"
allowfullscreen
style="border: 5px solid #3A97C3; border-radius: 20px;">
</iframe>

_The content above shows the data collections available with the openEO API at the Copernicus Data Space Ecosystem (CDSE). You can scroll and click on the names to discover more details._ [Open content in a new tab!](https://documentation.dataspace.copernicus.eu/APIs/openEO/Collections.html)

## How to search for data

Expand Down
Loading

0 comments on commit 7d317f9

Please sign in to comment.