diff --git a/.dockerignore b/.dockerignore index 8541c6f6b..9fe7f9c12 100644 --- a/.dockerignore +++ b/.dockerignore @@ -1,9 +1,10 @@ datacube_wms/wms_cfg_local.py -.pytest_cache -*/__pycache__ +**/.pytest_cache +**/__pycache__ +.hypothesis venv .venv -.pixi +**/.pixi .git diff --git a/Dockerfile.micromamba b/Dockerfile.micromamba index 1caa1e853..67960feec 100644 --- a/Dockerfile.micromamba +++ b/Dockerfile.micromamba @@ -1,7 +1,11 @@ FROM mambaorg/micromamba:1.5.8 COPY --chown=$MAMBA_USER:$MAMBA_USER env.micromamba.yaml /tmp/env.yaml RUN --mount=type=cache,target=/opt/conda/pkgs micromamba install -y -n base -f /tmp/env.yaml && \ - micromamba clean --all --yes + micromamba clean --all --yes --force-pkgs-dirs && \ +# find /home/mambauser/.mamba/pkgs -type d \( -name test -o -name tests \) -print0 | xargs -0 rm -rf && \ + find /opt/conda/lib -type d \( -name test -o -name tests \) -print0 | xargs -0 rm -rf && \ + rm -rf /opt/conda/lib/libpython3* /opt/conda/include /opt/conda/share/{gir-1.0,poppler,man} +# TODO: pieces of botocore (98Mb) and scipy (72Mb) can likely be removed ARG MAMBA_DOCKERFILE_ACTIVATE=1 # (otherwise python will not be found) diff --git a/README.rst b/README.rst index 3f63f19aa..d2fcbc8fc 100644 --- a/README.rst +++ b/README.rst @@ -1,6 +1,6 @@ -============ +========================== Datacube Open Web Services -============ +========================== .. image:: https://github.com/opendatacube/datacube-ows/actions/workflows/lint.yml/badge.svg :target: https://github.com/opendatacube/datacube-ows/actions/workflows/lint.yml @@ -59,7 +59,7 @@ Community This project welcomes community participation. -`Join the ODC Discord `__ if you need help +`Join the ODC Discord `__ if you need help setting up or using this project, or the Open Data Cube more generally. Please help us to keep the Open Data Cube community open and inclusive by @@ -204,11 +204,11 @@ The following instructions are for installing on a clean Linux system. * Create a configuration file for your service, and all data products you wish to publish in it. - `Detailed documentation of the configuration format can be found here.`_ + `Detailed documentation of the configuration format can be found here. `_ * Set environment variables as required. Environment variables that directly or indirectly affect the running of OWS - are `documented here`_. + are `documented here `_. * Run ``datacube-ows-update`` (in the Datacube virtual environment). diff --git a/docs/cfg_colourmap_styles.rst b/docs/cfg_colourmap_styles.rst index 9248b314d..aa9c10c0b 100644 --- a/docs/cfg_colourmap_styles.rst +++ b/docs/cfg_colourmap_styles.rst @@ -7,7 +7,7 @@ OWS Configuration - Colourmap Styles Colour-Map Styles ----------------- -Colour-map Styles are `styles `_ where +:doc:`Colour-map Styles ` are where each pixel is mapped to one particular colour from a fixed pallet by applying a logical decision tree to the flag data for that pixel. @@ -118,7 +118,7 @@ Rules Each Value Rule must also specify the rule to evaluate when it applies. For details, refer to the -`OWS Masking Syntax `_. +:doc:`OWS Masking Syntax `. ------ Legend @@ -339,7 +339,7 @@ Aggregator function =================== Alternately, you can define an aggregator function using OWS's -`function configuration format `_. +:doc:`function configuration format `. The function is passed a multi-date Xarray Dataset and is expected to return a timeless Dataset, which can then be rendered using either the single-date value-map, or a separate single-date value-map diff --git a/docs/cfg_colourramp_styles.rst b/docs/cfg_colourramp_styles.rst index 1a044c2c8..779bb066b 100644 --- a/docs/cfg_colourramp_styles.rst +++ b/docs/cfg_colourramp_styles.rst @@ -8,7 +8,7 @@ OWS Configuration - Colour-Ramp Styles Colour-Ramp Styles ------------------ -Colour-ramp Styles are `styles `_ where +Colour-ramp Styles are :doc:`styles ` where a single continuous index value is calculated from the raw data for each pixel, and that index value is mapped to a graduated colour ramp for display. @@ -19,10 +19,12 @@ Colour-ramp styles support the Colour-ramp styles support automatic legend generation. Specialised legend configuration is described `below <#legend-configuration>`__. -Note on spelling: I am Australian and spell it "colour". Most software -packages use the US spelling "color". Within the configuration file -format we use the software conventional spelling, but within the text -of this documentation, I use the UK/Australian spelling. +.. note:: On spelling + + I am Australian and spell it "colour". Most software + packages use the US spelling "color". Within the configuration file + format we use the software conventional spelling, but within the text + of this documentation, I use the UK/Australian spelling. --------------------------- @@ -67,11 +69,11 @@ index_function The ``index_function`` allows the user to declare a callback function to calculate the index value using OWS's -`function configuration format `_. +:doc:`function configuration format `. The function is expected to take an xarray Dataset containing all the bands in the `needed_bands list <#needed-bands-list>`__ (plus any additional arguments handled by the -`function configuration format `_); and returns +:doc:`function configuration format `); and returns an xarray Dataset containing the index value. A `small library `_ @@ -555,7 +557,7 @@ aggregator_function The ``aggregator_function`` entry is required for colour ramp style multi-date handlers. It is a function defined using OWS's -`function configuration format `_. +:doc:`function configuration format `. The first argument passed to the function depends on the value of the ``pass_raw_data`` element, as described above. The function should return an xarray DataArray with no time diff --git a/docs/cfg_component_styles.rst b/docs/cfg_component_styles.rst index 2c944a2c2..2ba275187 100644 --- a/docs/cfg_component_styles.rst +++ b/docs/cfg_component_styles.rst @@ -7,7 +7,7 @@ OWS Configuration - Components Component Styles ---------------- -Component Styles are `styles `_ where +Component Styles are :doc:`styles ` where each component channel of the image (red, green, blue and optionally alpha) is calculated independently from the data for that pixel. @@ -228,7 +228,7 @@ Callback Function Components +++++++++++++++++++++++++++++ In a callback function component, the user declares a callback function -using OWS's `function configuration format `_. +using OWS's :doc:`function configuration format `. The function must take an xarray Dataset containing the raw band data and return a xarray DataArray containing the channel data. It is diff --git a/docs/cfg_functions.rst b/docs/cfg_functions.rst index 002756624..47fd17139 100644 --- a/docs/cfg_functions.rst +++ b/docs/cfg_functions.rst @@ -283,6 +283,7 @@ E.g. Using arcsec of the sdev band as a local brightness control for an rgb imag The raw red,green,blue bands go to 3000. :: + "components": { "red": { "function": "datacube_ows.band_utils.single_band_arcsec", diff --git a/docs/cfg_global.rst b/docs/cfg_global.rst index e895d128c..057cd8ead 100644 --- a/docs/cfg_global.rst +++ b/docs/cfg_global.rst @@ -7,8 +7,8 @@ OWS Configuration - Global Section Global Section -------------- -The "global" section of the `root configuration object -`_ +The "global" section of the :doc:`root configuration object +` contains config entries that apply to all services and all layers/coverages. @@ -34,7 +34,7 @@ The following global configuration items are relevant to `_. Message File (message_file) -+++++++++++++++++++++++++++ +--------------------------- The "message_file" entry gives the path to the message file. @@ -50,20 +50,20 @@ Note: the `message_file` configuration entry is not required (except as an inter if `translations_directory` is set. The order of metadata resolution is: 1) If Internationalisation is active via the `translations_directory` config entry, take the - value from the *.mo file corresponding to the best available language match for the request + value from the \*.mo file corresponding to the best available language match for the request headers. 2) If a translation for the metadata entry is not available, or if internationalisation is not active, get the value from the `message_file` if it is configured. 3) Otherwise, fallback to the value in the main body of the config. Message Domain (message_domain) -+++++++++++++++++++++++++++++++ +------------------------------- The message domain used by internationalisation. Defaults to "ows_cfg". Translations Directory (translations_directory) -+++++++++++++++++++++++++++++++++++++++++++++++ +----------------------------------------------- The path to the directory containing translation subdirectories. Traditionally known as "locales". Defaults to None, meaning no translation support. @@ -73,7 +73,7 @@ Required for internationalisation. Supported Languages (supported_languages) -+++++++++++++++++++++++++++++++++++++++++ +----------------------------------------- A list of supported languages for internationalisation. Defaults to English only (``['en']``). The first language listed is treated as the default language, and it is assumed that all metadata @@ -276,7 +276,7 @@ logo All of the above elements are optional, but at least one must be provided if the attribution section exists. ----------------- + Attribution Logo ---------------- diff --git a/docs/cfg_hybrid_styles.rst b/docs/cfg_hybrid_styles.rst index a3b97ef57..f95bd4e7b 100644 --- a/docs/cfg_hybrid_styles.rst +++ b/docs/cfg_hybrid_styles.rst @@ -7,7 +7,7 @@ OWS Configuration - Hybrid Styles Hybrid Styles ------------- -Hybrid styles are an experimental type of `style `_ that +Hybrid styles are an experimental type of :doc:`style ` that return a linear combination of a component style and a colour ramp style. This can allow for a more easily visually interpreted image, but @@ -15,8 +15,8 @@ there are usually better ways to achieve the same effect on the client side. Hybrid styles support most [*]_ elements supported by either -`component styles `_ or -`colour ramp styles `_ and +:doc:`component styles ` or +:doc:`colour ramp styles ` and define two independent styles (one of each type) that are then blended according to the required `component_ratio` entry. diff --git a/docs/cfg_layers.rst b/docs/cfg_layers.rst index dab144425..710734865 100644 --- a/docs/cfg_layers.rst +++ b/docs/cfg_layers.rst @@ -7,8 +7,8 @@ OWS Configuration - Layers Layers Section ============== -The "layers" section of the `root configuration object -`_ +The "layers" section of the :doc:`root configuration object +` contains definitions of the various layers (WMS/WMTS) and coverages (WCS) that installation serves. @@ -245,8 +245,8 @@ E.g. } If the -`manual merge option in the -image_processing section discussed below<#manual-merge-manual-merge>`_ +:ref:`manual merge option in the +image_processing section discussed below ` is set to ``True``, then overlapping products are layered according to the priority order specified in ``product_names`` @@ -313,8 +313,8 @@ A date-aware product can be presented as a single-date mosaic layer with no publ dimension with the optional `mosaic_date_func` element. If supplied, the ``mosaic_date_func`` must be a function, declared -using OWS's `function configuration format -`_. +using OWS's :doc:`function configuration format +`. The Mosaic Date Function should take a list of available dates and return a tuple of two datetimes to be used in the dataset search operation. @@ -361,7 +361,7 @@ are mapped to user-accessible dates. The acceptable values are: * "summary" Data has time dimension based on the start date of start datetime of datasets, - which are expected to have a `00:00:00.0000+00" time portion. + which are expected to have a `00:00:00.0000+00` time portion. Used for daily, weekly, monthly, quarterly or annual summary data. @@ -417,7 +417,7 @@ Regular Time Axis (time_axis) The time axis is how OWS publishes the dates for which data is available. The default behaviour (``time_axis`` not specified or ``None``) is to use an irregular time axis, where the available dates -(as cached in `the OWS range tables `_) +(as cached in :doc:`the OWS range tables `) are listed individually. These long lists of dates lead to unnecessarily large capabilities documents for all supported protocols. @@ -525,8 +525,8 @@ of data urls for authentication (e.g. Microsoft Planetary Computer). They may a to handle various network proxying or data-migration scenarios where the URLs visible to the OWS server are different to the URLs at indexing time. -The "patch_url_function" config element is set using OWS's `function configuration format -`_. +The "patch_url_function" config element is set using OWS's :doc:`function configuration format +`. The function is expected to take a string containing an unpatched url and return a string containing the patched url. @@ -866,7 +866,7 @@ An additional element, ``describe_cache_maxage`` is also provided, which controls the cache control headers for WCS DescribeCoverage requests for the coverage/layer. This element is optional, and defaults to the value set in the -`top-level WCS section`_ +`top-level WCS section `_ ++++++++++++ max_datasets @@ -933,8 +933,8 @@ use: "extent_mask_func": "datacube_ows.ogc_utils.mask_by_val", If this is not appropriate or possible for your data, you can -set an alternative function using OWS's `function configuration format -`_. Some sample functions are included in ``datacube_ows.ogc_utils``. +set an alternative function using OWS's :doc:`function configuration format +`. Some sample functions are included in ``datacube_ows.ogc_utils``. The function is assumed to take two arguments, data (an xarray Dataset) and band (a band name). (Plus any additional arguments you may be passing in @@ -979,8 +979,8 @@ Fuse Function (fuse_func) +++++++++++++++++++++++++ Determines how multiple dataset arrays are compressed into a -single time array. Specified using OWS's `function configuration -format `_. +single time array. Specified using OWS's :doc:`function configuration +format `. The fuse function is passed through to directly to the datacube load_data() function - refer to the Open Data Cube documentation @@ -1241,7 +1241,7 @@ Custom Layer Includes (custom_includes) is optional and defaults to an empty dictionary (i.e. no custom data.) ``custom_includes`` is dictionary mapping keys to functions specified with -OWS's `function configuration format `_. +OWS's :doc:`function configuration format `. The keys of the "custom_includes" dictionary are the keys that will be included in the GetFeatureInfo responses. They should therefore be keys that are not included by @@ -1291,11 +1291,11 @@ is optional and defaults to an empty dictionary (i.e. no custom data.) Legacy ``include_custom`` entries can be over-ridden by new-style ``custom-includes``. ------------------------------------ +------------------------- Styling Section (styling) ------------------------------------ +------------------------- -The `"styling" section `_ describes the WMS and WMTS styles for +The :doc:`"styling" section ` describes the WMS and WMTS styles for the layer. diff --git a/docs/cfg_style_api.rst b/docs/cfg_style_api.rst index 71255a0b4..cec9fdd50 100644 --- a/docs/cfg_style_api.rst +++ b/docs/cfg_style_api.rst @@ -125,7 +125,7 @@ There are two API functions that provide this functionality: ``apply_ows_style`` For more detailed examples, refer to the -`styling how-to guide `_. +:doc:`styling how-to guide `. Saving or Displaying Images --------------------------- diff --git a/docs/cfg_styling.rst b/docs/cfg_styling.rst index 0ba6f622b..9e5867833 100644 --- a/docs/cfg_styling.rst +++ b/docs/cfg_styling.rst @@ -7,8 +7,8 @@ OWS Configuration - Layer Styles Styling Section --------------- -The "styling" sub-section of a `layer configuration section -`_ +The "styling" sub-section of a :doc:`layer configuration section +` contains definitions of the various styles that layer supports. @@ -55,23 +55,23 @@ Style Types There are four distinct possible types of style. -1. `Component Styles `_ +1. :doc:`Component Styles ` Each component channel of the image (red, green, blue and optionally alpha) is calculated independently from the data for that pixel. -2. `Colour Map Styles `_ +2. :doc:`Colour Map Styles ` Each pixel is mapped to one particular colour from a fixed pallet by applying a logical decision tree to the date for that pixel. -3. `Colour Ramp Styles `_ +3. :doc:`Colour Ramp Styles ` A single continuous index value is calculated from the data for each pixel, and that index value mapped to a graduated colour ramp for display. -4. `Hybrid Styles `_ +4. :doc:`Hybrid Styles ` A linear combination of a component style and a colour ramp style. @@ -175,7 +175,7 @@ The pq_masks entry is a list of mask definitions. Each mask definition contains `Flag Processing Section `_ for the layer. 2. A mask rule, using the - `OWS Masking Syntax `_ + :doc:`OWS Masking Syntax ` The mask rules in the pq_masks list are AND'd together. i.e. A pixel must match all of the mask rules in the list to remain visible. diff --git a/docs/cfg_wcs.rst b/docs/cfg_wcs.rst index 5db951ae1..dcb9556f3 100644 --- a/docs/cfg_wcs.rst +++ b/docs/cfg_wcs.rst @@ -7,8 +7,8 @@ OWS Configuration - WCS Section WCS Section -------------- -The ``wcs`` section of the `root configuration object -`_ +The ``wcs`` section of the :doc:`root configuration object +` contains config entries that apply to the WCS services for all coverages. @@ -59,7 +59,7 @@ NetCDF formats and look something like this: } }, -Renderer is set using OWS's `function configuration format `_. +Renderer is set using OWS's :doc:`function configuration format `. For WCS1, The function is expected to take the following arguments: * A WCSRequest object diff --git a/docs/cfg_wms.rst b/docs/cfg_wms.rst index 24f273bad..63cfb2147 100644 --- a/docs/cfg_wms.rst +++ b/docs/cfg_wms.rst @@ -7,8 +7,8 @@ OWS Configuration - WMS Section WMS Section -------------- -The "wms" section of the `root configuration object -`_ +The "wms" section of the :doc:`root configuration object +` contains config entries that apply to the WMS/WMTS services for all layers. diff --git a/docs/cfg_wmts.rst b/docs/cfg_wmts.rst index 798047ac8..3b217c747 100644 --- a/docs/cfg_wmts.rst +++ b/docs/cfg_wmts.rst @@ -7,11 +7,11 @@ OWS Configuration - WMTS Section WMTS Section ------------ -The "wmts" section of the `root configuration object -`_ +The "wmts" section of the :doc:`root configuration object +` contains config entries that apply only to WMTS service for all layers. -Note that the `wms `_ section +Note that the :doc:`wms ` section also contains configuration that applies to the WMTS service. All entries in the WMTS section are optional and the entire section can therefore be omitted. diff --git a/docs/conf.py b/docs/conf.py index 568f55992..5d90c1a3b 100755 --- a/docs/conf.py +++ b/docs/conf.py @@ -10,12 +10,6 @@ import os import sys -# If extensions (or modules to document with autodoc) are in another -# directory, add these directories to sys.path here. If the directory is -# relative to the documentation root, use os.path.abspath to make it -# absolute, like shown here. -# sys.path.insert(0, os.path.abspath('.')) - # Get the project root dir, which is the parent dir of this cwd = os.getcwd() project_root = os.path.dirname(cwd) @@ -34,16 +28,19 @@ # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom ones. -extensions = ['sphinx.ext.autodoc', 'sphinx.ext.viewcode', 'sphinx_click'] +extensions = [ + 'sphinx.ext.autodoc', + 'sphinx.ext.viewcode', + 'sphinx_click' +] # Add any paths that contain templates here, relative to this directory. templates_path = ['_templates'] # The suffix of source filenames. -source_suffix = '.rst' - -# The encoding of source files. -#source_encoding = 'utf-8-sig' +source_suffix = { + '.rst': 'restructuredtext' +} # The master toctree document. master_doc = 'index' @@ -61,173 +58,20 @@ # The full version, including alpha/beta/rc tags. release = datacube_ows.__version__ -# The language for content autogenerated by Sphinx. Refer to documentation -# for a list of supported languages. -# language = None - -# There are two options for replacing |today|: either, you set today to -# some non-false value, then it is used: -# today = '' -# Else, today_fmt is used as the format for a strftime call. -#today_fmt = '%B %d, %Y' - # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. -exclude_patterns = ['_build'] - -# The reST default role (used for this markup: `text`) to use for all -# documents. -#default_role = None - -# If true, '()' will be appended to :func: etc. cross-reference text. -#add_function_parentheses = True - -# If true, the current module name will be prepended to all description -# unit titles (such as .. function::). -#add_module_names = True - -# If true, sectionauthor and moduleauthor directives will be shown in the -# output. They are ignored by default. -#show_authors = False +exclude_patterns = ['_build', 'venv'] # The name of the Pygments (syntax highlighting) style to use. pygments_style = 'sphinx' -# A list of ignored prefixes for module index sorting. -#modindex_common_prefix = [] - -# If true, keep warnings as "system message" paragraphs in the built -# documents. -#keep_warnings = False - - # -- Options for HTML output ------------------------------------------- -# The theme to use for HTML and HTML Help pages. See the documentation for -# a list of builtin themes. html_theme = 'default' -# Theme options are theme-specific and customize the look and feel of a -# theme further. For a list of options available for each theme, see the -# documentation. -#html_theme_options = {} - -# Add any paths that contain custom themes here, relative to this directory. -#html_theme_path = [] - -# The name for this set of Sphinx documents. If None, it defaults to -# " v documentation". -#html_title = None - -# A shorter title for the navigation bar. Default is the same as -# html_title. -#html_short_title = None - -# The name of an image file (relative to this directory) to place at the -# top of the sidebar. -#html_logo = None - -# The name of an image file (within the static path) to use as favicon -# of the docs. This file should be a Windows icon file (.ico) being -# 16x16 or 32x32 pixels large. -#html_favicon = None - -# Add any paths that contain custom static files (such as style sheets) -# here, relative to this directory. They are copied after the builtin -# static files, so a file named "default.css" will overwrite the builtin -# "default.css". -html_static_path = ['_static'] - -# If not '', a 'Last updated on:' timestamp is inserted at every page -# bottom, using the given strftime format. -#html_last_updated_fmt = '%b %d, %Y' - -# If true, SmartyPants will be used to convert quotes and dashes to -# typographically correct entities. -#html_use_smartypants = True - -# Custom sidebar templates, maps document names to template names. -#html_sidebars = {} - -# Additional templates that should be rendered to pages, maps page names -# to template names. -#html_additional_pages = {} - -# If false, no module index is generated. -#html_domain_indices = True - -# If false, no index is generated. -#html_use_index = True - -# If true, the index is split into individual pages for each letter. -#html_split_index = False - -# If true, links to the reST sources are added to the pages. -#html_show_sourcelink = True - -# If true, "Created using Sphinx" is shown in the HTML footer. -# Default is True. -#html_show_sphinx = True - -# If true, "(C) Copyright ..." is shown in the HTML footer. -# Default is True. -#html_show_copyright = True - -# If true, an OpenSearch description file will be output, and all pages -# will contain a tag referring to it. The value of this option -# must be the base URL from which the finished HTML is served. -#html_use_opensearch = '' - -# This is the file name suffix for HTML files (e.g. ".xhtml"). -#html_file_suffix = None - # Output file base name for HTML help builder. htmlhelp_basename = 'datacube_owsdoc' - -# -- Options for LaTeX output ------------------------------------------ - -latex_elements = { - # The paper size ('letterpaper' or 'a4paper'). - # 'papersize': 'letterpaper', - - # The font size ('10pt', '11pt' or '12pt'). - # 'pointsize': '10pt', - - # Additional stuff for the LaTeX preamble. - # 'preamble': '', -} - -# Grouping the document tree into LaTeX files. List of tuples -# (source start file, target name, title, author, documentclass -# [howto/manual]). -latex_documents = [ - ('index', 'datacube_ows.tex', - u'datacube-ows Documentation', - u'Datacube OWS Team', 'manual'), -] - -# The name of an image file (relative to this directory) to place at -# the top of the title page. -#latex_logo = None - -# For "manual" documents, if this is true, then toplevel headings -# are parts, not chapters. -#latex_use_parts = False - -# If true, show page references after internal links. -#latex_show_pagerefs = False - -# If true, show URL addresses after external links. -#latex_show_urls = False - -# Documents to append as an appendix to all manuals. -#latex_appendices = [] - -# If false, no module index is generated. -#latex_domain_indices = True - - # -- Options for manual page output ------------------------------------ # One entry per manual page. List of tuples @@ -238,32 +82,3 @@ [u'Datacube OWS Team'], 1) ] -# If true, show URL addresses after external links. -#man_show_urls = False - - -# -- Options for Texinfo output ---------------------------------------- - -# Grouping the document tree into Texinfo files. List of tuples -# (source start file, target name, title, author, -# dir menu entry, description, category) -texinfo_documents = [ - ('index', 'datacube_ows', - u'datacube-ows Documentation', - u'Datacube OWS Team', - 'datacube_ows', - 'Web Services for the OpenDataCube.', - 'Miscellaneous'), -] - -# Documents to append as an appendix to all manuals. -#texinfo_appendices = [] - -# If false, no module index is generated. -#texinfo_domain_indices = True - -# How to display URL addresses: 'footnote', 'no', or 'inline'. -#texinfo_show_urls = 'footnote' - -# If true, do not generate a @detailmenu in the "Top" node's menu. -#texinfo_no_detailmenu = False diff --git a/docs/configuration.rst b/docs/configuration.rst index 94e685852..3a18cff97 100644 --- a/docs/configuration.rst +++ b/docs/configuration.rst @@ -7,7 +7,6 @@ OWS Configuration :hidden: :glob: - configuration cfg_global cfg_wms cfg_wmts @@ -21,7 +20,6 @@ OWS Configuration cfg_colourmap_styles cfg_hybrid_styles cfg_style_api - cfg_style_api_examples .. contents:: Table of Contents @@ -32,8 +30,8 @@ Introduction The behaviour of a datacube_ows server instance can be controlled, customised and extended by configuration files. -Datacube OWS configuration files can be written in Python (as a static serialisable python dictionary) or as JSON. -All examples in this documentation will always be presented in Python syntax. JSON syntax is almost identical, but +Datacube OWS configuration files can be written in Python (as a static serialisable Python dictionary) or as JSON. +All examples in this documentation will always be presented in Python syntax. JSON syntax is almost identical, but note that there are some important differences: 1. JSON string literals must always be delimited with double quotes - so no Python-style single quote strings! @@ -53,10 +51,10 @@ Where is Configuration Read From? Configuration is read by default from the ``ows_cfg object`` in ``datacube_ows/ows_cfg.py`` but this can be overridden by setting the ``$DATACUBE_OWS_CFG`` environment variable. -Configuration can be read from a python file, a json file, or a collection of python -and/or json files. +Configuration can be read from a Python file, a JSON file, or a collection of python +and/or JSON files. -A json file can optionally also be fetched from AWS S3 if access to S3 is configured correctly for datacube_ows +A JSON file can optionally also be fetched from AWS S3 if access to S3 is configured correctly for datacube_ows and the environment variable ``$DATACUBE_OWS_CFG_ALLOW_S3`` is set to ``YES``. Please note that this is not possible for configurations as python file. @@ -116,12 +114,10 @@ allowing cleaner organisation of configuration and facilitating reuse of common elements across different layers within the one configuration and between different configurations or deployment environments. -N.B. The examples here illustrate the inclusion directives only, and are not valid Datacube OWS configuration! +.. note:: The examples here illustrate the inclusion directives only, and are not valid Datacube OWS configuration! If you are simply loading config as a Python object, this can be directly achieved by normal programmatic techniques, -e.g.: - -:: +e.g.:: handy_config = { "desc": "This is a piece of config that might want to use in multiple places", @@ -142,7 +138,7 @@ e.g.: ] -If you want to reuse chunks of config in json, or wish to combine json with and python in your configuration, +If you want to reuse chunks of config in JSON, or wish to combine JSON with and Python in your configuration, the following convention applies in both Python and JSON: Any JSON or Python element that forms the full configuration tree or a subset of it, @@ -159,7 +155,7 @@ can be supplied in any of the following ways: "another_entry": "llama" } -2. Include a python object (by FQN - fully qualified name): +2. Include a Python object (by FQN - fully qualified name): :: @@ -172,7 +168,7 @@ can be supplied in any of the following ways: The path must be fully qualified. Relative Python imports are not supported. - N.B. It is up to you to ensure that the Python file in question is in your Python path and + .. note:: It is up to you to ensure that the Python file in question is in your Python path and that all package directories have a ``__init__.py`` file, etc. @@ -185,12 +181,12 @@ can be supplied in any of the following ways: "type": "json" } - N.B. Resolution of relative file paths is done in the following order: + .. note:: Resolution of relative file paths is done in the following order: - a) Relative to the working directory of the web app. - - b) If a JSON file is being included from another JSON file, relative to - directory in which the including file resides. + a) Relative to the working directory of the web app. + + b) If a JSON file is being included from another JSON file, relative to + directory in which the including file resides. Note that this does not just apply when the included python or json entity is a dictionary/object. Any of the above include directives could expand to an array, or even to single integer or string. @@ -259,8 +255,8 @@ The child objects can also be used in turn as the parents of subsequent layers, as long as cyclic dependencies are avoided. There are two types of named configuration object that support inheritance: -named `Layers `_ and -`styles `_. +named :doc:`Layers ` and +:doc:`Layer Styles `. The exact way to inherit by name differs depending on the object type so `see `_ the @@ -332,9 +328,7 @@ a common abstract base style that has an empty list for the color ramp. Eg. "color_ramp": [ ... ], # All New Color Ramp } -Or: - -:: +Or:: abstract_base_style = { "abstract": """ @@ -498,28 +492,28 @@ At the top level, the Datacube OWS configuration is a single dictionary with the ] } -The `global `_ section contains configuration that +The :doc:`global ` section contains configuration that applies to the whole server across all services and layers. -The `global `_ section is always required. +The :doc:`global ` section is always required. -The `wms `_ section contains configuration that applies to the WMS/WMTS +The :doc:`wms ` section contains configuration that applies to the WMS/WMTS services aross all layers. -The `wmts `_ section contains configuration that applies to the WMTS +The :doc:`wmts ` section contains configuration that applies to the WMTS services aross all layers. -The `wms `_ section can be omitted if only the WCS service is +The :doc:`wms ` section can be omitted if only the WCS service is activated (specified in the `global services `_ section), or if the default values for all entries are acceptable. -The `wmts `_ section is optional. +The :doc:`wmts ` section is optional. -The `wcs `_ section must be supplied if the WCS service is +The :doc:`wcs ` section must be supplied if the WCS service is activated (specified in the `global services `_ section). WMTS is implemented as a thin wrapper around the WMS implementation. Therefore configuration in the WMS section generally applies equally to WMTS. -The `layers `_ section +The :doc:`layers ` section contains a list of layer configurations. The configured layers define the layers (in WMS and WMTS) and coverages (in WCS) that the instance serves, and their behaviour. The layers section is always required. diff --git a/docs/environment_variables.rst b/docs/environment_variables.rst index c5b663dd6..f4b84ac88 100644 --- a/docs/environment_variables.rst +++ b/docs/environment_variables.rst @@ -132,8 +132,7 @@ Docker and Docker-compose ------------------------- The provided ``Dockerfile`` and ``docker-compose.yaml`` read additional -environment variables at build time. -Please refer to the `README `_ +environment variables at build time. Please refer to the :doc:`README ` for further details. Environment variables exclusive for docker-compose diff --git a/docs/index.rst b/docs/index.rst index ed5490437..45ac105db 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -14,7 +14,7 @@ Contents: styling_howto usage environment_variables - pyspy + performance contributing history diff --git a/docs/installation.rst b/docs/installation.rst index c5fb9b3af..a128ef7d3 100644 --- a/docs/installation.rst +++ b/docs/installation.rst @@ -16,16 +16,16 @@ Stable release datacube-ows is released to PyPI and the latest release can be found on https://pypi.org/project/datacube-ows/. -Install Postgis +Install PostGIS ---------------- -In addition to the database installed for `Datacube Core`, `datacube-ows` also require `postgis` installed. +In addition to the database installed for `Datacube Core`, `datacube-ows` also requires PostGIS_ installed. .. code-block:: console $ sudo apt-get install postgis Download datacube-ows source --------------------------- +---------------------------- The sources for datacube-ows can be downloaded from the `Github repo`_. You can either clone the public repository: @@ -66,11 +66,12 @@ To install datacube-ows, run: .. _datacube-core: https://datacube-core.readthedocs.io/en/latest/ .. _Github repo: https://github.com/opendatacube/datacube-ows .. _zip: https://github.com/opendatacube/datacube-ows/archive/master.zip +.. _PostGIS: https://postgis.net/ Update_range natively --------------------- -Refer to the `database documentation `_ documentation +Refer to the :doc:`database documentation ` documentation for information on how to setup and maintain a Datacube OWS database. From sources ( within Docker ) @@ -102,7 +103,7 @@ of the config file, and edit it to reflect your requirements. $ vi .env Create an external PostgreSQL Database for OWS use. (See -the `database documentation `__ for +the :doc:`database documentation ` for more information.) jUse this as a sidecar docker or natively on the host system. The following steps assume the database is on the host system for networking @@ -120,7 +121,7 @@ Update_range via docker ----------------------- Connect to the running docker to run datacube-ows-update/update_range.py -commands (see the `database documentation `__ for more +commands (see the :doc:`database documentation ` for more information). E.g. to set up a new database: diff --git a/docs/performance.rst b/docs/performance.rst index 3f1d05809..06f9d37c3 100644 --- a/docs/performance.rst +++ b/docs/performance.rst @@ -1,6 +1,6 @@ -===== +===================== Performance deep dive -===== +===================== ows_stats ========= diff --git a/docs/style_howto_color_map.rst b/docs/style_howto_color_map.rst index 5f2891d06..4b25eb429 100644 --- a/docs/style_howto_color_map.rst +++ b/docs/style_howto_color_map.rst @@ -370,6 +370,6 @@ equivalent: }, We've seen how to use transparency in colour-ramp styles in the last chapter, and in colour-map styles in this -one. `In the next chapter -`_ we explore other ways +one. :doc:`In the next chapter +` we explore other ways of achieving transparency in datacube-ows. diff --git a/docs/style_howto_color_ramp.rst b/docs/style_howto_color_ramp.rst index f9d7b6357..7c17bd243 100644 --- a/docs/style_howto_color_ramp.rst +++ b/docs/style_howto_color_ramp.rst @@ -243,6 +243,6 @@ where 0.0 is totally transparent and 1.0 (the default) is opaque. Note that you show the next layer down on the map. The full size view shows the image against a grey background on most browsers, which may help to convey the sense of transparency.) -`Next up -`_ -we will look at colour-map styles, which are useful for visualising discrete measurement bands. +:doc:`Next up, colour-map styles +`_ +only get you so far, so :doc:`next ` we start to look at how to apply more powerful maths to calculate components. diff --git a/docs/style_howto_components_nonlinear.rst b/docs/style_howto_components_nonlinear.rst index 903f215bb..212497a0f 100644 --- a/docs/style_howto_components_nonlinear.rst +++ b/docs/style_howto_components_nonlinear.rst @@ -9,7 +9,7 @@ Non-Linear Components: Functions -------------------------------- All the examples in the -`previous section `_ +:doc:`previous section ` involved using linearly scaled, linear combinations of bands to calculate channel values. If we want to evaluate components using more sophisticated calculations, we need to use python functions. @@ -97,7 +97,7 @@ Non-Linear Components: OWS Function Syntax and Scalable ------------------------------------------------------- You can use the ``@scalable`` decorator provided by the API, and OWS's -`extended function syntax `_ +:doc:`extended function syntax ` for a more streamlined solution to scaling: Example: red-ndvi-blue (half scale) @@ -191,6 +191,5 @@ Note that utility functions are referenced by name, rather than importing the na `View full size `_ -`Next up -`_ +:doc:`Next up ` we will look at colour ramp styles. diff --git a/docs/style_howto_legends.rst b/docs/style_howto_legends.rst index 35466ca51..220b5aa8f 100644 --- a/docs/style_howto_legends.rst +++ b/docs/style_howto_legends.rst @@ -16,7 +16,7 @@ Auto-legend-generation for Colour Ramp Styles +++++++++++++++++++++++++++++++++++++++++++++ The first type of style that supports automatic legend-generation is -`colour ramp styles `_. +:doc:`colour ramp styles `. For these examples, we will use a simple colour ramp style using a Matplotlib named colour ramp: @@ -478,7 +478,7 @@ Auto-legend-generation for Colour Map Styles ++++++++++++++++++++++++++++++++++++++++++++ Auto-generated legends are also available for -`colour map styles`_. +:doc:`colour map styles `. For these examples, we will be extending the following style example: diff --git a/docs/style_howto_transparency.rst b/docs/style_howto_transparency.rst index ec999da4d..df4db143e 100644 --- a/docs/style_howto_transparency.rst +++ b/docs/style_howto_transparency.rst @@ -310,6 +310,6 @@ transparent with values between 0 and 0.5 shown partially transparent: `View full size `_ -`In the next chapter -`_ we look at how to generate +:doc:`In the next chapter +` we look at how to generate legends for datacube-ows styles. diff --git a/docs/styling_howto.rst b/docs/styling_howto.rst index 3a949b0ec..dd89c677d 100644 --- a/docs/styling_howto.rst +++ b/docs/styling_howto.rst @@ -4,7 +4,6 @@ OWS Styling HOW-TO Guide .. toctree:: :maxdepth: 2 - :hidden: style_howto_components style_howto_components_nonlinear @@ -12,6 +11,7 @@ OWS Styling HOW-TO Guide style_howto_color_map style_howto_transparency style_howto_legends + styling_howto_jupyter .. contents:: Table of Contents @@ -29,12 +29,12 @@ of their data product and the use of the Open Data Cube (ODC). It aims to: 2. demonstrate how to configure OWS styles with real-world examples. For thorough technical description of the configuration format, -`see the configuration documentation -`_. +:doc:`see the configuration documentation +`. Setting up an ODC environment and installing datacube-ows is also beyond the scope of this HOWTO, but if you are using a JupyterHubs based system like DEA Sandbox, -a `Quick Start Guide `_ +a :doc:`Quick Start Guide ` is available. Introduction - Testing styles @@ -67,8 +67,8 @@ definition. .. note:: - If ``apply_ows_style_cfg()`` returns - :: + If ``apply_ows_style_cfg()`` returns:: + WMSException: Style stand_alone does not support requests with 2 dates set ``group_by='solar_day'`` @@ -121,8 +121,8 @@ If you are using a notebooks based environment like JupyterHub, you can display `View full size `_ -Refer to `the documentation `_ for more +Refer to :doc:`the Style API documentation ` for more information about the OWS styling API. -`Next `_ +:doc:`Next ` we start to look at how style configurations work. diff --git a/docs/usage.rst b/docs/usage.rst index 14655458f..e997f2c93 100644 --- a/docs/usage.rst +++ b/docs/usage.rst @@ -11,7 +11,7 @@ To use datacube-ows in a project:: import datacube_ows -To use the `stand-alone styling API `_:: +To use the :doc:`stand-alone styling API `:: from datacube_ows.styles.api import * @@ -22,15 +22,15 @@ OWS Command Line Tools Datacube-OWS provides two command line tools: * ``datacube-ows-update`` which is used for creating and maintaining - `OWS's database tables and views `_. + :doc:`OWS's database tables and views `. * ``datacube-ows-cfg`` which is used for managing - `OWS configuration files `_. + :doc:`OWS configuration files `. -.. click:: datacube_ows.update_ranges:main +.. click:: datacube_ows.update_ranges_impl:main :prog: datacube-ows-update :nested: full -.. click:: datacube_ows.cfg_parser:main +.. click:: datacube_ows.cfg_parser_impl:main :prog: datacube-ows-update :nested: full @@ -95,12 +95,12 @@ Update extents of a new product or to update a product in Datacube to make it ea $ datacube-ows-update --views $ datacube-ows-update alos_palsar_mosaic -Deploy the Digital Earth Africa OWS config available `here `_ +Deploy the `Digital Earth Africa OWS config `_ by copying to ows_cfg.py. Ideally load the config outside a docker container to iterate faster. Run GetCapabilities via curl to ensure data is present. -Perform GetMap via Qgis to ensure data is visible. +Perform GetMap via QGIS to ensure data is visible. .. code-block:: console diff --git a/env.micromamba.yaml b/env.micromamba.yaml index 370031589..185e7a474 100644 --- a/env.micromamba.yaml +++ b/env.micromamba.yaml @@ -14,7 +14,7 @@ dependencies: - scipy - flask-babel - regex - - matplotlib + - matplotlib-base # Not matplotlib, that pulls in a GUI backends - python-slugify - prometheus_flask_exporter - timezonefinder @@ -23,5 +23,6 @@ dependencies: - colour - setuptools_scm - bottleneck + - opentelemetry-distro - pip: - pyows