Merge branch 'tech_doc_2025' of https://github.com/NOAA-EDAB/tech-doc into tech_doc_2025

Author: BBeltz1

.github/workflows/manage_cache.yaml

@@ -21,7 +21,7 @@ jobs:
           sudo apt-get install  libgdal-dev libcurl4-gnutls-dev libgit2-dev libudunits2-dev
 
       - name: Check cached R packages
-        uses: actions/cache@v2
+        uses: actions/cache@v4
         id: cache
         with:
           path: ${{ env.R_LIBS_USER }}

_bookdown.yml

@@ -7,6 +7,7 @@ rmd_files:
     -   "chapters/sectionHeaders/methods.rmd"
     -   "chapters/erddap_query_and_build.Rmd"
     -   "chapters/Trend_analysis.Rmd"
+    -   "chapters/short_term_trend.Rmd"
     -   "chapters/regime_shift_analysis.Rmd"
     -   "chapters/survdat.rmd"
     -   "chapters/EPU.Rmd"

bibliography/short_term_trend.bib

@@ -21,9 +21,9 @@ Summarizing trends for ecosystem indicators is desirable, but the power of stati
 
 In a simulation study [@hardison2019], we explored the effect of time series length and autocorrelation strength on statistical power of three trend detection methods: a generalized least squares model selection approach, the Mann-Kendall test, and Mann-Kendall test with trend-free pre-whitening. Methods were applied to simulated time series of varying trend and autocorrelation strengths. Overall, when sample size was low (N = 10) there were high rates of false trend detection, and similarly, low rates of true trend detection. Both of these forms of error were further amplified by autocorrelation in the trend residuals. Based on these findings, we selected a minimum series length of N = 30 for indicator time series before assessing trend.
 
-We also chose to use a GLS model selection (GLS-MS) approach to evaluate indicator trends in the 2018 (and future) State of the Ecosystem reports, as this approach performed best overall in the simulation study. GLS-MS also allowed for both linear and quadratic model fits and quantification of uncertainty in trend estimates. The model selection procedure for the GLS approach fits four models to each time series and selects the best fitting model using AICc. The models are, 1) linear trend with uncorrelated residuals, 2) linear trend with correlated residuals, 3) quadratic trend with uncorrelated residuals, and 4) quadratic trend with correlated residuals. I.e., the models are of the form
+We also chose to use a GLS model selection (GLS-MS) approach to evaluate indicator trends in the 2018 (and future) State of the Ecosystem reports, as this approach performed best overall in the simulation study. GLS-MS allowed for linear model fits and quantification of uncertainty in trend estimates. The model selection procedure for the GLS approach fits two models to each time series and selects the best fitting model using AICc. The models are, 1) linear trend with uncorrelated residuals, 2) linear trend with correlated residuals. I.e., the models are of the form
 
-$$ Y_t = \alpha_0 + \alpha_1X_t + \alpha_2X_t^2 + \epsilon_t$$
+$$ Y_t = \alpha_0 + \alpha_1X_t +  \epsilon_t$$
 $$\epsilon_t = \rho\epsilon_{t-1} + \omega_t$$
 
 $$w_t \sim N(0, \sigma^2)$$

chapters/Trend_analysis.Rmd

@@ -0,0 +1,50 @@
+# Short Term Trend Analysis
+
+
+**Description**: Time series trend analysis for short time series
+
+**Found in**: State of the Ecosystem - Gulf of Maine & Georges Bank (2025+), State of the Ecosystem - Mid-Atlantic (2025+)
+
+**Indicator category**: Extensive analysis, not yet published
+
+**Contributor(s)**: Andy Beet
+
+**Data steward**: NA
+
+**Point of contact**: Andy Beet, <[email protected]>
+
+**Public availability statement**: NA
+
+
+## Methods
+
+In prep: **A.Beet "A test for short term trend detection in the presence of autocorrelation"**
+
+The specific model addressed here is of the form,
+
+\begin{equation}
+  Y_t = \beta_0 + \beta_1 t + \epsilon_t  
+\end{equation}
+
+where $\epsilon_t = \phi\epsilon_{t-1} + z_t$ is a stationary first order autoregressive process with $z_t \sim N(0,\sigma^2)$. Interest centers on testing the null hypothesis, $H_0:\beta_1 = 0$ against the alternative, $H_1:\beta_1 \neq 0$
+
+Testing for a trend in time series data has been addressed by many authors from a wide range of disciplines including economics, statistics, hydrology, ecology, fisheries, and epidemiology (@cochrane_application_1949, @prais_trend_1954, @beach_maximum_1978, @park_estimating_1980, @brillinger_trend_1994, @bence_analysis_1995, @woodward_improved_1997, @zhang_temperature_2000, @yue_influence_2002, @wang_linear_2015 ,@hardison_simulation_2019). These approaches have typically taken one of three paths; non parametric methods such as the Mann Kendall test and its pre-whitening variants (@hamed_modified_1998,  @zhang_temperature_2000, @yue_applicability_2002,  @wang_linear_2015); parametric methods involving data transformation such as @cochrane_application_1949, @prais_trend_1954, @woodward_improved_1997; parametric methods such as generalized least squares and maximum likelihood estimation (@beach_maximum_1978,  @davison_economic_1999, @pinheiro_mixed_2000). 
+
+It has been well documented that under the null hypothesis of no trend, $H_0:\beta_1=0$, in the presence of autocorrelation, parametric tests relying on asymptotic distribution theory reject the null hypothesis too frequently, leading to nominal significance levels that are too high, even for relatively long time series of length n = 100 (@woodward_improved_1997). Non parametric tests like those listed above also suffer the same problem. 
+
+We introduce a test that, like @beach_maximum_1978, uses maximum likelihood for parameter estimation, but differs in that the significance of the likelihood ratio statistic, LR, is assessed via a parametric bootstrap (@efron_introduction_1993). Parametric bootstrap procedures have been used in some of the aforementioned work. @woodward_improved_1997 uses an alternative statistic, the Cochrane-Orchutt statistic, for detecting a trend. @rayner_bootstrapping_1990 focuses on the significance of the AR(1) parameter and @bence_analysis_1995 focuses on adjusting confidence intervals. We use the parametric bootstrap as an alternative means of assessing the significance of the LR statistic.
+
+The Likelihood ratio statistic combined with a parametric bootstrap is employed to test for a linear trend in the presence of autocorrelation in the form of an AR(1) process. Small samples of size, n = 10, are of particular interest. 
+
+### Data source(s)
+NA
+
+### Data extraction
+NA
+
+### Data analysis
+
+Code used for the fitting and evaluation of short term trend can be found [here](https://github.com/NOAA-EDAB/arfit).
+
+**catalog link**
+No associated catalog page

chapters/short_term_trend.Rmd

@@ -6,7 +6,7 @@ site: bookdown::bookdown_site
 knit: "bookdown::render_book"
 always_allow_html: true
 documentclass: book
-bibliography: ["bibliography/introduction.bib","bibliography/aggregate_groups.bib","bibliography/seasonal_sst_anomaly_maps.bib","bibliography/Aquaculture.bib","bibliography/Bennet_indicator.bib","bibliography/bottom_temperature.bib","bibliography/bottom_temp_highres.bib","bibliography/Revenue_Diversity.bib","bibliography/ches_bay_water_quality.bib","bibliography/phytoplankton.bib","bibliography/ecosystem_overfishing.bib","bibliography/comm_eng.bib","bibliography/calanus_stage.bib","bibliography/ches_bay_temp.bib","bibliography/conceptmods.bib","bibliography/Condition.bib","bibliography/EPU.bib","bibliography/Expected_Number.bib","bibliography/cold_pool_index.bib","bibliography/sandlance.bib","bibliography/gulf_stream_index.bib","bibliography/habitat_diversity.bib","bibliography/habitat_vulnerability.bib","bibliography/Ich_div.bib","bibliography/long_term_sst.bib","bibliography/MAB_HAB.bib","bibliography/NE_HAB.bib","bibliography/habs.bib","bibliography/occupancy.bib","bibliography/productivity_tech_memo.bib","bibliography/RW.bib","bibliography/seabird_ne.bib","bibliography/seal_pup.bib","bibliography/slopewater_proportions.bib","bibliography/Species_dist.bib","bibliography/survey_data.bib","bibliography/thermal_hab_proj.bib","bibliography/trans_dates.bib","bibliography/trend_analysis.bib","bibliography/zooplankton.bib","bibliography/cold_pool_index.bib","bibliography/forage_energy_density.bib","bibliography/Forage_Fish_Biomass_Index.bib","bibliography/marine_heatwave.bib","bibliography/protected_species_hotspots.bib","bibliography/ocean_acidification.bib","bibliography/wind_habitat_occupancy.bib","bibliography/warm_core_rings.bib", "bibliography/glossary.bib","packages.bib"]
+bibliography: ["bibliography/introduction.bib","bibliography/aggregate_groups.bib","bibliography/seasonal_sst_anomaly_maps.bib","bibliography/Aquaculture.bib","bibliography/Bennet_indicator.bib","bibliography/bottom_temperature.bib","bibliography/bottom_temp_highres.bib","bibliography/Revenue_Diversity.bib","bibliography/ches_bay_water_quality.bib","bibliography/phytoplankton.bib","bibliography/ecosystem_overfishing.bib","bibliography/comm_eng.bib","bibliography/calanus_stage.bib","bibliography/ches_bay_temp.bib","bibliography/conceptmods.bib","bibliography/Condition.bib","bibliography/EPU.bib","bibliography/Expected_Number.bib","bibliography/cold_pool_index.bib","bibliography/sandlance.bib","bibliography/gulf_stream_index.bib","bibliography/habitat_diversity.bib","bibliography/habitat_vulnerability.bib","bibliography/Ich_div.bib","bibliography/long_term_sst.bib","bibliography/MAB_HAB.bib","bibliography/NE_HAB.bib","bibliography/habs.bib","bibliography/occupancy.bib","bibliography/productivity_tech_memo.bib","bibliography/RW.bib","bibliography/seabird_ne.bib","bibliography/seal_pup.bib","bibliography/slopewater_proportions.bib","bibliography/Species_dist.bib","bibliography/survey_data.bib","bibliography/thermal_hab_proj.bib","bibliography/trans_dates.bib","bibliography/trend_analysis.bib","bibliography/zooplankton.bib","bibliography/cold_pool_index.bib","bibliography/forage_energy_density.bib","bibliography/Forage_Fish_Biomass_Index.bib","bibliography/marine_heatwave.bib","bibliography/protected_species_hotspots.bib","bibliography/ocean_acidification.bib","bibliography/wind_habitat_occupancy.bib","bibliography/warm_core_rings.bib","bibliography/short_term_trend.bib", "bibliography/glossary.bib","packages.bib"]
 geometry: "left=1.0in, right=1.0in, top=1.0in, bottom=1.0in, includefoot"
 biblio-style: apalike
 link-citations: true

index.Rmd

@@ -138,7 +138,7 @@ @Manual{R-stocksmart
   title = {stocksmart: Provides access to NOAAs stock SMART data},
   author = {Andy Beet},
   year = {2023},
-  note = {R package version 0.6.12},
+  note = {R package version 0.6.30},
   url = {https://github.com/NOAA-EDAB/stocksmart},
 }