-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy path03-indicators.qmd
522 lines (290 loc) · 31.3 KB
/
03-indicators.qmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
# Indicators {#sec-indicators}
## The RAM-OP indicator set {#sec-ram-op-indicator-set}
RAM-OP surveys collect and report on data for a broad range of indicators relevant to older people.
These indicators cover the following dimensions:
* Demography and situation
* Food intake
* Severe food insecurity
* Disability
* Activities of daily living
* Mental health and well-being
* Dementia
* Health and health-seeking behaviour
* Sources of income
* Water, sanitation, and hygiene
* Anthropometry and screening coverage
* Visual impairment
Data for a small group of miscellaneous indicators are also collected and reported.
The RAM-OP indicator set has been designed on a modular basis. Each module is a set of indicators relating to a single dimension from the list given above and is collected using a dedicated set of questions and measurements. This means that the RAM-OP questionnaire also consists of a set of modules.
Whenever possible, RAM-OP uses standard and validated indicators and question sets.
Indicators are described below, showing the questionnaire components that are used to collect and record the data required, and flowcharts of the process used to derive indicators from the collected data. Standard symbols are used. For example:
```{mermaid}
%%| label: fig-indicators01
%%| fig-cap: Standard symbols used
%%| fig-alt: Standard symbols used
%%| fig-align: center
flowchart TD
A[**sum = a1 + a2**<br/><br/>*Assign a value or perform a calculation*]
B{**h1 = 0?**<br/><br/>*Make a decision Yes/No*}
A --> B
```
A non-standard symbol is used to show **recode operations**. A recode operation shows changes that are made to data so that it can be used to derive indicators without having to show many decision nodes in the flowchart. They are also used to specify what should be done with missing or out-of-range values. For example:
{#fig-indicators02 fig-alt="Example recode operations flowchart" fig-align="center"}
### Demography and situation {#sec-demography-situation}
The demography and situation indicators are used to describe the survey sample and are derived from this questionnaire component:
{#fig-questionnaire01 fig-alt="Demography and situation questionnaire module" fig-align="center"}
\newpage
Each of the questions yields a separate indicator:
{#fig-indicators03 fig-alt="Demography and situation indicators" fig-align="center"}
### Food intake {#sec-food-intake}
Food-intake indicators are derived from this questionnaire component. This data can be queried to yield a large number of useful indicators.
\newpage
{#fig-questionnaire02 fig-alt="Food intake questionnaire module" fig-align="center"}
\newpage
There are three related sets of diet-related indicators:
* meal frequency
* food groups consumed / dietary diversity
* indicators of nutrient consumption.
The indicator hierarchy is:
{#fig-indicators04 fig-alt="Diet-related indicators hierarchy" fig-align="center"}
The data on the number of meals taken in the previous twenty-four hours forms a *meal frequency score*.
Food intake data from each subject is combined into a *dietary diversity score*. The dietary diversity score is a crude measure of food security. The dietary diversity score ranges between zero (i.e. no food groups) and eleven (i.e. eleven food groups). Higher values of the dietary diversity sore are associated with better food security.
The meal frequency score and the dietary diversity score follow [@swindale_household_2006; @kennedy_guidelines_2010]:
> Swindale A, Bilinsky P, *Household Dietary Diversity Score (HDDS) for measurement of household food access: Indicator guide*.,Washington DC, Food and Nutrition Technical Assistance (FANTA) Project, 2006
> Kennedy G, Ballard T, Dop MC, *Guidelines for Measuring Household and Individual Dietary Diversity*, Rome, Food and Agricultural Organization, 2010
The data on the types of food consumed in the previous twenty-four hours are analysed in order to determine the diet’s content of specific micronutrients that are important for older people. This also follows @swindale_household_2006 and @kennedy_guidelines_2010, and @world_health_organization_management_2000:
> World Health Organisation, *The management of nutrition in major emergencies*, Geneva, WHO, 2000
### Meal frequency {#sec-meal-frequency}
The meal frequency score indicator is the answer given to the first food intake question:
{#fig-indicators05 fig-alt="First food intake question" fig-align="center"}
Meal frequency is a crude measure of food security.
Higher values of meal frequency are associated with better food security.
### Food groups and dietary diversity {#sec-diet-diversity}
Questions relating to the consumption of individual food items / food types are combined to create food groups and the number of food groups consumed are counted to create a dietary diversity score:
{#fig-indicators06 fig-alt="Workflow for creating food groups" fig-align="center"}
The consumption of the eleven individual food groups and the dietary diversity score are reported separately.
The dietary diversity score is a crude measure of food security. The dietary diversity score ranges between zero (no food groups) and eleven (eleven food groups). Higher values of the dietary diversity score are associated with better food security.
### Indicators of nutrient consumption
**Overview**
Questions and combinations of questions relating to the consumption of individual food items and food types can be used to determine whether the reported diet is likely to be provide sufficient nutrients of various types:
{#fig-indicators07 fig-alt="Workflow for nutrients uptake indicators" fig-align="center"}
Each indicator is formed using logical “or” operations (i.e. the indicator is true if **any** of the constituent foods are consumed). For example, the indicator for the consumption of iron rich foods:
{#fig-indicators08 fig-alt="Recoding for iron rich foods" fig-align="center"}
requires the consumption of one or more of green leafy vegetables, organ meats, meat, or fish and shellfish. Consumption of **any** of these foods is sufficient to indicate that the survey subject consumes iron rich food.
#### Protein rich foods
Indicators of consumption of protein rich foods from animal sources, plant source, and any / all sources are calculated as:
{#fig-indicators09 fig-alt="Recoding for protein-rich foods" fig-align="center"}
#### Vitamin A rich foods
Indicators of consumption of vitamin A rich foods from animal sources, plant source, and any / all sources are calculated as:
{#fig-indicators10 fig-alt="Recoding for vitamin A-rich foods" fig-align="center"}
\newpage
#### Iron rich foods
An indicator of consumption of iron rich foods from any / all sources is calculated as:
{#fig-indicators11 fig-alt="Recoding of iron-rich foods from any source" fig-align="center"}
#### Calcium rich foods
An indicator of consumption of calcium rich foods from any / all sources is calculated as:
{#fig-indicators12 fig-alt="Recoding of caclcium-rich foods" fig-align="center"}
#### Zinc rich foods
An indicator of consumption of zinc rich foods from any / all sources is calculated as:
{#fig-indicators13 fig-alt="Recoding of zinc-rich foods" fig-align="center"}
\newpage
#### Vitamin B rich foods
Indicators of consumption of vitamin B rich foods from any / all sources are calculated as:
{#fig-indicators14 fig-alt="Recoding if vitamin B-rich foods" fig-align="center"}
Note that the vitamin B complex indicator requires that at least one food from each of the B1, B2, B3, B6, and B12 rich food combinations is consumed.
### Severe food insecurity
An indicator of severe food insecurity (hunger) is derived from this questionnaire component:
{#fig-questionnaire03 fig-alt="Severe food insecurity questionnaire module" fig-align="center"}
and is calculated as:
{#fig-indicators15 fig-alt="Recoding of severe food insecurity indicator" fig-align="center"}
\newpage
This indicator is the *Household Hunger Scale (HHS)* and is a simple, well-validated, and widely used indicator of severe food insecurity [@ballard_household_2011; @ruel_measuring_2014]:
> Ballard T, Coates J, Swindale A, Deitchler M, *Household Hunger Scale: Indicator Definition and Measurement Guide*, Washington DC, FANTA-2 Bridge, FHI 360, 2011
> Ruel MT, Ballard TJ, Deitchler M, *Measuring and Tracking the Access Dimension of Food Security: Available Indicators and Recommendations for Future Investments*, Global Nutrition Report 2014: Technical Note 6, Washington DC, International Food Policy Research Institute, 2014
### Disability
Indicators of disability across six different domains are derived from this questionnaire component:
{#fig-questionnaire04 fig-alt="Disability questionnaire module" fig-align="center"}
\newpage
Individual disability indicators are reported for each domain (i.e. vision, hearing, mobility, remembering, self-care, and communication) of disability in the Washington Group’s short set of question designed to identify people with a disability in a census or survey format [@washington_group_on_disability_statistics_washington_2022].
Overall disability prevalence indicators are also reported.
Indicators of disability in each domain are calculated as:
{#fig-indicators16 fig-alt="Recoding of disability indicators" fig-align="center"}
Overall disability prevalence indicators are calculated as:
+----------+------------------------------------------------+
| `P0` = 1 | if no domain has `D1` = 1, |
| | else = 0 (no disability in any domain) |
+----------+------------------------------------------------+
| `P1` = 1 | if at least one domain has `D1` = 1, else = 0 |
+----------+------------------------------------------------+
| `P2` = 1 | if at least one domain has `D2` = 1, else = 0 |
+----------+------------------------------------------------+
| `P3` = 1 | if at least one domain has `D3` = 1, else = 0 |
+----------+------------------------------------------------+
| `PM` = 1 | if at more than one domain has `D1` = 1, |
| | else = 0 (M stands for “Multiple”) |
+----------+------------------------------------------------+
### Activities of daily living
Indicators of how well the subject copes with activities of daily living are derived from this questionnaire component:
{#fig-questionnaire05 fig-alt="Activities of daily living questionnaire module" fig-align="center"}
Individual *independence* indicators are reported for each dimension (i.e. bathing, dressing, toilet, mobility, continence, and eating) of daily living activities.
A composite indicator of the degree of *independence* (i.e. how well the subject can cope with activities of daily living) is also reported. This indicator is the *Katz Index of Independence in Activities of Daily Living* (or the *Katz Index of ADL* for short) and is a simple, well-validated, and widely used indicator of how well the subject can cope with activities of daily living [@katz_studies_1963; @katz_progress_1970; @katz_assessing_2015].
> Katz S, Ford AB, Moskowitz RW, Jackson BA, Jaffe MW, *Studies of illness in the aged. The Index of ADL: A standardized measure of biological and psychosocial function*, JAMA, 185(12), 1963, pp. 914-9
> Katz S, Down TD, Cash HR, Grotz, RC, *Progress in the development of the index of ADL*, The Gerontologist, 10(1), 1970, pp. 20-30
> Katz S, *Assessing self-maintenance: Activities of daily living, mobility and instrumental activities of daily living*, JAGS, 31(12), 1983, pp. 721-726
The Katz Index of ADL ranges between zero (complete dependence) and six (independence).
\newpage
The seventh question of this module, which is not part of the Katz Index of ADL, is reported separately and indicates whether the subject has someone to help them with activities of daily living:
**Activities of Daily Living**
+----+--------------------------------------------------+----------+----------+
| a7 | Is someone taking care of you or helping you with| 1 = Yes; | [__] |
| | everyday activities such as shopping, cooking, | 2 = No | |
| | bathing and dressing? | | |
+----+--------------------------------------------------+----------+----------+
It is not possible to know if the help available completely meets a subject's needs, but we can identify the proportion of subjects needing help with one or more activities of daily living who also report not having someone to help them:
{#fig-indicators17 fig-alt="Conceptual representation of indicator on unmet need" fig-align="center"}
This is an indicator of unmet need.
\newpage
Indicators of how well the subject can cope with activities of daily living and probable unmet need are calculated as:
{#fig-indicators18 fig-alt="Recode for unmet need indicator" fig-align="center"}
### Mental health and well-being
Indicators of mental health and well being are derived from this questionnaire component:
{#fig-quesetionnaire06 fig-alt="Mental health and well-being questionnaire module" fig-align="center"}
A score is calculated. This is the *Kessler K6 Psychological Distress Scale*. The score ranges from zero (indicating no psychological distress) to twenty-four (indicating severe psychological distress). A score of thirteen or more indicates serious psychological distress. The Kessler K6 Psychological Distress Scale is a widely recommended, widely used, accurate, reliable, and simple measure of psychological distress [@kessler_short_2002; @kessler_screening_2003].
> Kessler RC, Andrews G, Colpe LJ, Hiripi E, Mroczek, DK, Normand SLT, et al, “Short screening scales to monitor population prevalences and trends in non-specific psychological distress”, *Psychological Medicine*, 32(6), 2002, pp. 959–976
> Kessler RC, Barker PR, Colpe LJ, Epstein JF, Gfroerer JC, Hiripi E, “Screening for Serious Mental Illness in the General Population”, *Archives of General Psychiatry*, 60(2), 2003, pp. 184-189
Indicators of mental health and well-being are calculated as:
{#fig-indicators19 fig-alt="Recode of mental health and well-being indicators" fig-align="center"}
\newpage
### Dementia
An indicator of probable dementia is derived from this questionnaire component:
{#fig-questionnaire07 fig-alt="Dementia questionnaire module" fig-align="center"}
\newpage
The indicator of *probable* dementia is calculated as:
{#fig-indicators20 fig-alt="Recode of probably dementia indicator" fig-align="center"}
This indicator is derived from the Community Screening Instrument for Dementia (CSID) developed by the 10/66 Dementia Research Group. This is a simple, validated, and widely used indicator of probable dementia [@prince_brief_2010].
> Prince M, et al, “A brief dementia screener suitable for use by non-specialists in resource poor settings - The cross-cultural derivation and validation of the brief Community Screening Instrument for Dementia”, *International Journal of Geriatric Psychiatry*, 26(9), 2011, pp. 899–907
\newpage
### Health and health-seeking behaviour
Indicators of health and health-seeking behaviour for chronic and acute conditions are derived from this questionnaire component:
{#fig-questionnaire08 fig-alt="Health and health-seeking behaviour questionnaire module" fig-align="center"}
\newpage
Indicators of health and health-seeking behaviour for chronic conditions are calculated as:
{#fig-indicators21 fig-alt="Recode of health and health-seeking behaviour indicators for chronic conditions" fig-align="center"}
\newpage
Indicators of health and health-seeking behaviour for acute conditions are calculated as:
{#fig-indicators22 fig-alt="Recode of health and health-seeking behaviour indicators for acute conditions" fig-align="center"}
### Sources of income {#sec-sources-income}
Indicators related to sources of income are derived from this questionnaire component:
{#fig-questionnaire09 fig-alt="Sources of income questionnaire module" fig-align="center"}
and are calculated as:
{#fig-indicators23 fig-alt="Recode of sources of income indicators" fig-align="center"}
The grouped income sources (i.e. `m2a`, `m2b`, etc.) and individual income sources may vary between settings. The questionnaire component shown above has proved suitable for use in Ethiopia, South Sudan, and Tanzania.
\newpage
### Water, sanitation, and hygiene {#sec-wash}
Indicators relating to water, sanitation, and hygiene (WASH) are derived from this questionnaire component:
{#fig-questionnaire10 fig-alt="Water, sanitation, and hygiene questionnaire module" fig-align="center"}
\newpage
Indicators are calculated following @world_health_organization_core_2006.
> WHO / UNICEF, *Core Questions on Drinking-water and Sanitation for Household Surveys*, Geneva, WHO / UNICEF, 2006
Indicators relating to water, sanitation, and hygiene (WASH) are calculated as:
{#fig-indicators24 fig-alt="Recode of water, sanitation, and hygiene indicators" fig-align="center"}
\newpage
### Anthropometry and screening coverage {#sec-anthro}
Indicators relating to anthropometry and screening coverage are derived from this questionnaire component:
{#fig-questionnaire11 fig-alt="Anthropometry and screening questionnaire module" fig-align="center"}
And are calculated as:
{#fig-indicators25 fig-alt="Recode of anthropometry and screening indicators" fig-align="center"}
Raw MUAC data (i.e. not MUAC class) is collected, entered, and analysed. This requires that an adult MUAC tape (i.e. capable of measuring MUAC to 450 mm) is used.
The presence of bilateral oedema is assessed by pressing with your thumbs **both** feet of the older person for three seconds and checking whether this creates a lasting depression or “pit” on both feet. Bilateral pitting oedema in older people may not be “nutritional” oedema (as is almost always the case with children). Older people with bilateral pitting oedema should be advised to consult a doctor.
The prevalence of GAM, MAM, and SAM are estimated using a PROBIT estimator. This type of estimator provides better precision than a classic estimator at small sample sizes [@world_health_organization_physical_1995; @dale_assessment_2012; @blanton_probit_2013].
> World Health Organisation, *Physical Status: The use and interpretation of anthropometry. Report of a WHO expert committee*, WHO Technical Report Series 854, WHO, Geneva, 1995
> Dale NM, Myatt M, Prudhon C, Briend, A, “Assessment of the PROBIT approach for estimating the prevalence of global, moderate and severe acute malnutrition from population surveys”, *Public Health Nutrition*, 1–6. doi:10.1017/S1368980012003345, 2012
> Blanton CJ, Bilukha, OO, “The PROBIT approach in estimating the prevalence of wasting: revisiting bias and precision”, *Emerging Themes in Epidemiology*, 10(1), 2013, p. 8
The PROBIT estimator is described in Box 1.
MUAC-based case definitions for acute malnutrition are used:
+:--------:+:----------------------+
| **GAM** | MUAC < 210 mm |
+----------+-----------------------+
| **MAM** | 185 mm ≤ MUAC < 210mm |
+----------+-----------------------+
| **SAM** | MUAC < 185mm |
+----------+-----------------------+
These are standard case definitions for acute malnutrition in adults and recommended by HelpAge International for use in older people in humanitarian contexts.
**Note** : MUAC in adults should be measured on the non-dominant arm. This is usually the left arm. The importance of high levels of accuracy and precision at the individual level is of lesser importance in survey work compared to case-finding or diagnosis in clinical contexts, for example. This means that a simple rule such as “Always measure MUAC on the left arm” may be used.
\newpage
::: {.callout-note title="Using PROBIT"}
An estimate of GAM prevalence can be made using a classic estimator:
$$\text{prevalence} = \frac{\text{number of respondents with MUAC < 210 mm}}{\text{total number of respondents}}$$
The estimate of GAM prevalence made from the RAM-OP survey data is made using a PROBIT estimator. The PROBIT function is also known as the *inverse cumulative distribution* function. This function converts parameters of the distribution of an indicator (e.g. the mean and standard deviation of a *normally* distributed variable) into cumulative percentiles. This means that it is possible to use the normal PROBIT function with estimates of the mean and standard deviation of indicator values in a survey sample to predict (or estimate) the proportion of the population falling below a given threshold. For example, for data with a mean MUAC of 256 mm and a standard deviation of 28 mm the output of the normal PROBIT function for a threshold of 210 mm is 0.0502 meaning that 5.02% of the population are *predicted* (or *estimated*) to fall below the 210 mm threshold.
Both the classic and the PROBIT methods can be thought of as estimating area:
{#fig-auc fig-alt="Area under the curve" fig-align="center"}
The principal advantage of the PROBIT approach is that the required sample size is usually smaller than that required to estimate prevalence with a given precision using the classic method.
The PROBIT method assumes that MUAC is a normally distributed variable. If this is not the case then the distribution of MUAC is transformed towards normality.
The prevalence of SAM is estimated in a similar way to GAM. The prevalence of MAM is estimated as the difference between the GAM and SAM prevalence estimates:
$$\widehat{MAM prevalence} = \widehat{GAM prevalence} - \widehat{SAM prevalence}$$
:::
\newpage
### Visual impairment {#sec-visual-impairment}
An indicator of visual impairment is derived from this questionnaire component:
{#fig-quesetionnaire12 fig-alt="Visual impairment questionnaire module" fig-align="center"}
\newpage
And is calculated as:
{#fig-indicators27 fig-alt="Recode of visual impairment indicator" fig-align="center"}
The *“illiterate E”* or *“tumbling E”* (the preferred term) is a validated and widely used method for measuring visual acuity [@taylor_applying_1978; @kaiser_prospective_2009]
> Taylor HR, “Applying new design principles to the construction of an illiterate E chart”, *American Journal of Optometry & Physiological Optics*, 55:348, 1978
> Kaiser PK, “Prospective Evaluation of Visual Acuity Assessment: A Comparison of Snellen Versus ETDRS Charts in Clinical Practice (An AOS Thesis)”, *Transactions of the American Ophthalmological Society*, 107: 311–324, 2009
\newpage
The size of the “E” used:
{#fig-indicators28 fig-alt="The tumbling E" fig-align="center"}
as well as the distance used for the test (two metres) and the indicator calculation apply the WHO case definition of visual impairment (i.e. visual acuity < 6 / 18).
The tumbling E card should be laminated (i.e. plastic coated and have a two metre cord attached which helps to ensure that the visual acuity test is performed at the correct distance (see @fig-indicators29)).
After demonstrating to the respondent what the test is about (i.e. the subject should indicate which direction the branches of the 'E' are pointing), the test is administered at a distance of two meters, turning the card in four different directions, and asking the person to indicate which direction the branches of the “E” is pointing. If the subject wears glasses, they are allowed to use them during the test if they want to.
::: {.callout-note}
If the person is unable to correctly answer at least three times out of four, they have a visual impairment. A simple visual acuity test such as the 'tumbling E' test also does not indicate anything about an underlying disease such as glaucoma or the need for reading spectacles (presbyopia). These conditions are common in people aged 60 years or older. Subjects failing the visual acuity test should be counselled to visit an ophthalmologist for a detailed eye examination.
:::
\newpage
{#fig-indicators29 fig-alt="Equipment used to measure visual acuity" fig-align="center"}
\newpage
### Miscellaneous indicators {#sec-misc-indicators}
Data for a small group of miscellaneous indicators are also collected and reported. These are derived from these questions:
**Hunger – Ration - Relief**
+-----+-------------------------------------------------+----------+-------+
| f6 | Are you or anyone in your household receiving a | 1 = Yes; | [__] |
| | food ration on a regular basis? | 2 = No | |
+-----+-------------------------------------------------+----------+-------+
| f7 | Have you or another member of your household | 1 = Yes | [__] |
| | received non-food relief items such as soap, | 2 = No | |
| | bucket, water container, bedding, mosquito net, | | |
| | clothes, or plastic sheet in the previous four | | |
| | weeks? | | |
+----------+--------------------------------------------+----------+-------+
**Activities of Daily Living**
+-----+------------------------------------+-----------------+-------------+
| a8 | Do you have problems chewing food? | 1 = Yes; 2 = No | [__] |
+-----+------------------------------------+-----------------+-------------+
\newpage
and are calculated as:
{#fig-indicators30 fig-alt="Recode for miscellaneous indicators" fig-align="center"}
## A note on data management and data analysis {#sec-data-management-analysis}
This section has described how RAM-OP data is used to create a broad set of indicators. If you do not want to use the standard RAM-OP software to do this then you can use this information to create data entry systems and data management scripts for your favoured database or statistical analysis software. See the sections on [**RAM-OP datasets**](#sec-datasets) and [**RAM-OP questionnaire**](#sec-questionnaire) for more compact information on variable names and codes that you may find helpful.
It is important to note that data analysis procedures need to account for the sample design. All major statistical analysis software can do this (details vary). There are two things to note:
* The RAM-OP sample is a two-stage sample. Subjects are sampled from a small number of primary sampling units (PSUs).
* The RAM-OP sample is **not** prior weighted. This means that you will need to provide per-PSU sampling weights. These are usually the populations of the PSU.
You will need to specify this sample design to your statistical analysis software. If you fail to do this then your analysis may produce estimates that place undue weight to observations from smaller communities with confidence intervals with lower than nominal coverage (i.e. they will be too narrow).
The standard RAM-OP software uses *blocked weighted bootstrap* estimation approach:
* **Blocked** : The block corresponds to the PSU or cluster.
* **Weighted** : The RAM-OP sampling procedure does not use population proportional sampling to weight the sample prior to data collection as is done with SMART type surveys. This means that a posterior weighting procedure is required. The standard RAM-OP software uses a “roulette wheel” algorithm to weight (i.e. by population) the selection probability of PSUs in bootstrap replicates.
A total of `m'` PSUs are sampled *with-replacement* from the survey dataset where `m'` is the number of PSUs in the survey sample. Individual records within each PSU are then sampled *with-replacement*. A total of n' records are sampled *with-replacement* from each of the selected PSUs where `n'` is the number of individual records in a selected PSU. The resulting collection of records replicates the original survey in terms of both sample design and sample size. A large number of replicate surveys are taken (the standard RAM-OP software uses $r = 399$ replicate surveys but this can be changed). The required statistic (e.g. the mean of an indicator value) is applied to each replicate survey. The reported estimate consists of the 50th (point estimate), 2.5th (lower 95% confidence limit), and the 97.5th (upper 95% confidence limit) percentiles of the distribution of the statistic observed across all replicate surveys. The blocked weighted bootstrap procedure is outlined in Figure \@ref(fig:indicators31).
The principal advantages of using a bootstrap estimator are:
* Bootstrap estimators work well with small sample sizes.
* The method is *non-parametric* and uses empirical rather than theoretical distributions. There are no assumptions of things like normality to worry about.
* The method allows estimation of the sampling distribution of almost any statistic using only simple computational methods.
The standard RAM-OP data analysis software is described in the section [**Standard RAM-OP software**](#sec-software).
\newpage
{#fig-indicators31 fig-alt="The blocked weighted bootstrap used by the standard RAM-OP software" fig-align="center"}