Skip to content

Commit b6276fa

Browse files
pedrohserranopedrohserrano
committed
document improvemnts
1 parent 610760d commit b6276fa

File tree

1 file changed

+50
-82
lines changed

1 file changed

+50
-82
lines changed

‎sdmt-report.tex

Lines changed: 50 additions & 82 deletions
Original file line numberDiff line numberDiff line change
@@ -21,10 +21,10 @@
2121

2222

2323
\title{\LARGE \bf
24-
SDMT Alternative Analysis
24+
Symbol Digital Modalities Test (SDMT) a Mobile App Study Analysis
2525
}
2626

27-
\author{Pedro Vladimir Hern\'andez Serrano}
27+
\author{Pim van Oirschot, Pedro Vladimir Hern\'andez Serrano}
2828
\begin{document}
2929

3030
\maketitle
@@ -35,61 +35,45 @@
3535
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
3636
\begin{abstract}
3737

38+
On this document we will find the description of the Symbol Digital Modalities Test (SDMT), and how is been used for detect impairment on Multiple Sclerosis (MS) patients. It is going to be showed the experiment conducted with 23 volunteers, 15 HC and 8 MS, which they scored
3839
It was replicated the stages of reliability and validity
3940

4041
\end{abstract}
4142

4243
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
4344
\section{Introduction}
4445
\vspace{2mm}
45-
The SDMT has proven to be quite useful in neuropsychological MS research. It has excellent test–retest reliability, and alternate forms have been developed that are equivalent in difficulty. When the alternate forms are used, reliability is maintained. Validity research shows that SDMT is a good measure of processing speed or efficiency. SDMT is the neuropsychological test most sensitive to MS cognitive disorder and correlates very well with MRI measures of atrophy, lesion burden, and micro- structural pathology. The test very effectively repre- sents the core neuropsychological domain of processing speed.
46-
In comparison to other digit/symbol substitution tests, the digital version has the following advantages: limited practice effect due to the generation of new keys and forms at each session, standardisation of the presentation, precision in the presentation of the stimulus and scoring of the responses, and immediate classification of each subject score according to normative values
4746

48-
The main difference of responding the SDMT test on paper and responding on digital is thet the second one we have the precise moment every patient performs a task, so we might discover new features as time.
49-
We are interested on analyse the time of response every participant have, in addition, we want to check the practicing effect on trials over the time
47+
Cognitive impairment is one of the traits observed in 43-65\% of Multiple Sclerosis (MS) patients. Many batteries have been proposed to diagnose the cognitive impairment in MS. The alternative showed in this document is the use of Symbol Digital Modalities Test (SDMT), which only takes 90 seconds and aims to measure the information processing speed. As the name of the SDMT suggests, the subject is presented with a map of symbols and digits and asked to match symbol and digits. The total number of the correctly classified digits, reaction time and the total number of digits are noted for analysis.
48+
49+
The SDMT has proven to be quite useful in neuropsychological MS research. It has excellent test-retest reliability, and alternate forms have been developed that are equivalent in difficulty. When the alternate forms are used, reliability is maintained. Validity research shows that SDMT is a good measure of processing speed or efficiency. The test very effectively represents the domain of processing speed.
50+
51+
The main difference of responding the SDMT test on paper and responding on digital is that the second one we have the precise moment every patient performs a task, so we might discover new features as time. We are interested in analysing the time of response every participant have, in addition, we want to check the practicing effect on trials over the time.
5052

5153
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
5254
\section{Methods}
5355
\vspace{2mm}
5456

5557
\subsection{Study}
56-
We choose the score variable as the number of correct answers on every test
57-
58-
Study performed from 2017-05-31 to 2017-08-10
59-
60-
We expect that every person performs the test within 8 or 10 times, we found 9.479.47 on average.
61-
The number of days of the study on every participant should be 30 days, we found (9.47∗3.33=31.55)(9.47∗3.33=31.55) The expected number of trials times the expected days between every trial we can find an inference for the days.
62-
63-
Patients on MS group: 15 (65.0\%)
64-
Patients on HC group: 8 (35.0\%)
58+
\vspace{2mm}
6559

66-
Group Average Time of Response
67-
MS 3.82 Seconds
68-
HC 3.35 Seconds
60+
It's been conducted a study with 23 volunteers with different demographics, distributed on 15 Healthy individuals (65.0\%)and 8 individuals with a type of MS (35.0\%) . The Study performed from 2017-05-31 to 2017-08-10 and it's been instructed the people to respond the test through the application with restriction of time of the day.
6961

70-
Order 1 2 3 4 5 6 7 8 9
71-
MS Mult Star Plus Triangle Inf Square Hamburger Window Circle
72-
HC Plus Star Mult Triangle Inf Square Hamburger Window Circle
62+
We expect that every person performs the test within 8 or 10 times, we found that the participants actually respond 9.47 times on average alongside the study. The number of days of the study on every participant should be 30 days, we found out 31.55 the length of actual time the test was performed on every participant $(9.47∗3.33=31.55)$ The expected number of trials times the expected days between every trial we can find an inference for the days.
7363

74-
Average Number of Event per Group
75-
MS Group: 11.93 (SD 4.59)
76-
HC Group: 8.38 (SD 3.11)
64+
It's been chosen the score variable as the number of correct answers on every test, but also we recorded metrics regarding on time of response to every task in the test. We found that the average time of response on the MS group was 3.82 Seconds and 3.35 Seconds for the Healthy group.
7765

78-
Computing Intervals for every group
66+
In the study we also found the difficulty of the symbols in the test, sorted by the average time of response on each different symbol, and the order from thee most difficult (more time to response, 2.1 Seconds ), to the easiest one (the less time to response 1.6 Seconds), as follows: 1. Mult, 2. Star, 3. Plus, 4. Triangle, 5. Inf, 6. Square, 7. Hamburger, 8. Window, 9. Circle. Is important to say that this sequence is the same with MS individuals and HC group.
7967

80-
Average Score of MS Group: 46.22 (SD 7.88)
81-
SEM (Standard Error): 2.03
82-
True Score Interval:[42.2359279085235, 50.211286981849291]
68+
\subsection{Statistics}
69+
\vspace{2mm}
8370

84-
Average Score of HC Group: 52.92 (SD 8.22)
85-
SEM (Standard Error): 2.91
86-
True Score Interval:[47.223154935582976, 58.613704038776]
71+
The individuals in the study responded the test different number of times, we call every time a person make the test is an event, and if we summarize the number of event between the two groups we have for MS Group 11.93 (SD 4.59) of times that an individual make the test and 8.38 (SD 3.11) times for HC Group.
8772

88-
As we can see both distributions looks pretty Gaussian, so we might perform a test to prove that all the scores of MS patients have a normal distribution, the same with HC participants
73+
Regarding with the score variable itself, we got Average Score of MS Group: 46.22 (SD 7.88) with a Standard Error of (SEM 2.03), therefore computing intervals it's been obtained a True Score Interval as follows [42.23, 50.21]. The same calculations were performed on the Health individuals, we found an Average Score of 52.92 (SD 8.22)
74+
with a Standard Error of (SEM 2.91) making the True Score Interval over [47.22, 58.613].
8975

90-
Mean MS: 46.22 Interval: [42.2359279085235, 50.211286981849291]
91-
Mean HC: 52.92 Interval: [47.223154935582976, 58.613704038776]
92-
At this point it does not seems to be any difference in standard deviation on both groups, is clear to see an important difference between the 2 groups on average score
76+
Using the same logic we can plot the distributions of both groups, considering all the events of the participants, then we might perform a test to prove that all the scores of MS patients have a normal distribution, the same with HC participants, at this point it does not seems to be any difference in standard deviation on both groups.
9377

9478
\begin{figure}[ht]
9579
\includegraphics[width=9cm]{scores_distribution_grid.png}
@@ -98,73 +82,56 @@ \subsection{Study}
9882
\end{figure}
9983

10084
\vspace{2mm}
85+
\input{table1}
10186

102-
\subsection{Statistics}
103-
104-
\input{summary1}
105-
106-
87+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
10788
\section{Results}
10889
\vspace{2mm}
10990

11091
\subsection{Test-Restest Reliability}
92+
\vspace{2mm}
93+
94+
Benedict et al. BMC Neurology 2012 recomends a Test-Retest Reliability Assessment to be achieved by evaluatingan MS and/or healthy volunteer sample on two occasions separated by 1–3 weeks. This is thegold standard separation where the question is only test reliability, then Pearson’s correlation coefficient > 0.70 will usually be required. On this study was found an Average Days Between Test-retest: 14.7 (SD): 3.21, therefore it fits with the recomendation. We found the scores on SDMT Test-Retest Reliability as follows:
11195

112-
Average Days Between Test-retest: 14.7 (SD): 3.21
96+
Average score MS Group $0.84$ $p=0.0000849$
11397

114-
Test-Retest Reliability:
115-
MS: 0.84 p-value: 8.496653745316737e-05
116-
HC: 0.85 p-value: 0.006935033738858537
98+
Average score HC Group $0.85$ $p=0.00693$
99+
100+
The Standard Error of Measurement (SEM) was calculated for every group, therefore we obtain $SEM_{MS}=2.274$ and $SEM_{HC}=2.004$
117101

118102
\begin{figure}[ht]
119103
\includegraphics[width=8cm]{test-retest_density_grid.png}
120104
\caption{Test Re-test Density}
121105
\label{tab:density}
122106
\end{figure}
123107

124-
Is it possible to see the correlation between test and retest on both groups, also is intuitive to see that people is scorimng more on the second one, showing an effect of improving
125-
126-
Standar Error of Measurement:
127-
MS: 2.274577217805805
128-
HC: 2.004522634546915
129-
130-
The MDC, representing the magnitude of change necessary to exceed the measurement the error of two repeated measures at a specified CI was calculated for the 95% CI as:
131-
$$MDC_{95} = SEM( 1.96 )(\sqrt{2})$$
132-
133-
134-
Minimal Detectable Change:
135-
MS: 6.304806382168228
136-
HC: 5.556253267885786
137-
It means that if a measure is more than that so it's an outlier
108+
It is possible to see the correlation between test and retest on both groups, also is intuitive to see that people are scoring more on the second one, showing an effect of improving with the time. In the other hand we calculate the Minimal Detectable Change (MDC), that is representing the magnitude of change necessary to exceed the measurement the error of two repeated measures at a specified CI was calculated for the 95\% CI as:
109+
$$MDC_{95} = SEM( 1.96 )(\sqrt{2})$$. It's been found for each group $MDC_{MS}= 6.304$ and $MDC_{HC}=5.556$ which means that if a measure is more than that it's a possible outlier.
138110

111+
\subsection{Effect Size}
112+
\vspace{2mm}
139113

140-
Effect Size
114+
The first difference in means we looked for is between the two groups, MS and HC, using the $Cohen`s d-value$, compared with the results of Benedict 2016, obtained $d=1.11$ which is considered a Large difference in means (Cohen, 1988), we found $d=0.837$ which is considered Large too. After that we calculate the scores on different moment of the day, in order to see if there's a difference in performance.
141115

142-
Cohen`s d-value: 0.837
143-
Benedict 2016 d=1.11d=1.11 Large (Cohen, 1988)
144-
Orikami 2017 d=0.837d=0.837 Large (Cohen, 1988)
116+
\input{table2}
145117

146-
MS Morning: 45.7 (SD 7.61)
147-
MS Afternoon: 43.92 (SD 9.33)
148-
HC Morning: 52.36 (SD 8.82)
149-
HC Afternoon: 53.53 (SD 8.68)
150-
There's not significance difference between means, so it means that a person is going to score alike no matter the moment on the day. But in the other hand is clearly to see that respond in the morning for MS group is presumably less variance
118+
There's not significance difference between means, so it means that a person is going to score alike no matter the moment on the day. But in the other hand is clearly to see that respond in the morning for MS group is presumably less variant
151119

152-
Difference in Variance for MS: 66.52% p-value: 0.0285
120+
Difference in Variance for MS: $66.52\%$ $p=0.0285$
153121

154-
Difference in Variance for HC: 97.0% p-value: 0.466
122+
Difference in Variance for HC: $97.0\%$ $p=0.466$
155123

156-
There's an statistical significant difference between the variance between morning and afternoon on MS group, it means that people perform more constant over the time if they respond in the morning, in the other hand, it shows also a big difference for HC, but we got a big p-value, so we cannot reject the null hypothesis for HC group
124+
There is an statistical significant difference between the variance between morning and afternoon on MS group, it means that people perform more constant over the time if they respond in the morning, in the other hand, it shows also a big difference for HC, but we got a big p-value, so we cannot reject the null hypothesis for HC group
157125

126+
\subsection{Practicing Effect}
127+
\vspace{2mm}
158128

159-
Practicing Effect
160-
3.1 Effect on Median Time of Response
161-
We should find if there is statistical difference on the first median time a person performed and the last median time the person performed, we use the d-Cohen value to find an effect split on MS and HC groups
129+
The Effect on Median Time of Response it is been split on MS and HC groups trying to find a statistical difference on the first median time a person performed and the last median time the person performed, we used the d-Cohen value to find an effect.
162130

163131
Cohens d-value, First Median vs Last Median:
164132
MS: 0.754 Large
165133
HC: 0.974 Large
166134

167-
168135
Average Score of MS Group:
169136
First: 4186.53 (SD 803.87)
170137
Last: 4186.53 (SD 803.87)
@@ -175,11 +142,7 @@ \subsection{Test-Restest Reliability}
175142
Last: 3798.62 (SD 631.04)
176143
Cohens d-value: 1.037 Ttest pvalue=0.000176
177144

178-
\input{top10_fcc}
179-
180-
In order to prove validity, we need to be able to predict who is an MS person
181-
Predictive validity
182-
This is the degree to which a test accurately predicts a criterion that will occur in the future. For example, a prediction may be made on the basis of a new intelligence test, that high scorers at age 12 will be more likely to obtain university degrees several years later. If the prediction is born out then the test has predictive validity.
145+
In order to prove validity, we need to be able to predict who is an MS person called Predictive validity, it means the degree to which a test accurately predicts a criterion that will occur in the future.
183146

184147
Source Accuracy Specificity Sensitivity
185148
Akbar 2011 0.78 0.84 0.71
@@ -255,9 +218,14 @@ \section{Discussion}
255218

256219
\begin{thebibliography}{10}
257220

258-
\bibitem{c1} Daron Acemoglu and Asu Ozdaglar, Graph Theory and Social Networks, Lecture 2, MIT, September 14, 2009.
259-
\bibitem{c2} John Adrian Bondy, Graph Theory with Applications. Mathermathics: Amsterdam, 1976.
260-
\bibitem{c3} Sonal Raj, Neo4j High Performance. Packt Publishing, March 2015.
221+
\bibitem{c1} M. L\'opez-G/'ngora, L. Querol, and A. Escart/'in. A one-year follow-up study of the symbol digit modalities test (sdmt) and the paced auditory serial addition test (pasat) in relapsing-remitting multiple sclerosis: an appraisal of comparative longitudinal sensi- tivity. BMC Neurology, 15(1):1–8, 2015
222+
\bibitem{c2} N. Akbar, K. Honarmand, N. Kou, and A. Feinstein. Validity of a computerized version of the symbol digit modalities test in multiple sclerosis. Journal of Neurology, 258(3):373–379, 2011.
223+
\bibitem{c3} S. Hoffmann, M. Tittgemeyer, and D. Y. von Cramon. Cognitive impairment in multiple sclerosis. Current Opinion in Neurology, 20(3), 2007.
224+
\bibitem{c4} H. Lapshin, K. L. Lanctˆot, P. O’Connor, and A. Feinstein. Assessing the validity of a computer-generated cognitive screening instrument for patients with multiple sclerosis. Multiple Sclerosis Journal, 19(14):1905–1912, 2013.
225+
\bibitem{c5} A. Ruet, M. S. Deloire, J. Charr ́e-Morin, D. Hamel, and B. Brochet. A new comput- erised cognitive test for the detection of information processing speed impairment in multiple sclerosis. Multiple Sclerosis Journal, 2013.
226+
\bibitem{c6} B. M. Sandroff, R. E. Klaren, L. A. Pilutti, D. Dlugonski, R. H. B. Benedict, and R. W. Motl. Randomized controlled trial of physical activity, cognition, and walking in multiple sclerosis. Journal of Neurology, 261(2):363–372, 2014.
227+
\bibitem{c7} M. Moccia, R. Lanzillo, R. Palladino, K. C.-M. Chang, T. Costabile, C. Russo, A. De Rosa, A. Carotenuto, F. Sacc`a, G. T. Maniscalco, and V. Brescia Morra. Cogni- tive impairment at diagnosis predicts 10-year multiple sclerosis progression. Multiple Sclerosis Journal, 22(5):659–667, 2016.
228+
\bibitem{c8} Benedict et al. BMC Neurology 2012, 12:55 http://www.biomedcentral.com/1471-2377/12/55
261229
\end{thebibliography}
262230

263231

0 commit comments

Comments
 (0)