Jump to content

Statistics help wanted! Help me finish my masters! Anyone into this stuff?


Polytrix
 Share

Recommended Posts

Just tying my luck here! Anyone really good at research on statistics? I'm looking for some help to help me calculate effect sizes or something comparable so I can properly compare the results of 12 selected research papers.

 

My issue is that I've got comparable effect sizes for 4/12 (a good thing) without needing to calculate them myself from the available data in the articles but..the remaining 8 either don't present enough data for me to calculate effect sizes or use another system, namely McNemar test.

 

I'm horrendous at using SPSS and don't really know where to go from here but unless I present results in one universal way, I don't know how I can interpret things.

 

Thanks!

Link to comment
Share on other sites

Thank you! Will do.

Yeah, it's speech and language therapy single case studies comparing a certain type of treatment efficacy.  So at its simplest it's pre/post treatment scores as per some form of assessment. 

Link to comment
Share on other sites

12 minutes ago, Polytrix said:

Yeah, it's speech and language therapy single case studies comparing a certain type of treatment efficacy.  So at its simplest it's pre/post treatment scores as per some form of assessment. 

ok, that's way off from what i'm familiar with, but do drop a link to one of those papers.

spss is dogshit btw. if you're not too invested in it already and if your environment doesn't force you to, do switch to something sensible like stata (proprietary but very neat and complete package) or R if you plan to go balls deep into advanced quantitative stuff.

Link to comment
Share on other sites

I'm writing the methods section to a systematic review and I want to mention my approach to comparing Treatment Outcomes and Clinical Efficacy.

The title of the systematic review is: A systematic review into the efficacy of interventions utilising semantics in the treatment of bilingual Aphasia.

So basically I'm reporting on 12 selected research studies and the efficacy of a specific intervention type as mentioned in these studies.

4/12 articles report effect sizes which I can compare and interpret from established criteria but the remaining 8 either report McNemar X2/report ANOVA/T-test results/simply report pre/post intervention % score differences.

My thought at this stage is to try and use the data available in the non-effect-size reporting articles/convert other measures so I have 12 effect sizes to compare...problem is, I have no idea how to do this and don't know if I have sufficient data available to me to even do this as the research papers when not reporting effect sizes don't give me mean data in two groups and don't provide me with standard deviation scores. So I don't have data sets to work with.

What should I do?

 

Posted that elsewhere, sorry for white text situation!! ?

Link to comment
Share on other sites

Thanks man. Just for further information, I'm following the methods section of two similar systematic reviews (extracts below) so my thinking is kind of based on what they've detailed as to be logical when facing this challenge (i.e. different research papers reporting efficacy in different statistical ways): 

 

So one article giving me good data in an effectsize way for what I'm interested in would be this: 

 

https://www.researchgate.net/profile/Swathi_Kiran3/publication/240040890_Semantic_feature_analysis_treatment_in_Spanish-English_and_French-English_bilingual_aphasia/links/55ece3e108ae21d099c743a7/Semantic-feature-analysis-treatment-in-Spanish-English-and-French-English-bilingual-aphasia.pdf

 

Then there are a selection of articles giving what I think to be effect size data in a different way (McNemar), like this one: 

https://www.researchgate.net/profile/Monica_Norvik/publication/318740877_Cross-linguistic_transfer_effects_of_verb-production_therapy_in_two_cases_of_multilingual_aphasia/links/5d132996299bf1547c7f58df/Cross-linguistic-transfer-effects-of-verb-production-therapy-in-two-cases-of-multilingual-aphasia.pdf

 

And then there are less useful articles reporting efficacy in ways like this:

https://www.researchgate.net/profile/Lisa_Edmonds/publication/6878417_Effect_of_Semantic_Naming_Treatment_on_Crosslinguistic_Generalization_in_Bilingual_Aphasia/links/09e4150b65ea2decb3000000/Effect-of-Semantic-Naming-Treatment-on-Crosslinguistic-Generalization-in-Bilingual-Aphasia.pdf

 

1 - Each study was examined for the question(s) which it addressed and relevant pre- and post-therapy data were extracted.We computed statistical significance for the pre- and post- treatment scores using the McNemar’s change test (p < 0.05, Seigel & Castellan,1988).3 There were two primary reasons for performing the statistical computations. Some studies failed to report any statistical measure (e.g., Faroqi & Chengappa, 1996; Gil & Goral, 2004; Khamis, Venkert-Olenik, & Gil, 1996). A few other studies reported parametric statistical tests whose assumptions of normality and independence were not met by the study design and data. McNemar’s change test is a non-parametric test for paired nominal measures such as accuracy data, and has been used by several aphasiologists to compute statistical significance of treatment-induced changes in behavioral scores (Faroqi-Shah, 2008; Rochon, Laird, Bose, & Scofield, 2005). The use of a consistent statistical measure makes comparisons of statistical significance across studies more valid.

2 - As well as describing the treatment outcomes of included studies, the clinical efficacy of SFA was determined by calculating effect sizes. Effect sizes could be calculated only in those studies that reported sufficient data. To calculate, it was necessary to determine the individual values for the pretreatment and posttreatment phases for each set of trained items. Cohen’s d statistic was used to calculate effect size as described by Busk and Serlin (1992). The magnitude of change in performance was determined according to the benchmarks for lexical retrieval studies described by Beeson and Robey (2006). The benchmarks were 4.0, 7.0, and 10.1 for small, medium, and large effect sizes, respectively. Where Cohen’s d could not be calculated, the percent of nonoverlapping data (PND) was calculated. PND is the most widely used method of calculating effect size in single case experimental designs (Gast, 2010; Schlosser, Lee, & Wendt, 2008). PND is the percentage of Phase B data points (the treatment phase) that do not overlap with Phase A data points (baseline or no treatment). To determine the magnitude of effect, benchmarks put forth by Scruggs, Mastropieri, and Casto (1987) were used. PND scores higher than 90% were considered to demonstrate a highly effective treatment, PND of 70%–90% were interpreted as a moderate treatment outcome, and PND scores of 50%–70% were considered a questionable effect. PND scores less than 50% were interpreted as an ineffective intervention because performance during intervention had not affected behavior beyond baseline performance.

Link to comment
Share on other sites

yeah, this is way beyond me bro, never dealt with something like that. though i do i see that both the first article you linked and the third one (which doesn't bother with effect size metric at all) report the post treatment accuracy scores in % and whether the treatment was significant, so isn't it enough for proper comparison?

Link to comment
Share on other sites

I suppose I could go off significance values yes but that's not really a measure of efficacy I think. 

 

I want to be able to say that the approach taken in article X/Y/Z was more less effective than the approach taken in article XYZ.

 

Not sure maybe I'm just getting confused about al this. Cheers all the same dude.

Link to comment
Share on other sites

How can a report not publish effect size? Isn't that like the main thing one wants to put out there? It's even secondary to the significance score.

Anyway, one should be able to infer the effect size if you know all of the pvalue, sample size and variance (if applicable)

Link to comment
Share on other sites

Haha go for it.

I think it's because these are experimental single case studies and aren't always trying to be super analytical. Just makes life harder for me tho :( thanks for help people. I plan to calculate effect sizes when I can from the data or just use what I've got in a more general way.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...