Remember so far all you have done with your results is perform descriptive statistics. For example, measures of central tendency and dispersion. You may have then put this into graphs and/ or tables to make reading them easier. At this point you can see if your experiment/study has worked. The descriptive data will tell you that!

What it will not tell you is if the result is statistically significant. In other words, if you did the experiment/study again what is the probability or the chance of it occurring again?

 It is a bit like throwing a coin twenty times and recording several heads and tails you get. If for example you got 9 heads and 1 tail (this is your raw data) then you performed a mean and range (this is your descriptive data)

Could you say with any certainty whether you would get that result again? Probably not! It is what we call a chance result.

Unless you perform an inferential test on the results you will never know for certain what the actual probability of your results occurring again is.

If you were a real researcher you would now choose an inferential test…….. (Sign Test, Man Whitney U, Spearman’s Rho etc.) And you do complicated statistics and you get your inferential test result!!!!! You are now finally in the process of finding out if your result is significant.

 Only is psychology, we don’t call it a result (nothing is that easy) we call it either the observed value or the calculated value. Calculated makes sense, because you should have calculated your result. Observed doesn’t. I always say I need to observe my calculated value so I remember the two terms together.

After we have been given our calculated value we need to compare it to what is called a critical table value to see if it significant (occurred by chance).

Basically, you compare two numbers: The calculated/observed value against the critical/table value.

In this lesson we're going to look at the Sign Test the only statistical test that AQA might ask you to calculate now as I just said this is the third lesson for statistical testing and so a very brief recap of the content from the first two lessons

 Levels of Measurement - e.g., what kind of data you have:

                        I.         Nominal

                       II.         Ordinal

                     III.         Interval

                     IV.         Ratio

Calculated (observed) and Critical (table) and values,

e.g., the result of your test (calculated value compared to its probability value (critical value).

Related and unrelated tests,

e.g., related tests are for groups of participants that are related to each other in some way. For example with Matched Pairs Designs, the participants are matched on a variable (s) this means they relate to the groups studied in terms of some similarities (the matched variable(s).  Related also refers to Repeated Measures Design as the groups related to each other because they are the same people doing a condition twice.

The different categories of tests:

                        I.         Tests of difference (all experiments including Quasi.

                       II.         Correlations (all correlations)

                     III.         Tests of Association (all other non-experiments, e.g., observations, surveys, case studies

The difference between parametric and non-parametric data (non-parametric data is drawn from a population without a normal distribution. Parametric is the opposite.

How to write a statement of your results. The formal procedure for rejecting or accepting your Null Hypothesis.  

 UNDERSTANDING THE SIGN TEST

The sign test compares the sizes of two groups. It is a non-parametric or “distribution-free” test, which means the test doesn’t assume the data comes from a particular distribution, like the normal distribution. The sign test is a nonparametric test that can be used to test either a claim involving matched pairs or repeated measure sets of sample data or a claim involving nominal data with two categories,

STEP ONE

Firstly, we need to know that the Sign Test is used under the following conditions

• When we're doing a test of difference rather than correlation or test of association

• When we're using nominal data which is categorical data and not ordinal or ratio.

• Finally when we are using a related design - A related design means that we either have a repeated measures or a matched pairs design in our experiment.

 AIMS

A study that looks at the impact of CBT on depression

 HYPOTHESIS: “There will be a difference in depression score before and after a six-week course of CBT.” - it's a simple basic hypothesis but it'll do the job for now

HYPOTHESIS QUESTION

1. WHAT WOULD THE NULL HYPOTHESIS BE?

 PROCEDURE OF EXPERIMENT

In our study, we took ten patients and we measured their level of depression before and after a six-week course of CBT and we gave them a depression score before and after that course of CBT

 RAW RESULTS

 Okay so in the first column, you can see the depression score before therapy and in the second column you can see the depression score after therapy - what we want to know now is whether or not the difference in those scores is significant or not - (whether any differences occurred by chance) -

QUESTION?

How could we do this?

so step one in conducting the sign test is working out the difference between the two sets of data or conditions, e.g. what is the difference between the groups before and after therapy which involves subtracting one set of data from the other.

HAVE A GO YOURSELF

1) CAN YOU SUBTRACT THE RIGHT-HAND COLUMN ( Depression score after therapy) FROM THE LEFT-HAND COLUMN (Depression score BEFORE therapy).

2) Really important, either:

• Put a plus sign before the subtracted difference if it is a bigger number

• Put a minus sign before the subtracted difference if it is a smaller number.

• Put a zero there if there is no difference between the conditions.

 Here is my version

Just a couple of pointers for you.

Firstly, you don't need the exact figure, e.g., +3 or -2 etc. All you need to calculate the Sign Test is whether the column has a plus, minus, or zero in it, the numbers themselves are irrelevant (this is because the Sign Test is pretty basic as it doesn’t deal with real numbers). Incidentally, this is why the data is nominal and not interval because we are not using the calculated differences only whether the groups improved (+) or did not (-). In other words, simple categorical data is required for nominal inclusion.

All you need to do is tally up the number of pluses, minuses, and zeros that were calculated.

okay so in this case I've given you plus two plus  minus four and so on but actually all we really need is plus plus minus

the second thing is that it doesn't matter which column is subtracted from which yes these signs will be switched around but it doesn't matter because at the end of the day you'll still have the correct answer

now just a couple of pointers for you at this stage first thing is you don't need the exact figure you just need to know whether it's a plus or a minus because the plus and the minus are essentially your signs that you're gonna need for the sign test

The second thing is that it doesn't matter which column is subtracted from which yes these signs will be switched around but it really doesn't matter because at the end of the day you'll still have the correct answer

STEP TWO

ADD UP THE TOTAL NUMBER OF PLUSES, MINUSES AND ZEROS.

In our case, we have five pluses and three minuses

Or five minuses and three pluses if you did the column the other way around.

Now importantly where there is no difference the data can be ignored you're going to need that for later on so for all intents and purposes the participants are deleted if there is no difference in the before

STEP THREE

Find the least occurring sign. In our case, the least occurring sign is minus because it occurred only three times that means that our calculated or observed value of S is three.

Note but if we calculated with the columns switched the least occurring sign would be a plus as it occurred three times. The bottom line is that ultimately the number you are left with is always three .

Now another value that we're going to need going forward is our n value which is the amount of participants that we have now one first glance we have ten participants because it says so in the table right there but if you remember we have to discount all of those with a difference of zero

As stated before, for all intents and purposes the participants who scored zero are deleted from the study and so actually our n value is eight not ten because we have to discount the two where the difference was zero.

S = 3

n = 8

STEP FOUR

We now have to compare our calculated value to us critical value. now

The critical value can be found in the critical values table which will always be given to you in the examination.

To find the right value you have to ask yourself a series of questions

QUESTIONS TO NEEDED TO ESTABLISH THE CRITICAL TABLE VALUE

1. IS THE HYPOTHESIS ONE OR TWO-TAILED?

2. WHAT SIGNIFICANCE LEVEL WOULD YOU SET?

3. WHAT IS THE N VALUE ?

4. WHAT IS THE CALCULATED VALUE?

ANSWERS:

1. Is your test one-tailed or two-tailed?

   We know that our test is two-tailed because we're using a non-directional or 2-tailed hypothesis.

   Answer = 2 tailed

2. What significance level would you set? If you remember from previous lessons, unless you are told otherwise the significance level is always five percent.

   Answer = 0.05

3. What is the n value?

   In other words, how many participants to you have in the study?

   Answer n = 8

4. What is the calculated value?

   What value did you calculated to get your S value?

5. Answer = s = 3

   
   Based on the answers given above, look up the critical table value in the table below and write down what you think the value is.

Answer: The critical table value is 0

NOW YOU HAVE TWO NUMBERS:

THE CALCULATED VALUE AND THE CRITICAL VALUE

S= 3 AND THE CRITICAL VALUE IS 0

QUESTION

What do you do with these two numbers now? How do you find out if your results are statistically significant?

If something is statistically significant it means it did not occur by chance and that your study worked so you can accept your experimental/alternative hypothesis and reject your Null hypothesis. If it is significant at the 10% level it means that your results are more than 90 percent unlikely to be do with chance.

BUT

If something is NOT statistically significant it means it did occur by chance and that your study DID NOT WORK so you must reject your experimental/alternative hypothesis and accept your Null hypothesis. If it is not significant at the 10% level it means that your results are more than ten percent due to chance.

In the AQA exam and with most critical table values there is advice underneath each table regarding the rules for interpreting the calculated and critical values. generally speaking, In order for the ‘s’ value to be significant, you want the observed value (s) to be lower than the critical value (from the table).

Generally, if it is a test of difference, eg., The Sign Test, Man Whitney U, T tests and Wilcoxan Matched Pairs then the calculated value needs to be lower than the critical value to enable rejection of the Null hypothesis and acceptance of the experimental hypothesis. But if it is a non experiment then the calculated value needs to be higher than the critical value or Remember the important ‘r’ rule: When there is an ‘r’ in the name of the inferential test (e.g. spearman’s rho) you want the observed value to be greater than the critical value (from the table).

MORE QUESTIONS TYPICAL IN AQA EXAMINATIONS

Question 1: work out the calculated value of S from the data sets below.

2. WRITING OUT A RESULT STATEMENT

Using your set of critical/table values tables, decide whether the alternative/experimental hypothesis should be accepted or rejected and why. For the first one, complete the gaps to get the idea of how to write up your answers.

At the ------ level of significance, the critical/table value for a--------tailed test, when ---------=-------is ---------Since the observed value of ----is ------------which is -------than the critical/table value, the ---------- hypothesis can be ----------- and the .........hypothesis can be-----------------.

Example:

Sign Test, n=35, directional hypothesis at p=≤0.05, s = 27

At the 5% level of significance, the critical/table value for a 1 tailed test, when N = 35 is 12. Since the observed/calculated value of s is 27 which is more than the critical value, the experimental hypothesis can be rejected and the null hypothesis can be rejected.

Have a go yourself:

1. Sign test, directional hypothesis at p=≤ 0.01, N =17 and s = 3

2. Sign test, non-directional hypothesis at p=≤0.05, n=10, s=5

2. Psychological Research and Scientific Method January 2010 (This topic carries 35 marks).

STEM

A psychologist was interested in testing a new treatment for people with eating disorders. She put up adverts in several London clinics to recruit participants. Thirty people came forward and they were all given a structured interview by a trained therapist. . The therapist then calculated a numerical score for each participant as a measure of their current functioning. Where 50 indicated excellent, healthy functioning and zero indicated failure to function adequately. They then had therapy for six weeks. After six weeks, each participant was re-assessed using a structured interview conducted by the same trained therapist and given a new numerical score. For each participant, the psychologist calculated an whether they improved by subtracting the score at the start of the study from the score after six weeks. If there were more pluses than minuses the therapy was considered successful.

The psychologist used a statistical test to find out whether there was a significant difference in improvement between the treatment group and non-treatment groups. She found a significant difference at the 5% level for a one tailed test (p≤0.05).

QUESTION: Identify an appropriate statistical test for analysing the participants’ scores and explain why it would be a suitable test to use in this study (4 Marks) AO1 = 1 mark, AO2/3 = 3 marks

ANSWER:

Answer: One mark for identification of a suitable test. . It is likely that most candidates will identify a non-parametric test. The most appropriate test is the Sign Test. The 3 further marks are for an appropriate justification are:

• Name of test: Sign Test

• Repeated Measures Design

• Nominal data

• Test of difference

  A sign test was needed because it was a test of difference with a repeated measures design and nominal date ( the calcultion was only whether they improved or not no mathematical differences were needed.