“A valid test is always reliable but a reliable test is not necessarily valid”

Reliability and validity are two important characteristics of any measurement procedure.

Reliability has been defined as ‘the extent to which results are consistent over time… and if the results of a study can be reproduced under a similar methodology, then the research instrument is considered to be reliable.’ (Joppe 2000). This means a test is considered reliable if the same results are produced repeatedly, if it were to be carried out again. The more consistent the results produced, the higher the reliability of the measurement procedure.

Reliability is addressed in a variety of ways. These include:  

  1. Inter-Rater Reliability – is the variation in measurements taken by different people using same methods. In order to ensure reliability, the degree of variation must be small.
  2. Test-Retest Reliability – is established by comparing scores of the same individual, to calculate a correlation. There must be a strong correlation to ensure reliability.
  3. Split-Half Reliability – is obtained by dividing up the test into two comparable halves, in order to calculate consistency between the two scores. Consistency must be present in order for the test to be considered reliable.

Despite having methods to ensure reliability, there are issues which arise which can affect the reliability of the test and results. The researchers are human, and that means the experiment is open to human judgement and error. However, this can be solved by carefully reporting methodology in the study and, if using qualitative methods, double coding.

These solutions make reliability much easier to assess.

Validity, on the other hand, determines whether the research truly measures what it intended to measure, (Joppe, 2000).  This means it looks at the extent to which a test measures what it claims to measure, and therefore, answers the research question or hypothesis. How valid a test is, depends on the purpose of the research.

Validity is addressed in a variety of ways, and include:

  1. Content (face) Validity – is the degree to which a test measures an intended content area. It must measure what it claims to measure, in order to be considered valid.
  2. Concurrent Validity- measures the extent to which a correlation exists between a new measure and a standard measurement procedure.  The scores should be directly related in order to obtain validity.
  3. Convergent Validity – is the degree to which scores obtained from two different methods of measures. There must be a strong relationship, for validity to be demonstrated.

Despite having methods in place to ensure validity, there are threats. There are two main threats: experimenter bias and demand characteristics. Again, the researcher is human and this means the study will always be open to human error. The experimenter may influence the outcome of the research because of his/hers expectations regarding the results. This however, can be solved through the use of single blind or double blind studies, in which the researcher has no idea what the predicted outcome is. Again, the participants are human, and this can lead to the problem of demand characteristics, in which participants behave in a different way. Participants normally modify their behaviour in response to the fact they are participating in a study and are aware they are being measured. They strive to be a good participant. Although it is essentially impossible to prevent participants from modifying their behaviour, there are methods in place which can reduce this effect. Solutions include: using observations or concealing the measurement procedure.

Despite being very different, both reliability and validity are important in research.  As the saying goes ‘a valid test is always reliable but a reliable test is not necessarily valid’, but it is important to ensure that both reliability and validity are demonstrated.

About these ads
This entry was posted in Uncategorized. Bookmark the permalink.

9 Responses to “A valid test is always reliable but a reliable test is not necessarily valid”

  1. raw2392 says:

    Really good blog, you can tell you have a great understanding of reliability and validity and have clearly done some good research into these areas before writing this blog.
    Validity and reliability are both important when it comes to research and the output this research produces, if a test is not valid then the results that have come from it can not be trusted. Validity is essential for the results to be taken seriously. Like you have quoted ‘a valid test is always reliable but a reliable test is not necessarily valid’.
    Reliability is also important for a test to be repeated later on by different researchers and the same results to be found. This is important so that a test can be retested to see if the research can be published.
    A brilliant blog though and really explained the two concepts well! :)

  2. Pingback: Week 11 comments for TA | raw2392

  3. psud6e says:

    I think this blog was very good, with a clear understanding of reliability and validity – so much so, you were able to put it in simple terms I think the majority of people will be able to understand. Today, we seem to find that the reliability of research in psychology is only every tested by new researchers investigating a topic. When a researcher completes an experiment, and has gathered all their data, they don’t run the experiment again to look for the same results to test reliability; instead they compare it to other research in the same field. If other researchers support their findings, then the research can be assumed to be reliable. In the same way, reliability of ground-breaking research – stuff completely new to the field – is only found when another experiment in the same field is run.
    Validity, however, is a harder concept to check. To be honest, can we really be sure something is 100% reliable? We often use hypothetical constructs, especially in psychology, whereby we assume something means another thing. For example, in personality questionnaires, we ask questions that we assume are related to the personality type we are trying to measure.
    I definitely agree with you that reliability and validity are both important in testing, and a good example of this is in medicine. We want the drug trial to be reliable, because then we know that the drug is safe to use when it is manufactured, the next stage after the trialling process. We also want the trial to be valid. This means that we want it to test the effects of the drug. If it didn’t do this, we may not be able to see and measure the side effects, and therefore a drug that is actually dangerous may be manufactured. Therefore, reliability and validity are definitely important.

  4. Pingback: Week 10 and 11 Blog Comments | psud6e

  5. katrinspen says:

    Reblogged this on katrinspen and commented:
    This blog is so helpful on my research about Validity and Reliability in terms of Assessing Student’s Learning. :)

  6. Johnk576 says:

    Thank you for another informative website. Where else could I get that kind of info written in such a perfect way? I’ve a project that I am just now working on, and I have been on the look out for such information. cgcdcdbgckde

  7. teacher_inthemaking says:

    Thank you for this. I will have an exam later about Validity and Reliability and your blog gave a very comprehensive explanation about their importance. Indeed, “a valid test is always reliable but not all reliable test are valid.”

  8. It was a huge help. Thank you. :)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s