| Media homepage | Lester B Pearson College | IB Theory of Knowledge | Active Citizenship |
| Finding News | Evaluating News Critically | Considering Ethical Issues | Taking Action |
1. THE SURVEY QUESTION:
• According to what criteria are questions selected to be asked? Is the reason for the selection apparent?
• Are the terms precise? Will the respondent be likely to understand the words as the
questioner intended them?
• Do questions come with answers already implied? Could the respondent guess what
the "right answer" would be from the point of view of the questioner?
• Is the respondent asked to respond in qualitative or quantitative terms? Do any questions ask for quantitative answers to qualitative questions?
2. THE SAMPLING TECHNIQUE:
• Is the sample large enough for any meaningful conclusion? How large is "large enough"? (What is the "problem of induction"?)
• Is the sample representative of the population, or is it biased toward any sub-group of the population? e.g. door-to-door survey cuts out people not home at a given time, telephone surveys cut out people without telephones, postal questionnaires cut out people whose addresses have changed
• Is there any possibility of self-selection in respondents? e.g. People who feel successful may be more likely to reply to an alumni survey.
3. THE RELIABILITY OF THE RESPONDENT
• Can you know that the answers are accurate? Do the answers depend on estimate or on memory?
• Can you know that the respondent has enough background to answer questions in an informed way? In an opinion poll, can you know that the respondent has an opinion?
• Is it possible that the respondent might have lied -- to protect privacy, to play a joke, or to satisfy some other personal motive?
• Is it possible that the respondent might have given answers which are rather flattering to himself or herself? On the other hand, might he or she have been very modest?
4. THE CONCLUSIONS
• Average: Is it a mean, a median, or a mode?
• Do statistical co-relations necessarily imply causal connections?
• Is the difference cited between two figures significant enough to justify the conclusion?
• Do conclusions ever go beyond what is questioned in the survey? Is there any element of speculation, such as what might have been the case if circumstances had been different?
5. REPRESENTATIONS OF THE CONCLUSIONS
• Is any bias evident in the language of the conclusions?
• Is any bias evident in visual representations?
e.g. truncated graphs; pictorial graphs inconsistently comparing line to area, or area to volume; pictures used for associations or values
• Are numbers used to impress as much as to inform?
e.g. exact percentages can give an air of precision to approximate conclusions;
figures only loosely associated with the conclusion can give a ring of authority.
home
Critical Thinking Skills main page (with readings on media
Should I believe it?: A Guide to Evaluation
bias and manipulation of language
bias and manipulation of photographs
bias and manipulation of statistics
bias and manipulation of maps
logical fallacies