Guidelines for Constructing Research Surveys

Introduction

Some of you will do research through surveys in your thesis. Whether your questionnaire is in English of Japanese, there are several important factors you must consider when constructing your survey. Failure to follow these guidelines may result in a questionnaire with inaccurate results which will make it difficult to draw conclusions with any certainty.

Assignment

Decide on the topic for your survey and write your questions, following the guidelines numbered below.

Give your survey to other members of this class or to friends who understand enough English to respond. Sit with them and ask about places in the survey where they are confused or which appear unclear.

1. Be unambiguous. If respondents (people who are responding to your survey) can interpret a question differently from the meaning you have in mind, they will. For example, even a simple question such as, "How often do you visit a doctor?" can be very unclear. Are native healers, herbalists, acupuncturists, chiropractors, or public health clinics with nurses only "doctors"?

2. Use a level of vocabulary that your respondents will understand, but don't talk down to them. This is difficult if the people you are interviewing are of different educational levels.

3. Remember your respondents must know enough to be able to answer your questions. For example, if you ask an Asian foreigner living in Japan, "How many times have you been discriminated against in the past year?" they may not be able to remember how many times. However, if you ask, "Have you ever been discriminated against?" they will likely be able to answer yes or no.

4. Try to make your questionnaire look well planned. Group your questions logically and don't jump back to a previously covered topic for no apparent reason. You may annoy and switch off your respondents. Use a transition sentence when you move from subject to subject in your questionnaire. For example, "The previous section looked at English education in primary school. The next section is about junior high school."

5. Pay attention to contingencies and filter questions. Many questions contain several contingencies. For example, "Are you married?" may be followed by "Have you ever been married?" to cover people who may be divorced or widowed. You may want to make a contingency flow chart to map out the possibilities.

6. Use clear scales. Commonly used scales include: Strongly Agree,

Agree, Neutral, Disagree, Strongly Disagree. Try to give your respondents at least five choices to allow them to answer as accurately as possible. In addition, explain the meaning to reduce the chance of misunderstanding. Another form which allows respondents to show their feelings accurately. For example:

English Education in Primary Schools

Important 1 2 3 4 5 6 7 Unimportant

7. Try to package questions in self-administered questionnaires (see fig. 12.1). Be sure to make scales unambiguous (if you are asking how often people think they do something, do not say "regularly" when you mean "more than once a month"). Limit the list of questions to seven for any one section and go on to another type of question to prevent boredom.

8. Make the possible responses to a question exhaustive and exclusive, especially if you want respondents to choose only one answer. For example, the following is an example of what not to do:

How do you evaluate communication between your section at work and other sections?

a. There is much communication ____

b. There is sufficient communication ____

c. There is little communication ____

d. There is no communication ____

e. No basis for perception ____

The problem is that it is possible to choose b. and c. and have a logical answer.

Including, "don't know" as an answer gives an option to respondents who do not know the answer to one of your questions.

9. Keep unthreatening questions short. However, questions that are likely to intimidate respondents should have a long introduction to reduce the intimidation effect. The question after the preamble should be short. For example: "Corporal punishment for children is strongly condemned by some, but strongly supported by others. Which of the following statements is closest to your own feelings?"

10. Always provide alternatives, if possible.

11. Avoid loaded questions, that is questions that presupposes an action by the respondant. Any question that begins, "Don't you agree that . . ." is a loaded question. For example, the question, "When did you stop beating your children?" assumes that the respondent has been beating his/her children.

12. Don't use double-barreled questions. For example: "When did you leave home and go to work on your own for the first time?" This question assumes that people left home at the same time they went to work on their own. This may not be true.

13. Don't put false premises into questions. For example: "Which is preferable, for women to have a career or stay home to raise children.?" In this question, the writer has presumed that there are only two possible alternatives. There may be others.

14. Don't show emotional bias in the wording of questions. For example, "Should the government ban beer machines to prevent the tragedy of teen drinking?" The inclusion of the word "tragedy" shows the bias of the person who wrote the question.

15. When asking for opinions on controversial issues, be as specific as possible about the situation you are referring to. For example, "Do you approve of gun control?" might be better worded as, "Under what conditions do you approve of gun control?"

Translation and Back Translation

If you are planning to give a survey to two different language groups in their native language, you must follow the subsequent steps to insure that the meaning of the quesitons is the same in both questionaires.

a. Write the questionaire in your native language.

b. Have a bilingual person who is a native speaker of the second language translate the questionaire into the second language. Work closely with the translator so he/she can ask you questions about your meaning.

c. Have a bilingual person who his a native speaker of your language translate the questionaire back into your language. This translation should be almost exactly the same as the original questionnaire.

Adapted from H. Russell Bernard. Research Methods in Anthropology. 2nd ed. Walnut Creek: Altamira Press, 1995.