Survey
The understanding of a questionnaire can be developed by developing the thought process of a survey. A survey is basically asking questions to a cross-sectional representative of the society about a common issue at a given period of time. A survey enables the organization, which takes the survey, that what the public opinion about a certain topic is in a given period of time.
Constructing Questionnaires
A questionnaire can be used as a generic term for basically two types of set of questions which are dealt with in the modern world; they are self-administered questionnaires and the interview schedules completed by an interviewer. A questionnaire can be mailed or handed to the respondent which is to be filled by them without the interference of the interviewer. The other type, an interview schedule, is the interviewer asking the questions to the respondent and noting down their replies. The construction of a questionnaire can be a frustrating task as the room for error is significant; the respondent may not be able to answer some of the questions as they find them irrelevant or a form of advertising.
The questions need to put correctly and a form of trust should be established between the two parties participating in the interviewing process. The respondent may at times feel that the answers they give out could be used against them which would in turn cause an invasion of privacy. The concept of insecurities may come in as the respondent might think that their lack of knowledge may show in the survey which would open them up for judgment. Some particular individuals belonging to a specific minority class could end up thinking that all these surveys are pointed at them and they are always made the guinea pigs. There are a lot of things to keep in mind when constructing a questionnaire, these are some of the problems and the solution to them is to be a variety of questions which simple and straightforward answers.
Relevance
Each questionnaire has a unique way of construction as it has an aim for a specific group of individuals with a specific way to answer. There are many ways through which a questionnaire can be made relevant, one of the ways is that the ideologies of different groups may be compared but this requires a variety in the questions that are asked to a specific group. The aim of these questions is to have questions which give out a complete answer which is of satisfaction to the person conducting the survey; the information collected through the medium of the survey should be complete, valid and reliable. An easy trick which can be used to make a questionnaire better is to limit the responses, the more limited the responses are the easier it would be to analyze the answers and carry out an analytic study of the same.
Pitfalls in a Questionnaire
Every questionnaire has its shortcomings, below mentioned are some ways to sort them into being more productive.
1. Double – barreled Questions There shouldn’t be more than one question asked as it leads to confusion when the answer is given. For instance, if one is asked “Do you pray in the church or temple?”, one could answer no if they don’t pray anywhere, but a person would answer yes if he prays in either. That leads to a lot of confusion, hence words like “or” and “and” should be avoided.
2. Ambiguous Questions No researcher or questionnaire developer wants to make their questions ambiguous, but sometimes circumstances dictate them to be. These differences produce a sense of communication problems within an area. For instance, using slangs for a questionnaire for the older generation is going to be difficult as they won’t understand what the survey wants to ask them, hence leading to misleading answers.
3. Level of Wording It is very essential for an interviewer to phrase questions in such a way through which they are easily understood and the answers can be prompt. This is to ensure that the largest spectrum of people may be associated with the interview and that it could be easier for both the parties involved. The interviewer should also keep in mind to make themselves clear at all times as repetition causes frustration and a loss of funding as the time taken towards the study is increased significantly. Through this method the ambiguity of the survey is also masked to some extent. There still has been debate as some researchers feel that the interviewer should phrase their questions in the terms of slangs towards the audience to which it is intended to as to reduce the gap of communication.
4. Abstract vs. Factual Questions The questions asked should not be subjective to such an extent to where the answers become redundant. The questions should be phrased in way through which the answer is given in facts or figures as it is easier to analyze and the credibility question is taken out of play. Opinion based questions as very tricky as they make the respondent a bit insecure about their opinions and how the society would judge them on the basis of that.
5. Leading Questions A question should always be framed and asked in a very neutral form in order to be presented with an unbiased answer. If the question is not phrased carefully then it may lead to the artificial increase in probability of a particular response.
6. Sensitive or Threatening Questions This is probably the biggest threat for an interviewer as they don’t want to offend a respondent but sometimes their tone or the language which they use in the questionnaire corresponds to a different way of approaching the respondents reply. There could be situations where the respondent doesn’t want to answer but the interviewer is adamant on getting an answer, it is well advised to rephrase or step away from the respondent as it is a survey and not an interrogation.
Open and Closed Response Categories
Since, the wording part of the questionnaire is done; the interviewer needs to assess the answers given to the questions in the questionnaire. There are two major distinctions which are made on the basis of the answers which are received, i.e. open-ended questions and closed-ended questions.
1. Closed Ended Questions The responder has to select one or more of the specific categories provided by the researcher. The major advantage to this type of question is that the answer is always given in simple terms and the factor of redundancy is decreased. These answers can easily be compared and the process of analyzing them becomes easier and the results or findings can be mapped out easily. With the option of categories given to the respondent, it becomes easier for them to just pick one, now this could be a correct answer or an estimate, but it provides the interviewer with some information. The respondent needs to be very cooperative when dealing with such questions in order to give out substantial information. The disadvantages also have a substantial say in the process as it is a very time consuming and frustrating process to come with so many options in order for the respondent to answer. The factor of redundancy sometimes increases as some options may not suit the response of the interviewee which may lead to miscalculations when analyzing the data. There could be a lot of variations in the answers as the options might not be suitable for the response and may be interpreted differently. A major disadvantage may be that the respondent’s views may not align with the interviewer’s options and hence there could be conflict of interest making the interview less credible for research purposes.
2. Open Ended Questions The answers given through the open ended questions provide a clearer picture of the problem and the possible solutions which can be taken into the account regarding the problem. They provide the interviewee with more options regarding the questions asked by the interviewer, hence giving rise to a larger spectrum of solutions or data regarding the questions which were raised. It provides a summary regarding the answers given by the public regarding an issue, if one or two answers are omitted, it doesn’t cause too much of a nuisance. The major disadvantage regarding these questions are that the process is very time consuming and it can easily become a financial burden for the interviewer as people may have a lot of things to say regarding an issue. The problem of ambiguity with this type of questions is very prominent as there can be many answers to a given question by the people who have given the answers. Another issue is the large number of data which is gathered and needs to be processed; this data is very varied throughout the questionnaire.
Below mentioned are some of the types of closed ended question formats.
Question Order
Below mentioned are so rules through which one decides how many questions to put in a questionnaire and in what order to put them in.
1. The sensitive questions should be kept towards the end of the questionnaire so a trust is established with the respondent; hence the probability of an answer increases.
2. The easy to answer questions should be asked first so as to give the respondent confidence; this also increases the probability of the respondent answering more questions.
3. The information regarding subsequent questions should be gathered first so as to establish a stand for the questions to follow regarding the same topic.
4. The questions should be placed in a very logical manner, so as to make sense for the respondent.
5. The establishment of a response set should be avoided, this regards to the fact that if the respondent has a same thread of responses then the questions should be randomized as to get a variety of answers from the respondent.
6. There should a selection of questions to which answers are important; these questions should be kept in the opening part of the questionnaire.
Contingency Questions and Filter Questions
It is very essential for the interviewer to analyze their interviewee’s stance regarding a topic and only ask them the important questions. The contingency questions act as somewhat as filters in the questionnaire and take out the unnecessary details as to make the process less frustrating and time consuming for both the parties. For example, if the first question is “Do you believe in God?” and the answer is “No.” and the next question was supposed to be “What religion do you follow?” then the latter question may be avoided. Filter questions are also essential as they provide more insight on the answers given on certain questions. For instance, if the question is regarding abortions, then the interviewer might as the respondent to explain their stance in a more explanatory manner.
Instructions for Interviewers or Respondents
The instructions are usually enclosed in a document containing a questionnaire when it is mailed out and explained thoroughly when the interview is conducted. There are specific instructions for every question; hence it becomes a task to tell the respondent about all the questions. The filter and contingency questions need to accounted for and special instructions are given out for them for the questionnaire to function more smoothly. The questionnaire, when mailed, can’t have rules or instructions regarding all the questions as it all go unaccounted for. The face sheet also plays an important role as it keeps the administrative details of the interview intact, it answers questions such as “was the interview successful?” or “how long was the interview?”. There is also a final page which is filled by the interviewer without the knowledge of the respondent, regarding the respondent’s behavior and the answers they gave.
Pretesting
It is regarded as the final step of the questionnaire building process, it is used for feedback regarding the questions and is also typed in such a manner through which commenting on it may be made successful. The pretest is given the same regard as a final questionnaire and it is critically evaluated in the same manner. The researcher would make the necessary amendments regarding the comments on their pretested questionnaire. It is tested on a few respondents and further steps are taken according to the reaction of those respondents.
Chapter 7: Mailed Questionnaires
A very initial decision which is to be taken by the researcher conducting the survey is to decide whether the survey is to be done in a self-determined way or an interview study. The concept of a self-determined manner to carry out a survey is often associated with mailing it to the respondent. There are certain advantages and disadvantages regarding the mailing of questionnaires. They are as follows:
Advantages
1. It saves a huge amount of money.
2. The process saves time.
3. The questionnaire can be completed regarding the convenience of the respondent; hence there is no added pressure from the interviewer.
4. The assurance of anonymity remains; this is a welcome step for any of the respondents with a fear of judgment.
5. The factor of standardized wordings and no interviewer bias would be there.
6. The information is always secured through the mailing system of the State.
7. The information of the interview is highly accessible to the personnel wishing to access it.
Disadvantages
1. The lack of flexibility, there is no one to explain the questionnaire to the respondent; hence there could be information which gets translated differently.
2. The response rate is low as people generally assume these questionnaires to be of no use, hence they refute from answering them.
3. The aspects of the verbal behavior can only be measured and the non-verbal behavior isn’t taken into account.
4. There is no control over the order through which the questions are answered, hence disrupting the flow of the questionnaire.
5. There are a number of questions which remain unanswered during this process.
6. There is no control over the date of response.
7. A complex questionnaire can’t be processed easily through the eyes and mind of a respondent.
The Total Design Method
Dillman says that both mail and telephone surveys are considered as the “step children” of survey research and not as valuable as interview studies. He formulated something called the “Total Design Method” which enables the mail and telephonic surveys to achieve their full potential. There are two parts to the TDM; they are (a) a theoretical part, which explains the specifications of the research process and responses, (b) the practical part, which explains the detailed parts of the survey process. He also bought in the concept of social exchange, which states that a person would exhibit the same behavior as one exhibits on them. Hence, he said that the key concepts in the theoretical part of the TDM are cost, reward and trust. All the three concepts work in sync to give rise to the theoretical part of the TDM, whereas the practical may just be observance of the interviewee.
Factors Affecting Mail Surveys
1. Sponsorship of the questionnaire: The probability of getting a response increases as the organization conducting the survey is associated with a government or an even more powerful entity.
2. Attractiveness of the questionnaire: The usage of color to enhance the attractiveness of a questionnaire saw a better response as compared to a dull black and white printed questionnaire. There have been significant studies which show the impact of colors, but all have not yet been convinced.
3. Length of the questionnaire: A less cluttered format of the questionnaires receives a better response as reports a study which found out that there was a difference in the percentage of responses between a one page and a two page questionnaire. This difference was about 1.2%, which is really significant.
4. Nature of the accompanying letter requesting cooperation: There hasn’t been a study which satisfactorily proves the significance of formal or informal responses to a request letter, but the balance tips in favor of the informal response.
5. Ease of filling out the questionnaire and mailing it back: It is the job of the researcher to provide the adequate details regarding the mailing back of the questionnaire as well as making the questionnaire easy to fill by the respondent.
6. Inducements offered to reply: This is all up to the researcher on how he approaches to this problem; a common solution is to appeal to the sentimental side of the respondent and ask for the response as to “make a change in society”.
7. Nature of Respondents: The researcher needs to find out their audience and be aware as to whom exactly to send the questionnaire to. Generally, the above 90% responses are governed by the people who are targeted in groups; hence due to peer pressure a larger number of responses are gathered. There are some other factors too such as the time of mailing, the method of mailing and the time and technique as to how a follow needs to be done. These are already known by the researcher and are learnt from the help of superiors or through experience. The adequate response rate plays a key role in these surveys, as if there isn’t a good response then there is no good reason to carry out such surveys. The researcher needs to keep in mind that the organization is entrusting them with time and money hence it becomes essential for them to produce valid and reliable results. The reliability and validity of these results is essential as they are further used in studies and cited and even for the analysis of the data, it becomes essential for the team to know what sort of data they are working with.
Validity
The validity of a mailed questionnaire data is done be analyzing and comparing the findings of the questionnaire with already known facts. The rule of thumb dictates that the findings of the researcher be tested through the questionnaire and its answers, but due to the lack of funding, it becomes quite a daunting task. Studies show that it is near impossible to assess some of the data in the questionnaire as some of them may produce inconclusive results for different reasons altogether. The validity of a mailed questionnaire had only been found once when Colley found close correspondence between a questionnaire which asked which product people used the most and the sales data retrieved from the company that the product was from.
Reliability
Taking the reliability of a questionnaire is a huge task and quite impossible to do as well, the comparison is drawn between many questionnaires but these questionnaires haven’t been tested hence there is always an issue of miscalculations. The questionnaires are never consistent with their responses and thus make it hard to draw coherent conclusions. Hence, the reliability of these questionnaires are always questioned as they fail to showcase the proper findings of studies.
The Conclusion
There have been a number of aspects which have been discussed regarding questionnaires, the document provides a holistic view of what these questionnaires are supposed to look like and what purpose they should serve in order to optimize the results of a researcher. The various types are discussed with their pros and cons. Hence, the significance of a questionnaire has been discussed.
No comments:
Post a Comment