EDUS 660#7: Survey Says…

Food for Thought-Survey

Pay attention to the news on TV, radio, news magazines and/or newspapers and online.  What types of surveys or polls made the news this past week?  What do you know about the sample used in these surveys or polls (e.g., sample size, sampling frame, target population, etc)?  What is your opinion of the sample that was used? 

June is the month for high school graduations in the US.  While my graduation was many (cough) years ago, a high school English teacher made one key point that I have never forgotten.  She said that no one would care what I thought until I attended graduate school. This remark is a rather sad and sobering thought to consider as I embarked upon my college career. Interestingly enough, an organization did exist during that time frame that wished to know what seniors thought, how they felt and what was of importance to them.

monitoring the futureMonitoring the Future, a national study of American youth seeks to understand a population in secondary and college settings as well as those considered young adults.  Since its inception in 1975 surveying 8th-grade students, it has grown to encompass those at the end of high school as well as to  conduct follow-up surveys with respondents who participated in previous studies. The end of high school represents an important milestone in development for students. The research team selected this age as it is a logical place in which to consider how the influence of public education along with living in a parental setting may affect the attitudes and behaviors of students.

Content Areas and Questionnaire Design-A significant portion of the survey focuses on substance use. Respondents do not view the questionnaire, according to the report, as being a “drug use study.”  Different questionnaires are distributed to the participants.  The sequence is ordered which produces subsamples that are virtually identical.  The core or common variables for each of the six forms comprises one-third of the questionnaire. Researchers are able to link the core set of measures with the demographic measures statistically to all of the other measures.  Representativeness and Validity-The samples for this study should represent high school students in the 48 coterminous states.  Those students who drop out before the end of the senior year are missing from the cohort. The research team identified four (4) ways in which the survey data may not be fully accurate.  Some schools refused to participate. 100 % participation was not fully achieved of students sampled which may cause bias.  The validity of the survey could be questioned if participants made conscious and unconscious distortions when responding to a question. The accuracy of estimates could possibly mean that there are limitations in sample size and or design.  When schools refused to participate, a replacement school was located. Replacement schools were selected to match geographic area, urbanicity etc. This is a two-year study with two data collections. (2012:23)

Measurement-attitudes regarding those that one would expect to find in a survey of high school seniors: drug use, alcohol consumption, cigarette usage are surveyed.There are twenty different categories in all. Included within the twenty is “Happiness!”

Closed-ended questions varied in format. For example,

Imagine being asked to consider how happy you are as a senior?  Is the rest of the world as interested or consummed by the notion of happiness?
Imagine being asked to consider how happy you are as a senior? Is the rest of the world as interested or consumed by the notion of happiness?

Happiness: Taking all things together, how would you say things are these days-would you say you’re very happy, pretty happy, or not too happy these days? (Bachman, Johnston, and O’Malley, 2012:

3. Very happy  2.  Pretty happy  1.  Not too happy

Babbie suggests that survey questions be clear for the respondent (2013: 250). The question written above is one that I would expect a high school respondent able and willing to answer. The verbiage reflects the manner in which students would speak to each other.

Finding purpose and meaning in my life

1.  Not important  2.  Somewhat important  3.  Quite important  4.  Extremely important.

This is an example of a question that should be relevant to a student concluding high school.  The results of this type of question are useful for social scientists when contemplating the values of future generations (Babbie 2013:252).

Contingency Questions provided written suggestions.  For example,

Have you ever smoked cigarettes?    1.  Never-GO TO Q B006.

Procedures for Protecting Confidentiality: The theme of confidentiality and voluntary participation is described to participants in a descriptive flyer. At the start of the questionnaire, the administrator reinforces this component as many of the questions, particularly in the section concerning drug use are very personal. Teachers are discouraged from walking around the classroom while the survey is being completed. Participants are told to leave blanks where there may be a question that is objectionable for any reason. Names and addresses or respondents, while coded, are not able to be traced to the participant.  Of interest is the fact that the research team indicates where the data is stored (University of Michigan) and that a summary of the findings is mailed to the participants.  Wow!  I’ve participated in numerous studies and have yet to receive a summary of the findings even after requesting one.  Some students receive follow-up questionnaires in the future.

While the study that I located doesn’t fully fit the suggestion for this week’s food for thought, I did find it extremely useful to me. My task is to complete the Sampling Worksheet for my research group’s project. One component of our sampling is to collect data from preteen students.  The Monitoring the Future Survey provides several formats to consider when constructing the questionnaire for Group 13’s research project. In my opinion, the instance to survey high school students certainly provides social scientists with data that has a far-reaching impact for society.

Something to Chew on-

  • As a high school senior, how candid would you be when responding to questions about drug and alcohol usage?
  • This questionnaire is administered during school hours.  If given the chance to complete this survey in private, would your responses be different than those completed during the school day?
  • How influential were your friends at this time in determining your beliefs and attitudes?

Sources:

Babbie, E. (2015). The practice of social research. Cengage Learning.

 Monitoring the Future Publications

Johnston, L. D., Bachman, J. G., O’Malley, P. M., & Schulenberg, J. E. (2012). Monitoring the Future: A Continuing Study of American Youth (12th-Grade Survey), ICPSR34409-v2. Ann Arbor, MI: Inter-university Consortium for Political and Social Research [distributor], 11-20.

Advertisements

4 thoughts on “EDUS 660#7: Survey Says…

  1. I believe back when I was in high school and completing a questionnaire on school grounds I would be more conservative in my answers about drug and alcohol usage. I think if I participated in a mail-in survey I would tend to be more honest because the chance of the researches identifying me are rare. Participating in school there is more of a chance for risk. At the time, my friends were a huge influence in my beliefs and attitudes. Peer pressure plays a huge role during the high school years and self-esteem tends to be low. If I were to answer those same questions today, as a 40 something, I think I would be more honest because as you mature you are more willing to take risks and are less influenced by others around you.

    Like

    1. Quite frankly, I think that such a lengthy survey would be a little overwhelming. Students today are quite accustomed to completing tests online. I wonder if this organization will ever place the survey online? I would probably feel very self conscious and wonder if I was taking longer in answering my survey than others. What was asked in this survey was deeply personal, but most certainly helps to present a clear picture about where students stand in their thinking and feelings about a wide variety of topics.
      When I go to conferences now and a survey is administered to provide feedback, many individuals very quickly respond to surveys.

      Like

  2. This is a very well known study in social research circles. The data from all of the years of administration are freely available to VCU students by virtue of our membership in the Inter-University Consortium for Political and Social Research (https://www.icpsr.umich.edu/icpsrweb/landing.jsp). It would be interesting to read sampling details across the 20 years of the study. Broadly speaking, it’s a multi-stage cluster sample involving sampling geographic areas first, then schools within these geographic areas, then students within these schools who are seniors. As you point out though, there are always issues with participation rates, so adjustments sometimes have to be made. I’m glad that the study gave you some measurement ideas for your project!

    Like

    1. I’m glad to hear that this study is well-known. Thank you for providing the link. I’m sure that my group will look at the structure of the questions and the different styles when creating a survey for pre-teens.

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s