EDUS 660 #8: The Magic 8 Ball

Consider the eight sources of internal invalidity discussed. Make up an example (not in the chapter) to illustrate at least 4 of these sources of internal invalidity. Tag experimental.

ask-magic-8-ball-fortune-teller-yes-no-question-predictionGrowing up in the 1960’s, the old black and white movies were shown on the smaller networks.  The Three Stooges always provided me with an afternoon of old fashion entertainment. Their ridiculous slapstick comedy infused political satire during World War II.  I recognized the”Magic 8 Ball” in their film, You Nazty Spy, as a gadget young people enjoyed using for fun. The 3 Stooges call in Mattie Herring in to help them to understand the future.  When Mo cracks the ball on Curly’s head, a message falls out of it. Imagine that, the viewers say.

The Magic Ball, as a form of entertainment, answers questions with twenty possible choices with 10 positive, 10 negative and 5 that are neutral. There are stages within research when it would be helpful to have a tool, such as a magic ball, that would easily provide research groups with the correct method and course to choose. In my first blog, I recall stating that while I had some familiarity with research methods, I was certain that there was much that I didn’t know.

The sources of internal invalidity discussed in this podcast is an “8 Ball” for those engaged in experimental design.  

Prior to WWII, some Americans were considered Isolationists.  It may have been very difficult to imagine pursuing another war when remembering the results of “The Great War.” Using film as a device to sway public opinion is not a new tool to push a social action or agenda.  The Interview movie (2014) and Je Suis Charlie (2014) exemplify the desire to use the media to engage individuals in conversation regarding social action and concerns.  Babbie suggests that pretesting and post testing is a device used in even the simplest of designs.(2015:226). Exposing Americans to a Three Stooges film in 1940 could be a possible stimulus representing the independent variable attitudes toward intervention.  A test to measure the attitudes and beliefs of citizens prior to and after watching a satirical movie might reveal that opinions can change when viewing a film.                   (Ball #4: Pre and Post testing)

Participating in a university setting, I am made aware of research projects with regularity.  One such project was made known to me on several occasions this spring in VCU’s Daily Ram.  An investigator sought to enroll participants who were interested in mindfulness and yoga.  The stipulation for participation was that the individual needed to be involved in a romantic relationship.  Who can tell how long a romantic relationship can last?  What happens if a participant’s partner leaves the relationship voluntarily or involuntarily? Will that change affect the outcome of the research?  Will the participant be asked to cease their involvement in the study?) Good question.  (Ball #3: Maturation, an endogenous change)

Educators in public school settings know that one component of the week of work before school begins is the visitation of the previous test scores.  Ideology regarding student placement changes frequently.  Top tier, bottom tier, heterogeneous group and homogeneous grouping for content area instruction ride the wagon of schedule design. While teachers and students are unsure of where the chips will fall, what is often certain is that everyone is expected to make improvements.  When scores plateau, how does this affect the measurement tool selected by state organizations? (Okay, this isn’t an original idea on my part, however, read further Ball #4: Statistical Regression, an endogenous change). 

When the going gets tough, the game is changed.  I know of one school system where lines are redrawn, changing the composition of the school community.  In this same system, each time a school was closed for renovation, it would reopen as a themed school with a new community. Some students in this system receive a special curriculum, themed units of study, teachers who are all trained to implement special programs.  With these initiatives in place, students are still expected to complete the same standardized test procedures mandated by the state.  Where one group does well as a result of preselection, another may fail to thrive academically. Everyone is held to the same standard in the end (Ball #2: Selection of subjects, selection bias).

Many public classrooms have relented in terms of the use of social media during the school day.  It’s nearly impossible to keep some students from interacting with others during instructional time.  We had a joke in my classroom that when someone felt a buzz in their pants, they would suddenly need to use the restroom–real bad!  Translated this means that a student would head to the restroom to read and respond to a text.  While I could assume that the text was in regards to a social issue, how could I know that it was not to reveal the contents of an assessment given that day? If students share content information, do well on the assessment then it can be assumed that all is well.  There is no need to revisit this topic with the same intensity as a strand that is weak. Providing the same assessment to all students allows the administration to prepare for year-end state testing.  If students speak with each other during a state test, the test results are contaminated and the entire class may be required to retake the test. (Ball #7: Design Contamination).

Babbie reminds me that I use experiments in non-scientific inquiry every day.  Whether it is to try a new recipe, assembling a small appliance or attempting to generalize what is happening it the world around me I engage in experimentation (2015: 225). There is no “magic” method guaranteed to produce a flawless social experiment, the design method does provide the “magic” 8 concepts to consider in order to validate my work.

The Magic
The Magic “8” of Experimental Design:
1. History
2. Selection of Subjects
3. Maturation
4. Statistical Regression
5. Testing Effects
6. Experimental
7. Design
8. Instrumentation

Something to Chew on- a little bit of humor goes a long way, and travels in both negative and positive directions. At what point could it could it become be detrimental to an experiment and at which stage?  Can you recall a time when a social experiment in the real world may have failed as described by one of the “Magic Eight?


Babbie, E. (2015). The practice of social research. Cengage Learning.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s