Social Networking studies lack scientific diligence

While listening to CBC Radio this morning @JianGhomeshi of Q was talking about a recently released study that stated that Facebook users are narcissistic. The study was picked up by media all across Canada and the world including: The Globe and Mail, Mashable, the CBC and Yahoo to name a few.

This gets me thinking about the methodology and scientific validity of some studies around social technologies. I say some because data mining of online activities is different from subjective applications of archaic social psychology measurements on digital social technologies. This kind of study lacks the scientific diligence of something published so widely by so many voices of authority in media.

I don't want to take anything away from the author who I have a lot of respect for and know is going to be incredibly successful. Students should be encouraged to pursue innovative cyberpsychology studies.

Anything that receives as much press as this undergraduate study is an important thing in our digital culture. It is also important that as more and more studies begin to emerge about social networking we raise our level of analysis. The methodology and definitions used in studies around web technologies is often murky and can mislead people about the outcomes of those studies. This can be said of many studies.


Traditional media need to be more diligent about publishing the methodology of studies they discuss. The size of this study was only 100 voluntary participants. This sample is small and the fact they volunteered after being recruited on a University campus skews the results about social networks; a participatory and youth focused technology. How can we have a proper cross section of any group when only those on campus and willing to participate do?

As an example from higher education marketing a large institution in the US released a statistic that is much used by those advocating higher education marketing on social networks. That stat is:

99% of prospective students have a social networking profile.

I agree that social networks are the place for educational institutions to manage relationships with the next generations more effectively. 99% is a powerful stat so I dug into the methodology used by this large American Institution. There were a number of things that should have been published along with that stat.

Those who responded to the survey had already opted-in to participate in surveys via the internet with this institution. They were then sent an email inviting them to participate in an online survey. Those who participated clicked the link in the email and landed on a questionnaire asking them a number of questions about their use of social networks.

These steps all skewed the survey results. Much like the Canadian government intentions of changing the long form census from mandatory to voluntary the fact that people opt-in to take the surveys skews things immediately. In this case participants have to be technologically inclined and willing to take multiple actions with no obvious benefit to themselves: something most humans have little interest in [Disclosure: I fill out surveys all the time].

photo by: Kris Krug. remixed under creative commons license.
I don't know about you but I would think that 99% of people who took each of these participatory steps have an online social networking profile. It's this kind of methodology that hinders academic study of complex social technologies.
A new study saw participants singling out narcissists just by looking at their Facebook profiles. The study found that users with an abundance of friends, wall posts and attractive (usually sexual) photos often qualified as narcissists.
This sounds like something from the most recent study, right? This study was conducted by the University of Georgia in 2008. It states that people who have many pictures of themselves and lots of friends are narcissists. To me what makes these studies about Facebook users' narcissistic tendencies problematic is that they are putting the cart before the horse. Facebook doesn't make people narcissistic. It enables them to act out their narcissistic tendencies. It even encourages them to do so by its very nature. Even the average Facebook user doesn't need a survey to see this. It's plainly obvious and baked-in to the Facebook ecosystem and culture.

photo by: D'ashley Wilson. remixed under creative commons license.
In the most recent study [PDF] on Facebook narcissism and self-esteem the sample size was 100 individuals recruited from an Ontario university campus, aged between 18 and 25, 50% men, 50% women. The participants then had their Facebook pages 'rated' by the author of the study, a 22 year-old female undergraduate. Students were recruited on campus by being approached and asked to participate in a study exploring the use of Facebook. Wouldn't narcissistic Facebook users be more interested in participating than passive users or non-narcissistic users?

The study's author used the NPI or Narcissistic Personality Inventory metric to judge the levels of narcissism in the study's participants. The more comprehensive NPI measure, a 40-item forced-choice version, was passed over in favour of a "shorter unidimensional measure" the 16-item forced-choice questionnaire. Example items include 'I am more capable than other people' and 'There is a lot I can learn from other people'.

Although it was designed to measure these features in the general population the NPI measure is based on DSM-III (Diagnostic and Statistical Manual of Mental Disorders) clinical criteria and was created in the late 1980s. This measure was not designed for use in a digital world. People who have high NPI scores are said to 'value material things and like looking at themselves in the mirror'. Back to the new study:

"Five features of the participants Facebook pages were coded for the extent to which they were self-promoting. Self -promotion was distinguished as any descriptive or visual information that appeared to attempt to persuade others about one's own positive qualities. For example, facial expression (e.g., striking a pose or making a face)... The use of positive adjectives (e.g., nice, sexy, funny)".
Seriously? Striking a pose or making a face in a Facebook photo is 'self-promoting'? It's self-promotional to call yourself nice, funny or sexy? It's a struggle to understand how the judgement of the rater a 22 year-old undergraduate student can be used to report scientific results. The model is anything but objective and that's the challenge here. Studies of social networks need to use statistical information as opposed to objective measurement. Most social networking studies want to compare the way people act in digital social spaces to the way we act in real life and draw direct correlations. Digital social spaces were designed to enable self promotional activities.

The results of this study were then used by news organizations across North America and the world. This is the embarrassing part for the media. At this time media are so enamored with social technologies that they aren't conducting the critical analysis necessary for high quality journalism. Something that will preserve journalism and remain one of great value to the world. The media's current fear is driving reporting on sup-par quality information and stories.

This was shown by the balloon boy incident and most recently the worldwide exposure given to a christian cleric from the southern US who intended to burn the Koran on September 11th but recently said he wouldn't. He should never have had worldwide exposure. Less than 50 people listen to him weekly but the media gave him the opportunity to reach billions with his vitriolic intentions.

The social digital cultural revolution is real and its different than our offline social and cultural lives in terms of tempo and medium but it is still a social and cultural revolution.

"Instead of  [being] revealing, I think it just gives us a chance to edit ourselves and, in that way, conceal the real self. Facebook profiles are about the persona more so than the person ... Narcissism and voyeurism feed off each other in this case." - Ms. Sarah Nicole Prickett, 25.

According to this thorough collection of studies and insights the narcissism isn't due to the tools, like Facebook, but more from the way we are raising children these days.

"We need to stop endlessly repeating 'You're special' and having children repeat that back," said Professor Jean Twenge, author of Generation: Me and Living in the Age of Entitlement: The Narcissism Epidemic. "Kids are self-centered enough already," says Twenge.

Twenge and co-author Keith Campbell describe their study as the largest ever of its type and say students' NPI (Narcissism Personality Index) scores have risen steadily since the current test was introduced in 1982. By 2006, they said, two-thirds of the students had above average NPI scores, 30 % more than in 1982.

1 comment:

  1. The study that says facebook users are narcissistic is about as groundbreaking and useful as the one that says that people are happier on weekends. #FiledUnderNoShit

    ReplyDelete