February 12, 2005

Reading the Vanishing Newspaper, 6: Readability

In the last chapter of the "Vanishing Newspaper," I learned that reporters too dumb for sources. In Chapter 6 of Philip Meyer's new book, I find out that newspapers are too smart for their readers.

Talk about a conundrum of the damned.

Today's journalists, already the most educated crop of reporters and editors ever, need more ongoing education so they can interview with sophistication, research with understanding and report with credibility, but when it's time to write Meyer suggests they shed the sheepskins and scribble their stories at a sixth- to eighth-grade level, a range he identifies as the sweet spot of newspaper readability.

Meyer travels (as always) a statistic-laden route to identity the average newspaper reader as a junior-high schooler left behind. He parses the staff-written contents of 40 American newspapers with a common readability formula called the Flesch-Kincaid index. The index measures sentence structure and complexity of language, then gives the writing a number on a scale of 1 to 100, with 1 being "practically impossible to read" and 100 being, well, a stop sign, I suppose.

The Flesch-Kincaid tool is built into modern software like Microsoft Word, so it's easy to measure the readability of anything. Meyer offers some examples:

 Average newspaper story: 70-80 (about eighth-grade level).
 John F. Kennedy's inaugural address ("Ask not what "): 10.3.
 Patrick Henry's "Give me liberty or give me death" speech: 6.6.

What does Meyer find after crunching 2,125 stories from newspapers as diverse as the Grand Forks (North Dakota) Herald, the Philadelphia Inquirer and the Houston Chronicle? Only 25 percent of those stories could be understood by someone who reads at an eighth-grade level or lower. The other 75 percent needed an audience of people who at least struggled through high school English, including a few who opened the Borzoi Reader in college.

Meyer's conclusion:

"The case is clear. Many newspaper stories are too hard to read." (Emphasis added.)

Too hard to read? Too hard to understand because of clunky, jargon-rich writing - that I would agree with. Too boring because the content is institutional, not human? Too little useful information because the stories are under-reported and lack context? Those I would agree with also. But too hard to read because the language is too sophisticated and the sentences too complicated? I don't buy it.

I don't disagree with the underlying premise that more readable stories - stories with cleaner writing, stronger narrative and less institutional focus - are a necessary step toward continued relevancy for newspapers (more point of view, more voices from citizens and more transparency of sources are equally necessary), but I think Meyer is missing the point when he connects readability levels with readership.

Meyer creates a ranking for the 40 newspapers he surveyed based on the difference between their Flesch-Kincaid readability score and the education level of the people who live in their home counties. Those newspapers with the biggest difference - that is, those that write most below the reading level of the community - rank highest.

No. 1 on the list is the Grand Forks Herald with a reading level of 5.04 - a fifth-grade - five levels below its community; No. 4 is the Houston Chronicle with a tenth-grade level, just about on par with its community.

Meyer then compares the newspapers' reading-level gaps with their circulation and finds the papers with the biggest gaps have the deepest market penetration.

What I want to write is this: In other words, the dumber the writing, the higher the readership. But, I take that back because I'm sure it reflects some generational or educational bias I hold about the nature of journalism that conflicts with readability principles.

I'm also sure Meyer is not advocating dumber journalism, but instead proposing simpler writing. Is that a good strategy at a time when the rest of the media mob is racing to the bottom as fast as they can? I don't think so. It's time, I believe, for newspapers to distinguish themselves by providing the context, sophistication and depth not possible, or not preferable, on television.

It would be foolish to try to argue Meyer's research because he's got the depth of the data on his side. That said, I challenge the idea that people aren't reading newspapers because "many newspaper stories are too hard to read."

In fact, the most recent Audit Bureau of Circulation statements for the Grand
Forks Herald - Meyer's most readable newspaper - show the paper losing circulation, falling to 31,524 in September 2004 form 32,591 in March 2002.

Let's try an experiment. Here are the lead staff stories from Friday's Wall Street Journal and today's New York Times (web versions) run through the Flesch-Kincaid tool in Word:

 "Rare and Aggressive H.I.V. Reported in New York," New York Times - Reading ease, 48.2; grade level, 12.0.
 "In Path of Tsunami, Giant Cement Maker Struggles to Recover," Wall Street Journal - Reading ease, 47.4; grade level, 11.4.

And here are the recent ABC circulation numbers for both papers:

 New York Times -September 2004: 1,121, 057; March 2002: 1,194,491.
 Wall Street Journal - September 2004: 964,220; March 2002: 803,347.

Just using the numbers, it seems that what are arguably the nation's two best newspapers are also among the most unreadable., but they are gaining circulation while the country's "most readable" newspaper is losing it.

(CORRECTION: As Tom Johnson points out in the comments, I flipped the numbers on the New York Times, which, as you can see, in fact lost circulation in the period cited. My point, though, stands: That examples of what most journalists considerable to be quality journalism flunk the readability test.)

Finally, let's try this same test with the five most popular blogs (based on the Truth Laid Bear Ecosystem), just using the bloggers' own writing, not that of excerpts they are using:

 Eason Jordon post, 09:33 a.m., Instapundit - Reading ease, 55.7; grade level, 10.0.
 "Encore Performances: dKos Jazz," Daily Kos - Reading ease, 46.6; grade level, 12.0.
 "Remembering Mr. Lincoln," Powerline - Reading ease, 45.3; grade level, 12.0.
 "Easongate: A Retrospective," Michelle Malkin - Reading ease, 46.2; grade level, 10.9.
 "The news judgment of the public," Talking Points Memo - Reading ease, 53.9; grade level, 10.7.

Blog readership grew 54 percent in 2004, a number any newspaper publisher would envy for an entire decade, even though the most popular blogs are much more difficult to read - by readability standards - than newspapers.

Clearly, there exists an unquenched public appetite for robust writing, sophisticated commentary, analysis, debate and engagement that newspapers are not addressing - particularly at a local level. I'm not sure using a readability index is the best step toward satisfying that hunger.

I've had a difficult time (as you might be able to tell) expressing my concerns about Meyer's reliance on readability equations because I don't want to make light of his work and I do believe most newspapers are bastions of mediocre writing littered with jargon, acronyms and over-attribution. So, I want to give Meyer the last word. Here is a paragraph from the beginning of this chapter:

"Journalists today are well educated and have broad interests, and their natural inclination, if not checked by self-monitoring and good editing, is to write for each other." (Emphasis added.)

That's something we do agree on.

(This post scans 41.0 on the readability scale, an 11.4 grade level.)

Tags: , ,

Posted by Tim Porter at February 12, 2005 10:52 AM
Comments

Tim:

You wrote: "And here are the recent ABC circulation numbers for both papers:

New York Times -September 2004: 1,121, 057; March 2002: 1,194,491.
Wall Street Journal - September 2004: 964,220; March 2002: 803,347.

"... the nation's two best newspapers are also among the most unreadable, but they are gaining circulation while the country's "most readable" newspaper is losing it."

It sure looks to me, judging from the stats you use, that the NYT LOST roughly 70k readers in the two-year period.

-tom

Posted by: Tom Johnson on February 13, 2005 05:16 AM

There's an apples-to-oranges problem in the comparison of the Grand Forks Herald with the New York Times and the Wall Street Journal. The national newspapers are aimed at more specialized audiences and each has very low penetration of US households. If you look at the most recent ABC numbers, you will see that the most readable of the three, USA Today, is doing the best in circulation performance.

It would be great fun to repeat my quality measures in a comparison of the three national papers. But somone with no close ties to any of the papers should do it.

Posted by: Phil Meyer on February 13, 2005 10:04 AM

Phil:

Yes, the ABC numbers for USA Today indicate the relative best in terms of circulation performance. But given the recent reports of the subsets of defination tied to those types of numbers, I wonder how much of the performance is really hard-core, paid circ for USAToday? For example, surely there must be deep, deep discounting going on at all the hotels and airlines around the country and, indeed, the world. Is "paid circulation" a myth today at most newspapers?

(A couple years back, I was solicited to subscribe to the Oakland Trib. "The rate," I asked?
"That would be $9.95 delivered to your home," the caller said.
"And would that per week or month?"
"Oh no, that would be for a year," she replied.)

Do all these nuances of circulation matter any longer, and if so, to whom, other than circulation mangers and the ABC? Quien sabe?

Your book just arrived; I look forward to reading it.

-Tom

Posted by: Tom Johnson on February 13, 2005 03:45 PM

Hi, Tom.

It's true that ABC keeps changing its accounting rules to mask the loss of circulation. Currently, 25 percent of the list price has to be charged for a sale to count as paid circulation.It used to be 50. That applies to bulk sales to hotels as well as to sales to individuals. The latest rule change lets advertisers buy in bulk and give the papers away in targeted neighborhoods -- undermining the traditional concept of circulation as representing papers that a reader wanted badly enough to pay for them.
That makes it tough on those of us who want to make historical comparisons. But at least the rules change for everyone in the same way. So comparing newspapers with each other in the same unit of time still works.
The rules are posted at www.abcacess.com.

Cheers, Phil

Posted by: Phil Meyer on February 13, 2005 07:19 PM

I think the Flesch-Kincaid use of grade-level ratings is a little misleading. The measure is based entirely on the average syllables per word and average words per sentence. That probably says less about the sophistication of the ideas expressed than about the amount of work the reader will have to do to get the jist. I read somewhere (I haven't checked this out for myself) that James Fallows writing in The Atlantic -- hardly unsophisticated -- regularly checks in at about the 8th-grade level.

Posted by: John Perry on February 14, 2005 12:59 PM

John Perry:

Good observation! Flesch himself said that most people like to read a grade or two below their ability. It's easier. Moreover, the Flesch test was last renormed by Peter Kincaid in 1974. Based on my experience in teaching news writing to college sophomores, I believe that achievement has slipped another grade or two since then. In other words, what Flesch-Kincaid classified as 8th grade material now probably takes a 9th or 10th grade education to read.
Other tests look at more factors than word and syllable count, but they still correlate highly with Flesch-Kincaid. Some of them do, however yield higher grade-level scores, which would be expected if they were normed more recently.

Phil

Posted by: Phil Meyer on February 14, 2005 06:00 PM

Getting bogged down in readability scores is the kind of thing that has hamstrung newsrooms for a lot of years now: instead of report and write better, the instructions have come down to write "simpler." I admire the work that Phil has done (I really loved the book), and really hope that has central message (quality counts) is true. But I shudder at yet another editor coming across the readability research and imposing new "standards" on the newsroom. Bad research, under-reporting and boring writing won't be caught by Flesch-Kincaid or any other system

Posted by: Mark Hamilton on February 22, 2005 04:01 PM

Consider this example:

Reid doubtless realizes that while these tactics are good for instilling discipline and rallying the base, they are, following the lesson of the '02 and '04 cycles, worse than useless for winning elections. (Flesch-Kincaid Grade 12)

Better:

Reid knows these tactics instill discipline and rally the base. He also knows they lose elections. (Flesch-Kincaid Grade 5.9)

Which gets the message across better?

If you want to influence the public, swallow your pride and keep your sentences short, avoid big words, use active verbs.

Oh crap, no html. (How's that for a communicative thought?) The example's at http://voteforjudges.blogspot.com/2005/04/can-you-read-me-now.html.

Posted by: Karl Maher on April 25, 2005 10:20 PM
Post a comment