Sunday, July 19, 2009

How good are newspaper Web sites? A test of Eppy winner Azstarnet.com shows even the "best" may get a failing grade

Newspaper people often blame others - see Google, parasitic aggregators, etc. - for their troubles making enough money online. But could there be another problem in many cases? Could it be that newspaper Web sites in many - even most - cases aren't good enough to compete?

Mark Potts provides an excellent test for newspaper Web sites on his blog, recoveringjournalist.com.  Now you may quarrel with his standards, but I think it's fair to say that Mark is an expert in Web journalism. He was one of the co-founders of WashingtonPost.com and has a deep digital resume. I happen to think he's got a lot of good ideas about Web journalism.

I thought it would be an interesting exercise, since I no longer supervise a newspaper Web site of my own, to test one of  the best newspaper Web sites, as identified by the industry itself through the Eppy awards.  These awards, sponsored by Editor & Publisher and Mediaweek, honor "the best Web sites offered by newspapers, television, magazine and radio companies. " So to be fair, I didn't pick on an easy target. And I stayed away from the big guys - the NYTimes, WSJ, USA Today, which compete in a different league than most papers. I decided to apply Potts'  test to the Arizona Daily Star, Tucson's daily newspaper. That newspaper won this year for "Best News Web Site with fewer than 1 million unique monthly visitors." It is owned by Lee Enterprises, one of the country's leading newspaper companies and one that is paying attention to online journalism.

How did the Star do? It failed. It got a score of 41%, or 7 out of 17. Now, if I felt really generous and applied grade inflation, I could sneak it over 50% by reversing two scores that some might quibble with. Then it would get 9 out of 17. But the truth is I think I was generous giving it 7 passing grades. The score could easily have been lower. I think we can all agree, the results aren't very encouraging, especially for the "best news web site."

What follows is Potts' test with my comments and grade for each item. I understand that some may disagree with my thinking. But I tried to act like a new resident of Tucson using the Star's site to make my life better. I think that's reasonable. I think there's much that can be learned from this result. Perhaps that's for another blog post.

  1. Without using search, find continuing, in-context coverage of a long-running local story.  I decided to see whether the paper had continuing, in-context coverage of spring training and the issue of whether the Rockies and Diamondbacks would leave Tucson. I clicked on the latest story, "Teams ink deal to leave Tucson," and when I did I found high interest - 64 comments - with links to two related articles and a "Did You Know" question about spring training, but there was no link to a site that would have everything from analysis of the economic impact of spring training on Tucson to history of spring training in Tucson, photos and video, etc. Grade: Fail.
  2. Similarly, find a comprehensive package of information (even a collection of past stories) about a significant local icon or personality. Lute Olson seemed like a good choice, given his significance in Tucson as the longtime (and yes, former) coach of the University of Arizona Wildcats basketball team. So I typed his name in the site's search box and got one news story about Wildcats basketball. There was no link or refer to a page or site about Lute Olson or Wildcats basketball. It should be noted that only stories from the past week are free on this site, so perhaps that's the reason. Also, by this time I had clicked on the site enough that if I wanted to continue I needed to register, which I did. I did find at the top right hand corner of the page a small refer saying I could sign up for "Wildcat newsletters." I decided to go another route and go directly to the Wildcats basketball page to see if I could find a package of information on Olson. The drop-down menu on the navigation bar at the top of the page didn't have a clear link to Wildcats basketball. There's a pac10fanatic.com site, which was all schools and all sports. I tried it, but it didn't seem obvious what to do to find more on Olson there. So I tried "All sports." Strange as it may seem, I couldn't even find a basketball page, let alone a page on Olson. When I clicked on the columnists' page, I did find an Arizona Wildcats link, and it took me to a page that did have depth about UA sports. However, while there was a 2008-2009 basketball preview, it again wasn't obvious how to find out about Olson or the team taking shape for the 2009-2010 season. Grade: Fail.
  3. Locate all the coverage and information on the site about a specific local town. Casas Adobes was the first suburb of Tucson, begun in the mid-1940s, according to Wikipedia. Remember, I'm acting as a new resident who doesn't know much. The 2000 Census pegged its population at roughly 54,000. I typed its name into the site's search engine and did get a community calendar, one event and a "moving up" feature. There were also yellow page listings for the community and local web info. However, when I clicked on the community calendar link, I didn't find anything guiding me to more coverage and information on the community. So next I went to the Local News tab on the navigation bar. While I didn't find the name of the community, I did see a "neighbors" tab, so I tried it. That didn't do anything for me, but I did learn there that I could subscribe to a newsletter for what appeared to be any ZIP code in the Tucson area. I clicked on 85643 and got a few headlines, but no collection of material on even that ZIP. Grade: Fail.
  4. Starting on a story page (not the home page) quickly find other key information, e.g. the day's top headlines or most-read stories. (Remember, the vast number of readers don't enter your site from the home page, though print-focused newsies obsess about home pages.) I went to a hot, breaking news story about a Tucson man barricading himself in his house for four hours before surrendering. There was no list of the day's top headlines or of the most-read stories. Grade: Fail.
  5. Find a list of the best local restaurants, or ratings and reviews of a particular kind of cuisine, preferably by locality (extra credit: user reviews). BTW: This is why Yelp is really hurting newspapers. I went to the Food tab in the navigation bar and then clicked on the reviews tab. I found a search box where I could search by name, cuisine type, location and cost scale, but there was no list of best local restaurants or list of best restaurants by cuisine. I tried to type  barbecue under cuisine type to see whether that would help. It wouldn't work. I had my choice of 9 types of cuisine, or "all." So I picked "Italian/Pizza." I got a long list of restaurants, with a single sentence summary of the review, but the list didn't indicate which were the highest ranked (there was no rating system at all) or where they were. So I clicked on one to see what I would get on the review page. I could find nothing there explaining what the paper thought was the best Italian restaurant or best for different types of Italian cooking. Grade: Fail.
  6. Find a local movie listing, or better yet, a local theater listing and review (extra credit: user reviews). One click on the Entertainment tab of the navigation bar showed me there's a link for movie times. One click and I was there. Easy. It wasn't so easy for theater listings and reviews. There's no theater tab under the Entertainment tab of the main navigation bar. So I picked Events. That didn't seem to work. I tried Calendar. But the categories on that page didn't include theater. Let's give the Star credit for good movie listings anyway. Grade: Pass.
  7. Find something a family can do for fun this weekend. I went to the Entertainment tab of the main navigation bar and picked Events, thinking that should help. There's a tab on the left side that says, "Get out." I figured that would work. The listings were limited, but I could have found something to do. However there's nowhere I could see on this site to find "family" events. Again, the Star barely makes the grade, but it does enough that I can say I can find an event for a family. Grade: Pass. (Barely.)
  8. Find any location mentioned on the site on a map—wait, no, you're not allowed to leave the site. No MapQuest or Google maps! I decided to stick with the man who barricaded himself in the house before surrendering to police. There was nothing on the story page that would help me find the location on a map. No link in the story. No map accompanying the story. No indication that searching for location was possible. Grade: Fail.
  9. Using the site's search function, search for a term you know appeared in the newspaper in the past 24 hours. I tried Supreme Court, given the prominence of the hearings for nominee Sonia Sotomayor. I found story links from Friday, Thursday and Wednesday and a link to "More." There I found a total of 10 stories. Grade: Pass.
  10. Subscribe to your site's mobile alert function (you have one, right?) and see if it's truly useful. While you're at it, be sure to look at your site regularly on its iPhone or mobile version (you have one, right). Is it updated as frequently as the main site? There's a small link far down the home page on the left side. When I went to the site, it took me three clicks to get to a story. That's too long. I followed the story of the man barricaded in the house. Instead of continually updating one story, there are three stories on the recent updates page. I don't think that makes sense. But, the site does work, although nowhere near as nicely as major newspaper mobile sites. Grade: Pass. (Barely.)
  11. Find something in the paper's archives. I looked up Lute Olson's retirement. There were 77 articles. I could read a few lines before deciding whether to pay for the full article. Grade: Pass.
  12. How easy is it to e-mail a story, or print it out, or view it on a single page?  Easy. Grade: Pass.
  13. Find a way to quickly contact a specific reporter, or an editor, or anybody at the paper. The bylines on stories are not clickable if you want to use them to e-mail the reporter. I could not figure out how to contact the reporter on a story page. I couldn't find a link on the home page to contact editorial staff. I did click on subscriber services and it took me to Tucson.com, which appears to be the umbrella site for the Tucson newspapers. There were no editorial contacts listed there, though. Then I realized that under the Home tab on the main navigation bar there is a contact us tab. That leads to a search box that allows you to browse by last name, department or to search by last name. I found a friend who works there. But I'm sorry, this takes too long. Grade: Fail. 
  14. Find an ad you know is on the site. (This drives advertisers nuts, incidentally.) Display ads do not show up in the local search. I looked for the Ford ad I found on a business page. I couldn't find it. I could find no way to locate a directory of ads on the site. Grade: Fail.
  15. How easy is it to place a classified ad online—or to buy any kind of ad? To place a classified ad, you're taken to tucson.com,  the business site for the the Star and the former Tucson Citizen. You need a different login and password for this site even though you're already registered on the Star's site. That, to me, is an immediate failure. Grade: Fail.
  16. How easy is it to manage your print subscription online? It's doable. Grade: Pass.
  17. Using the site's search function, search for just about anything in the list above. One of the top stories on the web site was that a Grand Canyon death was ruled a suicide. So I typed Grand Canyon suicide in the search engine. It didn't find the story. Then I typed, Grand Canyon. No news stories came up. One of the latest updates on the site has this headline: Man's drive over Grand Canyon edge ruled suicide.  Grade: Fail.
  18. Now, try the same searches from Google. When I did the same with Google, the story that's reported on the Star's site is the fifth link in my first search under Grand Canyon suicide. It links to the San Jose Mercury News and 39 related articles. The Mercury News story is the same Associated Press story found on the Star's site. Clearly, Google is better at finding the topic than the Star's own search engine. Grade: Fail. But for purposes of this test, we won't count this one because it feels repetitive.

15 comments:

  1. This comment has been removed by a blog administrator.

    ReplyDelete
  2. John,

    I wish you gone thru the process of placing a classifed ad and comparing that process with Craigs List.

    I am sure the result would have been a double fail.

    ReplyDelete
  3. "I could not figure out how to contact the reporter on a story page."

    I'm not defending the Star's site, because I agree it's a horror. (And I live in Tucson.)

    But you get a Fail for not finding contact information for a story's author at the bottom of the story. It lists a phone number, email address, and, in the case of one reporter I noticed recently, a Twitter account.

    ReplyDelete
  4. Maybe if the Star's editors actually focused on designing a functional Web site past 1993 Web standards and stopped hanging on their laurels of fake awards, they would actually grow readership.

    The Star's Web site is an abomination, one that the staff widely regards as an embarrassment. I'm sure that a handful of "senior editors," who don't know what good Web design was if it him them over the head, think everything's hunky-dory. And I'm sure they probably ignore stinging criticism from their own reporters and readers because "they know best."

    It's a huge problem when you cannot effectively search for articles, wait 30 seconds at times for a page to load, read tables as garbled text, get thrown 404 errors, or are unable to navigate, say, the mobile site. Someone in Tucson needs to wake up, and fast.

    ReplyDelete
  5. John, thank you for pointing out the location of the contact information. But I still give the site a failing grade on this score because none of the links you describe are hot. I don't want to copy somebody's e-mail address to connect with them. I think it's reasonable to expect a site to make the e-mail, and frankly the twitter address, clickable. They're not. If I'm incorrect about that, please let me know.

    ReplyDelete
  6. John,

    I'm not sure site design is as important as it was even a few years ago. What is important is making site friendly to aggregators and allowing users to build up trust with individuals on the staff and to track ongoing stories (think plethora of RSS), and you are right the Washington Post does this far better than most.

    ReplyDelete
  7. It's important that top editors and publishers hear from the journalists on the front lines every day. The following comes from a Web journalist in his 20s working on the Web site of a print publication. I think his voice is worth listening to. I am posting this anonymously for him because he doesn't want to jeopardize his job. The following is an excerpt of his e-mail to me.

    "Thank God you wrote what you did about the Arizona Daily Star -- not to pick on them, but because it's just so perfect to show that an award-winning newspaper Web site is still a terrible Web site. I see this constantly in the newsrooms I've worked in -- people pat themselves on the back for making the tiniest changes, things that a reader would actually consider close to a bug fix than an upgrade (especially on a site that already looks ca. 1999) -- so they are always running five or so years behind mainstream thinking about how Web sites should look, feel, work, etc.

    "The problems also include a tick-the-box mentality -- we have a list of most-read stories; we have a restaurant list; we have a search button -- that doesn't consider whether those "features" actually even work for the readers. Who cares if you have all these things available if it's a huge pain to use them, or they don't even really work? But too many people in positions of authority either don't want to think about that or are incapable of doing so. And then there is the problem of editors being defensive -- oh we don't WANT to put too many links on the inside pages because we want people to go to our main page. News flash: making a Web site harder to use does not mean people will work harder to use it -- it means they will give up on it more quickly, and thus hurt your audience and revenue.

    "It's all just obscenely frustrating. I know how massively most news orgs are failing the Web test. Painful."

    ReplyDelete
  8. John:

    Your critique of our site is fair and points up useability problems that frustrate both our readers and the newsroom. We've tacked on lots of new content, but little work has been done the past few years on StarNet's design and infrastructure.

    Tucson's joint operating agreement came to an end in May, which means the Star's news site can now get access to the development resources we know it requires. (Under the JOA, our programmers, designers and other behind-the-scenes staff supported two news sites plus a third site for the business agency.)

    We'll unveil a redesign and new search functions, along with a new content management system and other improvements this fall. We hope you'll retest us then.

    So how did we win best news Web site? The contest judged our news content, not the useability of the site. Cleary, they should go hand-in-hand we need to excel at both.

    Bobbie Jo Buel
    executive editor, Arizona Daily Star

    July 20, 2009 6:54 PM

    ReplyDelete
  9. This comment has been removed by a blog administrator.

    ReplyDelete
  10. Ahhh, the folly of reporters (or ex-publishers) using Wikipedia.

    I can’t quibble with most of your review, but Test #3 has some problems.

    There is no Casas Adobes. It’s a neighborhood in the unincorporated county. There was an effort to incorporate it and the surrounding area in 1997 that was rejected by the courts.

    It has a Census CDP because the fledgling town had requested a Census count in 1997 for tax purposes (Arizona has a per capita state shared revenue system based) and the incorporation effort was still hung up in the courts when the 2000 count was being done.

    You didn’t find any stories about Casas Adobes because it doesn’t exist, there’s nothing to write about. The suburban towns in the metro area are Oro Valley, Marana and Sahuarita. If you search those keywords, you get stories about them. Plus, a large chunk of what is included in the Casas Adobes CDP has been annexed by Marana and Oro Valley.

    Also, the ZIP you gave in your post is for Willcox, Ariz., which is about 90 miles from Tucson. The ZIP for most of Casas Adobes is 85704. If you click on that ZIP, you get several stories and calendar items.

    So you should redo your test or give the ADS a pass for #3.

    And I’m not an ADS employee. I work for what’s left of the Tucson Citizen, which used to be the Star’s primary news competitor.

    Mark B. Evans

    ReplyDelete
  11. Kudos to the executive editor for accepting the criticism and pledging to change. JOA setups seem to be an especially frustrating hindrance to innovation. AZStarnet has done some innovative reporting despite being hobbled by site design. Of particular note is a mash-up they use to stream live video from an event with an adjacent Twitter feed to display comments from readers and reporters at the scene.

    Here's hoping the new CMS and redesign are a roaring success.

    Curt Chandler
    senior lecturer/multimedia
    Penn State university

    ReplyDelete
  12. hi! i like your website... Thanks for the sharing...easy to download

    ReplyDelete
  13. Hi,
    I like your article and itinspire to me create and read more article.Thanks for sharing useful information to us...

    r4 games

    ReplyDelete
  14. I have liked your thoughts, I am going to subscribe your blog and will come here again
    to know more. thanks,
    Your discussion is exactly the same i agree for . I will scribe your blog and will come back.

    <a href="http://dinariraqi.net>Dinar Iraqi</a>

    ReplyDelete