How good are newspaper Web sites? A test of Eppy winner Azstarnet.com shows even the "best" may get a failing grade

Newspaper people often blame others - see Google, parasitic aggregators, etc. - for their troubles making enough money online. But could there be another problem in many cases? Could it be that newspaper Web sites in many - even most - cases aren't good enough to compete?

Mark Potts provides an excellent test for newspaper Web sites on his blog, recoveringjournalist.com.  Now you may quarrel with his standards, but I think it's fair to say that Mark is an expert in Web journalism. He was one of the co-founders of WashingtonPost.com and has a deep digital resume. I happen to think he's got a lot of good ideas about Web journalism.

I thought it would be an interesting exercise, since I no longer supervise a newspaper Web site of my own, to test one of  the best newspaper Web sites, as identified by the industry itself through the Eppy awards.  These awards, sponsored by Editor & Publisher and Mediaweek, honor "the best Web sites offered by newspapers, television, magazine and radio companies. " So to be fair, I didn't pick on an easy target. And I stayed away from the big guys - the NYTimes, WSJ, USA Today, which compete in a different league than most papers. I decided to apply Potts'  test to the Arizona Daily Star, Tucson's daily newspaper. That newspaper won this year for "Best News Web Site with fewer than 1 million unique monthly visitors." It is owned by Lee Enterprises, one of the country's leading newspaper companies and one that is paying attention to online journalism.

How did the Star do? It failed. It got a score of 41%, or 7 out of 17. Now, if I felt really generous and applied grade inflation, I could sneak it over 50% by reversing two scores that some might quibble with. Then it would get 9 out of 17. But the truth is I think I was generous giving it 7 passing grades. The score could easily have been lower. I think we can all agree, the results aren't very encouraging, especially for the "best news web site."

What follows is Potts' test with my comments and grade for each item. I understand that some may disagree with my thinking. But I tried to act like a new resident of Tucson using the Star's site to make my life better. I think that's reasonable. I think there's much that can be learned from this result. Perhaps that's for another blog post.

1. Without using search, find continuing, in-context coverage of a long-running local story.  I decided to see whether the paper had continuing, in-context coverage of spring training and the issue of whether the Rockies and Diamondbacks would leave Tucson. I clicked on the latest story, "Teams ink deal to leave Tucson," and when I did I found high interest - 64 comments - with links to two related articles and a "Did You Know" question about spring training, but there was no link to a site that would have everything from analysis of the economic impact of spring training on Tucson to history of spring training in Tucson, photos and video, etc. Grade: Fail.
2. Similarly, find a comprehensive package of information (even a collection of past stories) about a significant local icon or personality. Lute Olson seemed like a good choice, given his significance in Tucson as the longtime (and yes, former) coach of the University of Arizona Wildcats basketball team. So I typed his name in the site's search box and got one news story about Wildcats basketball. There was no link or refer to a page or site about Lute Olson or Wildcats basketball. It should be noted that only stories from the past week are free on this site, so perhaps that's the reason. Also, by this time I had clicked on the site enough that if I wanted to continue I needed to register, which I did. I did find at the top right hand corner of the page a small refer saying I could sign up for "Wildcat newsletters." I decided to go another route and go directly to the Wildcats basketball page to see if I could find a package of information on Olson. The drop-down menu on the navigation bar at the top of the page didn't have a clear link to Wildcats basketball. There's a pac10fanatic.com site, which was all schools and all sports. I tried it, but it didn't seem obvious what to do to find more on Olson there. So I tried "All sports." Strange as it may seem, I couldn't even find a basketball page, let alone a page on Olson. When I clicked on the columnists' page, I did find an Arizona Wildcats link, and it took me to a page that did have depth about UA sports. However, while there was a 2008-2009 basketball preview, it again wasn't obvious how to find out about Olson or the team taking shape for the 2009-2010 season. Grade: Fail.
3. Locate all the coverage and information on the site about a specific local town. Casas Adobes was the first suburb of Tucson, begun in the mid-1940s, according to Wikipedia. Remember, I'm acting as a new resident who doesn't know much. The 2000 Census pegged its population at roughly 54,000. I typed its name into the site's search engine and did get a community calendar, one event and a "moving up" feature. There were also yellow page listings for the community and local web info. However, when I clicked on the community calendar link, I didn't find anything guiding me to more coverage and information on the community. So next I went to the Local News tab on the navigation bar. While I didn't find the name of the community, I did see a "neighbors" tab, so I tried it. That didn't do anything for me, but I did learn there that I could subscribe to a newsletter for what appeared to be any ZIP code in the Tucson area. I clicked on 85643 and got a few headlines, but no collection of material on even that ZIP. Grade: Fail.
4. Starting on a story page (not the home page) quickly find other key information, e.g. the day's top headlines or most-read stories. (Remember, the vast number of readers don't enter your site from the home page, though print-focused newsies obsess about home pages.) I went to a hot, breaking news story about a Tucson man barricading himself in his house for four hours before surrendering. There was no list of the day's top headlines or of the most-read stories. Grade: Fail.
5. Find a list of the best local restaurants, or ratings and reviews of a particular kind of cuisine, preferably by locality (extra credit: user reviews). BTW: This is why Yelp is really hurting newspapers. I went to the Food tab in the navigation bar and then clicked on the reviews tab. I found a search box where I could search by name, cuisine type, location and cost scale, but there was no list of best local restaurants or list of best restaurants by cuisine. I tried to type  barbecue under cuisine type to see whether that would help. It wouldn't work. I had my choice of 9 types of cuisine, or "all." So I picked "Italian/Pizza." I got a long list of restaurants, with a single sentence summary of the review, but the list didn't indicate which were the highest ranked (there was no rating system at all) or where they were. So I clicked on one to see what I would get on the review page. I could find nothing there explaining what the paper thought was the best Italian restaurant or best for different types of Italian cooking. Grade: Fail.
6. Find a local movie listing, or better yet, a local theater listing and review (extra credit: user reviews). One click on the Entertainment tab of the navigation bar showed me there's a link for movie times. One click and I was there. Easy. It wasn't so easy for theater listings and reviews. There's no theater tab under the Entertainment tab of the main navigation bar. So I picked Events. That didn't seem to work. I tried Calendar. But the categories on that page didn't include theater. Let's give the Star credit for good movie listings anyway. Grade: Pass.
7. Find something a family can do for fun this weekend. I went to the Entertainment tab of the main navigation bar and picked Events, thinking that should help. There's a tab on the left side that says, "Get out." I figured that would work. The listings were limited, but I could have found something to do. However there's nowhere I could see on this site to find "family" events. Again, the Star barely makes the grade, but it does enough that I can say I can find an event for a family. Grade: Pass. (Barely.)
8. Find any location mentioned on the site on a map—wait, no, you're not allowed to leave the site. No MapQuest or Google maps! I decided to stick with the man who barricaded himself in the house before surrendering to police. There was nothing on the story page that would help me find the location on a map. No link in the story. No map accompanying the story. No indication that searching for location was possible. Grade: Fail.
9. Using the site's search function, search for a term you know appeared in the newspaper in the past 24 hours. I tried Supreme Court, given the prominence of the hearings for nominee Sonia Sotomayor. I found story links from Friday, Thursday and Wednesday and a link to "More." There I found a total of 10 stories. Grade: Pass.
10. Subscribe to your site's mobile alert function (you have one, right?) and see if it's truly useful. While you're at it, be sure to look at your site regularly on its iPhone or mobile version (you have one, right). Is it updated as frequently as the main site? There's a small link far down the home page on the left side. When I went to the site, it took me three clicks to get to a story. That's too long. I followed the story of the man barricaded in the house. Instead of continually updating one story, there are three stories on the recent updates page. I don't think that makes sense. But, the site does work, although nowhere near as nicely as major newspaper mobile sites. Grade: Pass. (Barely.)
11. Find something in the paper's archives. I looked up Lute Olson's retirement. There were 77 articles. I could read a few lines before deciding whether to pay for the full article. Grade: Pass.
12. How easy is it to e-mail a story, or print it out, or view it on a single page?  Easy. Grade: Pass.
13. Find a way to quickly contact a specific reporter, or an editor, or anybody at the paper. The bylines on stories are not clickable if you want to use them to e-mail the reporter. I could not figure out how to contact the reporter on a story page. I couldn't find a link on the home page to contact editorial staff. I did click on subscriber services and it took me to Tucson.com, which appears to be the umbrella site for the Tucson newspapers. There were no editorial contacts listed there, though. Then I realized that under the Home tab on the main navigation bar there is a contact us tab. That leads to a search box that allows you to browse by last name, department or to search by last name. I found a friend who works there. But I'm sorry, this takes too long. Grade: Fail. 
14. Find an ad you know is on the site. (This drives advertisers nuts, incidentally.) Display ads do not show up in the local search. I looked for the Ford ad I found on a business page. I couldn't find it. I could find no way to locate a directory of ads on the site. Grade: Fail.
15. How easy is it to place a classified ad online—or to buy any kind of ad? To place a classified ad, you're taken to tucson.com,  the business site for the the Star and the former Tucson Citizen. You need a different login and password for this site even though you're already registered on the Star's site. That, to me, is an immediate failure. Grade: Fail.
16. How easy is it to manage your print subscription online? It's doable. Grade: Pass.
17. Using the site's search function, search for just about anything in the list above. One of the top stories on the web site was that a Grand Canyon death was ruled a suicide. So I typed Grand Canyon suicide in the search engine. It didn't find the story. Then I typed, Grand Canyon. No news stories came up. One of the latest updates on the site has this headline: Man's drive over Grand Canyon edge ruled suicide.  Grade: Fail.
18. Now, try the same searches from Google. When I did the same with Google, the story that's reported on the Star's site is the fifth link in my first search under Grand Canyon suicide. It links to the San Jose Mercury News and 39 related articles. The Mercury News story is the same Associated Press story found on the Star's site. Clearly, Google is better at finding the topic than the Star's own search engine. Grade: Fail. But for purposes of this test, we won't count this one because it feels repetitive.