So tomorrow is December 1, which is the deadline for entries into the Knight News Challenge, an annual contest for innovative ideas for the intersection of technology and news, funded by the John S. and James L. Knight Foundation.

I (along with my buddy Eric Marden) have an entry in this year’s competition.  It’s a project we’re calling News Grader.  Here’s the meat and potatoes of the proposal:

We propose building a web service which can intelligently parse and analyze text it has never seen before, and offer insight into the quality of content, along with the degree to which the filter is confident in its analysis: the “News Grade”. We envision making this available in much the same way as Akismet or OpenCalais provides their service – an open API, providing third-party developers an easy way to make use of News Grader’s analysis and quality ratings. This API will enable the easy integration of our system into a variety of formats which could include browser extensions, CMS plugins, and desktop/web applications such as RSS readers, news aggregators or social networking software.

We believe we can do most of the heavy lifting using well-known algorithms, including a modified version of Bayesian induction, the Porter stemmer algorithm, entity extraction algorithms, and manifold learning algorithms. We intend to identify and weight word clusters in an article, and compare a new article to the ratings of other articles with similar word clusters, amongst other techniques.

This automated process will be supplemented by a mechanism for users to provide structured feedback, which will allow the web service to “learn”, and which will increase the quality of the analysis provided over time. The more that people use the service, the smarter it will get. It also makes it more difficult to “game” the analysis of a given piece of content, since the machine intelligence will compare new content to other content it’s previously encountered, and will weight new feedback as only a portion of its analysis.

You can read our full proposal here.  If (and that’s a big if, since there are hundreds of entries) the KNC people end up being interested in the idea, you can be sure I’ll be writing more about it when we put together a business plan and timeline for the second round of the competition.

I’ve been pretty interested in mechanisms for machine learning lately, and since the KNC folks were specifically looking for some proposals about authenticity, trust, and content discrimination, I thought this idea might be up their alley.

Of course, some folks think that open-ended machine learning systems are an all-too-common startup idea which never seems to quite work out.  To those folks, I’d like to point out that useful expert systems have been relatively rare until pretty recently, simply because doing it right is computationally intensive.

Furthermore, I think it’s worth noting that where this sort of system has worked in the past, it’s worked really well.  Netflix, for instance, paid out a million-dollar prize in 2009 for improving their recommendation algorithms by a mere 10%.  Amazon relies on it’s recommendation system as a driver of sales.  There’s just no question that systems like these can work; the only questions are what do you want to measure, and how do you use the information?

Netflix and Amazon want to predict what an individual will think about a particular recommendation.  Will you buy it?  Will you like it?  And that’s a great idea — it drives commerce on these sites, and makes them more useful for users.

But the questions we’re interested in answering don’t rely on personalization; we’re not so much interested in what a particular user cares about.  We’re interested in predictive modeling of things like bias, completeness, and novelty, independent of the tastes of a particular user.  The question we’re asking is, “Can we discover good journalism, regardless of subject matter?”

We think the answer is going to be that we can.  We won’t know until we actually run the experiment, however.  As far as I can determine, nobody’s ever tried precisely what we’re proposing vis-a-vis journalism on the web.  Only time will tell if we get the opportunity to try.

**Update, 1/12/2010:  The Knight Foundation declined our proposal.  Anyone want to fund the idea?  Otherwise, I’d say it’s dead in the water. **

Web programmer Brett Yates works on his own terms at the COOP coworking space.  Photo: Ian Monroe/Medill

Web programmer Brett Yates works on his own terms at the COOP coworking space. Photo: Ian Monroe/Medill

Forget the cubicle, get off the couch, and ditch the coffee shop.

Freelance technology workers these days have a new way to get things done –coworking (enthusiasts insist there is no hyphen). A new coworking space, opened in January, aims to provide Chicago’s independent workers with a different kind of occupational environment.

The COOP is a hip office space where technology workers such as graphic designers, programmers, and Internet experts can rent a desk for a day, a week or a month at a time. Instead of punching the clock at a soulless cubicle farm, freelance workers can call their own shots. Instead of writing code on the couch at home, where a variety of diversions threaten to sabotage productivity, independent contractors and telecommuters can get the benefits of having an office without the expensive overhead of leasing their own space.

The COOP space, located on Fulton Market, once housed a chicken processing business. The exposed brick walls laden with art create a comfortable, relaxed workspace. Young entrepreneur Sam Rosen, 23, operates his web design business,One Design Company, in one section of the loft. The other portion is reserved for independent workers and freelancers to use for coworking. The easily reconfigurable loft space can comfortably house perhaps a half dozen such workers at a time.

Coworking enthusiasts say that the spaces aren’t just useful for getting work done, but also can be a resource for generating new business and for fleshing out new ideas. “It’s an incubator model,” said Rosen. “We’re not by any means asking for a stake in anyone’s venture, but the idea is if they come here and they’re starting up something, and it works, they’re going to need help. And we can help, or we know people to help. That’s cool, and that seems a lot more natural and nicer.”

The coworking movement began several years ago in California, but has rapidly become a global phenomenon. “It’s pretty widespread,” said Eric Marden, a freelance web programmer and coworking advocate. “It used to be where there would be one city in every state, and now multiple cities in the state, and sometimes multiple spaces in one city are happening, and that’s become more prominent. There’s a third one about to open in Austin.”

“It’s great,” said Brett Yates, a freelance computer programmer and patron of the COOP space. “I spent probably at least a year working out of my apartment, and doing that just kind of drove me insane. I started to find I was getting a lot less done – I’d go to coffee shops and get more done in two hours than in a full day at home.” That lack of productivity drove Yates to explore coworking. “I got out and kind of checked out a couple different places and this one seemed exactly like I was looking for.”

Chicago hasn’t yet gotten completely on board with the idea of coworking, but it’s catching on. “It was kind of a slow start for us, getting things out there and advertising,” said Linsey Burritt, a designer for One Design Company and a leader of the COOP. “People in Chicago haven’t heard about it as much as other cities.”

Workspaces at the COOP rent for $20 per day, $90 per week, or $300 per month, and include T-1 Internet access.

An informal “coworker visa” program lets members of one coworking space use the facilities of other coworking groups when they travel. Open sharing of ideas and resources is encouraged. “Any time we’ve communicated with anybody, people have come to us, or we’ve gone to them, people are with open arms,” Rosen said. “They’re like, ‘Here’s what I’ve got, let me help you. How can you help me?’ It makes the pie bigger.”

Marden was utilizing the COOP space to work on programming projects while visiting Chicago with his family. He helps run a similar space in Orlando, Fla., and traded ideas with Rosen and Burritt during his stay. “The coworking [in Florida] kind of grew out of our BarCamps [open conferences on technology and culture],” Marden said. “For us, it’s the physical hub now. Sort of the creative club house for all the stuff that was already happening all over the city, but it kind of lets us gel and have an area to work. We still all do our own thing, but we all kind of come together for that sort of stuff.”

Coworking may not yet be mainstream, but Rosen said it has caught the attention of many successful independent professionals. “We’re lucky because we’re busy, but you hear so much about how people are struggling,” Rosen said. “People who come here are not struggling. People who come in here are thriving. The people who are sitting down working, they’re here because they have too much work to do. They need a place to focus. That’s interesting.”

Note: This story was first published on the Medill Reports website on 4/28/2009, as well as on the Windy Citizen on 4/29/2008.  It’s republished here for my own archives.