Tony Byrne,( CMSWatch,which is now a component of The Real Story)  is always worth listening to. He has such a rich experience in understanding how software applications meet user needs and is so knowledgeable about specific features of different software application that he has been able to condense a complicated subject into a useful set of checklists. Originally (2001) focused on content management, the consultant practice now includes evaluations of vendors who are playing in the social media space.

In this E2.0 conference workshop “Insider’s Guide to Evaluating and Selecting Social Software”, Byrne begins by distinguishing the boundary between collaboration and networking. (I like distinctions, and this is a good one.) It helps to understand that some products are designed with a focus on networking, others with a focus on collaboration:

(C) 2010 The Real Story Group

What is happening, of course, is that vendors who start in one position are moving to the other as they rework the platform, add features, and so on. So the checklist addresses elements of each of these.  The CMSwatch framework has four checklists:

• What business scenarios (external: branded communities, professional networking; internal: project collaboration, KM, info sharing) does it support?
• What business services (wikis, blogs, file sharing, discussions) does does it support?
• What infrastructure capabilities (anti-spam, filtering, document repositories) does it support?
• What system administration functions (backup/restore, configuration, internationalization) does it provide?

These questions must all be answered in the context of software you are looking at.  CMSWatch provides an interesting taxonomy of collaboration and community software:

Tony goes on to provide insights into each of the platform vendors, with a strong concentration on Microsoft SP 2010.  Using his checklist, Tony can only conclude that SP is a jack of all trades and master of none. It does nothing really well, does pretty much everything (even if quirkily) and has a user interface that can only be navigated by experts. This next version does, however, integrate really well with MS Office products even though it’s not great on how it manages communities.

It appears that none of the platform vendors have successfully added a “social layer” to their existing applications. Nor have any of the ECM vendors successfully added a social layer. So much for the two types of vendors that started from the collaboration side of the boundary. On the other, networking side, the vendors are more agile and able to innovate by adding features as they see them emerge.

Whatever path in selection that you take, some of the key dos and don’ts:

• Understand the difference between platform and product. (infrastructure vs.  quick deployment)
• Try before you buy (Tony heartily disagrees with Andrew McAfee’s Drop Pilot position
• Understand the vendor as much as the software
• You may need to use different tools for different scenarios but focus on the scenarios

Tony concludes by suggesting that you can look at the main scenarios that you want to support and build a matrix of features needed for each.  In my work with software pilots, I have tried to focus on these use cases and design templates (or archetypes, as Tony calls them) for each type of community.

A well-done session on managing the complications of selecting tools, even if the market will stay unsettled — with new things coming — for the next three years.  ”A vibrant social software marketplace awaits.”