you are right, the tools evovle at a rapid pace. However, significant changes do not occur on a monthly basis and usually just come in major product updates. We are working on this trying to establish a process that will allow vendors to easily notify us about enhancements of their tools, so that we can reasses them. In addition we are optimizing our product review process which will allow us to add more tools to the Matrix despite of the limited time we have for working on our favorite side-project.

Best regards,

Bernd

You are making an important remark here. If we would post these ratings without an explanation of our categories and any indication how we came to that rating the numbers would be completely useless. We have however outlined our categories here and provide a short evaluation text for each rating on the individual toolpages (you find a linklist here and in the sidebar).

We will make this more transparent by providing a new matrix-table with links to the respective evaluation texts in the near future.

In the specific case that you mention (Lotus Connections, Data & Security) we did not like that “access controls are mostly just public or private and a definition of user roles needs a complicated integration with the Websphere Portal Server (WPS).”

Surely it is possible to justify a 10% higher or lower rating and we are aware that our assessments are not perfect. This is why we encourage people to understand the ratings as indicators of strength and weaknesses rather than absolute scientific statements.

Your conclusion that a given rating will not fit a particular customer environment is absolutely correct and valuable. The Social Software Matrix is provides a general overview only. For a detailed situational analysis and tool-selection experts should be consulted.

Best regards,

Bernd