Auditing skill acquisition with KM metrics

"The real contest is always between what you've done and what you're capable of doing.

You measure yourself against yourself and nobody else." - Geoffrey Gaberino

This saying holds good not only for individuals but for all performance-oriented processes such as knowledge management. In fact, the fundamental principle of knowledge management is that all knowledge and skills must be measured and validated. Strategists point out that while evolving KM model, emphasis is laid on the top layer of the model which involves development of metrics and tests. The focus here is not on training, but learning.

According to white papers on the ‘KM Metrics’, metrics are the measurements we use to establish that certain skills exist at acceptable levels within knowledge workers. Experts opine that it is important to establish at what level a skill has been mastered and not merely ‘known about’.

To illustrate better, consider this. A employee may have excellent knowledge of programming syntaxes, but may be limited in his/her knowledge about system architecture or Web technologies. Still, his job role may require knowledge about those very limitations he has. So, how do we assess performance of such unskilled people? This predicament may be applicable to the organisation as a whole. Only through a well thought out and outlined metrics can we set our goals, milestones and evaluate our progress as an organization.

Infosys technologies, Principal Knowledge Manager, Dr. Kochikar explains that a matured KM implementation must include quantitative measures of success. One set of metrics focuses on the level of KM activity, and may include measures such as

• volume and frequency of access to the various KM repositories,


• the effort spent by various groups in promoting KM, and


• the volume of contribution.

Another set of metrics focuses on internal value adds, such as the quality of knowledge being shared, gains that people perceive at an individual level, etc. However, he says, " The most crucial set of metrics, are those, which measure the business value that the KM implementation is bringing to the organization such as measures of cost and effort saved, new revenue opportunities created, quality and productivity gains, and customer retention and acquisition."

Metrics can include certain subjective assessments, self-assessments, customer evaluations, and various productivity benchmarks. These metrics provide tangible goals and career paths for workers. Technical skills are much easier to quantify and validate whereas soft skills such as customer service, teamwork, etc are much difficult to quantify. What matters most is the establishment of tests and reviews of the project implemented.

The idea behind this is very similar to those performance checks conducted in the academic institutions to keep the students up-to-date and alert. Likewise, enterprises conduct periodical reviews to ensure that the employer retains the knowledge he gained during training and this stresses the importance they attach towards the knowledge creation .

Often these metrics are unique to every organization. There is currently no universally accepted metric that measures KM implementation. Agrees Aztec Software and Technology Services, Director — Competency Development, Vidyut Shenoy, "Organizations have derived different measures to capture and track various aspects of the specific knowledge. Knowledge Currency Unit (KCU) is one such measure that an organization uses to track application specific bits of knowledge. KCUs can be tracked, exchanged and traded as currency. Possession or absence of KCUs could be treated appropriately."

Industry experts recommend that at the end of each project or implementation, a performance audit should be conducted to examine:

• What went well


• What was supposed to happen


• What actually happened


• Where does the differences lie


• What lesson are to be learnt

These learnings needs to be analyzed and organisations should see to it that the valuable info gained are incorporated into the next learning cycle and the process continued to develop a set of best practicesSays Microsoft (India), Marketing Manager, Pankaj Ukey, "The other key KM matrices would be in terms of:

• how compelling KM solutions / scenarios are and how fast are these delivered/released internally


• what kind of standards and best practices it helps generate amongst knowledge workers, so that the knowledge gets reused and help drive KW productivity


• Also, it would depend on how involved key business owners are in terms of helping define and share/reuse metrics for usage and usability

Often, the Return on Investment (ROI) of knowledge management can be very subjective and hard to measure. An interesting point here, as experts point out, would be to measure the rapidity or speed of knowledge creation and what point does ‘knowledge’ become outdated? How 'fresh’ is the knowledge?