Martijn S.
COO@Westbury/Execution/Following Through/Self-Service BI/Social Media/SIG Chapter Leader at Vivit
Why do organizations use so many metrics? Do they check if their already in use? Do organizations monitor the lifecylcle of a metric?
Clarification added August 6, 2008:
Our as Gary Lemke from CRM Advocate refrased my point that you need a Metric process and ownership in place: "That reminds me of another humorous movie, "Groundhog Day," where each morning the main character relives the previous predictable day in a seemingly endless loop. Just the act of stopping reports doesn't cut it. The act is a trigger to take inventory of what is really important as well as making the necessary changes to stop the waste. Are you living your own Groundhog Day?"
http://www.crmadvocate.com
Clarification added August 8, 2008:
Who in an organization decides what is measured? Who controles the defintion of metrics?
Answers (35)
Tim H.
Technology Consultant at Mousetech.com
Best Answers in: Software Development (12), Computers and Software (7), Web Development (6), Enterprise Software (5), Staffing and Recruiting (2), Education and Schools (1), Product Design (1), Career Management (1), Information Storage (1)
Metrics can tell you useful things. For example, I worked on a project that was heavily metered, and one thing we learned was that we had a fairly accurate idea of how long it took us to do work, but a given task would almost invariably be twice as much work as we estimated.
Unfortunately, it's all too common once a company gets into the metrics business to confuse metrics with reality. Metrics are comforting because they provide (the illusion of) concrete objective information, but what you don't measure, you don't see.
Furthermore, when metrics are too closely tied to rewards and punishments, people will work out ways to game the system.
Consider an all-too-common case - the corporate call center. Two of the most popular metrics in use there are call times and customer satisfaction surveys.
Call times seem like a good thing to monitor - the shorter the call, the more "productive" the employee. Until call times become so important that employees "accidentally" hang up on long calls, short customers on information, or leave an issue unresolved. Or even offend customers so that the customer will hang up in disgust.
The metric solution to this is customer satisfaction surveys and call monitoring. However, few customers are interested in taking the time to run through a formal survey - especially since typically what you'll get is more blinkered metering, where the questions asked aren't always that germane to the customer's real concerns. Call monitoring is likely to be outsourced to a low bidder who, in turn uses a set of fixed metrics, and so a spreading cloud of fuzz ends up surrounding the whole endeavor.
Sometimes metrics aren't the answer. One of the virtues of the "Management by Walking Around" approach is that it allows one to obtain a picture that's unconstrained by metrics, if not personal viewpoints.
Clarification added August 8, 2008:
Martijn finds my cynicism worth sharing:
There's sort of a business cliché that says that some people measure importance by how much paper they can stack around themselves. So the more reports, the better.
The quality of the reports is secondary, since often they don't get read anyway. I've actually worked in shops where some reports literally went from the printer to the trash can.
While good reports can help guide intelligent decisions, too often, reports are used unintelligently thanks to intellectual laziness or even incompetence. Classic Garbage In/Gospel Out.
Just human nature, I fear.
Organisations must learn to understand, how many metrics are making sense to get to their target. It is similar to a plane. The pilot has a lot of 'hard facts' to check the repetitive parts of the process. The soft facts, like having contact to the passengers through the flight attendants, deliver direct feedback on the current way.
The more metrics an organisation has, the more administrative work there is to keep them updated. At a certain point, there is too much work to keep them all together. Therefore, it is of huge importance to be aware, which metrics to use.
Most of the people in organizations using too many metrics don't really know what to mesure and what the value is of measuring some areas so they just measure everything. For most organisations it is very difficult to, for example, translate business strategy or objectives into valuable metrics or KPI's. My experience is that organisations tend to create operational metrics instead of metrics that they can use to actually manage a process or organisation. So the first step is understanding what the organisation would like to achieve and how metrics or KPI's can support the organisation by achieving their goals and objectives.
The main reason why organizations are using too many metrics is because there is no focus on what's important to measure. To figure out what is important to measure could be difficult. It involves knowing your strategical, tactical and operational goals and being to translate these goals into tangible metrics. Another reason is that a metric should be temporarily, meaning when a metric reaches its target and is stable the focus should change to other metric(s) (lifecycle). An important remark about a target is that an organization needs to know what is enough to reach their goals, because there also something called overperforming.
Erik H.
VP Products at Mirror42 | the people behind KPI Library
Best Answers in: Accounting (1), Budgeting (1), Internet Marketing (1)
I think this depends on how well an organization understands its performance drivers. If you do not know your performance drivers you will gather as much data as possible in the believe that you are in control of business performance (or even worse, to proof to stakeholders that you are in control).
Best in class companies understand their performance drivers (KPIs), monitor them, and adjust based on the outcome. Further more, they have regular cycles to check if performance drivers are still valid against company objectives.
Therefore, I think there is a simple test to check if an organization is in control: count the number of metrics.
Carol K.
Independent Consultant at Independent Consultant
Best Answers in: Personnel Policies (1), Computers and Software (1)
As a huge supporter of Continual Service Improvement, reporting and metrics are areas that I struggle with on most projects. It is true that most organizations tend towards one side of the scale or the other, either using too many metrics or not enough.
Those that use too many do so because the feeling exists that if a few numbers will prove they are doing their job then obviously more will really prove it. Those that use too few do so because they either do not know how to consolidate and measure items or they do not know what to measure.
For me, the most important aspect of setting up a metric is deciding what you need to know, what you want to accomplish with it, and how do you want to evaluate the metric. Using these three questions an organization should be able to determine what metrics they need, then how to get them, and finally what to do with the numbers once they have them. I do recommend to clients that they take a step back every 6 months or so and reevaluate the metrics in use. Unfortunately most clients are so excited about getting metrics that they become attached to the fact of the number and end up losing the meaning.
With clients that use too many metrics, I have found they have too many because they have lost focus of what the metrics should mean.
I recently had a client on the lesser end of the scale who had fallen into the trap of losing focus. They reported on the number of and the time to resolve p1 occurrences within the operational environment. It seems like a very simple and useful measurement. The issue arose because the operations groups classified all occurrences as errors. This meant there was no visibility into the type of error, the system generating the error, the relation of errors to changes, dependencies between systems experiencing errors and downstream consequences of errors.
This was a metric that was useful when they started reporting but rapidly became obsolete. Unfortunately they were tied so tightly to the idea of having a metric that they would not consider letting go of it. Nor would they consider altering the way the metric was calculated and reported. With this client, we did significant process and tool reengineering to provide better visibility into their environment and yet the last time I checked they were still reporting on the same metric.
Interesting "metrics" discussions.
But also understand that "metrics" includes compensation, how much is being paid for a specific level of performance! While some are quick to ask for more compensation there's considerable debate about how to measure the productivity provided for the compensation.
Assembly-line workers or "piece" workers can always look at having made 150 widgets a day that passed quality inspection. Technology and Information services and products should be no different. Managers need to know what level of output is being provided for the payroll expense. Failure to do so means the business suffers or goes under. As I believe many in this thread have addressed, the questions are varied; how to measure, what to measure, frequency of measurement, and revision of data over time.
I especially appreciate the MYBA approach in that people absolutely must know each other. But MYBA without objectivity creates opportunities for the failures of subjectivity. Translation: a manager can have one view of a person. But that view may be influenced by personalities that are unrelated to actual job performance.
Examples: I've noted many who just don't interact well with others, yet they're the jedi of getting the job done and getting the job done well. Conversely there are some who are gifted with people skills. They'll convince you they can walk on water and even create some just to prove they can walk on it. But when you step back and look at what they've actually done, they're all wet.
Based on my experience metrics in most organizations are used due to two reasons. One is to give the management at all levels much needed visibility and at least in some places though the benefits are not understood people impose it because it is industry best practice. In either case metrics management and their effectiveness is questionable. The primary reason behind this is that organizational maturity in handling metrics is generally less i.e The management sees this as a measure to judge people. On the other hand metrics are tailored to avoid the ire of the management. Another typical situation is the relevance of the metric is questionable in some situations. What needs to be done is to define metrics in a slightly more flexible manner. Secondly the management should take steps to curb any with hunts rather providing support and corrective measures to leverage fully on the usage of metrics. Induvidual groups in the organization need enough maturity to present their metrics without doctoring. The most important aspect is that those measured should have an active feedback mechanism to ensure that the metric is in line with what they do and redundant/unused metrics are removed and do not cause additional burden. I also feel innovative automatic metrics capture with user validation being the best measure possible.
As a Test Manager, part of my job involves defining the key metrics to capture and setting up tracking and reporting mechanisms during the testing lifecycle. The obvious ones to measure testing progress are the number of scripts that run, pass and fail during a cycle and the severity of the defects raised. However, to validate quality of design and development it is also important to measure number of iterations and rework turnaround time. Environment downtime should also be measured as should the number of productive mandays lost as a result. All statistics recorded should be fed into the Lessons Learned document to be reviewed at the post-Project Review workshop and maintained for future projects to use as benchmarks for estimating time and resources required. By the way, Groundhog Day is one of my favourite films.
Metrics are useful at a high-level. For example in project management, many organizations can get caught up in the details of developing, determining, or reviewing project metrics. The big project management topic today is earned value management (EVM). Good concept, in reality, very hard to justify what the Earned Value is in many projects - the cause is the over-complexity of the metrics involved in the process.
Simplification, if one discusses high-level metrics is a very good thing. Like taking a pulse, a doctor can determine how the patient is doing. The same can be done in project management metrics - take a pulse - if it's normal move on - otherwise there is a tendency to over analyze the results.
Depending on the size and complexity of the project, simplification of the metrics will result in greater results in the short and long term.
I'd say the number of metrics equals the number of purposes. Some are used for benchmarking purposes, others for taking decisions or simply for monitoring activities...
You can go to the http://kpilibrary.com website to have a look to most common metrics per category/domain. Some follow financial rules (IFRS for example), some follow standards (CoBit for example..). and it's not always easy to select the right metric...
Anyway, if you need to define/use metrics, i'd suggest you always ask the following questions :
-What is the objective of performing such a measure, (the output)
-What are the inputs
-What are the processes/tools/techniques to use to collect the data
-What are the way to ensure the results are correct
Hope this help...
Links:
Yadu K.
Experienced Business Development & Inside Sales manager; multi-linguist; open networker
Best Answers in: Change Management (2), Business Development (1)
Yadu K. suggests these experts on this topic:
Organisations use many metrics only when there is a gap between strategic intent & organisational readiness.Since the organisations are not 100% ready to operationally deliver the plans, management teams need metrics both leading & lagging indicators as cues to focus their energies on what is not working as per plan. The other reason can be lack of congruence between global initiatives & country/region specific priorities, this induces them to measure attributes both at a global level as well as regional/country level.
If you relate this to management behaviour, metrics induce rules of performance management & control on decisions. Thus at every layer of management, supervisors would introduce their own metrics to differentiate their level of control and influence performance management of people in the teams.
Arie V.
Experienced Change and Interim manager, Owner, Arivé Management Services.
Best Answers in: Organizational Development (11), Using LinkedIn (7), Change Management (5), Computers and Software (3), Staffing and Recruiting (2), Corporate Governance (2), Purchasing (1), Regulation and Compliance (1), Business Dining and Entertainment (1), Travel Tools (1), Education and Schools (1), Mentoring (1), Occupational Training (1), Government Policy (1), Business Development (1), Business Analytics (1), Manufacturing (1), Quality Management and Standards (1), Professional Books and Resources (1), Career Management (1), Communication and Public Speaking (1), Starting Up (1)
Hallo Martijn,
As an interim manager I know quite a lot of organisations and I have been confronted with an impressive amount of metrics.
It may sound cynical but my experience is that a lot of those metrics are produced to give managers the illusion that they are in control of their business. They have metrics, so they know what is going on, so they are in control and are good managers. However, reality is different.
In a lot of situations metrics are based on models that managers have about the world. And they try to adapt the organisational behavior to these models, which is not always effective. Let me give you an example.
In one of my assignments the manager responsible for sales wanted to have metrics about the number of outgoing phonecalls and the number of products sold. When sales did not grow any longer his solution was simple: the sales staff had to make more phonecalls and he announced that he would monitior the number of phonecalls closely.
You don't have to be the top scientist to predict that the number of phonecalls did increase. However sales went down. The reason: visits to potential clients were reduced and the duration of the phonecalls were reduced.
In my assignments I generally use 3 to 5 metrics for as long as changes in the values of these metrics are relevant for the business. When the situation has stabilised I stop using these metrics.
I have learned that a great help for deciding which metrics you need are two questions: What do you do when the metric is above the targer? What do you do when the metric is below the targer?
Don't be surprised if there is no concrete answer or the answers to the two questions are identical. Then you know that you don't need the metric.
Hope this helps and will give you a smile.
Arie
Metrics are driven top-down-Industry best practices Framework is combination of Balanced Scorecard and GQ(I)M. All the four quadrants of BSC will have multiple processes and metrics are required to report the efficiency and effectivness of the processes. Each metric should also have leading indicator for forecasting
E.g If we take Financial performance quadrant of BSC, There will be atleast 2 processes and for each of 2 -3processes there will be efficiency and effectivness metrics that makes it 4-6 for financial performance and for all the 4 quadrants of BSC it will be approx 24 metrics.
Blake R.
Blake Ratcliff - Director, Housing Construction and Community Development
Best Answers in: Internet Marketing (4), Commercial Real Estate (3), Staffing and Recruiting (2), Direct Marketing (2), Business Development (2), Small Business (2), Purchasing (1), Education and Schools (1), Mentoring (1), Government Policy (1), International Law (1), Offshoring and Outsourcing (1), Public Relations (1), Currency Markets (1), Equity Markets (1), Career Management (1), Using LinkedIn (1)
Because metric should (but don't always) drive performance.
Having been involved into an extended and intensive reporting cycle on an international (EMEA) servicedesk; I find most of the replies stated below a beacon of recognition. When we started out reporting on our KPI's (loads) we found that the figures didn't say anything specific, or even giving us some insight on what's going on exactly. Left us to start a process where we (as stated in some of the answers) started to ask ourselves a few basic questions.
- What do we want to do, and what can we do with the result (links in with Arie's reply).
- Which metric is going to help us improve our service delivery / outcome?
- What is the clarification (reason) of the good, moderate or bad result (getting inside the figures; analyzing the reason behind results have been really helpful during service reviews!).
Not to forget the frequency and complexity of the reports/metrics. For example; if you want to report your telephony SLA stats for multiple countries on a daily base. Do you want to inform all service-delivery managers within every region regarding all results? Sure; you can but what's the goal behind it? From experience I have noticed that this only generated a heavier workload as more people voiced their ad-hoc opinion. It became interesting seeing the figures on a monthly base and share experiences during service reviews.
Another thing that i'd like to mention is standardization; I am convinced that most large organizations still have multiple types (markup wise) of reports (Blues reports, Lights reports, Smiley reports, etc...) What do they say? basically the same! The difference here is the audience looking at the reports. By using as minimum style sheets as possible the reports turn into something recognizable and understandable throughout an organization. Naturally for these larger (international) organizations this is a tough nut to crack; but what a success it would be!
Metrics provide visibility of how something is used; that is, what is working and what is not. Understanding this can assist a business in setting strategic direction for process improvement or in developing solutions that more closely meet the needs of a business.
Generally, metrics are defined by management and should be relevant to the business. Once a solution has been implemented to capture the data to form the metrics, management should use the metrics to set a course of action, ultimately improving the current processes or to better understand what the business needs in the long term.
Metrics serve to provide direction and evidence to support an agenda, and once that agenda has been met, those metrics have reached their life cycle.
Gen F.
Supply Chain Optimisation
Best Answers in: Supply Chain Management (3), Quality Management and Standards (2), Internationalization and Localization (1), Corporate Governance (1), Enterprise Software (1)
Hi Martijin,
My answer is informed by working in trouble-shooting, project management and process optimisation over many, many organisations from corporate global entitiies to government to large privately held firms and innovative start-ups.
In general it is divisions or departments who multipy the metrics - in other words - the silos within organisations develop multiple measures in order to satisfy the perceived, implicit or (sometimes) explicit overall strategic objectives of the organisation as a whole. The outcome of this is (typically) a plethora of measures that don't necessarily add up to ensure that all parts of the organisation are, in fact, working toward the same overall goals.
Most typically what one finds is that many of the measures developed and implemented in each silo pull against the overall goal - and/or against goals developed and implemented in other parts of the organisation. Because they work and operate within a silo - frequently to different, not entirely coordinated, plans there is no analysis on whether the measures they use are actually driving the right behaviour in concert with other measures used within the organisation.
The keys to remember when establishing any sort of measures in an organisational sense, in my view, are:
* there should be between 5 and 8 overall organisational objectives
* no individual or team or department can ever be accountable for things that do not fall within their span of control
* every measure must be part of a building toward the top 5 - 8 objectives, if this cannot be demonstrated, get rid of the measure
* measurement on everything must be visible to everyone
* remember, measurement and reward drive behaviour - if you measure and reward sales people for top line sales, you will achieve high revenues....but will you make a profit?
The final comment I have is that metrics do have a life-cycle...the same measures will get tired and need replacement / revamping on a regular basis - and senior management needs to show their commitment to this.
Happy to chat further as measurement is a key element of the business optimisation projects my company, Ithaca, design and implement - hope my insights prove helpful!
Best regards,
Gen
it would be interesting to turn your question around a bit...
e.g. use an analogy to illuminate some thoughts.
a driver in a vehicle uses a simple set of metrics to determine:
speed (regulatory oversight); gas/oil levels (interesting how
each is statused); distance traveled (trip/total); engine rpm;
heat/coolant level; battery; etc.
along with the classic 'metrics' they have come out with a
newer set, e.g. fuel consumption - but what is one able to
do with that feedback, really? buy a newer vehicle?
some metrics are nice to know, but being actionable might
be a different story.
Not just that, but what would be actionable for the driver
versus his mechanic are also two different roles...
So your question seems to be turned around to me, i.e.
the cart before the horse.
It is not "why do they use so many metrics?", as much
as it is about "empowering folks in how/why/what to use
metrics for" - and yes, metric lifecycle could be important
sometimes, but it gets to be overly micro-managed in the
corporate world, simply because of the nature of control...
Donna G.
Senior Manager, Business Development - IT Programs at Northrop Grumman Information Systems
I agree with the others, Martijn. You can't improve it (reliably) unless you can measure it. I live mainly in the ITIL world where "metrics rule".
But to address your clarification from the ITIL standpoint, the "who decides what's measured", the decision shouldn't be made by a "who", really. Objectively, a map of "CSFs" ("critical success factors") should be prepared and agreed-upon by all key stakeholders (of course, there can be some discussion about WHO those key stakeholders are, but they should certainly include all process owners and the head of any organization that contributes to / is affected by a defined CSF). The CSFs need to be mapped to "KPIs" ("key success factors") that do effectively measure performance against those CSFs. This is the ITIL way, and how well you determine the CSFs and the KPIs that measure performance against them is KEY (otherwise your metrics are CRAP), but most process improvement disciplines and methodologies have some form of this mapping baked into them (CMMi, for instance).
For metrics to be effective, they need to have a "lifecycle", or - at least - be revisited often. Sometimes - if you look at the metrics realistically - you'll see that the way you should be measuring performance changes over time. Here's an example from MY enterprise:
One key measurement in most ITIL implementation is something called "First Call Resolution Rate" ("FCRR"). Because Tier I Support, or service-desk support over the phone, is the least expensive (and, done right, the very best way to handle problems), most enterprises want to maximize and increase the FCRR as much as possible. That's something we measure and report on as a KPI against the CSF of "reduction of per incident cost". But, to improve another KPI, customer satisfaction, and to deliver some added support for a new application, we decided to have a period during which each incident of certain types (often resolved by the service desk by phone) trigger dispatch of a Tier II technician to visit and work with the user. During this period, we adjusted our expected FCRR. In fact, for a short period, to monitor compliance to this change by the service-desk employees, we reversed the FCRR and looked for it to RISE. That FCRR is one key metric used in service desk agents performance evaluations - we just have to make sure that one "uniform" measurement isn't applied, since our FCRR targets and emphasis changes periodically.
To show you how much metrics are used for decision-making, follow the link I've attached. It's scary how much decisions depend on them! It makes any of us who are middle managers or consultants want to make sure that those metrics are measuring the right things the right way, for certain, since we know that those reports we give on the metrics will shape future decisions by upper management about important things (funding, priorities, etc.)
Links:
Metrics are used to measure at various levels. It sure helps to reduce subjectivity.
Why so many metrics:
Metrics gathered and analyzed are diffrent at various levels in the organization. What senior management wishes to see is different than that of the second level manager .
For example, in software development context, some of the metrics at middle management are the number of defects, number of source lines of code, number of requirement changes etc. At portfolio level, the metrics would be the number of projects behind schedule, attrition rate, field issues trend, patents per portfolio and so on.
Who decides what is to be measured:
Who decides what is to be measured? I would say that all the stakeholders at corresponding level have a say in deciding the metrics. Unless metrics based managamant has every stakeholder's buy in, it can not be successful.
Monitoing the life cycle:
Not sure what you are asking. Unless it is monitor there is no point in measuring. If defect rate is measured, the organization needs to make sure that it is within the control limits. If not take corrective steps and monitor to closure.
If number of NPIs at product line level or at brand level is 0 for last 2 years then organization has to act. But for this measure, the lifecycle will be different than for a defect rate at project level.
The down side of metrics nased managamenet:
Metrics based management is a nice story as long as it is used for proper purpose. For example a metric like number of source lines per hour of any particular language could be very objective if it is measured and organization-wide repository is maintained. It is very helpful for estimation of new project. But if code per hour is used to measure someone's performance, then it is incorrect. The people responsible for measuring, start manipulating and will ruin the entire estimation system you have in place.
Due to the pressure to measure too many parameters, often these metrics are not collected religiously and may not be very meaningful at granular levels. But to get macro trends these would still be very helpful and one can base her decisions on this.
However in organizations there are very objective metrics such as financial and other operational data. But all depends on how you use the data.
Conclusion:
Overall metrics and metrics based managamanet are essential for the health of the organization. But it is vital how we apply it to the organization.
There comes a time when qualitative justification of the work is just not good enough or causes sufficient discomfort because of its subjective nature. Then the race for metrics starts ("you can't manage what you can't measure" must be everyone's favourite saying :)
I don't see evidence of organizations monitoring the metrics lifecycle. Measurement soon degenerates into "we measure it because we have always been measuring it". Relevance isn't often questioned
Martijn,
It seems to me that too many organizations do not use any metrics at all. Actually, I don't think that the number of metrics is the most important thing, after all, 'paper is patient' (papier is geduldig). What organizations do with them is what counts. It is the difference between the 'check' and the 'act' of Deming's cycle.
When you look at Groundhog day, whatshisname has one key performance indicator. He needs to get the girl in order to get out of this horrible cycle of waking up in the same day every day. In order to do this, he does use his own deming cycle. He learns how to play the piano (in order to romance her) and he learns how to speak the proper / flattering words (all by trial and error). Saving the kid who falls out of the tree is practically a colleteral benefit.
In the end, all turns out well as you know. He gets the girl and escapes from the day. A good example of a self learning entity :-)
Organization use metrics to introduce process control and also benchmarking the projects performance and organizational thresholds. It is a practice which helps the project management team to check the health of project time to time.
It is also important to understand the success criteria for all the metrics. There is a strong possibility that from the set of metrics being captured in project delivery few of them are not even referred at any stage in the project or post delivery. Having said that I would like to see that the organization should also focus on streamlining the needs of metrics in pro-active way… which will eventually streamline the efforts on the metrics capturing….there is no hard and fast rule which can be set for all project but the organizations have to come forward to see what make sense for a particular project...So I feel its equally important to monitor lifecylcle of a metric specailly in big projects.
I would also like to add the Agile practices are on the way to address such issues….
Interesting clarification... I'd like to say that the stakeholder would typically define what's being measured. The problem here is that the stakeholder may not understand or know what -is available- to be monitored. Perhaps as owners of the data points, defining and publishing a menu of available metrics could be beneficial? Hey! A la carte metrics!
John Fuller says in this chain: "some metrics are nice to know, but being actionable might be a different story." His example was fuel consumption. Metrics may not only be provided "as requested" but can also be provided to modify behavior. Perhaps fuel consumption makes one a less aggressive driver, when that person sees their MPG drop below 20. Then again, perhaps not, if they don't mind paying for more fuel to support their driving habits. ;-)
Metrics that are provided "as requested" by a stakeholder can include a broad range of purposeful uses. Typically I've seen that requested metrics are knee-jerk responses to applied historical lessons learned -- meaning, the requestor has been burned by not knowing something. An example: the project went over budget: "give me a metric showing the budget utilization!" This is an effort to be proactive rather than reactive due to previous consequence. Who says manager's can't learn from mistakes!
Using the dashboard of a vehicle as an example (continuing John F's answer) shows that sometimes simplification of literally -millions- of data points down to a succinct few that matter -- at a point in time when they are relevant. How much gas do I have left (operational)? Is my engine running hot (operational)? How fast am I going (legal compliance)? what gear am I in (procedural)? Am I consuming too much fuel at this rate (budgetary)? ... along with -hidden- metrics we lovingly call "idiot lights" (whoops! oil pressure low?! out of gas?!) Some metrics (such as the out of gas example) apply to -other- metrics that we've ignored or let slip, (such as the fuel gauge) and serve as a warning indicator for those that may not have understood the importance of the metric (keeping the tank topped off.)
Too often we overload on metrics, it's my opinion that we should attempt to simplify the information to that which is salient for a given consumer and make sure the consumer actually understands the data being presented. Data is pretty useless without context. ("Hey, what does that little genie-oil-lamp thingie on my dashboard mean? It's been glowing red for a couple of weeks now!")
Just a few passing thoughts on metrics ;-)
I know, I know... I shouldn't use so many parentheses!! 80% of all metrics use parentheses! (Oh, did I mention that your metrics and statistics should be provable to survive scrutiny? Yeah, that too.)
John K.
Entrepreneur, Architect, Mensan.
Best Answers in: Wireless (3), Telecommunications (2), Business Development (1), Computer Networking (1), Information Security (1)
There are many metrics but relatively few KPI's constructed from these metrics. If you are prepared to work at it a new technology can bring just a few KPI's.
And yes me monitor KPI's and six sigma scores on a monthly basis.
Best regards
John Kelliher
Geoff F.
"Hands-on" Software Architect and Senior Developer
Best Answers in: Computers and Software (30), Software Development (26), Web Development (21), Wireless (13), Enterprise Software (11), E-Commerce (9), Telecommunications (8), Information Storage (6), Blogging (5), Information Security (5), Offshoring and Outsourcing (4), Biotech (4), Computer Networking (4), Using LinkedIn (4), Starting Up (2), Databases (2), Occupational Training (1), Government Policy (1), Staffing and Recruiting (1), Corporate Law (1), Internet Marketing (1), Graphic Design (1), Mobile Marketing (1), Public Relations (1), Search Marketing (1), Planning (1), Nonprofit Management (1), Social Enterpreneurship (1), Market Research and Definition (1), Energy and Development (1)
Consider the alternatives.
Would you prefer to argue for hour after hour in interminable meetings about things that cannot be measured or would you prefer to argue about things that can be measured badly?
The dirty little secret is that computer science is a social science and so we use rubber rulers. That's just how it is and, ... the pay is pretty good.
One of the main roles of management is exactly to provide and maintain a definition of success and then create the support structure for people to actually achieve it. Metrics are a measurement of such success (say, average turnaround time, turnover, margins, time to completion, etc) and thus ideally are carefully selected by the top management, and continuously maintained, improved and refined to provide a measurable definition of success. Metrics also encode the vital relationship between short-term survival and long-term sustainability and growth (which tend to be essential to any entity that wants to survive and prosper). Ultimately, the responsibility is in the CEO office. In some occasions the CEO may delegate the responsibility to VPs - especially in case a business is set up as a group of different lines with little in common - but he/she should always have at least a good idea of which metrics apply to which line, since they embody the direction of the business.
Finally: metrics are often a proxy for the real thing and they are valuable only if they are a "good" proxy - so the responsibility for their maintenance (change, replacement, improvement) also resides in the same place.
