Adriano Dias Leite C
A challenge driven character, deeply reflected in my work and life style. Nowadays a Security Manager at DHL.
How to measure security? What is an ideal SLA for security? I have my opinion, want to hear yours..
Clarification added July 11, 2007:
Gentlemen, I am really glad with your answers and i noticed there are several different points of view regarding the topic. Being a generic term, it will never have a precise answer, due to its dynamic concept.
The fact is that many colleagues face the same situation over and over again: They are asked to provide senior management with some reports on how "secure" we are.
I think that there is no real answer for this question, because the QUESTION itself is wrong. The question, as pointed by some here should be: What risks do we incur?
But again, risk is nothing so simple to measure as, for example, network downtime. How many times did you face a situation of a virus outbreak and, after it is eliminated, we are not able to precisely measure the damage? So if we cannot measure the damage after the happening, measuring BEFORE it happens is even harder...
But maybe the major risk we have is the lack of security awareness... if our servers are 99% compliant regarding patches, i should care about what this 1% means.
The question is still open, let's brainstorm and see if we find a common sense!
:)
Answers (9)
Information Security is, firstly, different for all organisations. You can try to make your company as secure as possible, and succeed but end up killing all business processes, wasting serious money and killing the ability of the business to adapt to changing market conditions and chase opportunities.
Secondly, it is an ongoing process. Every month there are new types of malware, new avenues of information leak (think linkedin or facebook or blogs), new vulnerabilities, new patches and at the same time your organisation is hopefully growing or changing.
There are some areas I think that no company should ignore: border protection, malware protection and patch management. Even if you are a single person running a business from home you should have some product(s) that offer(s) the above.
And your final defense - backups.
Right, you asked about measuring..
Use a vulnerability scanner to do a scan from the inside and outside of your network: Nessus, Retina, ISS all do a good job and generate usable reports. Use MBSA (free from Microsoft) or WSUS or a dedicated patch program to see how well you are doing patch-wise. Your anti-virus server should give you reports, compare these to your CMDB or Active Directory.
Once you have a baseline work with your IT team to bring the results to something good but not impossible to achieve. Then put in SLAs to keep it there. If you can't get it to where it should be, you should approach business and explain the risks and let them accept them or give the IT guys more money for more resources.
One more thing: most software reports can be exported to excel or csv. Use excel, it makes great pivot tables and graphs. The guys upstairs love graphs.
Last point: I haven't used the word risk very much in my answer. I have a problem with the CISA types who assume everything should be proceeded with a risk analysis. Some stuff can be, and some should be but there are some basic things that should be done even without a risk analysis. I agree that you should apply more resources to the server that contains your company's I.P. , and less to the workstation that your tea lady uses to order more tea but working out risks is more art than science in most companies and its better to get protecting than calculating.
First, it varies with your definition of security... having a SLA on security make no sense as security is not a product nor a flow or a process. Also, it is difficult to measure.
I have been asked a similar question recently, regarding the good KPI for security. For me, a secure system (or organization) is a system that behave at it is expected to and we can rely on it. So, that makes a lot of indicators needed to measure security. If we are just talking about network security or PC security, it could be a bit different but not so much. At least, a good indicator, is the number of incidents.
Also, security is an economic, a business matter. Your SLA must be linked to the cost of the downtime. If you can afford a downtime of your system for 24 hours, fine for you. If you can face a loss of confidentiality on your "sensitive" information. Fine too. If not, what are you ready to pay to avoid such incident? That is the basic of risk management.
So, to make a simple answer to a not simple question: the good SLA for security is... adequate :o)
Urmez D
Practice Manager, Enterprise Security Services at Wipro
Best Answers in: Direct Marketing (1), Internet Marketing (1)
A fairly open ended question with multiple perspectives and no one single way to measure security - the best way would possibly be to baseline against a standard and then evaluate/measure security controls that are existant / non existant.
Its difficult to project an ideal SLA model - but I'd generalise that to say that it could be the level of controls/checkpoints required to guarantee non deviation from intended security posture of an organization. From a security operations standpoint it could mean time required to open /close tickets, time for config/policy changes, availability of the monitoring services, time taken to patch vulnerabilities, update virus definition files.... and several other parameters that could be used to define SLA's
Trust this helps !
Michael S
Information security survivor
Best Answers in: Information Security (12), Software Development (2), Education and Schools (1), Risk Management (1), Personnel Policies (1), Ethics (1), Using LinkedIn (1)
First off dump the word security. It doesn't exist except possibly as a state of mind. How do you measure the unmeasurable?
Use the appropriate metric - risk. Risk management is all about making sound business decisions based on the basic risk versus reward ratio. Any science uses the basic formula for risk.
frequency of event times magnitude of loss per event.
Information Risk Management is all about defining "Information Security" in terms that can be applied to the above formula then modifying the event frequency and loss magnitude via appropriate security controls until the risk is within the tolerance of the organization.
Links:
The answer is quite simple: it is not possible to measure security. At least, not in this case. If you try to measure security, you'll probably fail, as your measurement result will heavily depend on your level of knowledge and available resources. What you might measure relatively precisely are different, small subsets of overall security. For example, you can measure Internet access security between your LAN and outside world - you'd probably have at least a simple firewall there, a device you might be able to have intimate knowledge of it's inner working, know the shortcomings etc. In this case, you might end up with quite precise measurement of *that* part of the security.
But, it gets much bigger. The first step above the simple hardware device, you're going to have to take care of the processess, people and various software solutions. While you might be able to keep tab on software and business processes, it is extremly difficult to assess the security implications of your employees (other than to distrust everyone). The more parameters you get into calculation, the more complex it gets. So either you go to some length to measure security on micro level (part of the network, a department, VLAN...) or you have to to go great lengths to have relatively accurate security overivew on global scale.
As for the SLA, it really depends on the price/requirements ratio - the more you want to cover, the more the price soar. If you're asking from an management point of view, that would be my answer. First decide what and how much you want to protect, then negotiate the price. But it is really important to understand what it is you're trying to cover with SLA.
Sowmyanarayanan V
Strategist,Leader, Management Consultant specializing in BCM,Operational Risk and Information Security
Adriano,
Personally I do not think security can have an SLA. How do we predict the detection of new vulnerabilites in our system and can we determine the number of such high/medium/low vulnerabilities every month? NO...right?
So there is an element of subjectivity involved. The security function should ideally set CIA objectives and aim towards meeting them. Self -assessments should be done to serve as a barometer and audits by internal audit/ external audit should be used as a tool to validate our progress.
Complement your CIA objectives with a risk assessment and adjust the objectives in coordination with the inherent risk involved. It is always better to take an inherent risk position, apply controls, test and then derive the residual risk, rather than take a residual risk position at the start.
The fundamental disconnect is the fact that security is a journey,never ending, hence a program management approach is the ideal one.SLA's are meant for Operations, and assume that things work in 'steady state' which is not the case here.
Thanks,
Sowmy
Sowmyanarayanan V also suggests these experts on this topic:
At Neteller we try to align the Security SLAs to other key business metrics, such as profit and availability.
As has been mentioned by another poster, absolute security comes at the price of complete loss of business and therefore it is always about risk management.
Firstly we have a detailed risk register of security related risks. We then assign a financial impact on all these risks either directly (loss of revenue through unavailability or fraud) or indirectly (reputational impact and loss of business). Obviously the latter is pretty hard to forecast, but we have got to a point where we are comfortable with the figures.
Once we have those risks identified and scored we simply measure the impact of realised security incidents vs the industry norm. This allows us to place ourselves on the performance curve vs. our partners, customers and competitors. The additional benefit of this is that we are able to provide strong business cases for new or enhanced initiatives for optimizing our security measures.
However the main benefit for us is that this aligns the performance of the various security teams with those of the rest of the business resulting in the business looking to security to contribute to the profitability of the business plan.
I agree with others that "security" by itself isn't susceptible to a meaningful SLA. You can measure the organization's adherence to its own security standard of due care. I don't think it makes sense to use the occurrence of security incidents as an SLA, but you can measure the efficiency (not necessarily effectiveness) of the response.
Chris H
Founder at WebPRpro
Best Answers in: Starting Up (4), Offshoring and Outsourcing (1), Planning (1), Small Business (1)
Adriano -
I have relatively little experience in the broader security market compared to this group, but I recently helped start a company to address a small (but we hope important) segment of security: web filtering.
At BrightCloud, we are working off of the assumption that eventually everyone will be asking for much tougher web filtering SLAs than they are now. We think there are two important metrics here: Having X% of URL requests being classified (coverage), and Y% of those classified URLs being classified correctly (precision).
To our surprise, when we wanted to do some quantitative competitive comparisons, we were not able to find any hard data on (or SLAs relating to) the coverage of web viewing that a web filtering solution would provide for end users.
We thought this was important enough that we had a third party conduct a study to compare the coverage & accuracy of the major web filtering solutions. The results are striking - most web filtering SLAs would probably be in the 50% range if they were even offered!
Chris
(A copy of the report is available below from our homepage for those who are interested in more details about the results.)
