Thoughts on a Military Cyber Command — Part Three— The Security Domains
So as some have noted, the previous article had remarkably little to do with the Military — this is in part because IT systems are not ‘significantly’ bespoke — sure the software may, and often will be, but for the most part the same components you find on army bases you will find in a normal data centre.
Although one would hope for armed guards without a sense of humour at one.
This article continues in a similar vein, and so its alternate title is (drumroll please) -
Security for people who don’t do security, but probably should at least understand in principle!
It’s intended for Officers and other Project Management types to give them a basis of the how and why of security, and the items they should be considering when delivering a new system. Its based on several conversations with a friend when we realised we were explaining the same things over and over again whenever we joined a new project team.
And we both maintain the greatest risk to security is a project manager with a deadline!
No small amount of security tends to be counter-intuitive “Hey thats a nice ID card you have — I now know your full name and company, and hey there’s your linkedin profile, and with that I find your other social media feeds, and now I have your address”
Or the old classic “Hey there X, gosh its been years, how is it going, are you still with what’s his name…” and I’ve probably cloned your security card and possibly pick pocketed your keys. And believe me there is no end to the number of scams I’ve seen in infosec -
Another point that has been brought up recently is that the services don’t have the IT systems that people think -
Well here is somewhere implementing a decent level of security might help, modern systems have bundled in a huge number of security related improvements, for example if you are working with a remote system, say GovCloud, you need forward secrecy, which has only been implemented on Windows 10.
Sure that Windows 10 with a modern browser will be locked down so hard that you will dream of the days of XP — but it should be running on some nice new kit — where most of the processing power is being taken up with malware scanning and in memory and on disk encryption — but you can’t have everything!
CISSP already divides security into 8/10 domains, this is useful for people already familiar with the how of security, but skips over a lot of the supporting areas and the detail, with this I’m hoping to go broader but shallower, however it's likely I’ll start ranting at some point
Anti-Malware
This is everyone’s first line of defence, someone gets an email, or gets hit by a rogue ad farm forcing redirects to ransomware or bitcoin mining sites.
Or alternatively someone sees a USB stick in the parking lot, or maybe spots an ultra-high end mouse and keyboard by the bins, and decides “nice, Ill have that”, and plugs it straight into their computer, or visits a trade show and is given an octopus cable that is compatible with their iDeviceM — and just needs a bit of juice before they head home.
All of these are common vectors for what has been traditionally viewed as anti-virus, and there are many, many people who will fall for this kind of attack, but it has to go beyond this.
If you are shipping files around, how are you sure that those files are not infected? If you allow any put operation, how can you be sure that what is being put is benign? If your vendor sends you some software updates, how can you be sure that the vendor hasn’t been compromised?
And so Anti-Malware that often goes way beyond traditional anti-virus, providing endpoint protection (where you browse to) execution policies (only approved software can be run) and many other features.
The main vendors I’m familiar with in this space are McAfee, and Kaspersky, now the latter has been on the receiving end of some rather vigorous commentary from their links to the Russian state and gathering data from places where it was deployed.
Both of these things are completely true, I’m not aware of any major corporation not having ties to their respective states, and all Anti-Malware vendors gather data to help produce the signatures to protect systems from the latest round of vulnerabilities — but bringing them up might be a ‘Bad Thing’
Other elements to consider is ensuring signatures are updated, isolating an infected system from the network and blocking a system with outdated definitions from the network until it can be brought back into compliance.
Asset Management
What do you have, where is it, what is running on it and so forth. Sure a physical server is clearly an asset, but what of a virtual machine, a docker container, an instance running in the cloud with a connection back to on premise systems?
12 years ago I found a Novell Netware server running happily under a pile of drop cloths where it had been for the previous 8 years, happily running sitting on the network, doing nothing.
Nowadays if you are running anything without patching for 8 years with some kind of exposure to the internet, its probably mining cryptocurrencies — and thats if you are lucky.
Knowing what your assets is critical from both a defensive and forensic standpoint, and should be the cornerstone of the change, service and incident management tooling, allowing you to go back over what has happened to the system and help figure out what happened when the system was compromised.
Its also important to know if someones laptop or mobile has been stolen or ‘gone missing’, sure full disk encryption works well along with Virtual Private Networks and Multi-factor authentication, but ultimately being able to lock out one of these remote endpoints as soon as possible is critical, it can also be useful in helping in any prosecution in the case of common threat.
There is no shortage of tooling on the market for service management, although ServiceFirst definitely seems to be keeping up with the changes in the industry and provide a well featured and documented API to help integrate with other systems.
Cryptography
It’s likely that many people reading this article have had occasion to meet a cryptographer, and may have wondered how they ended up in that state — mostly its due spending far too much time thinking about mathematics.
However cryptographers are just one part of cryptography operations, and often the smallest component. Even the largest organisation has only need for a small number — their job is to determine what ciphers must be used for what particular secrets based on the risk of those secrets.
A far larger number of people will be crypto admins, I covered the core of PKI in another story but in essence an encryption key comes without any metadata, but unlocks potentially very sensitive information, whereas a certificate is a key + metadata + trust system that identifies you — the admins ensure that this crypto material is managed and controlled in a suitable manner. It’s also likely that this team will run the Hardware Security Modules that should be embedded in any secure system.
Certificates will expire, keys need to be rotated, processes on the generation of both need to be defined and followed. Given that even not terribly recent architectural practises require every link between systems to be encrypted, this is likely to take up a considerable amount of time, but failing to implement crypto correctly can result in issues such as those facing Panera Bread
But the consequences are likely to be far far worse!
Tooling tends to have been dictated from the top down and often embedded in systems, however Lets Encrypt — which you can implement internally using their open source tooling, is probably one of the best pieces of software to come out in recent years. I’m also a great fan of OpenSSL and a python tool called sslyze that I use to ensure anything I build is up to the right standards.
Data categorisation and leakage
Hopefully everyone in the forces is familiar with this, so we’re going to focus on something more general.
Every document, every email, every mechanism of sharing and storing information needs to have some definition of who can access that information and where it can be used and sent. And these are classifications that potentially need to change quickly and in a controlled manner — for example financial reports will change from most secret to public overnight when they are filed.
The tooling for managing these classifications in a digital manner is to be frank nascent, some simply add the classification rating to the document or email and rely on training of the end user, a few may attempt to stop transmission to unintended sources, but it is not an exactly difficult thing to get around.
Copy & Pasting for example, or a screenshot — sure it's likely to be against the law or policies, but you are only going to find out afterwards, if you are lucky
And so managing the leak when it happens is even more critical, there is little that can be advised on here, but punitive punishments won't be helpful, if anything it will cause people to attempt to cover up a mistake — covering up a mistake however…
Environments
When you are building a system, ideally you will have separate environments for development work, testing and production. Development will have the lightest Logical Access Management (see below)controls to allow the developers of the solution to figure out how the application will work, production the strongest, ideally with no developer access.
Because the rate of change on development systems is so high and controls significantly lower, its not uncommon that they are removed from a lot of the monitoring done further along in the development cycle.
And this is fine as long as the risk is appropriately managed -
- No direct production data is used, what is, is anonymised/sanitised appropriately
- All releases are put through a replica of production to ensure that no ‘quick hacks’ have breached the standard secure build of the platform
Given the launch of GDPR later this year across Europe, anything that could be Personally Identifiable Information should be cleaned out of any data import into a dev/test environment, so names, addresses, dates of birth and so on — and it goes without saying passwords should never be stored in a format that can be read by anyone, otherwise you could end up facing the nightmare situation T-Mobile had recently.
Secret Management
This ties in closely with Cryptography, within any system there are going to be ‘secrets’ usually database passwords and cryptographic material.
The system will need these values so it can start the application and do the job it was intended to do, so at some point these secrets have to be fed into the system. Not so historically it was common for these values to be put into plaintext files which meant once you lost say one server of the system, the rest of the stack rapidly fell.
PHP has historically been famous for this, although poor practise causes no small number of problems, see the recent case of putting decryption keys into public object stores along with the backups encrypted with them.
Over the past few years several tools have come onto the market that move away from the plaintext or encoded password sitting on a drive somewhere, cyberark which tends to be used in enterprise settings has released a number of tools, although my own experience has been mixed and it is an extensive and complex suite of tools.
Hashicorp Vault and Consul I’ve used to much greater success, there are still controls that need to be designed in and the solution becoming unavailable can be a single point of failure, but both tools audit logging, rotation and control are hugely useful in building a much more secure system.
SLAM — Security Logging And Monitoring
There are two types of organisations in the world, those who know they have been hacked, and those who are ignorant of being hacked — being in the former group means that you should be recording everything that happens on a system so your Security Operations team can spot anything unusual.
Its likely something such as this was able to result in Guccifer 2.0 being identified as a GRU agent when they forgot to turn on their VPN connection.
The purpose of SLAM is an attempt to identify people logging onto the system at strange hours from odd locations and performing activities they shouldn’t be doing. I imagine everyone has had the experience of logging onto a social media account from a foreign country and receiving an alert or other notification asking ‘is this you?’
How much data are we talking about here? Simply put lots, thankfully storage is cheap, but extracting value out of it can be quite complex.
As ever there are tools available, SPLUNK is without doubt the dominant force in this space and rightly so, its a phenomenally powerful tool that is easy to integrate with and comes with a vast array of predefined analytical tools, and it is reassuringly expensive.
For those on a more constrained budget, the traditional open source approach has been the Elastic Search (log processing tool), Logstash (or Fluentd, moves the logs into Elastic) and Kibana (dashboard, presents the information form Elastic) stack — AKA the ELK, it needs a fair amount of work to setup and keep running, but this is definitely something you do not wish to be without!
There was a trend a few years ago of recording access attempts to a database, I wouldn’t recommend this especially on a public facing system as it creates a very easy way of performing a denial of service attack on that database. Log shipping, although creating a small window of vulnerability where the logs could be changed, or streaming to the remote analysis system is much more effective.
Logical Access Management (and password policies)
Yep, this is the thing everyone without exception hates, can you prove who you are (your password) and do you have rights to access this system (your role).
Both of these are (hopefully) enforced by a central system such as LDAP or Active Directory, with the rights of people to access the system frequently reviewed and if no longer needed removed, along with people joining and leaving the organisation being added and removed as quickly as possible.
This should be a solved problem in any large organisation, although tying into the service might be entertaining, but it is certainly a far better approach than to have to maintain separate files and records that might slip through the gap.
As for password policies, an overly complex policy just encourages people to work around the policy using post-it notes, this also tells me the system isn’t likely tied in with a central identity provider with individual accounts and audit trails
Network Controls
Do you know where every network port in your organisation is, every switch and hub, and the Physical/MAC addresses of every device in the company that can connect in?
Probably not, hell I’m not completely sure of my home network!
It also doesn’t help that someone can just plug in a wireless hub/switch and extend the network in undesirable ways. Thankfully because of the (comparative) drop in cost of networking equipment, dumb devices are increasingly rare, which in turn allows more effective controls to be put in place.
Network segmentation is a must, if you don’t recognise the device being plugged in, the equipment should shunt it to a quarantined network until the device is registered correctly. Promiscuous devices (such as hubs) should be blocked with a port restriction until it is approved, deny everything by default and only permit which is necessary.
If you have any internet facing services — just about anything that allows an unauthenticated and untrusted user to open a connection to your systems, these too should be in a separate, segmented network usually referred to as a DMZ.
End user devices should be on a separate network from production servers, which should be separate from your internet stack and so on.
Networks are one of your fundamental systems, and due to a pattern that was common over the past 15 years, it’s unlikely that you have a segmented network — and moving to one after the fact is ‘complex’
Network Trust Levels
Leaving segmentation aside, it's almost certain that you will be connecting your network to another by some mechanism at some point.
Again the services already have standards around this, namely TEMPEST hardening — although this mainly relates to side channel electromagnetic leakage.
Network trust levels will be a set of definitions about how you go about connecting to one of these other networks, the public internet for example would be completely untrusted with all access logged, monitored and controlled via proxies that would ideally decrypt any traffic for additional scanning.
A trusted network might be another base or office, connected via specially provisioned network links such as X25 or MPLS. This would be viewed as just an extension of your local network.
A semi-trusted network might be another agency or company, where although traffic flows over the public internet, it is heavily encrypted via a Virtual Private Network. This sort of network would only have very limited access to your own network. Such connections should have additional authentication mechanisms to be sure that only the people who are connecting in are connecting.
As with most Infosec there are no explicit rules on the hows and whys, but knowing to ask for the trust level definitions (if any) is likely to be a big timesaver.
Patching
More years than I would care to remember I was doing a consultancy job, where the new manager had a very firm viewpoint on patching — “It broke things”, and so he banned all security patches and had for a short while, really, really good uptime figures.
Until some script kiddie fragged the public websites of course.
Patching is one of those things that has to be done, across anything that runs software, be that embedded firmware all the way up to web browsers otherwise someone will compromise them, unpatched systems are by far the most common.
How common? Lets take a look at WannaCry that caused chaos last year, using the utterly fantastic tool Shodan I can scan for all unpatched, internet exposed systems that are still vulnerable right now.
Its almost impossible to restate this enough — patch your systems! It won’t stop an APT with a Zero Day exploit (we’ll cover that in the next article), but it will stop the vast majority of the noise
Privileged Access & Recertification
This is essentially Logical Access Management — but on steroids.
There will be people in your organisation that have elevated privileges such as ‘root’ and ‘Administrator’ access — these should be roles in your directory service. And of course these superusers should only have access to the systems they explicitly need access to.
Speaking as someone who has had this level of access, be assured if someone were to grab me off the street and use harsh words, let alone a wrench, I would squeal my passwords and the answers to any questions they might have.
Anyone who thinks otherwise is delusional.
This also applies to less dangerous systems, for example accounts not being deleted when someone leaves an organisation, or passwords/keys not being rotated.
Keeping track of who these people are, and their level of current access is critical. Its also good practise to ensure that if they are on leave, that their account is automatically placed on hold so they cannot log in — if you need to grant them access it can be done on an exception basis, but that also reveals another weakness in a continuity plan (but this is a bit out of scope).
It should also go without saying that anyone with this access should be using multi-factor authentication.
Secure Build
We covered this in part one, and it remains the cornerstone of all security you will implement, you can perform all these other actions to an incredibly high level, but if you don’t know to build your systems in a secure and consistent manner — well good luck!
Security Operations
Security Operations are your valiant and noble knights who ride out 24x7x365 to defend you from evildoers. Or they are a single dude with way too much on their plates such as being a DBA, Sysadmin, Developer and general IT dogsbody!
To paraphrase the great Eugene ‘Spaff’ Spafford, “If you are the security officer without any power to make changes, your job is to take the blame” — these guys get the blame when things go very, very wrong!
DDOS attack, these get the call, major vulnerability with a cute name like Heartbleed, its they who coordinate the fix, SLAM alerts due to suspicious behaviour, I’m sure you are seeing a pattern, laptops get hijacked by ransomware, guess who?
And if you have an internet facing service, these guys need to have an axe to take to the servers if they start leaking data.
Security Operations potentially should even have signoff on any new service going live to ensure that they have the relevant contact information in case of a breach, that malware has been installed, what the systems are that it connects to and that all relevant alerting feeds into them — they should be seen as just as much as a customer as the people paying for the service.
Security Review & Consultancy
Whereas Security Operations comes at the end of the project lifecycle, Security Consultancy is a function that should be in place right at the start.
These are the people who understand the organisations security frameworks, understand the risk management systems and the level of risk desired by the organisation, and should be guiding the project manager and any architects away from solutions that are too risky, as well as booking and interpreting the results of any penetration test.
The reality is of course is that a truly secure system is a myth, and there is always an element of risk that will exist and it will get signed off, but these folk should prevent anything too egregious getting through the net.
Security Training
My own personal bugbear, and in part why I’ve spent ‘quite’ a few hours writing these articles.
Security is a difficult subject to understand, its practitioners have a rather odd way of thinking and we can generally spot each other a fair way off — probably something to do with the propellerhead hats…
Its also one of those things system administrators will genuinely believe they know, because they did something that worked that was involved with security — they used openssl to self sign a certificate but don’t set any ciphers, or they think they enabled mutual authentication by importing the public keys into each respective systems trust store.
Short intermission as I go into a corner and cry — and yes I know most people reading this won’t understand what I just wrote above.
With the increased use of automation and the power provided by cloud solutions, what this really means is that insecure systems are just built and delivered faster, of course mistakes happen, but with a proper training curriculum for non-security people, which can be as simple as don’t keep public and private keys in the same public location, catastrophic errors can be avoided.
Sure your consultancy and penetration testing teams will catch some, but ensuring people are trained to a minimal level will take a significant weight off their shoulders by preventing an issue in the first place.
Source Code
You’ve built your infrastructure in lines with best practise, you’ve spent a large fortune on network segmentation, your pentest has come back clean and your architecture is going to feature in ‘What Security’ magazine as an exemplar of what security should be.
And then a developer incorrectly serialises an object or doesn’t validate an input on a single form, and all of a sudden your customer data is leaked onto the internet, you are the lead on the six o’clock news and the CEO is being called into a select committee to explain themselves.
Or 10 years later the Chinese demo a new jet that is remarkably similar to the one you’ve been developing for the past 25 years.
Either way you are going to be facing some ‘difficult’ conversations.
The code that actually provides whatever service you are building is your biggest risk and hardest thing to lock down. There is training for developers to understand how to write code in a secure manner, but writing software is one of the most alien activities its possible to undertake. And by very definition it has full access to just about all your customer data by one mechanism or another.
There have been a few tools introduced over the last few years that perform what is called static code analysis to scan and spot poor programming behaviours and where an attack vector might be introduced, the one I’m personally most familiar with is Checkmarx which plugs in nicely with a Continuous Delivery tool such as Jenkins , its not likely to catch 100% of all possible vectors, but something is better than nothing — although it will significantly extend your build time.
Supplier Security, Auditing & Management
Everything you are doing internally, you should be ensuring that any company you bring in to deliver any IT component is doing the same thing as you are — and more!
Now some companies make this very easy, providing SOC reports, along with compliance statements that you can use to provide evidence to your regulators/auditors that you are doing the right thing.
Others will probably give you a very confused look when you start asking about FEDRAMP, ISO and ITAR controls, and so its necessary you have to start asking difficult questions.
In many cases these third party organisations are probably doing a good job anyway — their documentation however might be a bit lax, which unfortunately is what any auditor really cares about.
Alternatively you might find that the Certificate Authority you’ve been using has its signing key on a publicly available server with no controls on who can access it, and they’ve not been recording who has asked for certificates to be signed in your name.
If you are wondering why this is an issue, please, really, read THIS.
Threat Intelligence and Management
Threat Intelligence is the closest those of us in infosec get to living out our secret agent daydreams, however we’re still stuck at desks in front of a computer doing so.
I’m sure everyone has heard of hacktivist groups, most operating under the anonymous name, and ‘The Dark Web’ (dramatic music). Its a fact of life that if you work for a large organisation or other agency that can possibly upset people, including other state actors, you are likely to come under attack.
Threat Intelligence spend an awful lot of time on ‘Dark Web’ message boards, watching known suspicious twitter feeds and lurking on IRC channels waiting to see if an ‘op’ is going to kick off against their principle and can alert them to execute defence protocols…
(sorry, couldn’t help myself)
On a more practical level, TI are important in determining the scope of a non-state actor/ATP threat, what size botnets are operating out there, how many connections and how much bandwidth can they consume, is the current defence plans suitable for resisting any attack? Can we identify who is behind the attack?
Often these services can be provided by the penetration testing firm you engage (You are having separate pentests conducted right? Even with an internal team, everyone misses something and a second pair of eyes is critical), and in many ways this is crucial as it means there can be no defence of incitement and the victim can have a clean pair of hands.
If that is the route you go down (and I recommend you do), make sure you have regular meetings and ensure that they are indeed doing their jobs.
If you are lucky enough to be identified as critical infrastructure, you will also have semi-regular meetings with an overworked civil servant in a slightly better than average suit who will tell you about what Other Government Agencies have heard, possibly, maybe, rumour has.
These chaps are being paid significantly less than your suppliers, so except for some really hush-hush, burn before reading stuff, you should be asking difficult questions of your suppliers.
Time Sources & NTP
Network Time Protocol is one thing guaranteed to have system administrators swearing, but it is crucial to any decent level of security.
Ensuing that all your servers/systems have the same time is one of the most important things you can do to ensure you have a decent level of security — it ensures that all of your logs are in the right order for one thing making correlation significantly simpler between log events.
Believe me, I’ve been on the other side of this, trying to tie thread id’s together — it’s not even remotely fun.
More importantly though a great many systems depend on accurate timestamps to permit or deny access, the key example of this is the Kerberos token (not going to explain it here — gasp of relief is deafening), where it grants access to a system for a specific amount of time, before requiring reauthentication.
The same with session length (similar to the kerberos ticket), but controlled from the application/server side. Any access living beyond the time needed for completion of the work needed introduces a potential attack vector.
NTP is the way this is done in the IT world, synchronisation is critical, it should be part of any secure build, but can be missed, mostly because its an absolute PITA to configure correctly.
Likewise ensuring that the NTP server you are using is under your control (although likely is in turn being synchronised off another clock) is crucial to prevent ‘someone’ introducing drift and getting into places you don’t want.
Again fortunately for my (Western, but welcome my readers from China) government based readers, most of the top tier timesources are under your control
Vulnerability Management
And finally, the V’s (pun intended)
Vulnerability management ties in directly with patching, there is always a gap between a security researcher figuring out how to hack into a system (which is what they really do, they just don’t horde the knowledge, or frag your system).
When a new vulnerability is identified it is assigned (or should be) a Common Vulnerabilities and Exposures number, along with a whole suite of information provided by the Mitre corporation, the full database can be found here
One of the most important aspects of this is the CVSS (Common Vulnerability Scoring System) number, which actually determines how bad of a security hole you have.
First thing in the morning, directly after my first litre of coffee, was spent going through this list to see if something is going to ruin my day, its a thankless task because as we all know too well, the messenger is indeed the one that gets shot!
Alternatively your vendor might let you know if there is an issue… (I’m just going to leave this hanging, draw your own conclusions).
When finding something that could impact our stack, we had to carry out our own study to determine the scope of the vulnerability, for example the infamous Heatbleed issue in OpenSSL missed a friend of mine by mere weeks because he was still using Netscape NSS to provide cryptographic security.
Once we understand the scope of the vulnerability and how it applies to our organisation, we then need to manage the patching to ensure that we know what remains vulnerable, and what the scope of our risk is.
Thankfully in the last year or so a much, much better solution other than trawling through the latest CVE list has come out, called SAUCS this has allowed me to generate alerts and dashboards of the things I need to care about with lots of nice dashboards and an API I can directly query.
I cannot recommend this (free) tool enough, it has been a genuine lifesaver!
Conclusion
I have been doing Infosec for now 2 decades (note how I keep it ambiguous enough that this could suggest only 11 years — HA), and it is only in the last few years it has become a genuine point of concern.
But because I have spent so many years in this space, an awful lot of it comes instinctively to me, and so I have almost certainly used shortcut terms and failed to explain things to a suitable level — if there is something I can do better, please tell me, this is not meant for a security professional, but for the person who needs to understand Information Security to better do their job.
To my non-western government readers, you chaps might be the opposition, but we all build from the same blocks, better security for one is better security for all — Setec Astronomy is a myth (despite being one of the best movies ever!!), weaker for one is weaker for all.
I’m also incredibly grateful that the length of this is half of what I projected it to be!
Many thanks for making it to the end, I am very aware of how long this is, and my thanks to the various coffee shops, bars and airports in 3 separate countries that have hosted me in writing this in various states of ‘agreeableness’.
Next up, probably after a ‘short’ break is the Advanced Persistent Threat (APT) AKA The Other Guys, AKA the guys I don’t need to worry about, but you probably do!
