BLOG

One of the most common problems I encountered in many years of incident investigation was the lack of certain portions of evidence at the very beginning of the investigative process. A cybersecurity incident investigation requires the analysis of a lot of data to determine with certainty the events that allowed for the incident and the activity of the perpetrator from start to finish. We typically know the victim system very early, and investigations tend to branch off from the victim to look for connections to other systems so we can locate the course of the attack and any additional systems the perpetrator may have accessed. These are common steps in the investigative process that cybersecurity professors already teach their students. The problem, however, lies in the availability of data that makes the investigative process easier and allows for a quicker timeline to secure and recover the victims and protect the network from similar future attacks. 

Teaching Students the Limits of Default Network Logging 

Network devices are configured by default to log all incoming and outgoing connections. Storage space typically limits these logs from being retained indefinitely on the device, and a common configuration is to have the logs overwrite themselves at a set interval so that storage never becomes full. This is a logical practice and one that is necessary to maintain operational capabilities. But we should be teaching our students to download these logs to an offline location before they are overwritten. Networks with a lot of traffic data could have these logs overwritten every hour, while those with less traffic may be able to stretch the overwrites to a full week. Neither of these is an effective strategy because network breaches are rarely noticed in real time, and investigators often need historical data to piece together the puzzle. Perimeter security devices, and sometimes even routers, fall into the purview of cybersecurity professionals, so classroom education should include log retention methods as best practices. 

How Attackers Use “Low and Slow” Techniques to Evade Detection 

In practical reality, students should be trained to understand that a patient hacker will use a “low and slow” method to probe a network over a long period of time to hide their activity in the middle of otherwise noisy logs. If done successfully, a hacker can remain undetected almost indefinitely and, if the logs are eventually overwritten, the final attack, if detected, will be impossible to trace back to the beginning of the activity. This is a major reason why professors should drive this point home in the classroom. Moving logs off of the network device and onto an external storage medium makes a plethora of historical data available to the investigator if it is ever needed. We have the added benefit today that external storage is relatively cheap compared to 20 years ago and takes up only a fraction of the space. There is really no good reason not to take these measures. 

Why Long-Term Log Retention Must Be Part of Your Cybersecurity Curriculum 

Every investigator wants the full lifecycle of the attack. In many hacking incidents, we do not take a lot of time investigating common attacks against unpatched systems because there is not a lot to learn in those situations. But if systems on the same network are attacked on a regular basis, the interest level goes up, as it tells us someone may have established persistent access. The ability to trace through connections over several months or longer can help determine the true source of attacks and correlate this activity with other negative effects on the network. It also allows for the building of a list of blocks that can be used to keep the hacker out of the network on a more permanent basis. Without long-term log capabilities, investigations end up being a lot of guesswork, and this is not the position where we want our students to be stuck. 

Using Log Data as Admissible Evidence in Cybercrime Investigations 

Guessing who the attacker is will rarely result in protection from future attacks. This is the method hackers want defenders to employ because it helps them maintain the upper hand. And from a law enforcement perspective, guessing will never put the bad guy behind bars. Quality log retention can be used in court cases to prove activity as long as their backups are maintained in a consistent process and stored in a way that they cannot be modified. Sometimes, investigators only care about what happened and how to keep it happening again. But law enforcement-based investigations want to put the perpetrator out of business with some form of punitive action. This is a higher standard to prove, but the groundwork can be set by configuring network devices to backup plans well before an attacker strikes. 

Teaching Students the Broader Operational Value of Log Analysis 

Another big advantage of log retention that students should be taught is that, even apart from security, log analysis can help detect configuration errors that lead to outages or other forms of downtime. Network engineers have used logs for centuries to trace problems, look for trends that can yield potential future issues, and to find places where new hardware is needed. So log retention has many potential benefits and can be one of the easiest adjustments to make in network security management. 

Training Future Investigators to Recognize Meaningful Log Patterns 

Professors can say, with a high degree of confidence, that the more data we have available during an incident investigation, the more likely we are to fully solve the mystery. Knowing where the attacker originated, the different ports they probed before an attack, and how long they have been poking around the network reveals the level of an attacker's sophistication. Script kiddies who run random attacks against secure systems can be spotted a mile away because the logs will show a lot of noisy activity. These are not the ones that students need to be concerned about. The ones cyber security professors should focus on in the classroom are those who know how to evade capture by hiding small amounts of data in communication logs. Saving and analyzing those logs is often the key to a successful investigation.