[.ca] Investigative Data Mining for Security and Criminal ... (ISBN 0750676132)

When on-message, an excellent intro to data mining:
I read "Investigative Data Mining for Security and Criminal Detection" (IDM) after attending the 2003 Recent Advances in Intrusion Detection (RAID) conference. Researchers at RAID mentioned "self-organizing maps," "neural networks," "machine learning," and other unfamiliar topics. Mena's book helped me understand these subjects in the context of performing data mining. If you steer clear of the author's discussion of intrusion detection in chapter 10, you'll find IDM enlightening and a little scary. Author Jesus Mena defines investigative data mining as "the visualization, organization, sorting, clustering, segmenting, and predicting of criminal behavior" (p.1). His book strays from this definition, as he also covers simply discovering patterns of activity for responding to events. Accomplishing this task requires investigative data warehousing, link analysis, software agents, text mining, neural networks, and machine learning. Mena addresses each technique in its own chapter, offering descriptions, case studies, and tools. Two types of data mining analysis exist: descriptive, such as a chart, graph, or decision tree; and predictive, obtained via neural networks and machine learning (p.261). Mena also describes mining via "top-down" vs "bottom-up" approaches. The first involves an analyst exploring data to support his theories. The second relies on software to find patterns in data not imagined by a human analyst (p.343). Mena is most effective when he writes about what he knows best. I loved chapter 9, where he explains cell phone, insurance, and financial frauds. Much of what he wrote applied directly to my interest in network security monitoring and intrusion detection. Chapter 10 (Intrusion Detection), however, is best ignored. Mena does not appear to understand computer security, and neither do his editors. He calls Snort a "freeware site-based system IDS," in contrast with "network-based IDSs such as RealSecure" (p.306). He labels tcpdump an "attack" tool and says "this is utility for eavesdropping for passwords" (his typos) (p.307) and describes "rhosts" in a "stealth" attack phase as "this utility will evaluate hosts and lists hosts and users who are trusted by the local host" (p.308). Mena isn't a "security guy," either; he lumps "threats and vulnerabilities" together as "weaknesses or flaws in a system, such as a hole in security or a back door" (p.14). A threat is one or more entities with capabilities and intentions sufficient to exploit vulnerabilities in information resources, while a vulnerability is a weakness in design, configuration, or deployment which allow threats to abuse, subvert, or break information resources. Overall, I really enjoyed IDM. Mena makes numerous fascinating insights. While his prose is somewhat repetitive, he explains the key points needed to get data mining newbies up to speed. In light of the recent revelations of jetBlue sharing data with the government, the techniques Mena describes are both powerful and disturbing.

Somewhat weak on details:
I was very excited when I bought the book, but was somewhat disappointed. The reason for that is the book is very light on details and tends to talk about things rather then on how things are done and how they work. The book does cover some tools but with no connection to concepts and with few details on how the tools do what they do. It does contain a lot of interesting material and s generally well written. Of the most interest to me was the intrusion detection chapter, but in addition to a well-known facts on IDS technology it provided few details on how exactly data mining helps. MITRE case study seems to mostly hint at things rather then show how they were done in this project. I did pick up some ideas from it. Anton Chuvakin, Ph.D., GCIA, GCIH is a Senior Security Analyst with a major information security company. His areas of infosec expertise include intrusion detection, UNIX security, forensics, honeypots, etc. In his spare time, he maintains his security portal info-secure.org

Interesting applications of data mining:
It's the second book of Jesus Mena that I read. The subject of this new one is a little bit opportunistic in the world and US actuality. The book is a sort of general presentation of applications in fraud investigation in terms of models, tools and usages. Of course to build such models the book is not enough detailed to do it but all the elements are given to you to go deeper in the subject. In cas this book is to read absolutly.

Excellent reference:
It used to be that only government agencies and corporate behemoths could maintain huge data warehouses. Now, that information is only a Web trip to Google away. With the combined power of Internet tools and cheap hard drives, search engines and archival databases can enable almost anyone to find information about almost anyone else. Today's challenge, however, is not finding or storing the data, but rather making sense of it. That's where Investigative Data Mining for Security and Criminal Detection comes in. It shows how myriad distributed data streams can be harnessed to fight crime. Through easy-to-read prose, the reader learns how to use both public and private databases and networks to find threats and minimize risks. Besides explaining how data mining is done, the book introduces the reader to such techniques as intelligent agents (software that performs user-delegated tasks autonomously), link analysis (a process involving the mapping of the associations between suspects and locations), and text mining (a process used to identify a document's content based on linguistic analysis) and how they can aid law enforcement. For example, law enforcement in the United Kingdom use text mining to "institutionalize the knowledge of criminal perpetrators and organized gangs and groups," author Jesús Mena writes. Case studies buttress these points. This work is one of the first books to show security professionals the power of data mining as an investigative tool. As such, it is itself a powerful tool for the industry.

Alternative Methodologies:
Are you interested in IDS's? If yes, perhaps you may already know that there are two main kinds of IDS's: based on "known bad behavior or abuse" or based on "behavior deviation". The first kind is very well known after several popular implementations like SNORT.On the plus side they are not prone to "false positives" but, however,on the minus side they are almost useless with new form of attacks. The second kind, in turn, is very prone to false positives and not yet well implemented, but eventually can handle quite well unexpected or new form of attacks. If you are interested in this second type of IDS's then "Investigative Data Mining for Security and Criminal Detection" is a MUST. From basic definitions to a case study, you are leaded through a wonderful tour that includes among others: Intelligent Agents Text Mining Neural Networks Machine Learning Criminal Patterns Intrusion Detection So, if you are just casually interested in "deviation" IDS's or a true researcher in related areas, this book undoubtedly will be useful and of great help.