Multi-Criteria Decision Modeling for Complex Operations

Next week we will be presenting a paper at the International Conference on Cross-Cultural Decision Making in Miami, Florida. I am looking forward to participating in a highly informative and interesting session, bridging modeling and simulation disciplines with socio-cultural data for military operations. In our paper entitled “Geospatial Campaign Management for Complex Operations”, we report initial findings from a research effort to understand the complexity of modern day insurgencies and the effects of counterinsurgency measures, integrating data-driven models, such as Bayesian belief networks, and goal-driven models, including multi-criteria decision analysis (MCDA), into a geospatial modeling environment in support of decision making for campaign management. Our Decision Modeler tool instantiates MCDA, a discipline for solving complex problems that involve a set of alternatives evaluated on the basis of various metrics. MCDA breaks a problem down into a goal or set of goals, objectives that need to be met to achieve that goal, factors that effect those objectives, and the metrics used to evaluate the factor. Since the selection of metrics for specified objectives and data for computing metrics are the biggest hurdles in using MCDA in practice, both the metrics and associated data are part of our tool's library for user reuse. Below is an image of the MCDA structure. Click on any of the images in the post to see more detail. Our decision modeling tool also incorporates a weighting system that enables analysts to apply their preferences to the metrics that are most critical for the mission. Linking these decision models in a shared space within the tool creates a repository of knowledge about progress along lines of effort in an operation, providing a source for knowledge transfer for units rotating into and out of the theater. The alternatives considered in the decision model are different courses of action that can be evaluated against metrics to determine the optimal action for accomplishing the commander’s goals. Of course, working in a complex human system such as the one found in counterinsurgency and stability operation environments, our tool is not meant to be a ‘black box’ model that simply reports to the user what to do, but rather the decision analysis provides insight through both qualitative and data-driven models about what courses of action will set the conditions for a more successful outcome based on the commander’s intent.

In evaluating our tool with users, we determined that one of the most important features involves the visualization of the tradeoffs for various courses of action in the decision model. To address this, we compute the uncertainty of data based on its distribution and propagate its effect analytically into the decision space, presenting it visually to the commander. A greater dispersion represents more uncertainty, while a clustered set of data points indicates more certainty regarding the cost and effectiveness metrics for a particular course of action. In this way, we are able to represent the high levels of uncertainty inherent in socio-cultural information without negatively impacting the ability of our tool to calculate a decision model. By incorporating a visual representation of uncertainty in the model, scenarios can then be played out to determine optimization for various courses of action based on data inputs and user preferences, translating model outputs into a form that can more readily be used by military users.

To demonstrate an example of how the visualization of uncertainty would work in the tool, in the image below we have analyzed two potential courses of action relating to the essential services line of effort with the objective of supporting healthcare initiatives in an area of operations. In this case, we are deciding where to focus our efforts, comparing two districts, Arghandab and Anar Dara in Southern Afghanistan. Here we are only examining a few potential metrics: the cost of building healthcare centers proposed by local development councils; the number of basic healthcare centers already in the district; and the number of people that identified a lack of healthcare as the major problem facing their village, a question that is collected in the Tactical Conflict and Assessment Planning Framework (TCAPF) data. Our MCDA tool would compute and display the effectiveness versus costs data points from metrics corresponding to the two proposed courses of action. We want to determine which district would optimize our goal of restoring essential services with the objective of supporting healthcare initiatives by leveraging the data inputs. In considering the uncertainty, we have represented the distribution in the ellipsoid around the data point. This allows a military planner to visually analyze and evaluate the potential courses of action based on cost versus effectiveness metrics, while accounting for the uncertainty of the data. In addition, the weighting system, sliders shown on the right hand of the image, allows a military planner to experiment to determine how a change in metrics will affect the proposed courses of action.

One of the key benefits of our approach is that it allows for real-time knowledge generation. By updating the model with new data the Decision Modeler will re-evaluate the outlined courses of action against the new information, allowing the user to view trends over time in the effectiveness and cost metrics for particular courses of action. In the example below, perhaps the cost estimates went up for the proposed course of action in Anar Dara given deterioration in the security situation that affected the ability of hiring contractors to execute the project. In Arghandab, the metric could have changed according to our collection of TCAPF data, emphasizing that more people responded that healthcare is the major problem facing their village, therefore, increasing the effectiveness against our objective if we built a healthcare center there. Given the increased need, the villagers have offered to provide labor at decreased cost and will contribute a certain percentage of funds to the project, therefore representing the decreased costs associated with Arghandab data points. In this way the tool will provide course of action forecasting based on an analysis of data for the purposes of proactively planning operations that optimize the commander’s objectives.

We will be presenting a more detailed analysis of our research results at the conference, so keep an eye out for links to our papers and presentation.

Shuffling methodology for sanitizing Afghanistan TCAPF microdata: a working paper

Sometime back in February 2010 I started a working paper titled "Shuffling_Methodology_for_Sanitizing_TCAPF_Microdata" (click to download as PDF) which outlined the methodology I used for data sanitization of TCAPF data.  The sanitization approach I discuss is applicable to cases where its desired to share unclassified data while preserving the privacy (and operational security) inherent in the data.

Essentially the data which was shared with us by USAID, although it was unclassified it had distribution restrictions due to the sensitive nature of the data which was collected by 24th MEU and other units in Afghanistan.  We felt compelled to publish the results from a bayesian analysis we performed on the data and thought it best to sanitize the data first and then publish the results from the cleansed data.  In order to do so, we had to maintain the analytical value of the data by preserving the distributional properties of the dataset for the results obtained to remain valid.  We had to balance this need for preserving analytical value with the privacy needs to withhold or obfuscate data fields deemed too sensitive to disclose.

The discussion in the paper where I go through a thought process of what could go wrong should get you thinking, at least.  I welcome your feedback and ideas in the comments below.

Phishing Websites Flux Their Way

The recent article "Tracking Devious Phishing Websites" in MIT Technology Review reports that 10 percent of phishing sites are using fast flux techniques to hide themselves. ICANN describes fast flux as ‘rapid and repeated changes to host and/or name server resource records, which result in rapidly changing the IP address to which the domain name of an Internet host or name server resolves’. Fast flux is used by botnets to conceal the Command and Control server to foil takedown. Such botnets are used in DDoS, spam, phishing, malware delivery and exfiltration. In particular, the use of fast flux increases the survival rate of a phishing botnet by about 27% as discussed in the Technology Review article.

Over the last 18 months, we have tracked over 280,000 fast flux domain, IP, and nameserver entities, and witnessed the fast flux infrastructures to evolve from nascent to widespread use. Our ACM paper "Behavioral Analysis of Fast Flux Service Networks" compares  the characteristics (e.g. size, lifespan, growth, etc.) of spam, phishing, and malware botnets. Figure below shows the lifespan distribution of fast flux malware (blue), phishing (green), and spam (red) botnets in our current collection. Here the x-axis scale shows the lifespan of an inactive fast flux botnet in number of days. The y-axis shows the number of inactive domains corresponding to specific lifespan measured in days.

phish.png

In comparison to botnets used for spam and malware, phishing botnets live less than a week. In contrast, spam botnets live up to 90 days whereas malware botnets live up to 30 days.We suspect that phishing botnets receive the attention of brand protection takedown services as they target well-established brands. In contrast, malware delivery and spam botnets distribute their pain across the general population, thus avoiding retaliation.

Current domain and/or IP blacklist approach may be useful for malware delivery and spam botnets as they tend to stay a while. In contrast, such blacklist approaches are clearly inadequate to cope with phishing botnets with a short lifespan. As industry research by Cyveillance suggests that “the majority of the damage caused by phishing attacks is realized during the first 24 hours after an attack is launched”, near real-time detection capability of phishing botnets is imperative.

ECPR 5th General Conference

Last week we attended and presented a paper at the European Consortium for Political Research (ECPR) 5th General Conference in Potsdam, Germany. ECPR is a scholarly association focused on the training, research and cross-national co-operation of political scientists. From our  viewpoint, the percentage of papers dealing with fragile states was significantly smaller than papers dealing with inward issues (i.e. EU) in contrast to the situation that we would normally see on our side of the Atlantic. In terms of exhibitors, the Bartelsmann Transformation Index (BTI) was of particular interest to our research on complex operations. BTI, which is published bi-annually, promotes democracy under the rule of law and market economy with social safeguards. For instance, Uruguay joined the top 10 performers while Poland fell out of this group in the most recent edition. Another exhibitor GIGA, which has a Focus Afrika publication, indicated that they will soon start publishing their data, which is great news to the research community. One of the interesting sessions addressed the question: Is a workable peace-building concept possible? Gilles Carbonnier's paper on the role of non-state actors in resource-rich fragile states in the context of the Extractive Industries Transparency Initiative. The paper defined a set of criteria such as proportionality, non-discrimination, neutrality and independence for humanitarian assistance to differentiate from development assistance. Although indicators for these metrics are sparse, the provincial distribution of economic aid can be effectively used a proxy for measuring these metrics. Thomas Biersteker's paper on peacekeeping in theory and practice gave a nice overview of the process in building the UN Peacebuilding Commission (UNPBC), which was created to address gaps in the global response to armed conflict and conflict recurrence. The commission's charter is to  support fragile societies recovering from the devastation of war within two years after the cessation of hostilities. Since its inception in 2005, UNPC has disbursed about $250M of funds mostly in African countries.

Our paper on rumors presented by Dr. Karen Guttieri was received well and generated several questions. Rumor - information that is unsubstantiated yet widely shared - is rife during social conflict. In this paper, we analyzed rumors reported in The Baghdad Mosquito after the United States-coalition invasion of Iraq in March 2003, and mapped rumor types against public opinion polling and timeline of events that includes both insurgency and inter-sectoral conflict. Our paper shows that rumors have the potential to develop actionable cultural intelligence. The analysis of rumors can identify specific concerns and fears of a population that explain behavior and affect local cooperation with US counterinsurgency efforts. Furthermore, rumors can be used to assess foreign public opinion and measure the effectiveness of a hearts and minds campaign. While we have focused on Iraq, the concept of incorporating rumors as an intelligence source is applicable to virtually any country as long as the content analysis and rumor remedies are tailored for the culture in which they occur.

Peter Kotzian's paper on social norms analyzed the importance of macro and micro level variables allowing the individual to change its beliefs about whether a particular norm is still valid or not. The empirical findings based on survey data from 24 countries show that there are no effects of social trust on norm compliance. What makes people comply with norms is not blind trust but the belief, based on information, that the norm is still effective; hence, it is rational to comply. David  Westlund's paper on rational belief changes for collective agents was an interesting formal model to study the emergent collective beliefs from the belief systems of individual agents. This model shows that the collective must believe exactly the same as at least one of its members. Dörte  Dinger's paper analyzed partner perceptions in German-Italian bilateral relations by studying the press coverage of the incident created by Berlusconi remarks.

The Inheritance

Perhaps now more than at any other time in our nation's history, the United States faces a multitude of strategic threats and challenges. Rogue regimes, militant Islamist networks, and changing power balances from rising nations such as China, to failing states such as Pakistan, threaten to upend the security and stability of the United States. 

 

As a research assistant for The Inheritance: The World Obama Confronts and the Challenges to American Power, a book by David E. Sanger, Chief Washington Correspondent for The New York Times, I had the opportunity to dive deep into issues ranging from Chinese military modernization to cyber-security to the Iranian nuclear program. My research took me into the Pakistani nuclear establishment and the militant threat emanating from the tribal areas to the post-invasion environment in Afghanistan and the personalities shaping the debate on counterinsurgency in the post-9/11 world. 

 

The democratization of technology involving nuclear materials, cyber-attacks, and biological agents, has provided non-state actors access to weapons that were previously the purview of states. The multifaceted nature of these complex issues will require greater interagency cooperation and knowledge transfer, in particular in the civil-military field. Securing the homeland from the threat of radiological weapons will require a robust intelligence effort abroad to root out shadowy networks dealing in such materials, such as those of A.Q. Khan, increased focus on securing at-risk facilities in Russia and the former Soviet states through initiatives like Cooperative Threat Reduction, and increasing collaboration between the scientific community and government entities such as the Domestic Nuclear Detection Office to bring cutting edge research and technology to the detection of radioactive materials crossing our borders. 

 

In the cyber-security realm, bolstering public-private partnerships between government entities such as the military and intelligence community, and corporations, financial institutions, and public utilities, often the targets of cyber-attacks, will be important in developing detection and response capabilities and formulating comprehensive rules of engagement. In addition to the military component of COIN operations, civilian teams specializing in security-sector reform, judicial and political affairs, economic development, and infrastructure, will be operating in the battlespace to bolster host government legitimacy, the center of gravity in the campaign. Given the shared responsibilities in the civil-military field on these issues, fostering knowledge integration and cooperation between the various branches of government, military, and civilian stakeholders is of paramount importance to ensuring unity of effort. 

 

The Inheritance is a researched-backed analysis of the challenges we currently face, a legacy of the opportunities missed after 9/11.  While I may be biased because of my involvement with the book, I strongly recommend it to anyone interested in understanding the challenges confronting Obama and the complexities of the geopolitical environment. 

Reflections on CATCH

I attended the Cybersecurity Applications and Technology Conference for Homeland Security conference on March 3-4, 2009 in Washington, DC. I had to leave on Sunday to escape the snowstorm but it was well worth the effort. The keynote speech American Crisis in Innovation by Pascal Levensohn was the most thought provoking presentation. (See related BusinessWeek blog.) Pascal articulated the broken ecosystem of innovation in USA, and argued forcefully about the need for promoting effective innovation partnerships between government and university research organizations, corporations, and entrepreneurs. Pascal quoted several statistics from Judy Estrin's book Closing the Innovation Gap. Estrin has empirically proven that America has relied too much on incremental innovation in recent years at the expense of the open-ended scientific research that eventually leads to truly breakthrough innovation. How true! NRL funded the development of GPS in 1970s when no one could foresee the applications it spawned today. How many American organizations are investing today in the GPSs of the future? More importantly, how many decision makers are heeding Levensohn's alarm? 

Another interesting session was the panel discussion on the second day. I was particularly impressed with the comments of DHS Cybersecurity Chief Rod Beckstrom, who called for the adoption of Web 2.0 platforms within the government and the development of a generalized model for sensorizing the Internet. I was sad to read that Rod Beckstrom resigned today. It's great loss for DHS.

Our presentation on Real-time Detection of Fast Flux Service Networks was received well. The presentation generated lots of questions, and considerable interest in our Fast Flux Monitor demo at the expo. Tina Williams of Unisys asked one of the more interesting questions: From the tens of thousands of IPs in your DB, what user segments (ISP, edu, enterprise...) have this problem? Is the solution policy or technology? There is no question that ISPs and universities in USA are most seriously inflicted with the fast flux problem. The enterprise has a botnet problem with its mobile workforce. The government has started doing a better job in protecting its machines being recruited into zombies. The solution is both technology and policy. You can't be aware of the problem without the technology. However, you still need to train your personnel for effective remedies.

One final note. Congratulations to Dr. Doug Maughan, who runs the cybersecurity R&D at DHS using a collaborative model. As Milcord, we have participated in this program for the last three years. Open collaboration did improve our botnet defense solution with the suggestions of our colleagues in this program. Collaborative research programs in information technology are rare within the government. I wish more Program Managers adopted such a philosophy.

DHS Conference on Cyber Security (CATCH)

How can an organization defend against cybercrime enabled by botnets operating as fast flux service networks? Milcord will present its solution for "Real-time Detection of Fast Flux Service Networks" and botnets at the Cybersecurity Applications and Technology Conference for Homeland Security conference scheduled March 3-4, 2009 in Washington, DC. Very soon afterwards we'll be announcing the beta release of our new product Fast Flux Monitor that was the foundation for our research investigation.  To find out more about our research, visit the [[Botnet Defense]] project page.

Here's the abstract:

Here we present the first empirical study of
detecting and classifying fast flux service networks
(FFSNs) in real time. FFSNs exploit a network of
compromised machines (zombies) for illegal activities
such as spam, phishing and malware delivery using
DNS record manipulation techniques. Previous studies
have focused on actively monitoring these activities
over a large window (days, months) to detect such
FFSNs and measure their footprint. In this paper, we
present a Fast Flux Monitor (FFM) that can detect and
classify a FFSN in the order of minutes using both
active and passive DNS monitoring, which
complements long term surveillance of FFSNs.

Battling Botnets

Dr. Alper Caglayan was quoted within Military Information Technology journal's recent issue in an article titled Battling Botnets: Whether the U.S. Military Should Establish Its Own Botnet Capability is Debatable, But Defending Against Them is a Necessity.

Some organizations have developed backlists of suspicious Web pages and sources of e-mail in order to protect their systems from malware, noted Alper Caglayan, a principal investigator at Milcord, a software solutions company.

A step further would be to restrict access to the system from all sources except those appearing on an approved list. “This would not allow anything on the computer unless it is registered with your organization and certified to be free of malware,” he said. “This eliminates the effort of trying to figure out whether something belongs on the blacklist or not.”

Click here to read the full article...

Milcord at MobiSensors'07

Milcord presented a position paper titled "A Commercial Perspective: Collaborating on Application Prototypes as anInfrastructure Provider"at the NSF Workshop on Data Management for Mobile Sensor Networks (MobiSensors).

Sensor data management and fusion is a technical component in a number of our projects across a range of applications and technologies, including: · Monitoring [[SPE|Earth Science]] Data – NASA · [[GEMI|Intelligent Video Surveillance]] – Army · Enemy [[Course of Action Forecasting|Course of Action]] Analysis – Army · Quality of Service in Tactical Networks – Air Force · [[Botnet Defense|Botnet Detection]] and Mitigation – DHS