As part of the DARPA SocialSim challenge, we address the problem of predicting behavioral phenomena including information spread involving hundreds of thousands of users across three major linked social networks: Twitter, Reddit and GitHub. Our approach develops a framework for data-driven agent simulation that begins with a discrete-event simulation of the environment populated with generic, flexible agents, then optimizes the decision model of the agents by combining a number of machine learning classification problems. The ML problems predict when an agent will take a certain action in its world and are designed to combine aspects of the agents, gathered from historical data, with dynamic aspects of the environment including the resources, such as tweets, that agents interact with at a given point in time. In this way, each of the agents makes individualized decisions based on their environment, neighbors and history during the simulation, although global simulation data is used to learn accurate generalizations. This approach showed the best performance of all participants in the DARPA challenge across a broad range of metrics. We describe the performance of models both with and without machine learning on measures of cross-platform information spread defined both at the level of the whole population and at the community level. The best-performing model overall combines learned agent behaviors with explicit modeling of bursts in global activity. Because of the general nature of our approach, it is applicable to a range of prediction problems that require modeling individualized, situational agent behavior from trace data that combines many agents.

Simulating and predicting planetary-scale techno-social sys- tems poses heavy computational and modeling challenges. The DARPA SocialSim program set the challenge to model the evolution of GitHub, a large collaborative software-development ecosystem, using massive multi- agent simulations. We here describe our best performing models and our agent-based simulation framework, which we are currently extending to allow simulating other planetary-scale techno-social systems. The chal- lenge problem measured participant’s ability, given 30 months of meta- data on user activity on GitHub, to predict the next months’ activity as measured by a broad range of metrics applied to ground truth, using agent-based simulation. The challenge required scaling to a simulation of roughly 3 million agents producing a combined 30 million actions, acting on 6 million repositories with commodity hardware. It was also impor- tant to use the data optimally to predict the agent’s next moves. We describe the agent framework and the data analysis employed by one of the winning teams in the challenge. Six different agent models were tested based on a variety of machine learning and statistical methods. While no single method proved the most accurate on every metric, the broadly most successful sampled from a stationary probability distribution of actions and repositories for each agent.

We model the evolution of GitHub, a large collaborative software-development ecosystem, using massive multi-agent simulations as a part of DARPA's SocialSim program. Our best performing models and our agent-based simulation framework are described here. Six different agent models were tested based on a variety of machine learning and statistical methods. The most successful models are based on sampling from a stationary probability distribution of actions and repositories for each agent.

In many domains, high-resolution agent-based simulations require experiments with a large number (tens or hundreds of millions) of computationally complex agents. Such large-scale experiments are usually run for efficiency on high-performance computers or clusters, and therefore agent-based simulation frameworks must support parallel distributed computations. The development of experiments with a large number of interconnected agents and a shared environment running in parallel on multiple compute nodes is especially challenging because it introduces the overhead of cross-process communications. In this paper we discuss the parallel distributed architecture of the farm agent-based simulation framework for social network simulations. To address the issue of shared environment synchronization we used a hybrid approach that distributes the simulation environment across compute nodes and keeps the shared portions of the environment synchronized via centralized memory storage. To minimize cross-process communication overhead, we allocate agents to processes via a graph partitioning algorithm that minimizes edge cuts in the communication graph, estimated in our domain by empirical data of past agent activities. The implementation of the toolkit used off the shelf components to support centralized storage and messaging/notification services. This architecture was used in a large-scale Github simulation with up to ten million agents. In experiments in this domain, the graph partitioning algorithm cut overall runtime by 67% on average.

Effective reasoning about the impact of security policy decisions requires understanding how human users actually behave, rather than assuming desirable but incorrect behavior. Simulation could help with this reasoning, but it requires building computational models of the relevant human behavior and validating that these models match what humans actually do. In this paper we describe our progress on building agent-based models of human behavior with passwords, and we demonstrate how these models reproduce phenomena shown in the empirical literature.

We discuss our ongoing work with an agent-based password simulation which models how site-enforced password requirements affect aggregate security when people interact with multiple authentication systems. We model two password memorization techniques: passphrase generation and spaced repetition. Our simulation suggests system-generated passphrases lead to lower aggregate security across services that enforce even moderate password requirements. Furthermore, allowing users to expand their password length over time via spaced repetition increases aggregate security.

Agent-based modeling can serve as a valuable asset to security personnel who wish to better understand the security landscape within their organization, especially as it relates to user behavior and circumvention. In this paper, we argue in favor of cognitive behavioral agent-based modeling for usable security, report on our work on developing an agent-based model for a password management scenario, perform a sensitivity analysis, which provides us with valuable insights into improving security (e.g., an organization that wishes to suppress one form of circumvention may want to endorse another), and provide directions for future work.

Security subsystems are often designed with flawed assumptions arising from system designers' faulty mental models. Designers tend to assume that users behave according to some textbook ideal, and to consider each potential exposure/interface in isolation. However, fieldwork continually shows that even well-intentioned users often depart from this ideal and circumvent controls in order to perform daily work tasks, and that "incorrect" user behaviors can create unexpected links between otherwise "independent" interfaces. When it comes to security features and parameters, designers try to find the choices that optimize security utility---except these flawed assumptions give rise to an incorrect curve, and lead to choices that actually make security worse, in practice. We propose that improving this situation requires giving designers more accurate models of real user behavior and how it influences aggregate system security. Agent-based modeling can be a fruitful first step here. In this paper, we study a particular instance of this problem, propose user-centric techniques designed to strengthen the security of systems while simultaneously improving the usability of them, and propose further directions of inquiry.

Users’ mental models of security, though possibly incorrect, embody patterns of reasoning about security that lead to systematic behaviors across tasks and may be shared across populations of users. Researchers have identified widely held mental models of security, usually with the purpose of improving communications and warnings about vulnerabilities. Here, we implement previously identified models in order to explore their use for predicting user behavior. We describe a general approach for implementing the models in agents that simulate human behavior within a network security test bed, and show that the implementations produce behaviors similar to those of users who hold them. The approach is relatively simple for researchers to implement new models within the agent platform to experiment with their effects in a multi-agent setting.

We describe an agent-based model of individual human behavior that combines a dual-process architecture with reactive planning and mental models in order to capture a wide range of human behavior, including both behavioral and conceptual errors. Human operator behavior is an important factor in resilient control of systems that has received relatively little attention. Models of human behavior and decision making are needed in order to test existing control systems under a range of conditions or analyze possible new approaches. While the model we describe has been developed and applied in the area of cyber security, it is relevant to a wide range of resilient control systems that include human operation. We discuss an application to modeling operator behavior in a nuclear power plant.

Individual and organizational computer security rests on how people interpret and use the security information they are presented. One challenge is determining whether a given URL is safe or not. This paper explores the visual behaviors that users employ to gauge URL safety. We conducted a user study on 20 participants wherein participants classified URLs as safe or unsafe while wearing an eye tracker that recorded eye gaze (where they look) and pupil dilation (a proxy for cognitive effort). Among other things, our findings suggest that: users have a cap on the amount of cognitive resources they are willing to expend on vetting a URL; they tend to believe that the presence of www in the domain name indicates that the URL is safe; and they do not carefully parse the URL beyond what they perceive as the domain name.

The existence of and market for notebooks designed for users to write down passwords illuminates a sharp contrast: what is often prescribed as proper password behavior—e.g., never write down passwords—differs from what many users actually do. These password logbooks and their reviews provide many unique and surprising insights into their users’ beliefs, motivations, and behaviors. We examine the password logbooks and analyze, using grounded theory, their reviews, to better understand how these users think and behave with respect to password authentication. Several themes emerge including: previous password management strategies, gifting, organizational strategies, password sharing, and dubious security advice. Some users argue these books enhance security