linux.conf.au 2014

Perth, Western Australia - 6th to 10th January 2014

linux.conf.au


<-- Back to schedule

Discovery and Monitoring without limit using the Assimilation Project

Project: Assimilation Project

The Assimilation Project (TAP) implements a radical monitoring architecture which naturally scales to 100K servers, while continuously discovering servers and services that were forgotten, detailed attributes of those things you know about, and integrates these two functions tightly together. Because monitoring is driven by discovery from a common Neo4j graph database, it is much simpler to configure and deploy initially and to maintain long-term. Because we also discover dependencies, diagnosing cascading failures is simplified. In addition, the monitoring is easily audited for completeness and systems can be audited for compliance with best practices and other policies.

TAP's discovery process has zero network footprint, which integrates well with network security policies. Approximately 30% of network breaches are accomplished through systems which are unknown or have been forgotten about. We discover those systems without sending out any network packets. For many sites, no configuration is required to perform discovery. In addition to servers and services, IP addresses, MAC addresses, and basic OS configurations, the easily extensible discovery process also discovers switch connections, dependencies, and anything else for which you've written a plugin.

This talk will give an overview of The Assimilation Project, its architecture, algorithms and implementation. It will also cover the current status and future plans of the project. It is ideal for system administrators interested in understanding and monitoring the systems they manage, or are interested in improving the quality of the services they offer. In addition, those interested in contributing to an up-and-coming open source project that makes the lives of administrators simpler, or those interested in improving security, or who are charged with compliance issues will also find this talk useful.

Alan Robertson

Alan is a long-time open source developer and frequently requested conference speaker. He worked for Bell Labs for 2 decades, SuSE for a year and IBM for a decade. He is best known for founding and his 10 years' leadership of the Linux-HA project for high-availability, now known as Pacemaker. More recently he founded and wrote much of the software for the Assimilation Project for extreme scale (>100K systems) discovery and monitoring project.