ESM Data Field Descriptions

Summary:

This document discusses improvements in ArcSight ESM v3.0 related to database management and event categorization, focusing on a normalized schema design that optimizes storage and retrieval efficiency. Key enhancements include splitting fields like 'protocol' and 'domain' into more specific categories, preserving original log entries for detailed analysis, and implementing a multi-dimensional categorization system for enhanced precision in security event monitoring. The document outlines the steps to view detailed event information on the ESM Console and provides an overview of data processing chains within the network hierarchy, highlighting the structured approach used to ensure accurate data handling and transmission from devices to the Original Connector.

Details:

The ArcSight Database serves as a central repository for all events, storing data such as severity, creation time, triggered rules, etc., within a normalized schema. This database is optimized for efficient storage and retrieval of enterprise event information, with the Manager being the only component capable of communicating with it. The database's schema design focuses on normalizing data to optimize resource usage, including improved field usage efficiency through stricter definitions for fields like 'protocol', which was split into more specific categories ('transport protocol' and 'application protocol'), as well as splitting 'domain' into 'NT domain', 'DNS domain', and 'Device domain'. These design changes have led to significant improvements in disk space consumption, making ArcSight ESM events consume less disk space compared to pre-v.3.0 versions while providing faster event storage and retrieval. The normalization of the database schema ensures that highly repetitive data is stored in side tables, reducing overall disk usage without sacrificing efficiency. ArcSight ESM v3.x introduces significant improvements that streamline data storage, retrieval, and processing for enhanced performance and accuracy in event monitoring and correlation. The system now maintains a set of predefined fields across all events, which remain constant over extended periods, reducing the need to store individual field values with each occurrence. This approach not only economizes on storage space but also accelerates the querying and reporting processes by referencing pre-set value sets within main event tables. Augmenting this efficiency is the preservation of the original log entry for every event, which can be selectively accessed based on user requirements. This feature allows detailed inspection when needed, facilitating more granular analysis without impacting overall performance or capacity. The implementation of these changes is adaptable at various integration levels (Turbo Level 1 and Turbo Level 2), ensuring flexibility in how data is managed and retrieved. The introduction of a multi-dimensional categorization system for events enhances precision by considering six distinct dimensions: the object acted upon, the action represented, the technique used, success or failure of the action, the security implications, and the type of device reporting the event. This method supports more targeted rule creation and data monitoring configurations, providing clearer insights into the nature of each event. For instance, an event classified as Snort SID 103 (BACKDOOR subseven 22) would now be categorized with greater detail across multiple dimensions, potentially allowing for a finer distinction between actions like attempted or successful backdoor installation and communication activities. The enhanced categorization system thus supports better correlation and analysis of security events, improving the overall effectiveness of the system in detecting and responding to potential threats more accurately. The text discusses a categorization model for events in network security, where events can be categorized as either actions or states. Actions involve attempts to exploit objects with various vulnerabilities, while states describe the status of specific objects. These events are significant for understanding the security profile of protected networks. The example provided involves an event related to scanning for pre-installed backdoors (subseven 22) and categorizing it based on the actions taken during a network intrusion detection process. The text also mentions that each attribute in processed events has several characteristics, including a Label used in the ESM Console, a unique Script Alias for referencing in filters or rules, Data Type to understand how to handle the data, and Turbo Level which indicates its importance (1 for essential, 2 for optional). The default turbo level is explained as follows:

• Turbo Level 1 ("fastest") refers to essential attributes that are critical for fast processing.

• Turbo Level 2 ("faster") denotes optional but important attributes that can be processed more slowly if necessary.

This categorization helps in understanding and managing the data collected by network security systems, allowing for focused analysis of potentially significant events related to backdoors or other vulnerabilities. To view detailed event information on the ESM Console, follow these steps: 1. Select the desired event in the grid view of an active channel. 2. Right-click and select "Show event details." The event's details will appear in the Event Inspector. 3. To see all event fields, ensure no field set is selected to display the full list of available fields. 4. If you need to clear a specific field set selection, use the drop-down menu above the event fields and choose "Clear." This option allows you to view all available fields without any restrictions. The article provides an overview of various categories within the device-to-Manager information chain on the ESM Console: 1. Connector Group - involved in the device-to-Manager chain, passing data through ArcSight Forwarding Connectors for Manager-to-Manager connections. 2. Attacker Group - not described further; possibly a placeholder or omitted section. 3. Category Group - not described further; likely part of the information architecture. 4. Destination Group - not described further; likely part of the information architecture. 5. Device Group - involved in the device-to-Manager chain, passing data through Manager SmartConnectors for Manager-to-Manager connections. 6. Device Custom Group - not described further; possibly a placeholder or omitted section. 7. Event Group - related to events displayed on the console. 8. Event Annotation Group - not described further; likely part of the information architecture. 9. File Group - not described further; likely part of the information architecture. 10. Final Device Group - involved in the device-to-Manager chain, acting as a trusted reporting point before reaching an Original Connector or through Manager hierarchy. The text describes a data processing chain involving various groups and connectors within a network hierarchy, starting from the device where an event is sensed by hardware. This chain involves passing information through multiple stages to reach the Original Connector. These stages include several group categories such as Flex Group, Manager Group, Old File Group, Request Group, Source Group, Target Group, Threat Group, Resource Attributes, and Geographical Attributes. The purpose of this structured approach appears to be to ensure that data is accurately processed, passed through trusted channels, and ultimately delivered to the Original Connector for further handling or transmission across the network.

Disclaimer:

The content in this post is for informational and educational purposes only. It may reference technologies, configurations, or products that are outdated or no longer supported. If there are any comments or feedback, kindly leave a message and will be responded.