ROCK SE Tips and Tricks

Summary:

The text provided is a summary of various tasks related to different systems and applications, including Syslog configuration, log management, email template customization in ESM (Enterprise Security Manager), and integration with MSSP services. Below, I'll break down the key points from this text for clarity and organization: ### Task 1: Install and Configure the Syslog File Reader Connector #### Steps: 1. **Prepare Environment**: Ensure you have a Linux machine with appropriate permissions where the log files are stored. The connector should be installed and running properly in another shell window. 2. **Access Logger Interface**: Navigate to the directory containing the .gz log files. Use commands like `gunzip -c logfile1.gz logfile2.gz logfile3.gz >> //concat.log` (or use gzcat if preferred) to concatenate these compressed log files into a single file named "concat.log". 3. **Verify Connector and Log Collection**: Ensure the connector is running, that "concat.log" is growing in size, and check if the "SyslogFileTest" receiver in Logger is collecting events. Review permissions on Linux and ensure proper configuration of the connector if necessary. 4. **Manage Growing File Size**: Recognize that "concat.log" will continue to grow as more .gz files are concatenated. Periodically delete and recreate "concat.log" as a 0-byte file in the same location after parsing it into Logger, allowing for continued concatenation of new log files. 5. **Using End Time for Searches**: Use the endTime field for historical log searches instead of human-readable formats. This is automatically indexed and can be filtered using a calendar interface in Logger's Search Composer. 6. **Search Setup**: Open "Search Composer" in Logger, set the filter options to use "endTime", choose an operator (usually "="), and enter specific date and time by clicking on a calendar icon next to the endTime field. 7. **Execute Search**: Perform the search using specified parameters based on the historical log entries corresponding to the selected end time. ### Task 2: Customizing Email Templates in ESM #### Steps: 1. **Understand the Need for Customization**: Recognize that default email templates provided with ESM may not meet expectations during trials or other uses, prompting a need for customization. 2. **Set Event Fields and Velocity Macros**: Create specific email notification templates based on different types of correlation alerts by setting event fields in the alert criteria and selecting appropriate velocity macros that adapt to these fields. 3. **Refer to Example from Ameritrade (TD Ameritrade)**: The example provided by Vigilant for Ameritrade can be used as a guide, showing how to tailor email templates according to different alert types using specific event fields and velocity macros. 4. **Dynamic Adjustment of Templates**: Use the method to dynamically adjust email templates according to various scenarios or alerts within the ESM system, improving user experience and engagement during critical security events. ### Additional Information: - The text includes references to other documents detailing how-tos for mapping MSSP customer data restrictions and general training materials for using Logger and ArcSight products. - Mention of Jive SBS community software version 4.0.11 as a related resource. This structured overview helps in understanding the tasks associated with different systems, applications, and tools involved in log management and security notifications.

Details:

This document, titled "SE Tips and Tricks," provides useful information for Sales Engineers (SEs), focusing on specific techniques related to integrating various systems such as ArcSight SIEM with NetWitness Investigator, configuring NetWitness CEF integration, and utilizing the Connector Syslog Simulator. The document also includes tips for creating demo events using a Replay Generator, detailing how to create a desktop shortcut to generate these events easily. Overall, this resource is intended to enhance the efficiency and effectiveness of SEs in their roles by providing practical guidance on working with different tools and systems within the cybersecurity domain. This document outlines a step-by-step process for using the Asset 3.5 Resource Generator tool and Event Streamer Manager (ESM) to import event data, specifically zones or assets for customers. The steps are as follows: 1. Extract the Resource Gen zip file into a working folder. 2. Place the "asset file".csv in the same folder. 3. Open ESM and set up a channel for the desired event source. 4. Run Event Streamer Manager (ESM) and invoke the Replay File Generator wizard to connect to your instance of ESM using standard credentials. 5. In the Replay File Generator, select the target file location and name, enter the start and end dates based on when events were played into ESM, choose an event filter specific to the desired events, and decide whether to sanitize IP addresses or hostnames if applicable. 6. Click "Next" twice to generate the replay file. 7. Move the generated replay file to C:\arcsight\agent\current and restart your demo connector. The new replay file should now be visible in ESM. 8. The process might require adjustments to filters to avoid including unwanted ArcSight correlation events or ASM events. This method, developed by Erik Barnett with guidance from Jim Rutherford, aims to simplify the import of zones or assets for customers who may have encountered difficulties with previous attempts using different tools. This document outlines steps for using ArcSight tools to import assets and create a dark IP list, along with instructions for generating a Dark IP address list from RADB.NET. Step-by-step guide: 1. Navigate to the folder containing ResourceGen and "Asset file".csv. 2. Open a Command Window and move to the specified folder. 3. Run the command: `java -cp . ResourceGenerator "assetfile".csv "assetfile".xml`. 4. Copy and place the new "asset file".xml into the Arcsight manager folder: arcsight40\manager\. 5. Move to the arcsight40\manager\bin folder and run the command: `arcsight archive -u "user name" -m "manager name" -i -f "Asset File".xml`. 6. Double-check the ESM manager for successful import. Ensure unique zone names without ampersands. 7. For generating a Dark IP list, create a shortcut with the command: `C:\arcsight40\console\current\bin\arcsight.bat whois -s whois.radb.net fltr-unallocated | grep "0.0.0/8^+" | gawk "/./ {print $1}" | sort -un | cut -d. -f1,2,3| grep -v "filter:" | gawk '{print $1 ".0/8"}' > ..\..\..\darkaddress.txt`. This script uses ArcSight's whois tool and Cygwin to fetch reserved address space from RADB.NET, filtering out unallocated IP ranges and generating a list of dark IPs in the specified directory. The text provided outlines a method for customizing an ArcSight console to include a user-created graphical representation similar to the images found in a PCI demonstration environment. This process involves manipulating configuration files and utilizing the ESM (Enterprise Security Manager) image editor within the ArcSight Console. The steps outlined are as follows: 1. Obtaining the drawing file, preferably in Visio format or a high-quality JPG at 150dpi. 2. Opening this file in Visio and exporting it as a JPG with dimensions set to 150dpi x 100dpi. 3. Modifying a specific line in the ArcSight configuration file (admin.ast) on the prospect's computer to enable the image editor. 4. Accessing the Image Editor within the ArcSight Console, creating a new entry with the exported JPG and resizing it appropriately. 5. Utilizing the help features of the Image Editor to add and customize chart types that reference specific filters saved in the system. 6. Saving the edited image using the customer's name. 7. Ensuring compliance with licensing agreements and adhering to best practices for data handling and privacy, especially when working within PCI-regulated environments. 8. Reviewing relevant documentation or using the built-in help features of the software for guidance on how to perform each step effectively. The text provides instructions and descriptions related to various tasks in managing systems and data, particularly within a security or management framework using ArcSight. Here's a summary of each point mentioned: 1. **Channel Viewer Setup**: This involves navigating through a platform feature called "Active Channel" (or similar). At the bottom right of the channel window, click on an icon labeled as a "Channel Viewer type." Then, select "Image Viewer>" followed by entering the customer's name. Ensure that filters set for different chart types are populated with data to facilitate proper viewing and management. 2. **Unlocking Asset Tree**: If you encounter issues with duplicate assets in the asset tree under ArcSight System Administration due to numerous entries, add a specific property to the server configuration file (server.properties). This involves adding certain URIs to the resource.lock.uri.exclude.list property and then restarting or bouncing the Manager application. The URIs include various paths under "/All Assets/ArcSight System Administration" which are excluded from locking mechanisms for easier cleanup of duplicates. 3. **How-To Docs by Paul Bowen**: A zip file containing a collection of tips provided by Paul Bowen, possibly useful during trials or other stages of setup and management. The content is not detailed in the summary but suggests it contains practical advice on various topics related to managing systems using ArcSight. 4. **Migrating ESM Zones to Logger Destinations**: This document outlines how to transfer an existing network model from Event Stream Manager (ESM) to connectors that send events to a Logger system. It focuses on migrating zones and configuring them with Logger destinations for better data segregation and management based on customer needs. 5. **ArcSight Logger RBAC MSSP Procedures**: Describes the process of implementing multi-customer mapping on a connector used in ArcSight Logger, specifically tailored for MSSP (Managed Security Service Provider) scenarios. It discusses how to use this setup to manage separate customer data with distinct RBAC (Role-Based Access Control), and suggests using a Syslog connector on the Connector Appliance as an alternative to traditional methods. 6. **Test Scenario Involving VMWare, Windows Logs, Snare NAT**: This test scenario involves setting up a VMware virtual machine running Windows that sends logs to two Network Address Translation (NAT) destinations using Snare software with two virtual interfaces for each customer's IP addresses. The purpose and details of the setup are not fully detailed in this summary but suggest it is part of testing or configuring systems related to event management and data flow within a network environment. Overall, these points provide insights into managing various aspects of an ArcSight system using different tools and configurations, which could be crucial for maintaining and troubleshooting large-scale security information and event management operations. The provided text is a communication from Mathew Varghese regarding the process of forwarding SYSLOG events from Cisco MARS to an Elasticsearch Management and Reporting System (ESM). He mentions that during a proof of concept (POC), events were successfully forwarded via SYSLOG, but suggests sharing this information in case it helps others who might face similar challenges. The document Mathew refers to is titled "ArcSight_Logger_RBAC_MSSP_Procedurev2.doc" and contains detailed steps on how to configure the Syslog connector for MARS to forward events correctly. The main focus of this procedure involves setting up a Syslog connector either through software installed by the customer or via the Connector Appliance, creating mappings for NAT IP addresses to populate the Customer URI field with appropriate customer names, configuring Search Group Filters and user groups with Logger rights and search permissions, and assigning these filters to respective groups. The steps outlined are as follows: 1. Install a Syslog connector either on a customer-procured system or the Connector Appliance. 2. Create a "mappings" file to parse NAT IP addresses and populate the Customer URI field with customer names. 3. Develop two Search Group Filters for each Customer URI (e.g., CustomerA-Restrict and CustomerB-Restrict). 4. Create two user groups named GroupA and GroupB, granting them Logger rights and search permissions. 5. Add two users, UserA and UserB, assigning them to their respective groups. 6. Assign the Search Group Filters to each group. 7. Finally, log in as each user to verify that they can only access customer-specific data. This procedure is part of a larger effort to manage and filter SYSLOG events efficiently within an organization's security infrastructure using ArcSight Logger 3.0 and historical syslog configurations. The article outlines a procedure for restoring historical UNIX Syslog files using Logger 3.0 Patch 1, which no longer supports Event Time Parsing for CEF UDP, CEF TCP, or SmartMessage receivers. As a result, forensic investigators must rely on the endTime displayed as EPOCH in queries. This method involves setting up a CEF UDP receiver and configuring it to read multiple historical UNIX syslogd files compressed into .GZ format. Here's a step-by-step breakdown of the procedure: 1. **Configure the Syslog Receiver**: Set up a CEF UDP receiver on Logger named "SyslogFileTest". This method is preferred over SmartMessage as it is faster due to no encryption involved. 2. **Log into Linux Box**: Access the Linux box with root permissions or using sudo if necessary. 3. **Create an Empty File**: On the directory where the connector will read the log file from, create a 0-byte file named "concat.log". 4. **Concatenate Log Files**: Since the Syslog connector can only read one file, concatenate multiple .GZ files into one. This is done using commands like `gunzip` to decompress and then using command line tools such as `cat` or `cp` to combine them if necessary. 5. **Install and Configure the Connector**: Install the Syslog File Reader connector in a directory with appropriate read/write permissions, configure it to read "concat.log" and send data to the "SyslogFileTest" receiver on Logger. 6. **Start the Connector**: Start an instance of the connector from the command line using the command `./arcsight agent`. This process allows for restoring historical Syslog data into Logger, even though it is no longer possible to parse event times directly. It's worth noting that if devices are sending events with incorrect future timestamps due to misconfigured device time, this method might not resolve indexing issues caused by such anomalies in the logs. To summarize the provided procedure for searching historical logs based on end time using a Logger tool, here's a step-by-step breakdown: 1. **Prepare Environment**: Ensure that you have access to a Linux machine where the log files are stored and that the connector is running properly in another shell window. Also, confirm that the external storage device containing historical log files is attached or copied to the Linux box. 2. **Access Logger Interface**: Open a new shell window and change the directory to where the .gz log files are located. Use commands like `gunzip -c logfile1.gz logfile2.gz logfile3.gz >> //concat.log` (or use gzcat if preferred) to concatenate these compressed log files into a single file named "concat.log". 3. **Verify Connector and Log Collection**: Ensure that the connector is running, that "concat.log" is growing in size, and check if the "SyslogFileTest" receiver in Logger is collecting events. If not, review permissions on Linux and ensure proper configuration of the connector. 4. **Manage Growing File Size**: Recognize that "concat.log" will continue to grow as more .gz files are concatenated. To avoid this issue, periodically delete and recreate "concat.log" as a 0-byte file in the same location after parsing it into Logger. This allows for continued concatenation of new log files. 5. **Using End Time for Searches**: When searching for historical logs, use the endTime field instead of human-readable formats like years or months. This is automatically indexed and can be filtered using a calendar interface in Logger's Search Composer. 6. **Search Setup**: Open "Search Composer" in Logger, select the Name dropdown under filter options, and set the field to "endTime". Choose an operator (usually "=") and enter the specific date and time for your search criteria by clicking on a calendar icon next to the endTime field. This will populate the field with the requested time format. 7. **Execute Search**: Perform the search using the specified parameters. The results should be based on the historical log entries corresponding to the selected end time. This procedure outlines how to efficiently manage and query large volumes of compressed log files stored in a Linux environment, ensuring that they are accessible for analysis through Logger's interface. The text discusses the customization of email templates within ESM (Enterprise Security Manager), which is used for correlation alert notifications. It mentions that the default email templates provided with ESM are basic and may not meet the expectations of customers during trials or other uses, prompting a need to customize them. The solution involves creating specific email notification templates based on different types of correlation alerts. This can be achieved by setting event fields in the alert criteria and selecting appropriate velocity macros that adapt to these fields. The document suggests referring to an example provided by Ameritrade (now known as TD Ameritrade) which involved producing a detailed template for customization purposes. The example, originally created by Vigilant for Ameritrade, is designed to help users understand how to tailor email templates according to the type of alert being triggered. This includes setting specific event fields and choosing templates based on those fields using velocity macros. The method described in this document allows for more tailored communication that can be dynamically adjusted according to different scenarios or alerts within the ESM system, which is particularly useful for improving user experience and engagement during critical security events. This document contains a variety of information related to different topics such as logging, MSSP (Managed Security Service Provider), Arcsight training, and more. It includes various documents authored by Gary Freeman, detailing how-tos for mapping and restricting MSSP customer data, as well as general training materials for using the Logger and ArcSight products. There are also mentions of other resources like Jive SBS community software version 4.0.11.

Disclaimer:

The content in this post is for informational and educational purposes only. It may reference technologies, configurations, or products that are outdated or no longer supported. If there are any comments or feedback, kindly leave a message and will be responded.