Mini Lesson on Active Lists Compressed
- Pavan Raja
- Apr 8, 2025
- 10 min read
Summary:
To address the issues described in your scenario, let's break down the steps you can take to troubleshoot and potentially resolve the problem:
### Step 1: Verify Variable Definition and Usage - **Check if the variable is defined correctly:** Ensure that the variable `DupOrder` (or any other relevant variable) is indeed defined as per the documentation provided. This includes checking its type (string, number, etc.) and ensuring it maps to the correct custom string or action from ArcSight. - **Verify where the variable is being used:** Make sure that the variable `DupOrder` is being used in a way that aligns with how you expect it to be used based on your configuration. This includes checking any rules, active lists, or field sets where this variable might be referenced.
### Step 2: Check Active Channel Configuration - **Review the active channel settings:** Ensure that there are no other active channels influencing the VE/video channel's behavior unintentionally. Verify if there is another channel that might be overriding settings for your VE/video channel. - **Field Set Association:** If you have a field set associated with the VE/video channel, ensure it includes the variable `DupOrder`. This can sometimes help in ensuring visibility and proper interaction of variables within the active channels.
### Step 3: Debugging and Testing - **Test Variable Visibility:** Create a simple rule or test event to check if the variable appears correctly when using a VE/video channel. This can be done by setting up a basic event with `DupOrder` defined, and then checking how it is displayed in the VE/video channel. - **Look for Error Messages:** Check any error messages or logs that might indicate why the variable is not being displayed as expected. These errors could provide clues about what might be going wrong.
### Step 4: Adjust Field Set and Variable Association - If you find issues with field set association, adjust the field set to include `DupOrder`. This can involve creating or modifying a field set in your ArcSight configuration where necessary adjustments need to be made for proper inclusion of this variable.
### Step 5: Consult Documentation and Support - **Review Documentation:** Refer back to the documentation provided by HPE (formerly known as Hewlett Packard Enterprise, now part of Dell Technologies) regarding any updates or clarifications that might address your issue with displaying variables in VE/video channels. - **Contact Support:** If you are still facing issues after trying these steps, consider reaching out to HPE ArcSight support for assistance. They can provide more detailed guidance based on specific configurations and requirements.
### Step 6: Check for Updates or Patches - Ensure that your version of ArcSight is up to date, as there may be patches or updates that address known issues with variable display in VE/video channels.
By following these steps, you should be able to identify the root cause of why the variable `DupOrder` cannot be displayed in your VE/video channel and take appropriate action to resolve this issue.
Details:
This document outlines a case study on fraud within a call center using ArcSight 4.0 SP2, focusing on how Active Lists can be utilized to detect fraudulent activities such as order manipulation by CSRs. The objective is to demonstrate the use of Active Lists for adding entries, verifying list membership, utilizing derived variables, and managing lists to prevent further fraud cases like the one described.
**Objective:**
The document aims to show how ArcSight's Active Lists can be used to detect and manage potential fraudulent activities in a call center environment where orders are manipulated by Customer Service Representatives (CSRs). The primary objective is to demonstrate techniques for verifying if an entry exists in a list, utilizing derived variables from the list, moving values between lists, and ultimately removing undesirable entries.
**Use Case:**
The scenario involves a CSR cancelling one order and immediately creating another under a different dealer's name as part of a fraudulent scheme involving finder’s fees. This case study illustrates how Active Lists can be used to track suspicious activities and prevent financial loss due to employee fraud.
**Implementation:**
1. **Install CEF File Reader Smart Agent**: Set up the ArcSight CEF (Common Event Format) file reader smart agent to read events in the specified format.
2. **Create demonstration orders file in CEF format**: Prepare a sample data set in CEF format that mimics order transactions, including cancellations and new orders.
3. **Create User and Notifications List**: Set up user accounts and configure notifications for alerts related to suspicious activities.
4. **Create Active Channels and Lists**: Define active channels like 'WorkOrders' and create lists such as 'Work Order', 'Work Order Cancellations', and 'Work Order Suspect'.
5. **Create Rules**: Develop rules that trigger when a new order is created or an existing order is cancelled, especially if they are part of fraudulent patterns.
6. **Move rules to Production**: Deploy the developed rules into production environments for real-time monitoring and alerting on suspicious activities.
**Appendix A - Orders.cef:**
This file contains sample data that was used to simulate actual orders in a call center environment, including cancellation events crucial for detecting fraudulent behavior.
The document concludes with instructions for testing the system's effectiveness using a test case trace and providing a schema of how data should be structured within the ArcSight platform. The goal is to provide a comprehensive guide on implementing Active Lists for fraud detection in real-time systems, ensuring that companies can swiftly identify and react to potential fraudulent activities involving their employees or external parties.
The provided document outlines a process for testing an assumption using application logs, focusing on the detection of suspected fraud in orders within an agile development environment. Here's a summary of the steps and components involved:
1. **Setup**:
Create three active lists: Orders, Cancelled Orders, and Suspected Fraud.
Implement a system to check if an order is on the CXL list (Cancelled Orders); if so, add it to the Suspected Fraud list; otherwise, place it on the Orders list.
Install the CEF File Reader Smart Agent software onto the system at `c:\arc_test`.
2. **Data Preparation**:
Create a demonstration orders file in CEF format and place it in a new directory named `c:\temp_orders`.
A mini-lesson titled "CEF for Use Case development / the shortcut to success" will be published soon, providing further details on how to use CEF.
3. **User Creation**:
Create a user named "Order/Administrator" with the following details: Last Name: Order, First Name: Order, Title: Fraud Analyst, Department: Audit Analyst.
Assign this user to a custom user group called "Order Fraud Group".
4. **Notification Setup**:
Create a notifications list and configure it as follows: Name: Order, Destination: Console, User/Group: /All Users/Administrator/Order.
5. **Active Channels**:
Create an active channel named "WorkOrders".
6. **Active Lists**:
Define three active lists:
WorkOrder (contains all received work orders),
WorkOrderCancellations (contains all cancelled work orders),
WorkOrderSuspect (contains orders that were in the cancellation list but are now new orders).
7. **Configuration**:
Set TTL hours to 0 for the active lists to prevent expiration, ensuring they remain active and up-to-date during testing.
This process is designed to facilitate the analysis of order data within a controlled environment using agile development practices, with specific attention to detecting and managing suspected fraud cases.
The provided text outlines the setup and creation of rules for managing a work order system using ArcSight, an incident management tool. Here's a summarized explanation of the process:
1. **Data Definitions**: The data related to each work order is defined including attributes such as `Ord_phone`, `Ord_num`, `Ord_date`, etc. These are crucial for tracking and managing each work order efficiently.
2. **Creating Rules**:
**Work Order New Rule**: This rule ensures that new orders are added to the active list only if they are not already present, preventing redundant entries. The key here is setting the type to "Type != Correlation" because this specific type is used internally by ArcSight for controlling rules and actions. If left unchecked, it could lead to multiple triggers of the rule due to adding rows to the active list, which would trigger a new device action.
**Work Order CXL Rule**: This rule handles the cancellation of work orders. It checks if an order is canceled (CXL ORDER) and not already in the specified "WorkOrderCancellations" active list. If conditions are met, it adds relevant details to this list.
**Work Order CXLNew Rule**: Similar to Work Order CXL but specifically for new cancellation events, also ensuring that these are not duplicates before being added to the active list.
3. **Variable and Field Setup**: A variable `DupOrder` is created to check if an order already exists in the active list using the phone number (`deviceCustomString1`) as a primary key for identification. This helps avoid duplicate entries. The value of this field can be tested by entering specific values (like "555-555-1111") during testing.
4. **Action Setup**: In both rules, the entire row read is added to the active list under the name "WorkOrder". This setup ensures that all relevant information about each order and its status is maintained in a single place for easy tracking and management across different systems.
This system helps streamline operations by providing clear guidelines on how to manage incoming orders and their cancellations, minimizing redundant entries and ensuring efficient handling of each case through automated processes.
The provided text outlines a series of actions and rules related to an event named "NEW ORDER." This event involves checking for potential fraud in order processing by analyzing various parameters such as device custom strings, numbers, actions, dates, addresses, sources, and more. Here's a summary of the key points:
1. **Event Identification**: The event is identified as "NEW ORDER" with specific attributes like Ord_phone (device CustomString1), Ord_num (device CustomNumber1), Ord_date (endTime), Ord_address (device Custom String2), and Ord_action (device Action).
2. **Initial Actions on First Event**: When the event is first detected, it triggers several actions:
It adds the event details to an active list named "WorkOrderCancellations."
Sets specific fields in the event with custom strings and numbers related to potential fraud detection (e.g., deviceCustomString3 set to $DupOrderord_num).
3. **Notification**: A notification is sent alerting about potential fraud, requiring acknowledgment. The message includes details of the order number involved.
4. **Aggregation and Variables**: The event data from "event1" is aggregated and used to populate variables for further analysis:
Specific fields are extracted and labeled as per the event details (e.g., ord_src, ord_phone).
Aggregation includes parameters like Device Custom Number3, Device Action, End Time, Device Custom String2, Device Custom String3, Severity, Device Custom String1, etc.
5. **Transition to Production**: The rules and settings from the "Public" environment are transferred to the "Real-Time rules" in the production environment using a LINK option.
6. **Configuration of Active Channel Grid**: The grid is configured to reflect specific column names related to device custom strings and numbers as required for monitoring fraud. These include DeviceCustomSting1, DeviceCustomString2, DeviceCustomString3, and DeviceCustomNumber1.
7. **Naming Conventions**: It emphasizes the importance of maintaining clear naming conventions for columns to avoid confusion or misinterpretation in the active channel grid setup.
This summary provides a condensed overview of how "NEW ORDER" event is handled from detection through analysis, aggregation, and ultimately into actionable intelligence within the system's fraud monitoring framework.
To address the task described, you need to create custom columns in a system by assigning specific names and retrieving data from predefined fields. Here's how you can do it step-by-step:
1. **Create Custom Columns**: Add new columns with the following specifications:
Column 1: Name = CUSTID, Field = $deviceCustomString1
Column 2: Name = Svc Address, Field = $deviceCustomString2
Column 3: Name = Order Source, Field = $deviceCustomString3
Column 4: Name = Order Number, Field = $deviceCustomNumber1
2. **Modify Default Columns**: Replace the default columns with these new custom columns by updating their names and fields as per your specifications.
3. **Implementing in a CEF File**: For example, if you have a CEF file named "Orders.cef" containing details like:
CEF:0|CallCenterX|ORDERS|1.0|NEW|NEW ORDER|1|cnt=1 end=04-NOV-08 08.34.09.160 AM act=NEW cs1=555-555-1111 cs1Label=ord_phone cn1=21 cn1Label=ord_num cs3=DIRECT cs3Label=ord_src
You will need to parse this data and map it to the new custom columns:
CUSTID should be mapped from $deviceCustomString1 (assuming its value is "555-555-1111")
Svc Address should be mapped from $deviceCustomString2 (assuming its value is "6001 Pinnacle")
Order Source should be mapped from $deviceCustomString3 (assuming its value is "DIRECT")
Order Number should be mapped from $deviceCustomNumber1 (assuming its value is "21")
4. **Updating the System**: Ensure that once these new columns are added, they reflect correctly in your system's user interface and any associated reports or databases.
5. **Testing**: Test the integration to ensure that all data appears as expected in the custom columns after mapping from CEF fields. Check for any discrepancies or errors during this process.
By following these steps, you will have successfully replaced default columns with new ones based on your requirements and mapped specific data from a CEF file into these customized columns.
The document outlines the usage of specific strings and actions within ArcSight, particularly focusing on device custom strings and active lists. Here's a summary of key points from the text:
1. **Variable Definitions:**
`ord_address` is defined as a string under `cs2`, and it corresponds to `DeviceCustomString2`.
`ord_action` is defined as a string under `act`, and it corresponds to `DeviceAction`. Additionally, it references `ord_src` under `cs3` which maps to `DeviceCustomString3`.
2. **Usage in Rules:**
The document emphasizes that the name of variables, including custom strings and actions, are case sensitive. For example, when creating a rule with the variable `DupOrder`, it should be used as follows:
```plaintext
DupOrder get_active_list_value("/All Active Lists/Public/WorkOrder")
```
Matching this against fields in an active list like "WorkOrder" will look for matches such as `Ord_phone` with the field `DeviceCustomString1`. If correctly entered, it returns a row where `Ord_num = 25`.
3. **Case Sensitivity and Naming:**
Although the variable function itself is case sensitive, the name being retrieved must be in lowercase. This means that if you set an event field action like:
```plaintext
set event field name Device Custom String1 Label
event field value $DupOrder ord_num
```
The active list fields such as `Ord_phone`, `Ord_num`, etc., although only the first letter is capitalized, still adhere to case sensitivity rules.
4. **Aggregation and Base Event Field:**
Variables must be included in the Aggregation tab of a rule. Specifically:
```plaintext
event1.DupOrder.ord_num
```
The base event field being matched, `DeviceCustomString1`, should also be listed here.
5. **Other Examples:**
The document suggests using Field Sets to display values in active channels by defining variables within a field set if needed.
6. **Summary of Key Points:**
Ensure that variable names are correctly spelled and case sensitive when used in rules.
Active list fields should be matched accurately, considering their case sensitivity.
Variables must be included in the Aggregation tab for effective rule functioning.
Consider using Field Sets to manage and display values efficiently across active channels.
This summary captures the main points regarding the use of custom strings and actions in ArcSight rules, emphasizing accuracy and proper naming conventions.
The issue at hand is related to a video (or VE) channel where a variable that has been defined cannot be displayed, even though the context of the active channel might be influencing its visibility. This problem could potentially be related to how the field set associated with the active channel interacts with the defined variable within this specific channel configuration.
The detailed description provided includes two parts:
1. The inability to display a predefined variable within the VE channel, which is being affected by another, possibly active channel.
2. An attempt to associate or relate the field set used in the active channel to the one where the variable is defined but not displaying correctly.
The summary of this issue can be stated as: "There's an apparent problem with a video/VE channel that prevents it from showing a previously defined variable, which seems to be influenced by another possibly active channel and a related field set misalignment."
Comments