Skip to content Business Need
The client used a static, rule-based, or query-based approach to monitor the activities of its trading members and clients. Given the dynamic nature of trading partner behaviors, client actions, and stock exchange functions, a static approach was inadequate for effective anomaly detection. Therefore, the client sought a dynamic, predictive anomaly detection system.
Business Challenge
The client, India’s leading exchange, processes a vast amount of trading data daily. The client’s customer base includes algorithmic and non-algorithmic trading members, each with unique behaviors further influenced by constantly shifting market sentiments. Detecting anomalies and analyzing them is critical for the client to preemptively resolve issues and gain valuable insights. However, their previous static, rule-based system faced numerous limitations:
- Unable to adapt to dynamic member behavior and fluctuating market conditions.
- Frequent false positives, requiring costly human intervention.
- Reactive approach, generating alerts only based on past rules.
- Limited to running once per day.
To overcome these challenges, the client required a dynamic anomaly detection system capable of:
- Identifying potential high-frequency traders.
- Detecting market abuse practices, including multi-leg reversal cases.
- Identifying algorithmically induced price crashes.
Business Solution
Leveraging our expertise in capital markets and advanced technology, NuSummit developed an in-house, dynamic AI/ML-based anomaly detection system tailored to the client’s needs.
This cloud-agnostic system built entirely on open-source platforms and 100% proprietary code, was implemented through the following steps:
- Exploratory Data Analysis and Attribute Selection: Identified 20 key attributes, such as trade ratio and trade value, influencing anomaly detection.
- Data Pre-processing and Cleansing: Ensured data quality and reliability.
- Data Enrichment and Attribute Computation: Enhanced the dataset with computed attributes.
- Creation of the ML Algorithm for Anomaly Detection: Incorporated the 20 influencers into the ML model.
- Model Training and Implementation: Deployed the model for real-time use.
The AI/ML-powered, self-learning system goes beyond previous reactive alerts by adapting to trading members’ evolving behaviors and shifting market sentiments in real-time. This predictive system has improved the surveillance workflow, expanded the types of anomalies detected, enhanced detection accuracy, and significantly reduced false positives.
The interactive anomaly detection dashboard summarizes outlier data for five use cases, enabling the surveillance team to examine details and take action as needed.
Capable of managing large data volumes, the system processes over 3 TB of data and 7+ billion messages daily, empowering the client to become more proactive in resolving anomalies through a user-friendly, predictive, and automated approach.
NuSummit harnessed the power of AI/ML to automate anomaly detection and helped the client become more proactive in resolving anomalies through a user-friendly, predictive, and automated anomaly detection system.
Tech Stack
Programming Languages and Frameworks:
- Python
- Flask
- .NET Core
- Greenplum
The AI/ML-driven anomaly detection system delivered the following key benefits:
- Near Real-Time, Predictive Anomaly Detection: Integrated real-time and historical data for better context.
- Broadened Detection Scope: Enhanced capability to detect more types of breaches, increasing investigative control.
- Reduced False Positives: Leveraged ML for improved accuracy in market surveillance.
- Cost Optimization: Reduced manual intervention and lowered operational costs.
- Improved User Experience and Speed: Enabled faster, more accurate insights and results.
- Immediate Alerts for Anomalies: Provided dynamic insights into unusual behaviors.
The system empowered the client to achieve domain-specific objectives, including:
- Identification of High-Frequency Traders: Determined whether identified traders adversely affected the market.
- Detection of Market Abuse: Monitored equity stock options over-the-market (OTM) contracts and identified multi-leg reversal cases and price distortions.
- Prevention of Algorithmically Induced Price Crashes: Enabled the exchange to proactively detect and mitigate potential market crashes.
