On January 7, the first stop of Changting Science and Technology Semantic Analysis 3.0 and Lei Chi 30 tour conference was held in Beijing.

The conference showcases the latest achievements of Changting semantic analysis technology, the newly upgraded version 30 of SafeLine and the best practices of WAF, reflecting the new breakthrough of Changting in the integration of network security and artificial intelligence technology. In addition, Zhao Weijing, senior research manager of IDC China, shared the theme of global network security technology hot spots and trends, providing an international perspective for the conference.

First, Semantic analysis 3.0 was released, and semantic analysis itself has the advantage of intergenerational

As the core technology of traffic detection products of Changting Technology, semantic analysis has generational advantages compared with traditional Web attack detection technologies based on rules and regular expressions.

Liu Jinzhao, CTO of Changting Technology

In the traditional Web attack detection, the regular and regular expression detection methods only try to capture the characteristics of the text content of the attack, but these characteristics can not reflect the essence of the attack, but are indirect characteristics. Semantic analysis techniques, like DNA techniques, accurately capture the essence of the "attack" : a syntactic code snippet that also contains malicious semantics. This is the fundamental reason why semantic analysis can take detection to another level.

From 1.0 to 2.0: Merging big models

Since Changting delivered the first commercial version of the semantic analysis engine in 2016, a total of nearly 10,000 engine iterations have been carried out, and a total of 18 detection engines for different attack means have been implemented.

In 2023, with the rise of AI large model, Changting innovatively combines semantic analysis technology with ChaitinAI security large model, ushering in a major leap in semantic analysis technology - Semantic analysis 2.0. In this release, the semantic analysis engine is responsible for extracting key semantic information from the attack, providing clear guidance for large models to understand the nature and intent of Web attacks. This fusion further improves the ability of semantic analysis to detect and interpret complex network attacks.

Reveal 3.0: "AI+ Data" drives a new era

Hidden behind the high accuracy and high performance of semantic analysis engine is a complex technical principle, each improvement needs to be carefully balanced, and the difficulty and complexity of iteration are increasing. How to further improve the performance of the semantic analysis engine?

After repeated verification, Changting once again found the path of innovation from the choice of technical routes, its core principle is the deep integration of semantic analysis technology and machine learning technology, semantic analysis 3.0 was born.

In the new engine architecture, semantic analysis is no longer directly output detection results, but focuses on extracting attack features in the request, including features at the lexical, syntactic and semantic levels. These strong safety features are relatively stable, reducing the need for engine maintenance. The extracted features are then fed into multiple AI models for determination. Based on these strong features, a high decision accuracy can be achieved without a complex and huge deep neural network model. Such a design preserves the advantages of both semantic analysis and machine learning while reducing the disadvantages of each.

The new engine in this architecture has significant advantages:

Excellent performance: Inheriting the speed advantage of semantic analysis, the comprehensive performance reaches more than 90% of the current semantic analysis 1.0 engine, and the actual engine takes a small proportion of time, which has no obvious impact on the overall performance.

Strong interpretability: strong syntactic features and semantic features combined with interpretable models can output the explanation of attacks in natural language, improving the transparency of security protection.

Easy maintenance: The maintenance frequency of the semantic analysis part is reduced, the performance is continuously improved by optimizing the distribution and quality of training data, and the maintenance threshold and cost are reduced.

Game breaking data challenge: Shadow mode + group labeling

With a good engine architecture, there is only one problem left to solve when it comes to maximizing performance in all directions: high-quality, secure data sets.

Obtaining high-quality data has always been a challenge in the security industry. Unlike industries such as e-commerce, the behavior of users visiting the website itself does not provide security labels, and security data labels can only be annotated by security experts from the logs, resulting in a small number of effective marks. The performance of AI models depends on data quality, which brings great challenges to the application of AI technology in the security industry.

To solve data problems, Semantic Analysis 3.0 borrows from autonomous driving technology and uses shadow mode and group labeling.

In shadow mode, a shadow engine is added to the existing detection engine. When the judgment results of the shadow engine and the main engine are different, relevant samples are saved for engine optimization.

Group labeling makes use of a large number of service data processed by deployed devices. When the model has poor performance against a specific attack load, it can find similar data from the massive data and optimize it to improve the detection capability of special attack vectors.

The above methods build a data-driven engine iteration process: after data acquisition, manual annotation and expansion are carried out to optimize sample distribution; Train and deploy new models with high-quality data; Continuously observe the difference during operation, expand the data and training again, and repeat the cycle to continuously improve the detection accuracy.

The data-driven iterative "engine" finally allows Semantic Analysis 3.0 to take detection performance to the next level:

Further improvement in accuracy: Semantic Analysis 3.0 successfully increased detection accuracy from 99.9% to 99.99% on validation data sets, with a significant reduction in false positives and missed positives.

Emergency response is faster: While it used to take three to seven days to deal with engine false positives and missed positives, a data-driven approach can ideally reduce this to two to eight hours. By simply analyzing the sample data, identifying the error loads, expanding the sample, and training the model, a new engine can be quickly produced, greatly increasing the speed of emergency response.

More precise model fine-tuning: As an AI engine, it can be fine-tuned for specific business scenarios. By fine-tuning the model, you can improve the adaptability to specific services, reduce missed and false positives, and store key feature data for easy management and protection of sensitive information.

Better recognition of unknown threats: Semantic Analysis 3.0 further improves the recognition of unknown threats (0-day). It inherits the advantages of semantic analysis to identify unknown attacks, and uses the stronger learning and generalization capabilities of AI models to respond more effectively to emerging cybersecurity threats.

Two, Lei Chi WAF heavy upgrade

At this conference, in addition to the innovative semantic analysis 3.0 technology, Changting Technology also released a full range of major upgrades to the SafeLine next generation Web application firewall 30 version. This upgrade focuses on the three major features of "innovation, intelligence and integration", which are mainly reflected in the following three aspects:

• Intelligent upgrade of the security core engine

• Pattern fusion and architecture upgrade

• WAAP solutions are converged and upgraded

Changting technology chief security product expert Guo Shichao

Intelligent upgrade of the security core engine

First of all, thanks to the transition of semantic analysis technology 3.0, the new version of Lei Chi 30 has improved by leaps and bounds in six aspects: performance, detection effect, user business fit, emergency response speed, 0day protection, and operation experience.

At the same time, another new technological breakthrough of Lei Chi 30 - streaming semantic analysis technology, has overcome the "big package bypass" problem that has plagued the industry for many years. The core breakthrough of this technology is to realize a number of innovations such as data fragment stream receiving technology, deep decoding stack snapshot technology, semantic analysis engine reentrant technology, and transform protocol parsing, decoding and pattern matching into a stream detection mode of "receiving, detecting and forwarding". Not only effectively solves the problem of "big package bypass";, but also significantly reduces the detection delay without sacrificing the detection effect, providing users with a better business experience.

In addition, in view of the HTTP SSE (Server-Sent Events) mechanism used in the popular large-model response return, Lei Chi 30 applies response flow detection and semantic caching technology, which can intercept or shield sensitive/violation information without affecting the large-model response effect. Further broaden the security protection capability of Lei Chi 30 in the field of emerging technologies.

Pattern fusion and architecture upgrade

1. Data plane engine upgrade: XDP enables superior performance

Lei Chi 30 fully implemented XDP technology on the data plane engine, and built the network layer protocol stack based on eBPF.

XDP eXpress Data Path, that is, fast data path, is a high-performance, programmable network packet processing framework provided by the Linux kernel. XDP will directly take over the RX packets of the network card, and process the packets quickly by running eBPF instructions in the kernel and seamlessly interfacing with the kernel protocol stack.

First of all, XDP has the technical advantages of high performance at the technical level, providing high performance packet processing efficiency, core throughput up to 24 million packets per second (Mpps), and kernel mode properties, can reduce the number of data copies, reduce system call overhead, no dedicated CPU, thereby improving the overall efficiency of the system

Secondly, XDP's kernel mode scheme is not subject to the support provided by upstream manufacturers and communities, and Lei Chi 30 has made a big step forward in security, autonomy and controllability. In contrast, the kernal-bypass solution based on DPDK can work properly or achieve the expected performance only after the user mode driver (PMD) ADAPTS to the underlying hardware. This part often relies on technical support from upstream hardware vendors and the DPDK community.

2. Mode upgrade: Get rid of hardware constraints and integrate diverse scenarios

The upgrade of the mode enables Lei Chi 30 to get rid of hardware dependence and achieve an integrated hardware and software architecture, which not only meets the requirements of soft and hard forms, but also is extremely friendly to the cloud and more in line with the cloud native concept.

The Lei Chi 30 mode has been adjusted and optimized:

• The transparent bridge and transparent agent are merged into a unified transparent mode, and automatic protection is not required for common sites, reducing maintenance costs.

• Split out an independent routing mode, which greatly enhances the capability in routing scenarios;

• All modes can support both hardware and software modes;

• Supports easy hot mode change on the page, and provides an "expert mode mode" to use four modes simultaneously on one WAF;

• The complex network mode also supports VM installation, meeting hyperconverged and special traffic access scenarios in the cloud, expanding the WAF usage mode, and providing users with more flexible and diversified choices.

3, chassis upgrade: unified management, seamless adaptation

Lei Chi has been based on Docker container technology since its birth, which is naturally competitive in cloud-native environments. However, Lei Chi 20 version has architectural design differences in different deployment scenarios, resulting in inconsistent user experience. Lei Chi 30 introduces the idea of domain-driven design (DDD) and refers to the Operator mode of K8s to upgrade the core configuration domain of control surface in an all-round way.

This upgrade achieves a consistent experience across scenarios. Regardless of single machine, cloud architecture, or mixed use of hardware cluster and software cluster, user operation and management can be seamlessly connected, flexibly adapted to various complex business scenarios, and maintainability and scalability are significantly improved. It can better meet complex service requirements without adding additional costs, ensuring availability and reliability in large-scale and complex scenarios, and providing users with a more stable and efficient network security management solution.

WAAP program upgrade: Integrated protection, build a strong safety net

Any single security protection function island can no longer meet the current complex and dynamic business security needs. WAAP of Leichi 30 is different from the simple combination of WAF, Bot protection, API security, CC protection and other functions, but through module integration, the seamless cooperation of various security modules is realized, forming an efficient and intelligent security protection system.

Zhou Xinyou, Vice President of Changting Technology

For example, in terms of API security, Lei Chi 30 implements a forward loop through semantic analysis engine and API security. On the one hand, the engine's deep decoding and semantic analysis capabilities help to extract more comprehensive request information, improve the accuracy of API sensitive data identification, user identification and risk detection; At the same time, the API recognition results can enhance the decoding performance of the semantic engine, achieve accurate protection against abnormal traffic and reduce the false positive rate through Schema verification and API baseline. API security can also improve the intelligence level of other modules, using machine learning to analyze API call patterns and model them, intelligently and dynamically adjust the threshold policies of Bot protection and CC protection according to the learning results, and respond to new attacks in real time.

In addition, the WAAP solution of Lei Chi 30 can be flexibly deployed in both hardware and software clusters or cloud scenarios, providing users with flexible solution options.

Third, global cybersecurity technology hotspots and trends

At the conference, Zhao Weijing, senior research manager of IDC China, also brought the theme sharing of "Global network security Technology Hot Spots and Trends". He pointed out that at a time when there are many uncertainties in the development of the global security market, artificial intelligence will accelerate the innovation and practice of cybersecurity technology. By 2030, AI will generate nearly $20 trillion in cumulative economic revenue. Among them, GenAI will also promote the accelerated development of the network security industry, and more enterprises will be more willing to use GenAI to apply to the security applications in the organization, and embed GenAI as a security assistant in security tools to improve security efficiency. In security fields such as security operations, application security, data security, risk/exposure management, and security compliance, GenAI has many application directions and space.

Zhao Weijing said that in the direction of application security, more enterprises are willing to widely adopt Web application protection solutions, and WAF as the basic solution is the first choice of customers. At the same time, WAAP solutions, which mainly include WAF, API security, DDoS mitigation, Bot management and other capabilities, have also attracted the interest of many customers.

Zhao Weijing, senior research manager, IDC China

Since its establishment, Changting Technology has been recognized as two major labels in the industry. One is excellent performance in the field of actual combat attack and defense, and the other is attracting attention as a technology-driven intelligent security company. The release of Semantic Analysis 3.0 is another milestone for Changting to embrace the new generation of detection technology in the AI era and continue to promote industry change in the direction of intelligent security. In the future, Changting will adhere to the concept of "knowledge and attack, good defense, intelligent security", and continue to promote the full stack security innovation at all levels from the application layer, to the network architecture layer, and then to the kernel protocol stack.