AIOps Kafka Interface Pre-processing engine to masquerade sensitive data in Log

Description:
During Log ingestion to AIOps through Kafaka interface customer experiencing that all the LOG data are being ingested to the AIOps for Log Anomaly Detection. In which all the sensitive data like username, password, email address, account no., address are being seen in logs. Given the increasing importance of data privacy and compliance, customer required the ability to centrally masquerade or sanitise private data at the ingestion point would be a highly valuable feature. It would allow teams to confidently forward logs to AIOps without risking exposure of sensitive information. So customer required this function to masquerade/mask the sensitive data during the log ingestion to AIOps through the Kafka interface. 

Impact:
Currently log ingestion to the AIOps is on hold due to data privacy concerns. So the LAD capability is totally stopped by the customer. Which is highly degrading the confidence of customer on the feature of the AIOps and specially the Log Anomaly Detection.

Any losses of revenue for this idea?
Yes. It is the basic requirements which was not taken care. Right now it is difficult for me to quantify the losses, but lack of capability to mask/masquerade sensitive data in logs in AIOps will lead to the customer's data privacy policy violation, which will impact the confidence on AIOps.
We are at POC stage, but to move to the production we need to have these solutions in place.


Customer: T-system, German