+12125658899
info@cybermarrow.com

Data Analytics

Customer challenge

A leading telecom company providing a wide range of telecom products and services including internet access, voice, entertainment, healthcare, video, and IPTV television. Customer was centralizing the enterprise grade data within Cloud through an ongoing data migration program that involves three different workstreams – Data Lake and Office 365. It aims to move PetaBytes of historical data and 5000+ pipelines from multiple on-prem Hadoop and RDBMS sources to Cloud.

Our team was engaged to provide architectural and design guidance and recommendations, with a focus on improving the reliability, operability, and security of cloud resources. Additionally, the customer sought technical advisory on Cloud products and services to improve operational efficiency of its migrated workloads and help its analytics team scale and increase their productivity.

How we helped

Throughout the engagement, our team worked in collaboration with the client’s Data Engineering team to address Cloud migration and adoption challenges. Some of the activities completed by the our team are listed below:

PII Tagging and DLP Optimization: Our team reviewed PII utility and provided recommendations which helped improve the accuracy of the tool by more than 2x. The proposed enhancements helped improve the overall PII process automation, reduce execution time, and improve the ease of usage for the end user.
Dynamic Data Masking: Conducted walkthroughs of the dynamic data masking and implementation of row and column level access controls on the datasets.
Serverless Spark: Guided the customer’s team to use Serverless Spark and enable the feature in the Datahub projects to expedite the migration of existing Spark/Pyspark pipelines from on-prem to cloud.
Streaming with Dataflow: Assisted customer’s team in understanding Dataflow backend system and provided development knowledge of windowing, limit logging, and DLT approach. Recommended client to take streaming as a separate workstream.
Data Quality with Collibra: Provided ad hoc guidance on Data Governance team on integrating Collibra metadata tool with cloud services.
Program Governance support: Provide migration status and potential risks for all stakeholder.
Result

The customer was able to meet all the migration deadlines:

Modernization of Data Lake enterprise tables to cloud.
Modernization of data warehouse
Office 365 migration

Customer’s datahub platform became one secure, reliable and scalable location for enterprise grade data so the customer teams can derive faster insights to support their business and drive real value. Because of the architectural stability of the Datahub platform, other client’s teams are interested in replicating it.