Haziq Matlan

Software QA Analyst with 5 years of experience in ETL and Big Data testing. Proficient in validating large-scale PySpark ETL pipelines, Delta Lake versioning, and complex cloud data migrations. 

Adept at architecting automated validation frameworks, enforcing data integrity across batch and streaming pipelines covering from raw-to-gold layers of multi-terabyte datasets. 

Collaborating with data engineering and product teams in Agile environments to ensure reliable and high-quality data.

• Validated a large-scale data migration project, transferring legacy systems and historical data to a new cloud-based platform, enhancing data accessibility and reliability.
• Planned and designed comprehensive data validation testing strategies across AWS S3, Databricks, Postgres, and Elasticsearch environments.
• Planned and architected end-to-end QA processes for large-scale Databricks ETL pipelines, ensuring data accuracy and consistency across multi-stage ingestion and transformation of structured data (CSV, Parquet). 
• Developed and maintained automated validation scripts in Pyspark and SQL queries, seamlessly integrated into CI/CD pipelines to enable continuous validation for every release.
• Validated incremental data loads to ensure data integrity and prevent data loss during multi-stage ETL processing, across Raw, Bronze, Silver, and Gold layers.
• Performed Delta Lake versioning tests to validate rollback functionality and ensure accuracy of time-travel queries.
• Planned and documented comprehensive test strategies for validating the Entity Resolution (ER) library, ensuring accurate matching logic and scoring outputs across large datasets.
• Retrieved, analysed, and reported data statistics across multiple platforms — including record counts, aggregated datapoint metrics, feature job runtimes, and success performance — leveraging AWS CloudWatch for monitoring and insights.
• Collaborated closely with cross-functional teams, to troubleshoot pipeline failures, optimize performance, enhance test efficiency and align testing with business requirement on multi-terabyte datasets.

Participated in Agile ceremonies and contributed to sprint planning and retrospectives.

Bachelor of Technology (Honours): Information and Communication Technology (ICT) 
Universiti Teknologi PETRONAS (UTP) – Malaysia