Data Engineer Job Description

Focus is good comms , Python Spark. Knowledge of AtScale a Big Big Plus.

Job Responsibility

- Design and implement scalable data pipelines using PySpark/Python.

- Develop and optimize SQL queries for data extraction, transformation, and loading (ETL) processes.

- Utilize Databricks on AWS to build and manage data workflows.

- Able to read Legacy code and understand it

- Excellent know how on preparing script for data validation

- Monitor and troubleshoot data pipelines to ensure smooth operation.

- Document data processes and workflows for future reference and maintenance.

- Work closely with existing team to understand the requirement and deliver accordingly

Technical Skills :

- Strong proficiency in PySpark and Python programming.

- Extensive experience with SQL and database management.

- Hands-on experience with Databricks on AWS.

- Familiarity with AWS services such as S3, Lambda, Glue, and Redshift.

- Knowledge of data warehousing concepts and ETL processes.

- Excellent problem-solving skills and attention to detail.

- Strong communication and collaboration skills.

Job Types: Contract, Temporary

Pay: $60.00 per hour

Expected hours: 40 per week

Schedule:

Work Location: In person

CAPY Tech Solutions

New Jobs