Azure Data Engineer
Location: Columbia, MD
This position requires the candidate to be in the Columbia office 1 day a week
Data Engineers (Data Insights Team) – 4+ years of experience; candidates should be local for these positions (Hybrid work environment – Thursdays in office)
· 4+ years college degree (required)
· 4+ years of working with data integration (required)
· 2+ years of advanced programming in PySpark
· Experience working with Databricks (preferred)
Essential Duties and Responsibilities
Include the following. Other duties may be assigned.
- Design, build, and maintain large, complex data processing pipelines that meet business requirements
- Ensure the quality of deliverables by developing automated controls and performing unit, integration, and user acceptance testing
- Develop scalable and re-usable frameworks for ingestion and transformation of large data sets
- Troubleshoot and perform root cause analysis on data pipeline issues
- Work with source system owners and business owners to incorporate business changes into data pipelines
- Create comprehensive documentation of data workflows, processes, and infrastructure
Qualifications
To perform this job successfully, the incumbent must be able to perform each essential duty satisfactorily. The requirements listed below are representative of the knowledge, skill, and/or ability required. Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions.
- Advanced programming skills in PySpark, Python, Spark SQL
- Thorough understanding of Data Lakes, raw/enriched/curated layer concepts, and ETL within Azure framework
- Proven development experience in building complex data pipelines for lakehouse/data warehouses using Agile methodology
- Experience in architecture, design, and implementation using Databricks and Azure Data Factory
Experience and/or Education Requirements
- 4+ year college degree (required)
- 4+ years of working with data integration (required)
- 2+ years of advanced programming in PySpark
- Experience working with Databricks (preferred)
Dexian is a leading provider of staffing, IT, and workforce solutions with over 12,000 employees and 70 locations worldwide. As one of the largest IT staffing companies and the 2nd largest minority-owned staffing company in the U.S., Dexian was formed in 2023 through the merger of DISYS and Signature Consultants. Combining the best elements of its core companies, Dexian's platform connects talent, technology, and organizations to produce game-changing results that help everyone achieve their ambitions and goals.
Dexian's brands include Dexian DISYS, Dexian Signature Consultants, Dexian Government Solutions, Dexian Talent Development and Dexian IT Solutions. Visit https://dexian.com/ to learn more.
Dexian is an Equal Opportunity Employer that recruits and hires qualified candidates without regard to race, religion, sex, sexual orientation, gender identity, age, national origin, ancestry, citizenship, disability, or veteran status
Job Type: Full-time
Pay: From $95,045.00 per year
Benefits:
Compensation Package:
Schedule:
Ability to Commute:
- Columbia, MD 21046 (Required)
Ability to Relocate:
- Columbia, MD 21046: Relocate before starting work (Required)
Work Location: In person
