Role, Key Accountabilities, and Requirements: Senior Azure Data Engineer
The Senior Azure Data Engineer is a senior technologist with at least ten years of expertise in data design, development, and architecture. They will be highly skilled in the Azure Cloud Environment and possess excellent communication skills complemented by strong functional and consultative skills. The senior technologist will work collaboratively in a dynamic, multidisciplinary team while interacting daily with the client, driving solutions.
Key Accountabilities
● Design, develop, and maintain data pipelines and workflows for ingesting, processing, and analyzing large-scale data.
● Provide thought leadership and collaborate daily with the team and client.
● Proactively contribute to technical solutions, participate as a multidisciplinary team member to analyze sponsor requirements and designs, implement software, and perform analyses to address these requirements.
● Engage with other data engineers frequently to share relevant knowledge, opinions, and recommendations, working to fulfill deliverables as a team.
● Utilize the Azure Databricks platform to deploy, configure, and manage the platform and services.
● Develop, publish, and schedule reports and dashboards to meet business requirements.
● Assist business users with functional and data requirements to enhance data models.
● Develop self-service models and data analytics using Power BI service
● Development and support of visualization tools
● Develop tabular, multidimensional models compatible with standards
● T-SQL queries, creating clustered and non-clustered Indexing, working on DTS, Data Cleansing, and Data Migration.
● Integrations with CSV, Excel, and intake of files and consumption of data from APIs and web services.
● Perform other duties as assigned.
Skill Requirements
● Senior-level experience in Python, Azure, and Databricks.
● Experience with financial data, POS, payments, and transactions in the design and use of models.
● A high-level data engineering competency in design and development. The ability to effectively handle concurrent technical tasks with conflicting priorities and approach complex problems enthusiastically and creatively.
● The ability to change focus when necessary and to work independently and implement research concepts in a multi-disciplinary team environment, where commitments and deadlines are essential to project success.
● Excellent interpersonal skills are necessary to interact with all levels of personnel.
● Excellent verbal and written communication skills are necessary to effectively collaborate in a team environment and present and explain technical information.
● Excellent analytical, problem-solving, and decision-making skills to develop creative solutions to complex problems.
● Design
- Design, develop, and maintain data pipelines and workflows for ingesting, processing, and analyzing large-scale data.
- Design and implement ETL systems to handle a variety of data sources.
- Responsible for design methodology and project documentation.
- Elicit requirements and propose improvements; trace and track requirements to completion.
- Design of SQL Server Integration Services (SSIS) packages, including various control flow tasks and data flow transformation.
- Concepts
■ Exceptional knowledge and understanding of capacities and workspaces.
■ Dashboards, reports, workbooks, datasets, and data flows.
- Create, modify, and publish dashboards to the Power BI service.
- Prototyping, designing, and requirement analysis.
- Participate in planning activities, including defining and organizing tasks, roles, and responsibilities, and prioritization of deliverables.
● Development
- Strong Python skills complimented with the use of R.
- Work with the Apache Spark API PySpark.
- Experience with Databricks.
- Experience with Azure DataLake Storage (ADLS)
- Experience in TimeTravel.
- Experience with Datatables.
- Experience with AutoLoader would be a plus.
- Strong skills with IDEs JupyterLab, RStudio, or equivalent.
- Customize, extend, and embed Power BI analytics.
- Write, edit, and debug complex T-SQL queries.
- Development of SQL Server Integration Services (SSIS) packages, including various control flow tasks and data flow transformation.
- Apply comprehensive APIs and use SDK libraries.
- SSRS, SSIS, SSAS, and other BI stack technologies.
- Build and deliver dashboard and analytical solutions using Power BI and related Azure data services.
- Develop custom visuals.
- Data prep, data modeling, and data visualization.
- Data transformation and consolidating data from disparate sources.
- Develop tabular and multidimensional models.
- Data modeling and integration through extracting, transforming, and loading (ETL) data from various sources.
- Write calculations using Data Analysis Expressions (DAX).
- Advanced knowledge in all phases of the Software Development Life Cycle (SDLC).
- Advanced knowledge of Agile/Scrum development methodologies.
- Conduct testing as needed to ensure that the solution meets the requirements.
- Update and maintain related documentation (e.g., user guides, FAQ, site improvement documentation).
● Architecture
- The architecture of datasets and dashboards.
- The full range of modern data architecture, from ingestion to storage to consumption.
- Drive enterprise architecture and technology roadmaps.
- Power Platform architecture, development & troubleshooting.
- Service architecture design and administration.
- Guidance to Data Engineers on best practices for developing high-quality datasets and visually stunning reports.
Certification Requirements
● Hold multiple Azure certifications.
● Hold multiple process certifications, i.e., Scrum.
● Obtain certifications immediately or show dedication to acquiring them. The employment offer letter will explicitly outline this requirement and must be agreed upon, initialed, and signed as a condition of employment. Non-compliance may lead to termination.
Professional Development
● Training Roadmap.
● Learning Agreement.
Compensation
● Competitive salary.
Benefits Package
● Company-paid medical/dental/vision/life coverage.
● 401K with company match.
Pet Health Package
● Pet Telehealth Services.
● On-Demand Medical and Veterinary Care.
● Pet prescription discounts of up to 75%.
● 24/7/365 access and consultation.
The Perks
● Gym stipend.
● Take the time when you need it - flex days for vacation, sick, and self.
● Flexible work hours.
● Shorter days near the holidays.
● Attend company-sponsored events and team outings.
● Participate in employee care and recognition programs.
● Wellness Program
● Fenway Group is a 7-time Dallas Morning News Top 100 Workplaces recipient.
Background Screening
● A background check is required.
● A drug screen is required.
Citizenship
● U.S. citizenship is required.
Education
● BS degree in Computer Science or a related field.
● Advanced degree preferred.
Job Type: Full-time
Pay: $127,245.00 - $138,254.00 per year
Benefits:
- 401(k)
- 401(k) matching
- Dental insurance
- Flexible schedule
- Flexible spending account
- Health insurance
- Health savings account
- Life insurance
- Paid time off
- Professional development assistance
- Referral program
- Retirement plan
- Vision insurance
Compensation package:
- Bonus opportunities
- Performance bonus
- Yearly pay
Experience level:
Schedule:
Application Question(s):
- Do you have experience with Databricks?
Experience:
- technical: 10 years (Required)
Work Location: Remote