Data Engineer

Data Fabrica • Washington, DC, US • 1d ago

Job Overview
This role will involve supporting the management and analysis of large, complex data streams from various sources, including structured, semi-structured, and unstructured data.

Key components of this requirement are outlined below. Only candidates with prior FAA experience will be considered, with a strong preference for those who have worked at the FAA recently.

Responsibilities

Support for Multiple Data Sources:

Data Streams: Support diverse data streams, which include sensor data, transactional data, operational data and more.
Integration: Ensure seamless integration of data across various platforms, databases, and cloud services.
Data Transformation and Cleansing: Assist in transforming, cleaning, and preparing the data for analysis, ensuring it’s in the proper format for use in Big Data platforms and analytics tools.

Big Data Management:

Data Storage: Provide expertise in managing large-scale data storage solutions.
Scalability: Implement solutions that can accommodate growing data volumes without compromising performance.
Data Processing Frameworks: Work with Big Data processing frameworks to process large datasets efficiently in real-time or batch modes.
Data Quality: Ensure high data quality standards; including monitoring for completeness, consistency, and accuracy across all datasets.

Real-time Data Analytics:

Streaming Data: Support real-time analytics for data streams from sources like Azure Stream Analytics and AWS Kinesis.
Decision Support: Create analytics solutions that provide real-time decision support for operations and systems, improving efficiency, and response time.

Reporting, Dashboards, and Visualization:

BI Tools such as Tableau, Power BI and ArcGIS for generating meaningful visualizations and interactive dashboards that communicate insights derived from big data analysis.
Tailored Reporting: Design custom reports and dashboards that cater to different stakeholders, from operational teams to strategic planners, ensuring that their insights are easily accessible and actionable.
Data Storytelling: Ensure that complex data findings are presented in an understandable, compelling way through dashboards, charts, and presentations that provide a clear narrative.

Security, Compliance, and Governance:

Data Security: Implement robust security measures to protect sensitive data.
Data Governance: Define and implement data governance practices to ensure consistency, accountability, and compliance with requirements.

Expertise in Key Tools and Platforms:

Must have experience with and certifications in a variety of platforms and tools including:

Alteryx (data blending and advanced analytics)
Tableau, Power BI (data visualization and reporting)
ArcGIS (geospatial data analysis)
Palantir Foundry (data integration and advanced analytics)
Azure AI, AWS AI, SageMaker (cloud-based AI/ML tools)
JupyterHub, Azure Notebook Workspaces (data science and notebook environments)

Skills

Bachelor’s degree and twelve (5+) years of relevant experience.
Bachelor’s degree must be in information systems, business systems, management information systems, IT Management, or other IT degree, engineering, math, and/or science.
Formal certification around expertise is desirable.
Proficiency in designing and managing data warehouses.
Strong experience with big data technologies such as Hadoop, Spark, or similar frameworks.
Solid programming skills in Python for developing scalable applications.
Familiarity with ETL tools like Informatica for effective data integration.
Knowledge of RESTful API development for efficient data retrieval.
Ability to make decisions in accordance with established policies, guidelines, and standards.
Strong organizational skills with the ability to multi-task, manage time effectively, and handle tight deadlines.
Ability to work with confidential and proprietary information using utmost discretion..

Job Type: Full-time

Pay: $100,000.00 - $158,566.00 per year

Benefits: