Data Engineer

Roles & Responsibilities

• Data Pipeline Development

  Design, develop, and maintain scalable data pipelines that enable efficient data extraction, transformation, and loading (ETL) from various sources into centralized data systems.

• Data Quality Assurance

  Implement robust data validation and cleansing processes to ensure data quality and integrity, identifying and resolving data issues swiftly to maintain accuracy.

• Database Management

  Manage and optimize databases including relational and non-relational systems, ensuring optimal performance, security, and accessibility of data for stakeholders.

• Collaboration with Teams

  Work closely with data scientists, analysts, and business stakeholders to understand data requirements and provide solutions that support data-driven decision-making.

• Technical Documentation

  Develop clear and comprehensive technical documentation for data workflows, ETL processes, and data schema changes, ensuring alignment and knowledge sharing across teams.

• Cloud Integration

  Integrate on-premises data infrastructure with cloud-based data platforms, facilitating seamless migration, storage, and analysis of data using cloud technologies.

• Performance Monitoring

  Establish metrics and dashboards to continuously monitor data pipeline performance, identifying bottlenecks and implementing solutions to enhance data processing efficiency.

Typical Required Skills and Qualifications

• 3+ years of experience in data engineering or a related field

• Proficiency in SQL and experience with ETL tools

• Strong knowledge of data modeling and data warehousing concepts

• Experience with big data technologies (e.g., Hadoop, Spark) and cloud platforms (e.g., AWS, Azure)

• Familiarity with programming languages such as Python or Java