Lead Data Engineer - SQL

Back to job search
  • Location:

    Eden Prairie

  • Job type:

    Contract

  • Job ref:

    810605

  • Published:

    23 days ago

  • Duration:

    6months to hire

  • Expiry date:

    02/11/2023

  • Startdate:

    ASAP

Lead Data Engineer

Summary

Design, develop, implement and maintain scalable ETL processes on Azure Cloud using Azure Synapse implement systems and processes to monitor data quality Ad hoc analyses troubleshooting data-related issues Develop and maintain propriety data warehouse tools Offer technical guidance to internal teams and business partners Work with offshore team, business analysts and other data engineering team members to ensure alignment of requirements, methodologies and best practices Participate in scrum process and deliver stories/features according to the schedule Participate in product support activities as needed by the team Individual contributor and Technical lead for Data Engineering - Data pipelines, Data modelling and Data warehouse.

How many members will you be working with?

  • 10 - 12 team members. 1 PM, 3 Product owners 2 Sprint teams consisting - 1 Scrum Masters and 12 developers.

Top responsibilities for this position:

  • Identify opportunities for Data Engineering and AI to enhance the core product platform, select the best machine learning techniques to the specific business problem and then build the models that solve the problem.
  • Architect and design DW and Analytics solutions and cloud services.
  • Own the end-end process, from recognizing the problem to implementing the solution.
  • Establish DataOps principles and best practices.

Requirements :

  • Undergraduate degree or equivalent experience. BE/BTech or equivalent experience BE/BTech or equivalent experience in Data engineering
  • Advanced SQL knowledge, capable to write optimized queries for faster data workflows.
  • Proven work experience in Python, SQL, Any RDBMS.
  • Experience in designing solutions for multiple large data warehouses with a good understanding of cluster and parallel architecture as well as high-scale or distributed RDBMS
  • Must be extremely well versed with handling large volume data and work using different tools to derive the required solution.
  • To be able to work in a fast-paced agile development environment.

Preferred Qualifications:

  • Experience with AzureML
  • Expert in Azure Synapse, Azure Container Registry, Azure App services
  • 5+ years of hands-on software engineering experience.
  • Strong knowledge of Azure Data Lake, Synapse, ADF, Power BI.
  • Strong knowledge of Data Modelling techniques like Data Vault 2.0
  • Understanding of CI/CD framework is an added advantage.
  • Experience with Git / Azure DevOps Ability to interpret/write custom shell scripts.
  • Python scripting is a plus.

Digital Intelligence Systems, LLC (DISYS) is an Equal Opportunity Employer that recruits and hires qualified candidates without regard to race, religion, sex, sexual orientation, gender identity, age, national origin, ancestry, citizenship, disability, or veteran status.