Azure Databricks Engineer

Key Responsibilities:

  • Design and implement scalable data processes focused on fulfilling product requirements.
  • Large complex components; influencing overall product architecture and patterns.
  • Autonomously implement component design in line with pre-defined Data and (ELT/ETL) architectural patterns.
  • Partner with Business, Technical and St***gic Product stakeholders to manage project commitments in an agile framework; rapidly delivering value to our customers via technology solutions.
  • Develop, construct, test, document and maintain data pipelines.
  • Identify ways to improve data reliability, efficiency, and quality.
  • Design and develop resilient, reliable, scalable and self-healing solutions to meet and exceed customer requirements.
  • Ensure that all parts of the application eco-system are thoroughly and effectively covered with telemetry.
  • Focus on automation, quality and streamlining new and existing data processing.
  • Create data monitoring capabilities for each business process and work with data consumers on updates to data processes.
  • Develop data pipelines using Python, Spark and/or Scala.
  • Automate and orchest*** data pipelines using Azure Data Factory or Delta Live Tables
  • Help maintain the integrity and security of the company data.
  • Communicate clearly and effectively in oral and written forms and be able to present and demonst*** work to technical and non-technical stakeholders.


Required Qualifications:

  • Undergraduate degree or equivalent experience
  • Minimum 5 to 7 years of IT experience in Software Engineering.
  • Minimum 3 to 5 years of experience in big data processing for batch and/or streaming data; data includes file systems, data structures/databases, automation, security, messaging, movement, etc.
  • Minimum 3 to 5 years of experience with Python and Spark in developing data processing pipelines.
  • Minimum 3 to 5 years of ETL programming experience in Databricks using Scala/Java.
  • Minimum 3 to 5 years of supporting extensive data analysis using advanced SQL concepts and window functions
  • Minimum 2 to 3 years of experience working on large scale programs, with multiple concurrent projects.
  • Minimum 2 to 3 years of experience with Agile methodologies and Test-Driven Development.
  • Experience with Continuous Development/ Integration (CI/CD) /DevOps skills, (Jenkins, Git, Azure DevOps/ Git Actions).
  • Strong written and oral communications along with presentation and interpersonal skills
  • Ability to lead and delegate work across other members of the data engineering team


Preferred Qualifications:

  • Minimum 2 to 3 years of cloud experience, preferably Azure.
  • Minimum 2 to 3 years experience with Databricks or other big data platforms.
  • Minimum 2 to 3 years automation/orchestration experience using Azure Data Factory
  • Experience and familiarity with data across Healthcare Provider domains (i.e. Patient, Provider, Encounter, Billing, Claims, Eligibility etc.)
  • Familiarity/Experience with Clinical data exchange formats like HL7, FHIR, CCD etc.
  • Familiarity/experience with FHIR resources and data model
  • Experience in Big data processing in healthcare domain
  • Cloud development and computing.
  • Knowledge of cutting to edge technologies (AI, ML, Blockchain, Wearables, IOT).
  • Knowledge/Experience with Microservice design, Java Spring, Kafka, RabbitMQ.
  • Ability/Willingness to explore/learn new technologies and techniques on the job

 


Required Skills : Databricks Spark/Scala • Azure Data Factory • Azure background specifically- not open to AWS at this point,
Basic Qualification :
Additional Skills :
Background Check :Yes
Drug Screen :Yes
Notes :
Selling points for candidate :
Project Verification Info :
Candidate must be your W2 Employee :Yes
Exclusive to Client :No
Face to face interview required :No
Candidate must be local :No
Candidate must be authorized to work without sponsorship ::No
Interview times set : :No
Type of project :Development/Engineering
Master Job Title :Data Analyst
Branch Code :Minneapolis

Indotronix is an Equal Opportunity Employer

Let Us Do the Heavy Lifting!
Upload your resume and we'll reach out when a job fits your skills.
Job Code
JPC - 152497
Posted Date
2023-02-24 07:01:05
Experience
5-7 years
Primary Skills
AWS, Spark/Scala, Azure Data Factory
Salary
$61.19-$61.19

Hear from our employees:

Hear from our employees: