Microsoft Azure Data Engineer Job at Purple Drive, Boston, MA

V3ZQSjBvdStLeGhUM3l4VUxQcmZWa052SHc9PQ==
  • Purple Drive
  • Boston, MA

Job Description

Microsoft Azure~Digital

6-9 yrs
Data Engineer - Databricks
- Must have at least 8+ years of extensive Data Engineering Experience in Databricks, ETL/ELT using Data pipelines, SQL/Procedure experience.
- Must have 8+ Years of hands-on development proficiency in implementing Databricks solutions using Scala and Spark with Data frames and Notebooks/SQL.
- Must have 8+ Years of hands-on development and performance tuning/enhancement experience in Scala/PySpark and SQL.
- Must have 8+ Years of hands-on development experience in working on Azure Cloud.
- Must have strong experience with Databricks, including developing and optimizing Spark jobs, data transformations, and data processing workflows.
- Must have experience and a good understanding on the below topics
-- Databricks Delta Lake storage
-- Unity Catalog
-- Autoloader
-- Lakehouse Architecture
-- Medallion Architecture
-- Databricks Serverless Options.
-- Dataset and Data frame
-- Databricks Clusters Configuration and Sizing
-- Databricks Job Orchestration
-- Spark Structured Streaming
- Must have experience with complex and large volume data streaming and data transformations using Azure Databricks.
- Must have extensive experience in Datawarehouse/Data Lakehouse implementations.
- Must have experience with monitoring, debugging, and resolving issues in live Databricks jobs.
- Must have strong hands-on expertise in troubleshooting DevOps pipelines, Azure, and AKS services

- The candidate should be able to provide Data Ingestion, Streaming and Transformations solutions and do development work.
- Experience in Apache Big Data implementations with programming proficiency like PySpark, R or Java prior to Databricks is advantageous.
- Good to have experience with containerization technologies such as Docker and Kubernetes.
- Strong critical thinking, communication, and problem-solving skills is essential.

Job Tags

Similar Jobs

The Morning Star Company

Proprietary Security Officer - Security Guard Job at The Morning Star Company

 ...Hour Depending on Experience The Mission To protect and secure Morning Star colleagues, facilities, and assets by providing an...  .... # Must be able to work flexible hours, including evenings, overnights, weekends, and holidays. Benefits Competitive hourly wage... 

Brightstar Lottery

Software Eng (Dev) III Job at Brightstar Lottery

 ...Brightstar is an innovative, forward-thinking global leader in lottery that builds on our renowned expertise in delivering secure technology and producing reliable, comprehensive solutions for our customers. As a premier pure play global lottery company, our best-in-class... 

Sharpcontra

Public Relations Assistant Job at Sharpcontra

 ...a highly organized and detail-oriented Public Relations Assistant to support our communications...  ...work ~ Proficiency in Microsoft Office (Word, PowerPoint, Excel)~ Ability to...  ...team environment Additional Information Benefits Competitive salary ($... 

Aramark

General Utility Worker - UH Hilton - UH - Hilton Job at Aramark

Are you self-motivated and proud of the work you do? Here at Aramark, we take pride in the level of service and safety we provide! As a General Utility Worker on our team of other service stars, youll take on the important task of maintaining the cleanliness of dishes...

Herbruck Poultry Ranch

Driver Job at Herbruck Poultry Ranch

Position Summary: To perform all functions of delivery of products to customers to the highest of Herbrucks quality standards. Responsibilities:~ Assist in loading of trucks at the stores~ Deliver product to customers with excellent customer service...