HiringCafe
🏢 Offshore Consulting Shops

Spyglass MTG is a specialized IT consulting firm hiring for billable client-facing project delivery rather than internal corporate functions.

This company was flagged and excluded from default search results. Proceed with caution.

LM
Posted 3mo ago

Lead Data Engineer - Databricks & Microsoft Fabric

Lighthouse MTG LLC
View all open roles
Lincoln or Maine or New Hampshire or Vermont or Massachusetts or Rhode Island or Connecticut
RemoteContract
Responsibilities
  • designing platforms
  • building pipelines
  • optimizing workloads
Requirements
  • 7+ years data engineering experience with Databricks
  • Microsoft Fabric
  • Apache Spark
  • Python/PySpark, and Azure
  • Strong pipeline
  • Lakehouse
  • ETL/ELT
  • Data modeling
  • Governance, and leadership skills
Technical tools mentioned
DatabricksMicrosoft FabricApache SparkPySparkPythonAzureCI/CD

Job description

Description

 Location: Remote (Preference for candidates located in New England)
Engagement: Contract or Contract-to-Hire 


Lead Data Engineer

Databricks & Microsoft Fabric


Overview

Spyglass MTG is seeking a highly skilled Lead Data Engineer with strong expertise in Databricks and Microsoft Fabric to help design, build, and scale modern data platforms in Azure.

This role is both strategic and hands-on. The ideal candidate will lead the development of enterprise data solutions while actively building scalable pipelines, lakehouse architectures, and data platforms that support analytics, reporting, and AI initiatives.

The Lead Data Engineer will work closely with data architects, analysts, and business stakeholders to deliver high-performance data solutions that enable organizations to leverage their data more effectively.


Responsibilities

• Design and implement modern data platforms using Databricks and Microsoft Fabric
• Build and maintain scalable data pipelines and ingestion frameworks
• Develop and optimize Spark and PySpark workloads within Databricks
• Implement lakehouse and data warehouse architectures
• Design and manage data models supporting analytics and reporting
• Collaborate with data scientists and analytics teams to support AI and advanced analytics initiatives
• Implement data quality, governance, and security best practices
• Provide technical leadership and mentorship to data engineering team members
• Optimize performance, reliability, and scalability of data platforms


Requirements


• 7+ years of experience in data engineering or data platform development 

• Strong hands-on experience with Databricks and Apache Spark  

• Experience implementing Microsoft Fabric data solutions  

• Proficiency in Python and PySpark  

• Experience working with Azure data services and cloud-based data platforms  

• Experience designing modern lakehouse or data warehouse architectures  

• Strong understanding of data pipeline design, ETL/ELT processes, and data modeling


Preferred

• Experience supporting AI or machine learning workloads  

• Experience working in enterprise data environments  

• Familiarity with CI/CD and data platform automation 

Similar jobs

Data Engineer roles near Lincoln, Rhode Island
7h
Save
Mark Applied
Hide
Data Engineer
Shrewsbury, Massachusetts, United States
$96k-$124k/yr HybridFull Time
Central One Federal Credit Union
Central One Federal Credit Union: Provides personal banking, lending, and insurance services to members.
5+ YOEBachelor's degree or equivalent experience; 5+ years in data engineering, analytics engineering, or data platform development. Requires SQL, Python preferred, Azure, Microsoft Fabric, Power BI, databases, and data governance expertise.
Microsoft Azure, Microsoft Fabric, Microsoft Data Factory, Microsoft Synapse, Microsoft Power BI, DAX, Microsoft Purview, SQL, Python, Oracle, Microsoft SQL Server, Azure RBAC, CI/CD, DevOps
Job PostingView all
4d
Save
Mark Applied
Hide
Sr. Data Engineer - Analytical Development
Boston or Cambridge
$137k-$215k/yr HybridFull Time
Takeda
TakedaTokyo Stock Exchange / New York Stock Exchange: 4502 / TAK: Researches, develops, and manufactures prescription pharmaceutical medicines.
8+ YOEComputer science, software engineering, or related degree; 8+ years combined analytical experience with SQL, AWS, cloud, Python, DevOps, and R Studio; ETL, statistical, and software engineering expertise.
SQL, AWS, Python, R Studio, R, Scala, C, C++, Spark, Databricks, Spotfire, Tableau, Power BI, Minitab, MATLAB, ETL
Job PostingView all
4d
Save
Mark Applied
Hide
Sr. Data Engineer
Boston or New York or Santa Monica or Washington or Los Angeles or San Francisco or Philadelphia or California
$150k-$160k/yr OnsiteFull Time
Goodwin Procter
Goodwin Procter: Legal services for innovators and investors in high-growth industries.
5+ YOEFive or more years in senior data engineering or database architecture; advanced SQL, Python, data integration, ETL/ELT, cloud platforms, relational databases, APIs, Git, Agile, and performance tuning.
Snowflake, SQL, Python, JSON, XML, Git, Aderant Expert, Foundation, iManage
Job PostingView all
5d
Save
Mark Applied
Hide
Google Senior Data Engineer
Albany or Arlington or Atlanta or Austin or Beaverton or Bentonville or Boston or Carmel or Charlotte or Chicago or Cincinnati or Cleveland or Columbus or Culver City or Denver or Des Moines or Detroit or Hartford or Houston or Irving or Kirkland or Miami or Milwaukee or Minneapolis or Morristown or Mountain View or Nashville or New York City or Oklahoma City or Overland Park or Philadelphia or Pittsburgh or Raleigh or Redmond or Sacramento or San Diego or San Francisco or Scottsdale or Seattle or St. Louis or St. Petersburg or Walnut Creek or United States
$80k-$266k/yr HybridFull Time
Accenture
AccentureNYSE: ACN: Global professional services firm providing consulting and technology solutions.
5+ YOERequires 5+ years in data engineering, analytics, or ML; 4+ years with GCP; 5+ years with SQL and pipelines; 3+ years with Python or AI tools; and a bachelor's degree or equivalent.
Google Cloud Platform (GCP), BigQuery, Looker, Vertex AI, Gemini Foundation Models, Gemini Enterprise, Dataflow, Dataproc, Pub/Sub, Cloud Storage, Looker Studio, Model APIs, Embeddings, Dataplex, IAM, SQL, Python, Git
Job PostingView all
6d
Save
Mark Applied
Hide
IT DATA Engineer IV NOT A REMOTE POSITION $75 IMMED OPENING
Westborough, Massachusetts, United States
$75/hr OnsiteContract
Scope Services
Scope Services: Provides infrastructure and workforce solutions for the utility industry.
Manages Gas SCADA and RTU software, develops Aveva Enterprise SCADA screens and databases, programs field RTUs, maintains communications links, and supports capital projects.
Aveva Enterprise SCADA 2023, SCADA, Remote Terminal Units (RTUs)
Job PostingView all
6d
Save
Mark Applied
Hide
Google Senior Data Engineer
Albany or Milwaukee or Dallas or Columbus or Kirkland or Cincinnati or New York City or Cleveland or Oklahoma City or Austin or Chicago or St. Petersburg or Hartford or Pittsburgh or St. Louis or Miami or Sacramento or Raleigh or Minneapolis or Mountain View or Scottsdale or San Francisco or Morristown or Denver or Boston or Philadelphia or Des Moines or Overland Park or Los Angeles or Charlotte or Walnut Creek or Carmel or Seattle or Houston or Arlington or Atlanta or Redmond or Bentonville or Beaverton or Nashville or Detroit or San Diego or United States
$80k-$266k/yr HybridFull Time
Accenture
AccentureNYSE: ACN: Global provider of management consulting and technology services.
5+ YOERequires 5+ years in data engineering, analytics, or ML; 4+ years on GCP; 5+ years with SQL, modeling, and pipelines; 3+ years with Python or AI tools; bachelor's degree or equivalent.
Google Cloud Platform (GCP), BigQuery, Looker, Vertex AI, Gemini Foundation Models, Gemini Enterprise, Model APIs, Embeddings, Dataflow, Dataproc, Pub/Sub, Cloud Storage, Looker Studio, Dataplex, IAM, SQL, Python, CI/CD, Git, RAG
Job PostingView all
1w
Save
Mark Applied
Hide
Data Engineer
Boston, Massachusetts, United States
HybridFull Time
Geode Capital Management
Geode Capital Management: An institutional asset manager providing core beta exposures across equity and niche asset classes.
5+ YOEBachelor's degree in computer science or a similar field; 5+ years of data or software engineering experience; Python, DBT, SQL, data modeling, database design, and ETL/ELT skills.
Python, DBT, Snowflake, AWS, Airflow, SQL
Job PostingView all
1w
Save
Mark Applied
Hide
Data Engineer
California or Boston
$90k-$167k/yr RemoteFull Time
Manulife
ManulifeTSX: MFC: Provides insurance, wealth management, and investment services globally.
3+ YOEMaster’s degree in computer science, engineering, information systems, or related field; 3+ years with Python, SQL, ETL, cloud, distributed computing, databases, governance, APIs, machine learning, and monitoring.
Python, SQL, ETL, Microsoft Azure, AWS, Spark, Databricks, Synapse, Apache Airflow, Power BI, Tableau, MongoDB, REST APIs, Splunk, CloudWatch, Docker
Job PostingView all