Not a proof-of-concept notebook — production data platforms moving real volume: a cardiac-device analytics program on the Hadoop ecosystem, a Fortune 500 bank’s big-data pipelines, satellite-imagery analytics on Databricks and Spark, and a healthcare data lake migrated from AWS to Google Cloud. Leopard Data designs the lakehouse, builds the pipelines, distributes the compute, and makes the data trustworthy — from raw ingest to the model and the dashboard.
Six disciplines, every one of them shipped on a real platform carrying real volume.
Object-store lakes on S3, ADLS, and GCS with open table formats — Apache Iceberg, Delta, and columnar Parquet/ORC — so the same data serves SQL, Spark, and ML without copies. Partitioning, compaction, and schema evolution designed in from day one.
Apache Spark, the Hadoop/MapReduce ecosystem, and Hive for batch at scale — on Databricks, AWS EMR, AWS Glue, and Azure Synapse. We tune the jobs that actually run nightly, not the ones that look good in a demo.
Cloud warehouses and query engines — BigQuery, Amazon Redshift, Azure Synapse, and Athena/Trino over the lake. Modeled marts, materialized aggregates, and BI that answers the question in seconds instead of overnight.
Ingest, transform, and serve — Azure Data Factory, AWS Step Functions, Luigi, and Airflow-style DAGs with dbt-style modeling. Idempotent, observable, and re-runnable, with preprocessor and post-processor stages that recover cleanly from failure.
Feature engineering across hundreds of thousands of features and billions of calculations — distributed with Ray, Anyscale, and KEDA-driven Kubernetes so a job that would never finish on one box finishes on a fleet. The data layer that feeds the AI & ML work.
HIPAA-aware data handling, lineage, encryption with KMS and Secret Manager, and partition/format choices that keep scan costs sane. Big data that auditors and finance can both live with.
The lakehouse, the distributed compute, and the cross-cloud migration — from raw ingest to a served result.
Source systems land as immutable raw files in object storage; Spark and Glue clean and conform them into open Iceberg/Parquet tables; modeled marts publish into the warehouse; and BI and ML read from the same governed layer. One copy of the truth, many readers.
flowchart LR
SRC["Source systems -- apps, devices, files, APIs"] --> ING["Ingest -- batch and stream landing"]
ING --> RAW["Raw zone -- object storage, immutable Parquet"]
RAW --> PROC["Spark and Glue -- clean, conform, enrich"]
PROC --> LAKE["Curated lakehouse -- Iceberg and Delta tables"]
LAKE --> WH["Warehouse -- BigQuery, Redshift, Synapse"]
LAKE --> ML["ML and feature store"]
WH --> BI["BI and dashboards"]
For a Fortune 500 industrial client, a feature-ranking and forecasting workload had to test up to 200,000 features across billions of calculations. A driver fans work onto a queue; KEDA scales runner pods on queue depth; Ray parallelizes the math across the cluster; results land in PostgreSQL and rank the best-correlated trends.
flowchart TD
USER["Data scientist -- ranking job"] --> DRV["Main driver -- reads config, writes DB"]
DRV --> Q["Work queue -- millions of feature jobs"]
Q --> KEDA["KEDA scaler -- watches queue depth"]
subgraph CL["Distributed cluster -- AWS EKS"]
KEDA -->|"scale 0 to N"| POD["Target and model runner pods"]
POD --> RAY["Ray head and workers -- parallel compute"]
end
POD --> DB[("PostgreSQL -- results")]
RAY --> DB
DB --> RANK["Ranked features -- best correlated trends"]
On a flight-analytics program, sensors across the airframe stream telemetry off each helicopter; the data is collected and processed in the Hadoop ecosystem — landed on HDFS, crunched with MapReduce and Hive — then turned into flight metrics and surfaced to customers on an interactive dashboard. Raw vibration, engine, and flight data becomes something an operator can actually read.
flowchart LR
HELI["Helicopter -- onboard sensors"] --> TEL["Flight telemetry -- vibration, engine, position"]
TEL --> ING["Ingest -- collection and landing"]
ING --> HDFS["Hadoop HDFS -- raw flight data"]
HDFS --> PROC["MapReduce and Hive -- process and aggregate"]
PROC --> METRICS["Flight analytics -- health and performance metrics"]
METRICS --> DASH["Customer dashboard -- per-aircraft insights"]
For a national healthcare technology platform, we helped move a clinical data lake from AWS to Google Cloud — mapping Redshift, EMR, Glue, Athena, and Iceberg/Parquet to BigQuery, Dataflow, and Cloud Storage, with FHIR/HL7 healthcare data on the Google Healthcare API. AI-assisted analysis accelerated the port; an abstraction layer kept eventing portable.
flowchart LR
subgraph AWS["AWS -- source"]
S3["S3 data lake -- Parquet and Iceberg"] --> EMR["EMR and Glue -- Spark ETL"]
EMR --> RS["Redshift warehouse"]
S3 --> ATH["Athena queries"]
end
S3 --> MIG["Migration -- AI-assisted mapping and validation"]
RS --> MIG
subgraph GCP["Google Cloud -- target"]
MIG --> GCS["Cloud Storage data lake"]
GCS --> DF["Dataflow -- transform"]
DF --> BQ["BigQuery warehouse"]
GCS --> HC["Healthcare API -- FHIR and HL7"]
end
The big-data technologies we run — the same tools across every engagement.
Six platforms across aviation, healthcare, banking, insurance, and industry. All of them shipped.
Leopard Data led a 160-developer program — five teams, two onshore and three offshore — building the big-data processing pipeline for a heart-analytics device platform on the Hadoop ecosystem, with .NET and SQL Server across an Azure/on-prem hybrid. High-volume device telemetry turned into clinical analytics at scale.
As Business Solutions Architect on a 60+ person program, Leopard Data drove the architecture for a 10-million-user banking information system — big-data pipelines on Azure and AKS with security architecture across distributed services. Built for the throughput and the audit trail a bank actually requires.
Leopard Data was Solutions Architect on a property-risk analytics platform leading a 50-developer team, processing satellite imagery and geospatial data through Databricks, Hive, and Spark with Azure Synapse and Data Factory, plus Python/Luigi preprocessor and post-processor stages over the big-data layer.
For the largest private company in the U.S., Leopard Data engineered the distribution layer for a feature-ranking and forecasting engine running hundreds of millions of calculations over data-science models, parallelized with Ray and Anyscale and scaled with KEDA on AWS EKS — the big-data compute behind the ML.
Read the full case studyLeopard Data built a big-data analytics pipeline for helicopter flight data — telemetry off the airframe collected and processed across the Hadoop ecosystem with .NET, Python, and SQL Server on Linux, then surfaced to customers as per-aircraft health and performance dashboards. High-volume sensor data turned into operational insight.
For a national healthcare technology platform, Leopard Data helped migrate a clinical data lake from AWS to Google Cloud — mapping Redshift, EMR, Glue, Athena, and Iceberg/Parquet to BigQuery, Dataflow, and Cloud Storage, with FHIR/HL7 data on the Google Healthcare API and AI-assisted tooling to accelerate the port.
Read the full case studyFrom lakehouse design and Spark/Hadoop pipelines to warehousing, distributed ML compute, and cross-cloud migration, Leopard Data ships the real thing. Corp-to-Corp engagements out of Plano, TX.