You are viewing a preview of this job. Log in or register to view more details about this job.

Data Engineer

Apollo MIS
Email

Apollo is building out a data lake using an already-established medallion architecture, and expanding it as we bring on new clients and migrate existing ones. We're looking for a Data Engineer to lead this effort — owning the data lake's architecture and health, leading external data imports as we onboard new clients, and leading all API-based data imports, which flow directly into the data lake rather than our transactional database.

This is a leadership role on the data side: you'll be the person setting direction on how data flows in, how it's structured, and how ETL pipelines are built and maintained, while also staying hands-on with the pipelines and queries themselves.

Responsibilities 

Lead the data lake build-out and expansion using our existing medallion architecture, onboarding both new and existing clients Own external data imports as part of client onboarding Lead all API-based data imports, ensuring API data is imported directly into the data lake rather than the transactional database Design, build, and maintain ETL pipelines using Python Write and optimize complex SQL queries against large datasets Build web scraping tools to source external data as needed Set technical direction and best practices for the team's data engineering work Requirements Experience with Azure Synapse (data lake) Experience with Azure Data Lake Studio Professional SQL experience, preferablyin a leadership role Python experience, preferrably in production Experience building and maintaining ETL pipelines ETL experience using Python Web scraping experience Comfort owning deployments, not just writing code Experience with Git/GitHub, Docker, or model deployment tools Ability to clearly communicate insights to both technical and non-technical stakeholders Ability to work independently and collaboratively in a fast-paced environment Able to work 40 hours per week, in-office Nice to Have Experience with PySpark Experience with Kafka Experience with Flink Microsoft 365 experience Our Evaluation Method ‍

Our process includes a practical component — we're interested in how quickly you can get up to speed on an existing database/data lake architecture, and we'll ask you to work through some complex SQL queries.

Please submit your resume and a cover letter to info@apollov2.com.

ApolloV2 is an equal opportunity employer with a commitment to hiring people with diverse backgrounds. We do not discriminate based on age, civil or family status, disability, ethnicity, gender, race, religion, or sexual orientation.