Data Engineer
Deputy
- Sydney, NSW
- Permanent
- Full-time
- Design, implement, test, deploy, and maintain stable, secure, and scalable data engineering solutions and pipelines in support of data and analytics projects, including integrating new sources of data into our data warehouse.
- Maintain and build on our data warehouse and analytics environment.
- Produce scalable, replicable code and engineering solutions that help automate repetitive data management tasks.
- Identify, design, and implement process improvements such as: improving database performance, optimising data delivery and re-designing infrastructure for greater scalability, and maintainability among many things
- Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources within our AWS and data warehouse ecosystem.
- Implement and monitor best in class security measures in our data warehouse and analytics environment, with an eye towards the evolving threat landscape.
- Work with stakeholders including the Engineering, Product, Data Analytics and Design teams to assist with data-related technical issues and support their data infrastructure needs.
- Work with data and analytics experts to strive for greater functionality in our data systems.
- Help data analysts troubleshoot their SQL, Python, or R code.
- Strong command of relational databases and SQL. Extract, Transform, and Load (ETL) data into a relational database. We use Snowflake, Redshift, DynamoDB, MySQL and DBT.
- Proficiency with Python or R, especially for data manipulation and analysis, and ability to build, maintain and deploy sequences of automated processes with these tools.
- Advance data manipulation skills: read in data, process and clean it, transform and recode it, merge different data sets together.
- Demonstrated ability to write clear code that is well-documented and stored in a version control system. We use Git.
- Use APIs to push and pull data from various data systems and platforms.
- Experience in building and maintaining a dimensional data model using type 2 dimensions.
- Demonstrated ability to learn new techniques and troubleshoot code without support, ex. find answers to common programming challenges on Google. In other words, be able to learn on the job.
- Excellent listening, interpersonal, communication and problem solving skills.
- Demonstrated ability to work effectively in teams, in both a lead and support role.
- Demonstrated ability to work independently and be a self-starter.
- Software engineering, data science, data analytics experience
- Experience in Docker
- Knowledge of building data lake or data warehouse.
- Knowledge of handling streaming data.
- Experience in Infrastructure as Code and CICD practice.
- ML or predictive analytics experience.
- Experience in web or mobile application development.