Senior Python Spark Engineer Locations: Dublin, Ireland
Time Type: Full time
Posted on: Posted 7 Days Ago
Job Requisition ID: R-223196
Our Purpose We work to connect and power an inclusive, digital economy that benefits everyone, everywhere by making transactions safe, simple, smart, and accessible. Using secure data and networks, partnerships, and passion, our innovations and solutions help individuals, financial institutions, governments, and businesses realize their greatest potential. Our decency quotient, or DQ, drives our culture and everything we do inside and outside of our company. We cultivate a culture of inclusion for all employees that respects their individual strengths, views, and experiences. We believe that our differences enable us to be a better team – one that makes better decisions, drives innovation, and delivers better business results.
Title and Summary Senior Python Spark Engineer. Founded by Mastercard and IBM in 2018, Truata specializes in privacy-enhancing technologies for privacy risk assessment, de-identification, and true anonymization of data. Truata was one of the first companies in the EU to receive independently accredited ISO 27701 certification, the international standard for data privacy management. In 2023, Mastercard acquired full ownership of Truata with Truata continuing to operate as an independent entity within the Mastercard Data Insights & Analytics Organisation.
Truata is now recruiting for a Senior Python Spark Engineer for our Analytics Platform team.
The Role As a Senior Software Engineer, you will be required to architect and build Truata's Core Platform to support advanced GDPR compliant big data and data analytic applications. You will be a member of a highly capable software team designing and developing applications for big data handling, data wrangling, anonymization, and data analytics. Our PaaS solutions will be deployed in cloud native environments providing highly scalable and secure data management.
As engineering culture is important to us, a great candidate will be intellectually curious and honest, passionate and hungry to expand their skills, and a great team player with a high sense of ownership of the delivered software, while developing high-quality team-oriented solutions to complex software engineering problems. This role is meant to be computer programming intensive.
Key Responsibilities: Participate in the architecture, design, and implementation of customer oriented big data analytics platform using Python based on Spark. Ensure the long-term quality in terms of architectural designs and code quality. Communicate frequently with product management, data pipelines, data science, and other teams to coordinate work efforts. Full software lifecycle participation including design, development, testing, bug fixing, cloud deployment, and maintenance process automation. Participate in a highly collaborative Agile Scrum team-based software development process. What you need: 4-6 years' experience working in software application development as a programmer. Strong and solid knowledge of software engineering principles and practices. University degree in Computer Science or equivalent is a MUST. Advanced degree preferred. Strong experience developing Spark applications using Python. Expertise with Big Data Hadoop platforms like Databricks, Cloudera, Teradata, etc., and solid fundamental understanding of the Hadoop architecture. Demonstrated professional expertise in understanding and implementing OO design patterns, developing container-based applications in a microservices architecture, using the state-of-the-art software engineering best practices. Experience working with relational and NoSQL databases (such as tuning and optimizing complex queries for highly scalable systems) and query languages (specifically Hive/SparkSQL and ANSI SQL). Experience building large scale Spark applications & data pipelines, ideally with Batch processing running on Hadoop clusters. Creativity and passion for tackling challenging data problems and willingness to work in a start-up environment. Experience with Git or other similar version control software. Experience and commitment in developing high quality, robust, scalable, and maintainable software. Knowledge of testing frameworks, CI/CD, test automation, and software monitoring and alerting. Experience working in a high performant collaborative agile team environment. Curious, and eager to learn, willing to expand their technical skills while helping other team members to grow. Strong communication and interpersonal skills. What would also be great to have: Experience with messaging queues such as Kafka, RabbitMQ or JMS and reactive architecture paradigm. Experience designing and developing highly available fault tolerant software. Experience in architecture and development of data models and data dictionaries in big data systems. Familiar with at least one large cloud computing provider like Azure, AWS, or IBM cloud, in this order of preference. Experience designing and implementing highly secure software and solutions. Knowledge with secure vulnerability tools such as Checkmarx, Snyk or Nexus IQ. Proven hands-on coding experience with data analytics platforms and workflow engines, including SPSS, Knime, RapidMiner, SAS, Dataiku etc. Experience in operating a UNIX shell and reading and developing simple shell scripts. 1-2 years' experience designing software systems, and mentoring junior software engineers. Corporate Security Responsibility All activities involving access to Mastercard assets, information, and networks come with an inherent risk to the organization and, therefore, it is expected that every person working for, or on behalf of, Mastercard is responsible for information security and must:
Abide by Mastercard's security policies and practices; Ensure the confidentiality and integrity of the information being accessed; Report any suspected information security violation or breach; Complete all periodic mandatory security trainings in accordance with Mastercard's guidelines.
#J-18808-Ljbffr