Data Engineer (GCP)

What You'll Be Doing:

• Design and implement data processing systems using GCP and other distributed frameworks like Hadoop, Spark, Snowflake, Airflow, or other similar technologies.

• Build data pipelines to ingest data from various sources such as databases, APIs, or streaming platforms.

• Integrate and transform data to ensure its compatibility with the target data model or format. 

• Design and optimize data storage architectures, with One lake, data lakes, data warehouses, Serverless and any distributed file systems.

• Implement techniques like partitioning, compression, or indexing to optimize data storage and retrieval.

• Identify and resolve bottlenecks, tuning queries, and implementing caching strategies to enhance data retrieval speed and overall system efficiency. 

• Design and implement data models that support efficient data storage, retrieval, and analysis.

• Collaborate with data scientists and analysts to understand their requirements and provide them with well-structured and optimized data for analysis and modeling purposes.  

• Utilize frameworks like Spark to perform distributed computing tasks, such as parallel processing, distributed data processing, or machine learning algorithms 

• Implement security measures to protect sensitive data and ensuring compliance with data privacy regulations.

• Establish data governance practices to maintain data integrity, quality, and consistency.

• Monitor system performance, identify anomalies, and conduct root cause analysis to ensure smooth and uninterrupted data operations. 

• Collaborate with cross-functional teams including data scientists, analysts, and business stakeholders to understand their requirements and provide technical solutions. Communicating complex technical concepts to non-technical stakeholders in a clear and concise manner. 

• Ability to work under Agile and Scrum development methodologies 

• Stay updated with emerging technologies, tools, and techniques in the field of big data engineering. Exploring and recommending new technologies to enhance data processing, storage, and analysis capabilities. 

• Train and mentor junior data engineers, providing guidance and knowledge transfer.

What We're Looking For:

• Strong analytical thinking and problem-solving skills

• Strong communication skillset – ability to translate technical details to business/non-technical stakeholders

• Extensive experience in designing and building data pipelines (ELT/ETL) for large-scale datasets. Familiarity with tools like Databricks, Apache Nifi, Apache Airflow, or Informatica is advantageous

• Proficiency in programming languages such as Python, R or Scala is essential.

• In-depth knowledge and experience with distributed systems and technologies, including Apache Hadoop, Spark, Hive or similar frameworks.

• Familiarity with cloud-based platforms like AWS, Azure, or Google Cloud is highly desirable. 

• Solid understanding of data processing techniques such as batch processing, real-time streaming, and data integration. Experience with data analytics tools and frameworks like Apache Kafka, Apache Flink, or Apache Storm is a plus.

• Experience with Git repository maintenance and DevOps concepts

• Familiarity with building, testing, and deploying process

• Additional certifications in big data technologies or cloud platforms are advantageous

Missing one or two of these qualifications? We still want to hear from you! If you bring a positive mindset, we'll provide an environment where you feel valued and empowered to learn and grow.

Offer:

• Stable employment. On the market since 2008, 1800+ talents currently on board in 7 global sites.

• “Office as an option” model. You can choose to work remotely or in the office.

• Flexibility regarding working hours and your preferred form of contract.

• Comprehensive online onboarding program with a “Buddy” from day 1.

• Cooperation with top-tier engineers and experts.

• Certificate training programs. Lingarians earn 500+ technology certificates yearly.

• Upskilling support. Capability development programs, Competency Centers, knowledge sharing sessions, community webinars, 110+ training opportunities yearly.

• Grow as we grow as a company. 76% of our managers are internal promotions.

• A diverse, inclusive, and values-driven community.

• Autonomy to choose the way you work. We trust your ideas.

• Create our community together. Refer your friends to receive bonuses.

• Activities to support your well-being and health.

• Plenty of opportunities to donate to charities and support the environment.

• Modern office equipment. Purchased for you or available to borrow, depending on your location.

• Great Place to Work Certified Employer in the Philippines