Software Engineer (Data Platform) - Merpay

Data Platform Engineer

Job Description

At Mercari, we believe in circulating all forms of value to unleash the potential in all people. We aim to create a world where anyone can unleash their potential by connecting people all over the world and using technology to create opportunities for individuals to realize their dreams and contribute to society.

As a Data Platform Engineer on the Merpay team, you will be responsible for designing, developing, and operating data infrastructure and pipelines to support data utilization for the entire Mercari Group. This includes not only Merpay but also the Mercari marketplace app. You will work on a wide range of areas such as marketing, machine learning, and R&D, utilizing data for business decisions and other processes.

You will work as part of the Data Platform Team, an engineering organization that builds the necessary systems for various domains that utilize data. Your responsibilities will include collecting data from microservices, developing data pipelines, constructing Data Lakes for storing collected data, managing permissions and resources on DWH, developing tools to support data utilization, and improving system performance and automating operations.

You will have the opportunity to work on large-scale data infrastructure and contribute to the development of an ecosystem to support data-driven business expansion. You will be involved in the entire data platform creation process, from design to development to operation.

Responsibilities

• Collecting data from microservices

  • Developing batch-based data pipelines
  • Developing streaming-based data pipelines
  • Developing SDK for logging within microservices

• Data Lake, DWH

  • Constructing Data Lakes for storing collected data
  • Managing partial permissions and resources on DWH

• Data utilization

  • Developing tools to support use of data in Data Lakes and on DWH
  • Developing a platform to provide stream data processing function

• Company-wide

  • Improving performance, aiming for low latency/high throughput according to application and middleware development, operation, and requirements
  • Developing tools to automate operations and/or lower costs
  • Identifying and solving technical issues with the system caused by engineering

Required Experience

• A shared belief in Mercari and Merpay’s missions and values

• A degree in computer science or a related field, or else five or more years of practical experience in software development

• Experience designing, developing, and operating large-scale services and/or distributed systems

• At least two of the following:

  • Experience developing in the cloud with AWS, GCP, etc.
  • Experience developing systems utilizing container technologies such as Kubernetes
  • Development experience using message queues like Cloud Pub/Sub and Apache Kafka
  • Experience in data processing development using distributed processing frameworks like Apache Flink and Apache Spark
  • Experience with ETL system using workflow engines like Airflow and Digdag
  • Experience designing application logs for large-scale services

Preferred Experience

• Experience working at a financial institution, Fintech company, or EC company
• Ability to identify the cause of technical system issues (e.g. drop in performance) and to resolve them
• Knowledge of network protocols such as TCP/IP, HTTP, gRPC, etc.
• Experience developing and operating software using two or more of the following: Go, Java, Scala, or Python
• Developed data collection systems using Treasure Data-created OSS (such as Fluentd, Embulk)
• Experience in development using DWH like BigQuery, Redshift, or Snowflake
• Experience in development using RDBMS, including MySQL and RDS
• Experience in development or cluster operation using the Hadoop ecosystem
• Experience developing software based on a microservice architecture
• Experience publishing and contributing to OSS

Language Requirements

• English: Basic (CEFR - A2) optional
• Japanese: Independent (CEFR - B2)

Screening Criteria

• Has a strong, shared belief in Mercari and Merpay’s missions and values
• Capable of going bold and possessing a strong sense of responsibility in regards to their work
• Possesses excellent communications skills which can push organizations and projects forward
• Able to collect information independently and identify the essence of issues to solve problems through engineering
• Thinks independently, acts independently, and leads operations with All for One mentality for the sake of the team and while holding on to professional mindset

Technical Assessment Criteria

• Documentation skills
• Testing
• Ability to code (efficiency, readability, maintainability)
• Design aptitude

Working Conditions

Employment Status

Full-time

• Probationary period: First 3 months after joining the company. (During this period your contract conditions will be the same as that of a permanent employee.)

Office

Roppongi

• Smoking is prohibited within our offices
• Mercari has introduced a work style policy called “Your Choice.” Each member is free to choose whether they want to work in the office or work fully remote. Exceptions made for certain kinds of work.

Work Hours

• Full flextime (no “core time” or “flex time”)

  *Does not apply to all positions

Holidays

• Two days off per week (as well as national holidays, New Year's break, etc.)
• Paid leave, congratulatory and bereavement leave, relax days, sick leave

Salary

• Annual salary paid in 12 monthly installments (including fixed overtime allowance)
• Based on skills, experience, and abilities
• Reviewed twice a year

Benefits

• Complete health and social insurance
• Incentive program
• Support systems, including those that benefit the employee’s family members

Support

• Relocation support
• Language learning support
• Translation/interpretation support

Media

Corporate Sites

• Mercari, Inc.
• Merpay, Inc.
• Souzoh, Inc.
• Mercoin, Inc.
• Mercari US

Owned Media

• Mercari Careers
• Mercari Engineering Portal
• Mercan - Introduces the people that make Mercari
• Mercari US Careers
• Mercari US Blog