- About the RoleWe are looking for a Senior Data Engineer to join our team in Bali. In this role, you will play a key part in building and scaling production-grade web scrapers and supporting the Lead Data Engineer in technical decisions, project planning, and team coordination. You will also mentor junior and medior engineers while ensuring best practices are followed.
Responsibilities
Lead the development and maintenance of scalable, production-grade web scrapers.Design, build, and maintain reusable scraper libraries and frameworks.Define coding standards, workflows, and processes with the Lead Engineer.Implement robust monitoring, testing, and alerting systems to ensure reliability.Manage and optimize cloud infrastructure (AWS/GCP/Azure) for large-scale scraping workloads.Optimize compute, storage, and proxy usage for performance and cost efficiency.Mentor and review code of junior and medior engineers.Collaborate with stakeholders to prioritize backlog items and integrate scraper outputs into pipelines.Act as second in command to the Lead Data Engineer for technical and project support.Must-Have SkillsStrong proficiency in Python and SQL, with hands-on web scraping experience.Deep knowledge of frameworks like Scrapy, Playwright, Selenium, and anti-bot strategies.Experience designing reusable libraries, coding standards, and modular scraper frameworks.Proficiency with cloud infrastructure (AWS/GCP/Azure), including compute, storage, and security basics.Experience with Docker & Kubernetes for containerization and orchestration.Familiarity with CI/CD pipelines, Git workflows, and IaC tools (Terraform/CloudFormation).Strong debugging, performance tuning, and large-scale scraper optimization skills.Excellent English communication skills (written & verbal).Good-to-Have SkillsWorkflow orchestration (Airflow, Prefect, Dagster).Observability/monitoring tools (Prometheus, Grafana, CloudWatch, ELK).Cost optimization in cloud and proxy-heavy workloads.Knowledge of data governance, RBAC, and compliance.Exposure to downstream data engineering pipelines (ETL/ELT, APIs, databases).
Benefits
Competitive monthly salary.Re
Location
support: one-off reimbursement for flights and moving expenses (for candidates relocating from other parts of Indonesia).Opportunity to work in a collaborative, data-driven environment with growth opportunities.