Big Data Developer

Responsibilities: Design, develop, and maintain large-scale data processing pipelines using Big Data technologies (e.g., Hadoop, Spark, Python, Scala).Design, develop, test, implement, and support technical solutions within an Agile development environment, utilizing distributed microservices architecture, big-data technologies (HIVE, Presto, Spark, Hadoop, Trino), and full-stack systems. Partner with cross-functional teams to understand application requirements and testing scenarios, delivering scalable technical solutions that meet business objectives. Build and maintain high-performance API platforms on AWS infrastructure, enabling interactive-speed access to big data while adhering to AWS Well-Architected Framework principles. Leverage AI tools and technologies to enhance development efficiency, code quality, and solution innovation throughout the software development lifecycle. Drive technical innovation by staying current with emerging technology trends, including AI/ML advancements, evaluating new tools and frameworks, and actively contributing to internal and external technology communities. Demonstrate a growth mindset, take ownership of deliverables, maintain customer-centric focus, and exercise sound technical judgment in decision-making. Must have skills: Python and/or Scala Big data (we work with terabytes per day)AWSSQLOrchestration, deployment understanding Kubernetes (troubleshooting, configuring for scale), can be AWS, GCP, or Azure Kubernetes. Should have maintained existing Kubernetes infrastructure GenAI for use in engineering (ex: Kiro), LLMs, etc.

Qualifications: 8+ years of professional software engineering experience with demonstrated leadership capabilities.3+ years of hands-on development experience in Data Engineering, Python / Scala / Java, SQL.3+ years working with Big Data technologies such as Hadoop, Spark, or Presto, etc.2+ years developing REST/web services and writing complex SQL queries for relational databases. Proven experience with AWS services including EMR (Hive/Presto), Glue, Athena, ECS, Lambda, and S3.Proficiency with Dev Ops practices and CI/CD pipelines using tools like Jenkins, Cloud Formation, etc. Working knowledge of scripting languages. Bachelor’s degree in computer science, information technology, engineering, or related technical field.#LI-CGTS#TS-3142