Data Engineer

Responsibilities: Designed, developed, and maintained scalable data pipelines to extract, transform, and load data from multiple sources. Built ETL/ELT workflows using Python, SQL, PySpark, and Azure Data Factory. Created and optimized data processing solutions with Apache Spark and Databricks. Worked with structured and unstructured data from relational databases, REST APIs, flat files, and cloud storage. Implemented data integration solutions aligned to business intelligence and analytics requirements. Developed complex SQL queries, stored procedures, views, and functions to transform and process large datasets. Implemented data validation, cleansing, transformation, and data quality checks to maintain accurate datasets. Built and maintained data warehouses, data marts, and dimensional data models. Integrated data from enterprise systems using REST APIs and database connectors. Monitored data pipelines, diagnosed failures, and resolved performance and data processing issues. Implemented incremental loading, partitioning, and optimization techniques to improve pipeline performance. Used Snowflake for data loading, transformation, query optimization, and warehouse management. Implemented data security, access controls, and governance practices for sensitive business information. Used Git and Azure DevOps for source control, CI/CD, and deployment management. Participated in Agile/Scrum ceremonies, technical discussions, code reviews, and production support.

Requirements: Bachelor's degree in Computer Science, Information Technology, or a related field. Experience in data engineering, ETL development, and implementing data pipelines. Strong programming knowledge in Python, SQL, and PySpark. Experience working with cloud platforms and data warehousing technologies. Understanding of data modeling, database concepts, and data integration. Strong analytical, problem-solving, debugging, and communication skills. Technologies Python, SQL, PySparkApache Spark, Hadoop, Databricks Azure Data Factory, Informatica, SSISMicrosoft Azure, AWSAzure Data Lake, Azure Data Lake Storage, Amazon S3Snowflake, Azure Synapse Analytics, SQL Server PostgreSQL, MySQL, OracleStar Schema, Snowflake Schema, Dimensional Modeling Git, Azure DevOps, CI/CD, Jenkins Power BIAgile, Scrum, SDLCREST APIsFor this onsite role in San Jose, CA, you will use a mix of cloud data platforms, ETL/ELT tooling, and warehouse technologies to deliver dependable data pipelines and analytics-ready data.