Data engineer with AWS

HARAMAIN SYSTEMS INC. • Contract • San Francisco, CA, US • $50 - $60 / hour • 5m ago

Position Senior Data Engineer With AWS

Location: Bay Area

Type: Contract

Required

Please find the Job Description:

Bachelor’s in Computer Science or related disciplines
8+ years of relevant experience in the field of Data Engineering.
Strong in ETL Data warehousing and SQL.
Experience with SnowSQL development
Knowledge of Snow Pipe implementation
Expert level in working with SQL against Snowflake data warehouse
experience in AWS or Azure or Google cloud
Proficient in building data platforms (architecture, storage, management, monitoring)
Experience working with different file formats like Parquet, ORC, Avro, JSON etc.
Familiarity with code versioning tools Git, perforce, SVN etc.
Knowledge of Data Structures and Algorithm.
Strong Python programming skills with ability to implement OOPs and functional programming. Knowledge of Scala/Java would be plus.
Strong knowledge on RDBMS and NoSQL databases with the ability to implement them from scratch. Knowledge of Graph databases will be a plus.
Strong expertise in building & optimizing data pipelines, architectures, and data sets.
Familiarity with big data infrastructure inclusive of MapReduce, Hive, HDFS, YARN, HBase, Oozie, etc.
Experience building ETL pipelines using ETL tools like Informatica, TalendD, Pentaho, etc.
Knowledge and experience of using orchestration frameworks like Airflow, Oozie, Luigi, etc.
Experience using Spark, and building jobs using Python/Scala/Java.
Knowledge building stream processing platforms using Spark Streaming, Storm, etc. Knowledge of Kafka/Flink+Beam would be a plus.
Knowledge of building REST API endpoints for data consumption.
Knowledge of implementing CI/CD on the pipelines is a plus.

Compensation: From $50.00 to $60.00 per hour