Must Have Skills:
· Good in Spark, Scala Programming, Spark SQL, Spark Streaming
· Experience in Databricks and AWS
· Experience in Spark query tuning and performance optimization
· Good Knowledge in Java
· Experience working with - HDFS, S3, Cassandra, and/or DynamoDB
· Expertise with functional programming using Scala
· Hive Query Language with performance aspects Strong grip of Hadoop and its eco system
· Understanding the basic suet that provides the solution to Big Data Is mandatory
· Experience/ knowledge in big data using Scala spark, ML, Kafka, Elastic search will be plus.
· Basic components of Hadoop which are HDFS and MapReduce is must
· Should have Data accessing tools like high pick scoop
· Master data management and monitoring tools like Flume Zookeeper and Guzzi are Mandatory
· Ability to install and configure Hadoop eco system
· Produce unit tests for Spark transformations and helper methods
Good To Have Skills:
· Proficient in client communication
· Linux Shell Scripting
· Data Warehousing concepts
· Knowledge in Kafka, Kinesis, Hive, Pig.
· Experience in Version Control tools like SVN, GITHUB as well as experiences in Production deployment process.