fbpx

Data Engineer

Full time @Digital Micro Services in IT- Software / DB / QA / Web / Graphics / GIS
  • 29 Siebel Ave, Colombo 00500, Sri Lanka
  • Apply Before : 12th August 2022
  • Share:

Job Detail

  • Job ID 39024
  • Career Level Executive
  • Experience 1 Year
  • Qualifications Degree Bachelor

Job Description

KEY RESPONSIBILITIES

  • Writing Code in Scala or Python
  • Design, Develop and test BigData solutions using selected tool
  • Read, extract, transform, stage and load data
  • Writing code/scripts, web scraping, calling APIs, SQL, queries, etc.
  • Process unstructured data and to generate modelled data
  • Build self monitoring and self healing/zero touch operations
  • Ability to design solutions independently based on high-level architecture
  • Collaborate with other development and research teams

PREFERRED QUALIFICATIONS

  • 1+ years of experience as a Big Data Engineer /Software Engineer or similar role
  • Bachelor’s Degree in Computer Science or equivalent preferred
  • Programming experience, ideally in Python, Scala or Java
  • Willingness to learn new programming languages
  • Experience with Spark/Hadoop Ecosystem
  • Experience with building stream-processing systems, such Spark-Streaming
  • Experience with integration of data from multiple data sources
  • Experience with various messaging systems, such as Kafka
  • Knowledge of statistics
  • Knowledge of data cleaning, wrangling, visualization and reporting
  • Experience in Apache Nifi, PostgreSQL, MapReduce, TimescaleDU, data mining, machine learning, natural language processing, or information retrieval is a plus
  • Experience processing large amounts of structured and unstructured data
  • Integrating data from multiple sources
  • Experience with Big Data ML toolkits, such as SparkML, Mahout, or H20 is a plus

Other jobs you may like