• Web Development
  • DevOps
  • Big Data

Professional Status

Consultant
Available

About Me

DevOps and Backend Consultancy - http://hurranet.com/services

Experience

Senior Software Engineer

Rakuten Marketing
September 2013 to November 2015
Responsibilities completed
  • Cloudera • CDH • HDFS • Hadoop • Storm • Flume • Chef • Ansible • AWS • Jenkins • Logstash • Java • RESTful APIs • Node.js • Python • PHP • Symfony ⁍
  • ------ Big Data Java Engineer ------
  • Wrote performant Storm topology to stream high-volume data (roughly 7 - 10k rows per second) from Oracle to HDFS in real-time. (some portions are open source now, available at http://github.com/linkshare )
  • W​rote Flume sinks to geocode and stream click data from Apache logs to Hive tables.
  • ​Wrote unit and functional tests for Storm topology
    components, included Oracle mock using in-memory H2, and Cucumber JVM feature files for integration and functional tests.
  • Big Data TechCon 2014 Certified.
  • ------ DevOps Engineer ------
  • Wrote Chef cookbooks, Python/Shell scripts to fully automate the provisioning of a complex, interdependent 100-terabyte CDH cluster
  • Built custom CSDs and Cloudera Parcels for Redis, Storm and Pentaho BI server. (open source now, available at http://github.com/linkshare )
  • A​utomated deployment of Oozie workflows and HDFS scripts using Jenkins and Git hooks
  • A​utomated SOPs to manage/operate Storm streams using Jenkins.
  • Exposure to Lambda Architecture, with focus on real-time layer and streaming
  • Integrated Storm with Logstash.
  • Wrote many Chef cookbooks to provision various application servers both on AWS and data centers. Leveraging advanced concepts such as encrypted data bags, Chef Search API, LWRPs, Berkshelf ...etc
  • ------ Back-end/Full Stack Developer ------
  • Strictly-compliant RESTful API using PHP/Symfony
  • Strong exposure to MongoDB, including concepts such as sharding and replica sets.
  • Node.JS real-time data streaming application, capable of streaming up to 5k records/second
  • CAS Server and Shibboleth IDP3 integration for SAML2 SSO implementation.
  • RESTful APIs using Node.JS, Express and MongoDB
Company Description

Rakuten Marketing provides eCommerce businesses with affiliate marketing services. Rakuten Affiliate Network clients include Macy's, Walmart, Vince Camuto, Sportscraft, and Viator.

The Rakuten Marketing network (formerly Linkshare) is largest pay for performance affiliate marketing network on the Internet.

Company website

http://marketing.rakuten.com