Abhishek Kumar

As a Data Engineer/Solution Developer, I provide solutions for Big Data related requirements. The task includes creating Algorithms/Programs/Utility tools in Hadoop Environment for processing Big Data and manage/Store them in Big Data environment like Hive/Hbase. Meet the adhoc analytical requirements on the huge data by developing programs for finding specific solutions to the issues encountered. Also, the responsibility involves transforming already existing Ab-Initio ETL Data flow to Big Data environment by developing Hadoop Jobs (Map-reduce programs, Hive Queries and Pig Scripts). Tools and Technologies: MapR Hadoop, Hive, Pig, Python, Perl, Shell Scripting, Java, MySql, Hbase, SQL

Currently I am working as Data Quality Analyst/Data Engineer for Arkansas Department of Education, Division of Research and Technology. As part of my work I have developed a Data Platform called Arkansas Longitudinal Data System, the system is used for Entity Resolution (Data Integration) service. System is capable of running millions of records as per ad-hoc as well as batch processing requirements. System is capable of performing Identity matching, Identity Capture and Identity Update on the data sets. The system is being used by different state agencies for Entity Resolution needs for example department of education uses the system for student data matching and similarly other agencies too. Also, as part of another assignment I have developed a Data Profiling tool called ADE-DPS (Data Profiling System), specifically used for Department of Education. the tool does the specific profiling job as well as helps producing Business Rules for the Entity Resolution System and helps determining the accuracy of Entity Resolution process with the feature of Entropy Calculation. Tools Technologies/Concepts: C# Programming, SQL Server, Data Profiling, Data Matching, Data Analysis, Algorithm development, Data Cleaning, Entity Resolution, Data Management, Data Quality, Telerik