Big Data – detailed technical migration plan(may need to specify the command needed) for moving data from Hive to Camel-MongoDB, Spark and Kafka, Sentry and/or IAM (Identity Access Manager) for AWS.
Paper details:
Computer Science – Big Data Module Project: (1100words) Let’s suppose you have a data table in Hadoop’s Hive system. You are now requested to make a detailed plan including all the step and commands that you would need to migrate data from Hive to other applications within the Hadoop environment: Follow all the steps below and specifically include all the command and execution that you would need to complete the following task 1,2,3: (400 words)It is proposed to migrate the current system away from Hive to the NoSQL scheme offered by Camel-MongoDB. Explain the prime considerations and actions that need to be taken for such a migration. (400 words)It is intended to convert the static visualisation dashboards previously created to live ones via the use of streaming schemes offered by Spark and Kafka. Outline what steps would need to be taken to accomplish this. (300 words)Explain how the use of Sentry and/or IAM (Identity Access Manager) for AWS may be used to help secure the cluster and the Hadoop deployment in an enterprise environment.
