{"id":179,"date":"2022-03-14T17:27:21","date_gmt":"2022-03-14T17:27:21","guid":{"rendered":"http:\/\/vargas-solar.com\/cloud-bigdata\/?page_id=179"},"modified":"2022-03-14T17:35:35","modified_gmt":"2022-03-14T17:35:35","slug":"executing-map-reduce-programs-on-hadoop-environments","status":"publish","type":"page","link":"http:\/\/vargas-solar.com\/cloud-bigdata\/executing-map-reduce-programs-on-hadoop-environments\/","title":{"rendered":"Executing Map Reduce Programs on Hadoop Environments"},"content":{"rendered":"\n<h2 class=\"wp-block-heading\"><em>Objective<\/em><\/h2>\n\n\n\n<p>The general objective of this exercise is to perform the first steps on the use of a Hadoop Environment for executing map-reduce programs (written in Python). This first exercise with show how to install a one node Hadoop setting on Collab and observe how to implement and run a map-reduce program.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Material<\/h2>\n\n\n\n<ul class=\"wp-block-list\"><li>Google Collab account<\/li><li><a href=\"https:\/\/github.com\/gevargas\/bigdata-management\/blob\/master\/Intro_Hadoop.ipynb\">https:\/\/github.com\/gevargas\/bigdata-management\/blob\/master\/Intro_Hadoop.ipynb<\/a><\/li><\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">&nbsp;<\/h2>\n\n\n\n<h2 class=\"wp-block-heading\">Description<\/h2>\n\n\n\n<p>The main steps of the exercise are very simple. At first, this exercise does not run on a cluster but on one CPU allocated by default by google cloud. It helps to concentrate on the way the map and reduce functions are specified and how a program is designed on the map-reduce model.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">To Do and To Hand In<\/h2>\n\n\n\n<ul class=\"wp-block-list\"><li>Propose a UML component diagram of the Hadoop environment installed on Collab.<\/li><li>Propose a UML component diagram of the two map-reduce count words programs tested in the lab.<\/li><li>Explain how the first example implementing a grep operation with a regular expression is executed.<\/li><li>Explain the way the program \u201ccount words\u201d is executed in the example.<\/li><li>What is the role of google drive in these examples?<\/li><\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Objective The general objective of this exercise is to perform the first steps on the use of a Hadoop Environment for executing map-reduce programs (written in Python). This first exercise with show how to install a one node Hadoop setting on Collab and observe how to implement and run a map-reduce program. Material Google Collab [&hellip;]<\/p>\n","protected":false},"author":11,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"page-templates\/full-width.php","meta":{"footnotes":""},"class_list":["post-179","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"http:\/\/vargas-solar.com\/cloud-bigdata\/wp-json\/wp\/v2\/pages\/179","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/vargas-solar.com\/cloud-bigdata\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"http:\/\/vargas-solar.com\/cloud-bigdata\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"http:\/\/vargas-solar.com\/cloud-bigdata\/wp-json\/wp\/v2\/users\/11"}],"replies":[{"embeddable":true,"href":"http:\/\/vargas-solar.com\/cloud-bigdata\/wp-json\/wp\/v2\/comments?post=179"}],"version-history":[{"count":1,"href":"http:\/\/vargas-solar.com\/cloud-bigdata\/wp-json\/wp\/v2\/pages\/179\/revisions"}],"predecessor-version":[{"id":180,"href":"http:\/\/vargas-solar.com\/cloud-bigdata\/wp-json\/wp\/v2\/pages\/179\/revisions\/180"}],"wp:attachment":[{"href":"http:\/\/vargas-solar.com\/cloud-bigdata\/wp-json\/wp\/v2\/media?parent=179"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}