{"id":61,"date":"2025-10-29T10:54:52","date_gmt":"2025-10-29T10:54:52","guid":{"rendered":"http:\/\/vargas-solar.com\/data-ware-lake-houses-bi\/?page_id=61"},"modified":"2025-10-29T10:58:24","modified_gmt":"2025-10-29T10:58:24","slug":"big-data-storage-capstone-project","status":"publish","type":"page","link":"https:\/\/vargas-solar.com\/data-ware-lake-houses-bi\/big-data-storage-capstone-project\/","title":{"rendered":"Big Data Storage Capstone Project"},"content":{"rendered":"\n<h2 class=\"wp-block-heading\">Principle<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Work in teams of 4 people.<\/li>\n\n\n\n<li><strong>T<\/strong>opic of your choice description, including story, objective, indicators, and analytics questions (indicators) &#8211; discuss with the lecturer about your idea.<\/li>\n\n\n\n<li>Combine more than one dataset (see below)<\/li>\n\n\n\n<li>Preferably adhere to sustainable development goals (https:\/\/www.agenda-2030.fr\/en\/the-17-goals\/)<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">Methodology<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Use \u22652&nbsp;<strong>open-data providers<\/strong>&nbsp;(with \u22652&nbsp;<strong>releases\/snapshots<\/strong>&nbsp;where possible), land them in a&nbsp;<strong>mini data lake<\/strong>with a small&nbsp;<strong>metadata catalogue<\/strong>, transform them with&nbsp;<strong>Talend<\/strong>&nbsp;into a&nbsp;<strong>relational data warehouse<\/strong>&nbsp;(star or snowflake), and publish a&nbsp;<strong>Tableau dashboard<\/strong>&nbsp;that runs&nbsp;<strong>backend OLAP queries<\/strong>&nbsp;to compute your business indicators (see details below).<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Final Deliverables<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Digital poster explaining the approach and results:&nbsp;<strong>before<\/strong><strong>02\/12\/2025<\/strong>&nbsp;at midnight<ul><li>Division of the work within the team (separate document)<\/li><\/ul><ul><li>General idea of the project<\/li><\/ul><ul><li>Data model adopted<\/li><\/ul>\n<ul class=\"wp-block-list\">\n<li>Processes created<\/li>\n<\/ul>\n<\/li>\n\n\n\n<li>Demo fest&nbsp;<strong>05\/12\/2025<\/strong>&nbsp;(dashboard and results obtained)<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"310\" src=\"http:\/\/vargas-solar.com\/data-ware-lake-houses-bi\/wp-content\/uploads\/sites\/64\/2025\/10\/schedule-1024x310.jpg\" alt=\"\" class=\"wp-image-68\" srcset=\"https:\/\/vargas-solar.com\/data-ware-lake-houses-bi\/wp-content\/uploads\/sites\/64\/2025\/10\/schedule-1024x310.jpg 1024w, https:\/\/vargas-solar.com\/data-ware-lake-houses-bi\/wp-content\/uploads\/sites\/64\/2025\/10\/schedule-300x91.jpg 300w, https:\/\/vargas-solar.com\/data-ware-lake-houses-bi\/wp-content\/uploads\/sites\/64\/2025\/10\/schedule-768x233.jpg 768w, https:\/\/vargas-solar.com\/data-ware-lake-houses-bi\/wp-content\/uploads\/sites\/64\/2025\/10\/schedule-1536x465.jpg 1536w, https:\/\/vargas-solar.com\/data-ware-lake-houses-bi\/wp-content\/uploads\/sites\/64\/2025\/10\/schedule-2048x621.jpg 2048w, https:\/\/vargas-solar.com\/data-ware-lake-houses-bi\/wp-content\/uploads\/sites\/64\/2025\/10\/schedule-500x152.jpg 500w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Figure&nbsp;1&nbsp;Partial deliverables calendar<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Evaluation Criteria<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Originality of the concept and the degree of alignment with the Sustainable Development Goals<\/li>\n\n\n\n<li>Use of different data sets described in the data lake<\/li>\n\n\n\n<li>An exploration interface for choosing portions of data for building a DW<\/li>\n\n\n\n<li>Data preparation and LT (Loading and Transformation) process to populate DW<\/li>\n\n\n\n<li>Analytics queries (OLAP)<\/li>\n\n\n\n<li>Relevance and interest of the indicators produced<\/li>\n\n\n\n<li>Level of difficulty (complexity of database, processes, etc.)<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">Material<\/h2>\n\n\n\n<h4 class=\"wp-block-heading\">List of tutorials<\/h4>\n\n\n\n<p class=\"wp-block-paragraph\">Follow this link to see an example for Talend:&nbsp;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>http:\/\/www.osaxis.fr\/talend-open-studio-par-lexemple\/<\/li>\n\n\n\n<li>https:\/\/business-intelligence.developpez.com\/tutoriels\/etl-open-source\/?page=Introduction<\/li>\n\n\n\n<li>https:\/\/fr.talend.com\/products\/data-preparation<\/li>\n<\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">Open data collections&nbsp;<\/h4>\n\n\n\n<ul class=\"wp-block-list\">\n<li>United Nations Open Data&nbsp;<a href=\"http:\/\/data.un.org\/Explorer.aspx\">http:\/\/data.un.org\/Explorer.aspx<\/a><\/li>\n\n\n\n<li>UN women&nbsp;<a href=\"https:\/\/data.unwomen.org\/\">https:\/\/data.unwomen.org<\/a><\/li>\n\n\n\n<li>Open Medic&nbsp;<a href=\"https:\/\/www.assurance-maladie.ameli.fr\/etudes-et-donnees\/open-medic-base-complete-depenses-medicaments\">https:\/\/www.assurance-maladie.ameli.fr\/etudes-et-donnees\/open-medic-base-complete-depenses-medicaments<\/a><\/li>\n\n\n\n<li>Open platform for French public data&nbsp;<a href=\"https:\/\/www.data.gouv.fr\/\">https:\/\/www.data.gouv.fr<\/a><\/li>\n\n\n\n<li>Metropolitan data for Greater Lyon&nbsp;<a href=\"https:\/\/data.grandlyon.com\/portail\/fr\/accueil\">https:\/\/data.grandlyon.com\/portail\/fr\/accueil<\/a><\/li>\n\n\n\n<li>Public Health France&nbsp;<a href=\"https:\/\/www.data.gouv.fr\/organizations\/sante-publique-france\/\">https:\/\/www.data.gouv.fr\/organizations\/sante-publique-france\/<\/a><\/li>\n\n\n\n<li>The Global Health Observatory&nbsp;<a href=\"https:\/\/www.who.int\/data\/gho\/\">https:\/\/www.who.int\/data\/gho\/<\/a><\/li>\n\n\n\n<li>Open data Enseignement sup\u00e9rieur et recherche&nbsp;<a href=\"https:\/\/data.enseignementsup-recherche.gouv.fr\/page\/home\/\">https:\/\/data.enseignementsup-recherche.gouv.fr\/page\/home\/<\/a><\/li>\n\n\n\n<li>SNCF open data&nbsp;<a href=\"https:\/\/ressources.data.sncf.com\/explore\/?sort=modified\">https:\/\/ressources.data.sncf.com\/explore\/?sort=modified<\/a><\/li>\n\n\n\n<li>Portail des donn\u00e9es ouvertes de l\u2019Union europ\u00e9enne&nbsp;<a href=\"https:\/\/data.europa.eu\/en\">https:\/\/data.europa.eu\/en<\/a><\/li>\n\n\n\n<li>U.S. Government\u2019s open data&nbsp;<a href=\"https:\/\/data.gov\/\">https:\/\/data.gov<\/a><\/li>\n\n\n\n<li>UK Government\u2019s open data&nbsp;<a href=\"https:\/\/www.data.gov.uk\/search\">https:\/\/www.data.gov.uk\/search<\/a><\/li>\n\n\n\n<li>Amazon Web Services Public Data Sets https:\/\/registry.opendata.aws<\/li>\n\n\n\n<li>Free Open Data Sources INSEE&nbsp;<a href=\"https:\/\/www.insee.fr\/fr\/statistiques?debut=0&amp;categorie=3\">https:\/\/www.insee.fr\/fr\/statistiques?debut=0&amp;categorie=3<\/a><ul><li>Working population aged 25 to 54 in employment by sector of activity and gender by municipality and department&nbsp;<a href=\"https:\/\/www.insee.fr\/fr\/statistiques\/fichier\/1893185\/pop-act2554-empl-sa-sexe-cd-trav-6816.zip\">https:\/\/www.insee.fr\/fr\/statistiques\/fichier\/1893185\/pop-act2554-empl-sa-sexe-cd-trav-6816.zip<\/a><\/li><\/ul>\n<ul class=\"wp-block-list\">\n<li>Codes and names of municipalities&nbsp;<a href=\"https:\/\/www.insee.fr\/fr\/statistiques\/fichier\/3720946\/communes-01012019-csv.zip\">https:\/\/www.insee.fr\/fr\/statistiques\/fichier\/3720946\/communes-01012019-csv.zip<\/a><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"applewebdata:\/\/3EFFF357-E5EC-41DC-923C-986450EFFF04#_ftnref1\"><sup>[1]<\/sup><\/a>&nbsp;https:\/\/www.tableau.com\/fr-fr<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"applewebdata:\/\/3EFFF357-E5EC-41DC-923C-986450EFFF04#_ftnref2\"><sup>[2]<\/sup><\/a>&nbsp;https:\/\/www.qlik.com\/us\/home<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Principle Methodology Use \u22652&nbsp;open-data providers&nbsp;(with \u22652&nbsp;releases\/snapshots&nbsp;where possible), land them in a&nbsp;mini data lakewith a small&nbsp;metadata catalogue, transform them with&nbsp;Talend&nbsp;into a&nbsp;relational data warehouse&nbsp;(star or snowflake), and publish a&nbsp;Tableau dashboard&nbsp;that runs&nbsp;backend OLAP queries&nbsp;to compute your business indicators (see details below). Final Deliverables &hellip; <a href=\"https:\/\/vargas-solar.com\/data-ware-lake-houses-bi\/big-data-storage-capstone-project\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":11,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-61","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/vargas-solar.com\/data-ware-lake-houses-bi\/wp-json\/wp\/v2\/pages\/61","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/vargas-solar.com\/data-ware-lake-houses-bi\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/vargas-solar.com\/data-ware-lake-houses-bi\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/vargas-solar.com\/data-ware-lake-houses-bi\/wp-json\/wp\/v2\/users\/11"}],"replies":[{"embeddable":true,"href":"https:\/\/vargas-solar.com\/data-ware-lake-houses-bi\/wp-json\/wp\/v2\/comments?post=61"}],"version-history":[{"count":2,"href":"https:\/\/vargas-solar.com\/data-ware-lake-houses-bi\/wp-json\/wp\/v2\/pages\/61\/revisions"}],"predecessor-version":[{"id":69,"href":"https:\/\/vargas-solar.com\/data-ware-lake-houses-bi\/wp-json\/wp\/v2\/pages\/61\/revisions\/69"}],"wp:attachment":[{"href":"https:\/\/vargas-solar.com\/data-ware-lake-houses-bi\/wp-json\/wp\/v2\/media?parent=61"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}