{"id":3195,"date":"2021-06-21T13:56:46","date_gmt":"2021-06-21T06:56:46","guid":{"rendered":"http:\/\/gcloudvn.wam.vn\/?p=3195"},"modified":"2023-03-31T16:31:18","modified_gmt":"2023-03-31T09:31:18","slug":"5-cach-dieu-chinh-sieu-thong-so-vertex-vizier-cai-thien-mo-hinh-ml","status":"publish","type":"post","link":"https:\/\/gcloudvn.com\/en\/kienthuc\/5-cach-dieu-chinh-sieu-thong-so-vertex-vizier-cai-thien-mo-hinh-ml\/","title":{"rendered":"5 ways Vertex Vizier hyperparameter tuning improves ML . models"},"content":{"rendered":"<p style=\"text-align: justify;\">Recently, <a href=\"https:\/\/gcloudvn.com\/en\/google-cloud-platform\/\">Google Cloud Platform<\/a> debuted <a href=\"https:\/\/cloud.google.com\/vertex-ai\" target=\"_blank\" rel=\"nofollow noopener\">Vertex AI<\/a>\u00a0to help you move machine learning (ML) from test to production faster and manage your models with confidence \u2014 accelerating your ability to improve outcomes at your organization.<\/p>\n<p style=\"text-align: justify;\">But they know many of you are just getting started with ML and have a lot to learn! In parallel with building the Vertex AI platform, teams are reducing\u00a0<a href=\"https:\/\/cloud.google.com\/architecture\/ml-on-gcp-best-practices\" target=\"_blank\" rel=\"nofollow noopener\">best practices content<\/a>\u00a0maybe to help you get up to speed. Also, Google has a dedicated event on June 10th,\u00a0<a href=\"https:\/\/cloudonair.withgoogle.com\/events\/summit-ml-practitioners\" target=\"_blank\" rel=\"nofollow noopener\">ML Summit App<\/a>, with sessions on how to apply ML technology in your projects, as well as develop your skills in the field.<\/p>\n<p style=\"text-align: justify;\"><img loading=\"lazy\" decoding=\"async\" class=\"lazy loaded aligncenter wp-image-19226 size-large\" src=\"https:\/\/gcloudvn.com\/wp-content\/uploads\/2021\/06\/News-5-1024x427.jpg?x11264\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" srcset=\"https:\/\/gcloudvn.com\/wp-content\/uploads\/2021\/06\/News-5-1024x427.jpg 1024w, https:\/\/gcloudvn.com\/wp-content\/uploads\/2021\/06\/News-5-300x125.jpg 300w, https:\/\/gcloudvn.com\/wp-content\/uploads\/2021\/06\/News-5-768x321.jpg 768w, https:\/\/gcloudvn.com\/wp-content\/uploads\/2021\/06\/News-5-1536x641.jpg 1536w, https:\/\/gcloudvn.com\/wp-content\/uploads\/2021\/06\/News-5-2048x855.jpg 2048w\" alt=\"5 ways to tune the Vertex Vizier hyperparameters improve the 1 . ML model\" width=\"1024\" height=\"427\" data-src=\"https:\/\/gcloudvn.com\/wp-content\/uploads\/2021\/06\/News-5-1024x427.jpg?x11264\" data-srcset=\"https:\/\/gcloudvn.com\/wp-content\/uploads\/2021\/06\/News-5-1024x427.jpg 1024w, https:\/\/gcloudvn.com\/wp-content\/uploads\/2021\/06\/News-5-300x125.jpg 300w, https:\/\/gcloudvn.com\/wp-content\/uploads\/2021\/06\/News-5-768x321.jpg 768w, https:\/\/gcloudvn.com\/wp-content\/uploads\/2021\/06\/News-5-1536x641.jpg 1536w, https:\/\/gcloudvn.com\/wp-content\/uploads\/2021\/06\/News-5-2048x855.jpg 2048w\" data-sizes=\"(max-width: 1024px) 100vw, 1024px\" data-was-processed=\"true\" \/><\/p>\n<p style=\"text-align: justify;\">In the meantime, Google itself couldn&#039;t resist a quick lesson on hyperparameter tuning, because:<\/p>\n<p style=\"text-align: justify;\"><b>(a)<\/b>\u00a0it&#039;s awesome<\/p>\n<p style=\"text-align: justify;\"><b>(b)<\/b>\u00a0You will impress your colleagues<\/p>\n<p style=\"text-align: justify;\"><b>(c)<\/b>\u00a0Google Cloud has a number of unique technologies that have been tested in the field and<\/p>\n<p style=\"text-align: justify;\"><b>(d)<\/b>\u00a0you&#039;ll save time by getting better ML models into production faster.<\/p>\n<p style=\"text-align: justify;\">On average, Vertex Vizier finds optimal parameters for complex functions in 80% fewer tests than traditional methods.<\/p>\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_80 counter-hierarchy ez-toc-counter ez-toc-grey ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of contents<\/p>\n<span class=\"ez-toc-title-toggle\"><a href=\"#\" class=\"ez-toc-pull-right ez-toc-btn ez-toc-btn-xs ez-toc-btn-default ez-toc-toggle\" aria-label=\"Toggle Table of Content\"><span class=\"ez-toc-js-icon-con\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewbox=\"0 0 24 24\" version=\"1.2\" baseprofile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/gcloudvn.com\/en\/kienthuc\/5-cach-dieu-chinh-sieu-thong-so-vertex-vizier-cai-thien-mo-hinh-ml\/#Vi_vay_no_vo_cung_tuyet_voi_nhung_no_la_gi\" >So it&#039;s amazing, but what is it?<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/gcloudvn.com\/en\/kienthuc\/5-cach-dieu-chinh-sieu-thong-so-vertex-vizier-cai-thien-mo-hinh-ml\/#Vertex_Vizier_cho_phep_dieu_chinh_sieu_tham_so_tu_dong_theo_mot_so_cach_Dieu_chinh_sieu_tham_so\" >Vertex Vizier allows automatic hyperparameter tuning in several ways: Hyperparameter tuning<\/a><\/li><\/ul><\/nav><\/div>\n<h2 style=\"text-align: justify;\"><span class=\"ez-toc-section\" id=\"Vi_vay_no_vo_cung_tuyet_voi_nhung_no_la_gi\"><\/span><strong>So it&#039;s amazing, but what is it?<\/strong><span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p style=\"text-align: justify;\">While machine learning models automatically learn from data, they still require user-defined nodes to guide the learning process. For example, these nodes, often referred to as hyperparameters, control the trade-off between training accuracy and generalization. An example of a hyperparameter is the\u00a0<a href=\"https:\/\/towardsdatascience.com\/optimizers-for-training-neural-network-59450d71caf6#:~:text=Optimizers%20are%20algorithms%20or%20methods,help%20to%20get%20results%20faster\" target=\"_blank\" rel=\"nofollow noopener\">optimizer<\/a>\u00a0being used, its\u00a0\u00a0<a href=\"https:\/\/en.wikipedia.org\/wiki\/Learning_rate#:~:text=In%20machine%20learning%20and%20statistics,minimum%20of%20a%20loss%20function.&amp;text=In%20the%20adaptive%20control%20literature,commonly%20referred%20to%20as%20gain.\" target=\"_blank\" rel=\"nofollow noopener\">learning rate<\/a>,\u00a0<a href=\"https:\/\/towardsdatascience.com\/regularization-an-important-concept-in-machine-learning-5891628907ea\" target=\"_blank\" rel=\"nofollow noopener\">regularization parameters<\/a>, number of hidden layers in\u00a0<a href=\"https:\/\/en.wikipedia.org\/wiki\/Deep_learning#Deep_neural_networks\" target=\"_blank\" rel=\"nofollow noopener\">DN<\/a>\u00a0and their size.<\/p>\n<p style=\"text-align: justify;\">Setting hyperparameters to their optimal values for a given data set can make a big difference in model quality. Usually, the optimal hyperparameter values are found through grid search with a small number of combinations or tedious manual testing.\u00a0<a href=\"https:\/\/en.wikipedia.org\/wiki\/Hyperparameter_optimization\" target=\"_blank\" rel=\"nofollow noopener\">Hyperparameter tuning<\/a>\u00a0automate this work for you by searching for the best configuration of hyperparameters for optimal model performance.<\/p>\n<p style=\"text-align: justify;\"><img loading=\"lazy\" decoding=\"async\" class=\"lazy loaded aligncenter wp-image-19232 size-large\" src=\"https:\/\/gcloudvn.com\/wp-content\/uploads\/2021\/06\/News-6-1024x510.jpg?x11264\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" srcset=\"https:\/\/gcloudvn.com\/wp-content\/uploads\/2021\/06\/News-6-1024x510.jpg 1024w, https:\/\/gcloudvn.com\/wp-content\/uploads\/2021\/06\/News-6-300x150.jpg 300w, https:\/\/gcloudvn.com\/wp-content\/uploads\/2021\/06\/News-6-768x383.jpg 768w, https:\/\/gcloudvn.com\/wp-content\/uploads\/2021\/06\/News-6-1536x765.jpg 1536w, https:\/\/gcloudvn.com\/wp-content\/uploads\/2021\/06\/News-6.jpg 1814w\" alt=\"5 ways Vertex Vizier hyperparameter tuning improves 2 . ML models\" width=\"1024\" height=\"510\" data-src=\"https:\/\/gcloudvn.com\/wp-content\/uploads\/2021\/06\/News-6-1024x510.jpg?x11264\" data-srcset=\"https:\/\/gcloudvn.com\/wp-content\/uploads\/2021\/06\/News-6-1024x510.jpg 1024w, https:\/\/gcloudvn.com\/wp-content\/uploads\/2021\/06\/News-6-300x150.jpg 300w, https:\/\/gcloudvn.com\/wp-content\/uploads\/2021\/06\/News-6-768x383.jpg 768w, https:\/\/gcloudvn.com\/wp-content\/uploads\/2021\/06\/News-6-1536x765.jpg 1536w, https:\/\/gcloudvn.com\/wp-content\/uploads\/2021\/06\/News-6.jpg 1814w\" data-sizes=\"(max-width: 1024px) 100vw, 1024px\" data-was-processed=\"true\" \/><\/p>\n<h2 style=\"text-align: justify;\"><span class=\"ez-toc-section\" id=\"Vertex_Vizier_cho_phep_dieu_chinh_sieu_tham_so_tu_dong_theo_mot_so_cach_Dieu_chinh_sieu_tham_so\"><\/span><strong>Vertex Vizier allows automatic hyperparameter tuning in several ways: Hyperparameter tuning<\/strong><span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p style=\"text-align: justify;\"><strong>1.<\/strong>\u00a0\u201ctraditional\u201d: by the way, it means finding the optimal value of the hyperparameter by measuring\u00a0<b>a single target metric<\/b>\u00a0is the output of the ML model. For example, Vizier chooses the number of hidden layers and their size, the optimizer, and its learning rate, with the goal of maximizing model accuracy.<\/p>\n<p style=\"text-align: justify;\"><strong>2.<\/strong>\u00a0Once the hyperparameters are evaluated, the models are trained and evaluated on parts of the data set. If the evaluation metrics are streamed to Vizier (e.g. as a function of the epoch) when the model is trained, Vizier&#039;s\u00a0<b>early stopping<\/b>\u00a0\u00a0the algorithm can predict the final target value and recommend premature stopping of unsatisfactory trials. This conserves computing resources and speeds up convergence.<\/p>\n<p style=\"text-align: justify;\"><strong>3.<\/strong>\u00a0Usually, the models are adjusted sequentially over different data sets. Vizier&#039;s built-in features\u00a0<b>transfer learning<\/b>\u00a0key points from previous hyperparameter tuning studies and utilize them for faster convergence into subsequent hyperparameter tuning studies.<\/p>\n<p style=\"text-align: justify;\"><strong>4.<\/strong>\u00a0<a href=\"https:\/\/cloud.google.com\/automl\" target=\"_blank\" rel=\"nofollow noopener\">AutoML<\/a>\u00a0is a variant of # 1, where Vertex Vizier does both\u00a0<b>model selection\u00a0<\/b>and also tune the architecture\/non-architecture modifier hyperparameters. AutoML usually requires more code than on Vertex Vizier (for data entry, etc.), but in most cases Vizier is the \u201cengine\u201d behind the process. AutoML is implemented by defining a tree format like (<a href=\"https:\/\/en.wikipedia.org\/wiki\/Directed_acyclic_graph\" target=\"_blank\" rel=\"nofollow noopener\">DAG<\/a>\u00a0search space), instead of a \u201cflat\u201d search space (as in # 1). Note that you can use the DAG search space for any other purpose where searching on the hierarchical space makes sense.<\/p>\n<p style=\"text-align: justify;\"><strong>5.<\/strong>\u00a0Sometimes you may want to optimize for more than one metric. For example, Google Cloud wants to optimize model accuracy while minimizing model latency. Vizier can\u00a0<b>find\u00a0<\/b><b>\u00a0<\/b><a href=\"https:\/\/en.wikipedia.org\/wiki\/Multi-objective_optimization\" target=\"_blank\" rel=\"nofollow noopener\"><b>Pareto frontier<\/b><\/a>, which offers a balance for many metrics, allowing the user to choose the appropriate balance. Simple example: I want to create a more accurate model, but want to minimize the delivery delay. I don&#039;t know in advance what the balance between the two metrics is. Vizier can be used to explore and draw equilibrium curves, so the user can choose the most appropriate one. For example, \u201ca delay of 200 milliseconds will only reduce accuracy by 0.5%\u201d<\/p>\n<p style=\"text-align: right;\"><strong>Source: Gimasys<\/strong><\/p>","protected":false},"excerpt":{"rendered":"<p>G\u1ea7n \u0111\u00e2y, Google Cloud Platform \u0111\u00e3 ra m\u1eaft Vertex AI\u00a0\u0111\u1ec3 gi\u00fap b\u1ea1n chuy\u1ec3n c\u00f4ng ngh\u1ec7 m\u00e1y h\u1ecdc (ML) t\u1eeb th\u1eed nghi\u1ec7m sang s\u1ea3n xu\u1ea5t nhanh h\u01a1n v\u00e0 t\u1ef1 tin qu\u1ea3n l\u00fd m\u00f4 h\u00ecnh c\u1ee7a m\u00ecnh \u2014 t\u0103ng t\u1ed1c kh\u1ea3 n\u0103ng&hellip;<\/p>","protected":false},"author":1,"featured_media":3196,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"inline_featured_image":false,"footnotes":""},"categories":[1],"tags":[],"class_list":["post-3195","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-kienthuc","entry","has-media"],"_links":{"self":[{"href":"https:\/\/gcloudvn.com\/en\/wp-json\/wp\/v2\/posts\/3195","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/gcloudvn.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/gcloudvn.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/gcloudvn.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/gcloudvn.com\/en\/wp-json\/wp\/v2\/comments?post=3195"}],"version-history":[{"count":0,"href":"https:\/\/gcloudvn.com\/en\/wp-json\/wp\/v2\/posts\/3195\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/gcloudvn.com\/en\/wp-json\/wp\/v2\/media\/3196"}],"wp:attachment":[{"href":"https:\/\/gcloudvn.com\/en\/wp-json\/wp\/v2\/media?parent=3195"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/gcloudvn.com\/en\/wp-json\/wp\/v2\/categories?post=3195"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/gcloudvn.com\/en\/wp-json\/wp\/v2\/tags?post=3195"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}