{"id":6401,"date":"2020-05-15T11:45:52","date_gmt":"2020-05-15T04:45:52","guid":{"rendered":"http:\/\/gcloudvn.wam.vn\/google-cloud-ho-tro-de-dang-truy-cap-den-phan-tich-luong-voi-sql-ai-thoi-gian-thuc-va-hon-the-nua\/"},"modified":"2023-09-13T16:07:22","modified_gmt":"2023-09-13T09:07:22","slug":"google-cloud-ho-tro-de-dang-truy-cap-den-phan-tich-luong-voi-sql-ai-thoi-gian-thuc-va-hon-the-nua","status":"publish","type":"post","link":"https:\/\/gcloudvn.com\/en\/kienthuc\/google-cloud-ho-tro-de-dang-truy-cap-den-phan-tich-luong-voi-sql-ai-thoi-gian-thuc-va-hon-the-nua\/","title":{"rendered":"Google Cloud provides easy access to flow analytics with SQL, real-time AI and more"},"content":{"rendered":"<p style=\"text-align: justify;\"><span style=\"font-weight: 400;\">During challenging and uncertain times, businesses around the world must think creatively and create more value with fewer resources to maintain reliable and efficient systems for their business. customers have needs. In terms of data analytics, it&#039;s important to find ways for startup engineering teams and operations teams to work on unique scenarios to maintain the required level of productivity. Balancing the development of modern, high-value threading pipelines with cost-effective batch workflow maintenance and optimization is an important goal for many teams. Google has implemented new capabilities to make it easy for developers and operations teams to access flow analytics.<\/span><\/p>\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_80 counter-hierarchy ez-toc-counter ez-toc-grey ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of contents<\/p>\n<span class=\"ez-toc-title-toggle\"><a href=\"#\" class=\"ez-toc-pull-right ez-toc-btn ez-toc-btn-xs ez-toc-btn-default ez-toc-toggle\" aria-label=\"Toggle Table of Content\"><span class=\"ez-toc-js-icon-con\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewbox=\"0 0 24 24\" version=\"1.2\" baseprofile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/gcloudvn.com\/en\/kienthuc\/google-cloud-ho-tro-de-dang-truy-cap-den-phan-tich-luong-voi-sql-ai-thoi-gian-thuc-va-hon-the-nua\/#Cac_diem_noi_bat_trong_cac_lan_ra_mat_nay_bao_gom\" >Highlights of these launches include:<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/gcloudvn.com\/en\/kienthuc\/google-cloud-ho-tro-de-dang-truy-cap-den-phan-tich-luong-voi-sql-ai-thoi-gian-thuc-va-hon-the-nua\/#Co_nhung_diem_gi_moi_trong_phan_tich_luong\" >What&#039;s New in Stream Analytics?<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/gcloudvn.com\/en\/kienthuc\/google-cloud-ho-tro-de-dang-truy-cap-den-phan-tich-luong-voi-sql-ai-thoi-gian-thuc-va-hon-the-nua\/#Su_ra_mat_cua_Dataflow_SQL\" >The launch of Dataflow SQL<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/gcloudvn.com\/en\/kienthuc\/google-cloud-ho-tro-de-dang-truy-cap-den-phan-tich-luong-voi-sql-ai-thoi-gian-thuc-va-hon-the-nua\/#Phat_trien_duong_ong_lap_trong_Jupyter_notebook\" >Iterative pipeline development in Jupyter notebook<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/gcloudvn.com\/en\/kienthuc\/google-cloud-ho-tro-de-dang-truy-cap-den-phan-tich-luong-voi-sql-ai-thoi-gian-thuc-va-hon-the-nua\/#Chia_se_duong_dan_va_ty_le_voi_cac_mau_flex\" >Share paths and scales with flex templates<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/gcloudvn.com\/en\/kienthuc\/google-cloud-ho-tro-de-dang-truy-cap-den-phan-tich-luong-voi-sql-ai-thoi-gian-thuc-va-hon-the-nua\/#Ra_mat_Pub_Sub_deal_letter_topics\" >Launching Pub \/ Sub deal letter topics<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/gcloudvn.com\/en\/kienthuc\/google-cloud-ho-tro-de-dang-truy-cap-den-phan-tich-luong-voi-sql-ai-thoi-gian-thuc-va-hon-the-nua\/#Toi_uu_hoa_xu_ly_du_lieu_luong_voi_thu_thap_du_lieu_thay_doi_CDC\" >Streamline data processing optimization with variable data collection (CDC)<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/gcloudvn.com\/en\/kienthuc\/google-cloud-ho-tro-de-dang-truy-cap-den-phan-tich-luong-voi-sql-ai-thoi-gian-thuc-va-hon-the-nua\/#Tich_hop_voi_Cloud_Platform_AI\" >Integration with Cloud Platform AI<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-9\" href=\"https:\/\/gcloudvn.com\/en\/kienthuc\/google-cloud-ho-tro-de-dang-truy-cap-den-phan-tich-luong-voi-sql-ai-thoi-gian-thuc-va-hon-the-nua\/#Giam_do_phuc_tap_trong_van_hanh_voi_bang_dieu_khien_quan_sat\" >Reduce operational complexity with an observation panel<\/a><\/li><\/ul><\/li><\/ul><\/nav><\/div>\n<h2 style=\"text-align: justify;\"><span class=\"ez-toc-section\" id=\"Cac_diem_noi_bat_trong_cac_lan_ra_mat_nay_bao_gom\"><\/span><b>Highlights of these launches include:<\/b><span class=\"ez-toc-section-end\"><\/span><\/h2>\n<ul style=\"text-align: justify;\">\n<li><span style=\"font-weight: 400;\"> \u00a0 \u00a0 \u00a0 Flow pipelines are developed directly in the BigQuery web UI with general availability of Dataflow SQL<\/span><\/li>\n<li><span style=\"font-weight: 400;\"> \u00a0 \u00a0 \u00a0 Dataflow integration with AI Platform enables simple development of advanced analytics use cases<\/span><\/li>\n<li><span style=\"font-weight: 400;\"> \u00a0 \u00a0 \u00a0 Enhanced monitoring with dashboard with visibility<\/span><\/li>\n<\/ul>\n<p style=\"text-align: justify;\"><span style=\"font-weight: 400;\">Built on the automation infrastructure of Pub\/Sub, Dataflow and <a href=\"https:\/\/gcloudvn.com\/en\/bigquery\/\">BigQuery<\/a>, Google Cloud&#039;s stream processing platform provides the resources that engineering and operations teams need to ingest, process, and analyze fluctuating volumes of data in real time to gain insights real-time business. We are very honored when <\/span><a href=\"https:\/\/cloud.google.com\/forrester-streaming-analytics\" target=\"_blank\" rel=\"nofollow noopener\"><span style=\"font-weight: 400;\">Forrester Wave\u2122: Streaming Analytics<\/span><\/a><span style=\"font-weight: 400;\">, the Q3 2019 report named Google Cloud as the leader in the space. These launches build on and reinforce the capabilities that drive that recognition.<\/span><\/p>\n<h2 style=\"text-align: justify;\"><span class=\"ez-toc-section\" id=\"Co_nhung_diem_gi_moi_trong_phan_tich_luong\"><\/span><b>What&#039;s New in Stream Analytics?<\/b><span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p style=\"text-align: justify;\"><span style=\"font-weight: 400;\">Development for batch data and stream pipelines is made even easier with major releases on both Dataflow and Pub\/Sub. You can get from idea to path and manage iteratively to meet customer needs efficiently.<\/span><\/p>\n<ul style=\"text-align: justify;\">\n<li style=\"font-weight: 400;\">\n<h3><span class=\"ez-toc-section\" id=\"Su_ra_mat_cua_Dataflow_SQL\"><\/span><span style=\"font-weight: 400;\">The launch of Dataflow SQL<\/span><span style=\"font-weight: 400;\"><br \/>\n<\/span><span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p><span style=\"font-weight: 400;\">Dataflow SQL cho ph\u00e9p c\u00e1c nh\u00e0 ph\u00e2n t\u00edch d\u1eef li\u1ec7u v\u00e0 k\u1ef9 s\u01b0 d\u1eef li\u1ec7u s\u1eed d\u1ee5ng c\u00e1c k\u1ef9 n\u0103ng SQL c\u1ee7a h\u1ecd \u0111\u1ec3 ph\u00e1t tri\u1ec3n c\u00e1c \u0111\u01b0\u1eddng d\u1eabn truy\u1ec1n d\u1eef li\u1ec7u Dataflow ngay tr\u00ean giao di\u1ec7n ng\u01b0\u1eddi d\u00f9ng web BigQuery. C\u00e1c \u0111\u01b0\u1eddng d\u1eabn SQL Dataflow c\u1ee7a b\u1ea1n c\u00f3 quy\u1ec1n truy c\u1eadp \u0111\u1ea7y \u0111\u1ee7 v\u00e0o t\u1ef1 \u0111\u1ed9ng h\u00f3a, c\u1eeda s\u1ed5 d\u1ef1a tr\u00ean th\u1eddi gian, c\u00f4ng c\u1ee5 x\u1eed l\u00fd tr\u1ef1c tuy\u1ebfn v\u00e0 x\u1eed l\u00fd d\u1eef li\u1ec7u song song. B\u1ea1n c\u00f3 th\u1ec3 tham gia truy\u1ec1n d\u1eef li\u1ec7u t\u1eeb Pub \/ Sub v\u1edbi c\u00e1c t\u1ec7p trong Cloud Storage ho\u1eb7c c\u00e1c b\u1ea3ng trong BigQuery, vi\u1ebft k\u1ebft qu\u1ea3 v\u00e0o BigQuery ho\u1eb7c Pub \/ Sub v\u00e0 x\u00e2y d\u1ef1ng b\u1ea3ng \u0111i\u1ec1u khi\u1ec3n th\u1eddi gian th\u1ef1c b\u1eb1ng Google Sheets ho\u1eb7c c\u00e1c c\u00f4ng c\u1ee5 BI kh\u00e1c. Ngo\u00e0i ra, c\u00f2n c\u00f3 m\u1ed9t giao di\u1ec7n d\u00f2ng l\u1ec7nh \u0111\u01b0\u1ee3c th\u00eam v\u00e0o g\u1ea7n \u0111\u00e2y \u0111\u1ec3 vi\u1ebft l\u1ec7nh c\u00e1c c\u00f4ng vi\u1ec7c s\u1ea3n xu\u1ea5t c\u1ee7a b\u1ea1n v\u1edbi s\u1ef1 h\u1ed7 tr\u1ee3 \u0111\u1ea7y \u0111\u1ee7 c\u1ee7a c\u00e1c tham s\u1ed1 truy v\u1ea5n v\u00e0 b\u1ea1n c\u00f3 th\u1ec3 d\u1ef1a v\u00e0o t\u00edch h\u1ee3p danh m\u1ee5c d\u1eef li\u1ec7u v\u00e0 tr\u00ecnh so\u1ea1n th\u1ea3o l\u01b0\u1ee3c \u0111\u1ed3 t\u00edch h\u1ee3p \u0111\u1ec3 qu\u1ea3n l\u00fd l\u01b0\u1ee3c \u0111\u1ed3.<\/span><\/li>\n<\/ul>\n<p style=\"text-align: justify;\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-15934 size-full\" src=\"https:\/\/gcloudvn.com\/wp-content\/uploads\/2020\/05\/news-10.png\" alt=\"Google Cloud provides easy access to flow analytics with SQL, real-time AI and more 1\" width=\"512\" height=\"371\" \/><\/p>\n<ul style=\"text-align: justify;\">\n<li style=\"font-weight: 400;\">\n<h3><span class=\"ez-toc-section\" id=\"Phat_trien_duong_ong_lap_trong_Jupyter_notebook\"><\/span><span style=\"font-weight: 400;\">Iterative pipeline development in Jupyter notebook<\/span><span style=\"font-weight: 400;\"><br \/>\n<\/span><span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p><span style=\"font-weight: 400;\">With notebooks, developers can now iteratively build pipelines from scratch with the Notebook AI platform and deploy with the Dataflow runner. Authoring Apache Beam pipelines step-by-step by examining the path graphs in the read-experience-in-iteration (REPL) workflow. Available through <\/span><span style=\"font-weight: 400;\">Google&#039;s AI Platform<\/span><span style=\"font-weight: 400;\">, Notebook lets you write pipelines in a visual environment with the latest machine learning and data science frameworks so you can develop better customer experiences with ease.\u00a0<\/span><\/li>\n<\/ul>\n<ul style=\"text-align: justify;\">\n<li style=\"font-weight: 400;\">\n<h3><span class=\"ez-toc-section\" id=\"Chia_se_duong_dan_va_ty_le_voi_cac_mau_flex\"><\/span><span style=\"font-weight: 400;\">Share paths and scales with flex templates<\/span><span style=\"font-weight: 400;\"><br \/>\n<\/span><span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p><span style=\"font-weight: 400;\"><a href=\"https:\/\/cloud.google.com\/dataflow\/docs\/guides\/templates\/using-flex-templates\" target=\"_blank\" rel=\"nofollow noopener\">Dataflow templates let you easily share <\/a>your links with team members and across your organization, or take advantage of the many templates provided by Google to perform simple but useful data processing tasks. With flex templates, you can create a template from any Dataflow pipeline.<\/span><\/li>\n<\/ul>\n<ul style=\"text-align: justify;\">\n<li style=\"font-weight: 400;\">\n<h3><span class=\"ez-toc-section\" id=\"Ra_mat_Pub_Sub_deal_letter_topics\"><\/span><span style=\"font-weight: 400;\">Launching Pub \/ Sub deal letter topics<\/span><span style=\"font-weight: 400;\"><br \/>\n<\/span><span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p><span style=\"font-weight: 400;\">Operating reliable flow paths and event-driven systems has become simpler with the general availability of <\/span><a href=\"https:\/\/cloud.google.com\/pubsub\/docs\/dead-letter-topics\" target=\"_blank\" rel=\"nofollow noopener\"><span style=\"font-weight: 400;\">dead letter topics<\/span><\/a><span style=\"font-weight: 400;\"> for Pub\/Sub. A common problem in these systems are \u201cdead letters,\u201d or messages that cannot be processed by the subscriber application. A dead letter topic allows such messages to be set aside for offline testing and debugging so that the rest of the message can be handled without delay.<\/span><\/li>\n<\/ul>\n<ul style=\"text-align: justify;\">\n<li style=\"font-weight: 400;\">\n<h3><span class=\"ez-toc-section\" id=\"Toi_uu_hoa_xu_ly_du_lieu_luong_voi_thu_thap_du_lieu_thay_doi_CDC\"><\/span><span style=\"font-weight: 400;\">Streamline data processing optimization with variable data collection (CDC)<\/span><span style=\"font-weight: 400;\"><br \/>\n<\/span><span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p><span style=\"font-weight: 400;\">One way to optimize processing of stream data is to focus on working with only the changed data instead of all available data. This is where change data collection (CDC) comes in handy. The Dataflow team has developed a sample solution that allows you to import a stream of change data coming from any type of MySQL database on versions 5.6 and later (self-managed, on-premises, etc.) Model it with datasets in BigQuery using Dataflow.<\/span><\/li>\n<\/ul>\n<ul style=\"text-align: justify;\">\n<li style=\"font-weight: 400;\">\n<h3><span class=\"ez-toc-section\" id=\"Tich_hop_voi_Cloud_Platform_AI\"><\/span><span style=\"font-weight: 400;\">Integration with Cloud Platform AI <\/span><span style=\"font-weight: 400;\"><br \/>\n<\/span><span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p><span style=\"font-weight: 400;\">Now you can take advantage of easy integration with <\/span><span style=\"font-weight: 400;\">AI Platform APIs<\/span><span style=\"font-weight: 400;\"> and access to libraries for implementing advanced analytics use cases. AI Platform and Dataflow capabilities include video clip classification, image classification, natural text analysis, data loss prevention, and several other streaming predictive use cases.<\/span><\/li>\n<\/ul>\n<p style=\"text-align: justify;\"><span style=\"font-weight: 400;\">Ease and speed shouldn&#039;t come to those building and starting data pipelines, but neither should those managing and maintaining them. Google has also enhanced the monitoring experience for Dataflow, which aims to empower operational teams even more.<\/span><\/p>\n<ul style=\"text-align: justify;\">\n<li style=\"font-weight: 400;\">\n<h3><span class=\"ez-toc-section\" id=\"Giam_do_phuc_tap_trong_van_hanh_voi_bang_dieu_khien_quan_sat\"><\/span><span style=\"font-weight: 400;\">Reduce operational complexity with an observation panel<\/span><span style=\"font-weight: 400;\"><br \/>\n<\/span><span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p><a href=\"https:\/\/cloud.google.com\/dataflow\/docs\/guides\/using-monitoring-intf#accessing_job_monitoring_charts\" target=\"_blank\" rel=\"nofollow noopener\"><span style=\"font-weight: 400;\">Observation panel<\/span><\/a><span style=\"font-weight: 400;\"> and Dataflow inline monitoring gives you direct access to job metrics to help troubleshoot batch and flow pipelines. You can access monitoring graphs at both step- and worker-level visibility, and set alerts for conditions like stale data and high system latency. Below, look at an example:<\/span><\/li>\n<\/ul>\n<p style=\"text-align: justify;\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-15935 size-full\" src=\"https:\/\/gcloudvn.com\/wp-content\/uploads\/2020\/05\/news-11.png\" alt=\"Google Cloud provides easy access to flow analytics with SQL, real-time AI, and more 2\" width=\"512\" height=\"434\" \/><\/p>\n<p style=\"text-align: justify;\"><span style=\"font-weight: 400;\">Getting started with stream analytics is now easier than ever. The first step to getting started with testing and experimentation is to move some data onto the platform. Check out the Pub\/Sub Quick Start docs to get moving with real-time texting and typing with <a href=\"https:\/\/gcloudvn.com\/en\/google-cloud-platform\/\">Cloud GCP<\/a>.<\/span><\/p>\n<p style=\"text-align: right;\"><strong>Source: Gimasys<\/strong><\/p>","protected":false},"excerpt":{"rendered":"<p>Trong th\u1eddi gian th\u1eed th\u00e1ch v\u00e0 kh\u00f4ng ch\u1eafc ch\u1eafn, c\u00e1c doanh nghi\u1ec7p tr\u00ean to\u00e0n th\u1ebf gi\u1edbi ph\u1ea3i suy ngh\u0129 m\u1ed9t c\u00e1ch s\u00e1ng t\u1ea1o v\u00e0 t\u1ea1o ra nhi\u1ec1u gi\u00e1 tr\u1ecb h\u01a1n khi \u00edt ti\u00eau t\u1ed1n ngu\u1ed3n l\u1ef1c h\u01a1n \u0111\u1ec3 duy tr\u00ec&hellip;<\/p>","protected":false},"author":1,"featured_media":6402,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"inline_featured_image":false,"footnotes":""},"categories":[1,135],"tags":[],"class_list":["post-6401","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-kienthuc","category-google-cloud-platform","entry","has-media"],"_links":{"self":[{"href":"https:\/\/gcloudvn.com\/en\/wp-json\/wp\/v2\/posts\/6401","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/gcloudvn.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/gcloudvn.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/gcloudvn.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/gcloudvn.com\/en\/wp-json\/wp\/v2\/comments?post=6401"}],"version-history":[{"count":0,"href":"https:\/\/gcloudvn.com\/en\/wp-json\/wp\/v2\/posts\/6401\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/gcloudvn.com\/en\/wp-json\/wp\/v2\/media\/6402"}],"wp:attachment":[{"href":"https:\/\/gcloudvn.com\/en\/wp-json\/wp\/v2\/media?parent=6401"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/gcloudvn.com\/en\/wp-json\/wp\/v2\/categories?post=6401"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/gcloudvn.com\/en\/wp-json\/wp\/v2\/tags?post=6401"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}