{"id":9551,"date":"2022-09-30T14:33:16","date_gmt":"2022-09-30T07:33:16","guid":{"rendered":"https:\/\/gcloudvn.com\/?p=9551"},"modified":"2023-07-12T14:32:16","modified_gmt":"2023-07-12T07:32:16","slug":"introducing-datastream-for-bigquery","status":"publish","type":"post","link":"https:\/\/gcloudvn.com\/en\/kienthuc\/introducing-datastream-for-bigquery\/","title":{"rendered":"Introducing Datastream for Google BigQuery"},"content":{"rendered":"<figure id=\"attachment_9543\" aria-describedby=\"caption-attachment-9543\" style=\"width: 600px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-9543\" src=\"https:\/\/gcloudvn.com\/wp-content\/uploads\/2022\/09\/Screenshot_18-1024x404.png\" alt=\"Datastream for Google BigQuery\" width=\"600\" height=\"237\" srcset=\"https:\/\/gcloudvn.com\/wp-content\/uploads\/2022\/09\/Screenshot_18-1024x404.png 1024w, https:\/\/gcloudvn.com\/wp-content\/uploads\/2022\/09\/Screenshot_18-300x118.png 300w, https:\/\/gcloudvn.com\/wp-content\/uploads\/2022\/09\/Screenshot_18-768x303.png 768w, https:\/\/gcloudvn.com\/wp-content\/uploads\/2022\/09\/Screenshot_18-18x7.png 18w, https:\/\/gcloudvn.com\/wp-content\/uploads\/2022\/09\/Screenshot_18.png 1097w\" sizes=\"auto, (max-width: 600px) 100vw, 600px\" \/><figcaption id=\"caption-attachment-9543\" class=\"wp-caption-text\">Datastream for Google BigQuery<\/figcaption><\/figure>\n<p style=\"text-align: justify;\"><span style=\"font-weight: 400;\">In today\u2019s competitive environment, organizations need to quickly and easily make decisions based on real-time data. That\u2019s why we\u2019re announcing <\/span><span style=\"font-weight: 400;\"><strong>Datastream for BigQuery<\/strong>, <\/span><span style=\"font-weight: 400;\">now available in preview, featuring seamless replication from active database sources such as AlloyDB for PostgreSQL, PostgreSQL, MySQL and Oracle, directly into <a href=\"https:\/\/gcloudvn.com\/en\/bigquery\/\">BigQuery<\/a> \u2013 Google Cloud serverless data warehouse. Datastream for BigQuery is Google&#039;s next big step toward realizing our vision of a unified data cloud, bringing together database, analytics, and machine learning into a single platform that delivers at scale. , speed, security and simplicity that modern businesses need. With a serverless autoscaling architecture, Datastream allows you to easily set up an ELT (Extract, Load, Transform) pipeline to replicate data with low latency allowing for insights on real time.\u00a0<\/span><\/p>\n<p style=\"text-align: justify;\"><span style=\"font-weight: 400;\">Consider the case of a large supermarket chain, with 100\u2019s of stores spread out across the region. Each individual store runs its own local point-of-sale and stock management systems, collecting data throughout the day about transactions and stock-levels in the store. To provide visibility and help streamline the chain\u2019s daily operations, the IT department set up a nightly batch process to collect and consolidate all of the data from the stores into a central data warehouse, so that reports on the stores\u2019 performance could be ready for review in the morning. Maintaining this process took time and resources from the data engineering team, and as the chain grows and more data needs to be processed, this process ended up taking so long that reports would only be ready late into the day. Organizations like this are looking for a modern solution that allows effortless replication of operational data to their data warehouse, enabling real-time decision making; Datastream for BigQuery is that solution.<\/span><\/p>\n<p style=\"text-align: justify;\"><strong>&gt; Reference:\u00a0<\/strong><\/p>\n<ul style=\"text-align: justify;\">\n<li><a href=\"https:\/\/gcloudvn.com\/en\/kienthuc\/datastream-la-gi\/\">What is Datastream?<\/a><\/li>\n<li><a href=\"https:\/\/gcloudvn.com\/en\/kienthuc\/toi-uu-bigquery-voi-nguon-du-lieu-trong-google-cloud-vmware-engine\/\">Optimizing BigQuery with data sources in Google Cloud VMware Engine<\/a><\/li>\n<\/ul>\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_80 counter-hierarchy ez-toc-counter ez-toc-grey ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of contents<\/p>\n<span class=\"ez-toc-title-toggle\"><a href=\"#\" class=\"ez-toc-pull-right ez-toc-btn ez-toc-btn-xs ez-toc-btn-default ez-toc-toggle\" aria-label=\"Toggle Table of Content\"><span class=\"ez-toc-js-icon-con\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewbox=\"0 0 24 24\" version=\"1.2\" baseprofile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/gcloudvn.com\/en\/kienthuc\/introducing-datastream-for-bigquery\/#Datastream_tang_toc_qua_trinh_ra_quyet_dinh_theo_huong_du_lieu_trong_BigQuery\" >Datastream accelerates data-driven decision making in BigQuery<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/gcloudvn.com\/en\/kienthuc\/introducing-datastream-for-bigquery\/#Dinh_gia_theo_cap_dua_tren_khoi_luong_moi\" >New volume-based tiered pricing<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/gcloudvn.com\/en\/kienthuc\/introducing-datastream-for-bigquery\/#Bat_dau_su_dung_Datastream_ngay_hom_nay\" >Start using Datastream today<\/a><\/li><\/ul><\/nav><\/div>\n<h2 style=\"text-align: justify;\"><span class=\"ez-toc-section\" id=\"Datastream_tang_toc_qua_trinh_ra_quyet_dinh_theo_huong_du_lieu_trong_BigQuery\"><\/span><b>Datastream accelerates data-driven decision making in BigQuery<\/b><span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p style=\"text-align: justify;\"><span style=\"font-weight: 400;\">Developed in close cooperation with the team <\/span>BigQuery by Google Cloud, Datastream for BigQuery offers a unique, truly seamless and easy-to-use experience that enables real-time insights in BigQuery in just a few steps.<\/p>\n<p style=\"text-align: justify;\"><span style=\"font-weight: 400;\">Using BigQuery\u2019s newly developed Change Data Capture (CDC) and Storage Write API\u2019s UPSERT functionality, Datastream efficiently replicates updates directly from source systems into BigQuery tables in real-time. You no longer have to waste valuable resources building and managing complex data pipelines, self-managed staging tables, tricky DML merge logic, or manual conversion from database-specific data types into BigQuery data types. Just configure your source database, connection type, and destination in BigQuery and you\u2019re all set. Datastream for BigQuery will backfill historical data and continuously replicate new changes as they happen. And as database schemas shift, Datastream seamlessly handles schema changes and automatically adds new tables and columns to BigQuery.\u00a0<\/span><\/p>\n<h2 style=\"text-align: justify;\"><span class=\"ez-toc-section\" id=\"Dinh_gia_theo_cap_dua_tren_khoi_luong_moi\"><\/span><b>New volume-based tiered pricing<\/b><span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p style=\"text-align: justify;\"><span style=\"font-weight: 400;\">We are also excited to announce the launch of volume-based tiered pricing that makes it more affordable for customers moving larger volumes of data. Volume-based tiered pricing will be applied automatically based on actual usage to unlock the power of Datastream.\u00a0<\/span><\/p>\n<p style=\"text-align: justify;\"><span style=\"font-weight: 400;\">Klook<\/span><span style=\"font-weight: 400;\">a leading travel and leisure e-commerce platform for experiences and services, processes vast amounts of data across a range of applications and databases. Using BigQuery, Klook\u2019s data team produces daily reports and analysis for their management team to help drive better business decisions. \u201cDealing with complex data environments and ingesting data from different sources into our data warehouse is very challenging\u201d, says Stacy Zhu, Senior Manager for Data at Klook. \u201cPrior to adopting Datastream, we had a team of data engineers dedicated to the task of ingesting data into BigQuery, and we spent a lot of time and effort making sure that the data was accurate. With Datastream, our data analysts can have accurate data readily available to them in BigQuery with a simple click. We enjoy Datastream's ease-of-use, and its performance helps us achieve large scale ELT data processing.\u201d<\/span><\/p>\n<figure id=\"attachment_9539\" aria-describedby=\"caption-attachment-9539\" style=\"width: 600px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-9539\" src=\"https:\/\/gcloudvn.com\/wp-content\/uploads\/2022\/09\/Datastream.max-1400x1400-1-1024x471.png\" alt=\"Volume based tiered pricing\" width=\"600\" height=\"276\" srcset=\"https:\/\/gcloudvn.com\/wp-content\/uploads\/2022\/09\/Datastream.max-1400x1400-1-1024x471.png 1024w, https:\/\/gcloudvn.com\/wp-content\/uploads\/2022\/09\/Datastream.max-1400x1400-1-300x138.png 300w, https:\/\/gcloudvn.com\/wp-content\/uploads\/2022\/09\/Datastream.max-1400x1400-1-768x353.png 768w, https:\/\/gcloudvn.com\/wp-content\/uploads\/2022\/09\/Datastream.max-1400x1400-1-18x8.png 18w, https:\/\/gcloudvn.com\/wp-content\/uploads\/2022\/09\/Datastream.max-1400x1400-1.png 1400w\" sizes=\"auto, (max-width: 600px) 100vw, 600px\" \/><figcaption id=\"caption-attachment-9539\" class=\"wp-caption-text\">Volume based tiered pricing<\/figcaption><\/figure>\n<p style=\"text-align: justify;\"><span style=\"font-weight: 400;\">Achievers, an award-winning employee engagement software and platform, is another customer who recently used Datastream. \u201cThe achievers used a lot of Google Cloud VM (GCE) and <a href=\"https:\/\/gcloudvn.com\/en\/google-kubernetes-engine-gke\/\">Google Kubernetes Engine<\/a> (GKE)\u201d. \u201cWith the help of Datastream, Achievers will stream our data into BigQuery and enable our analysts and data scientists to start using BigQuery for intelligent analytics, helping us take data warehousing to the next level.\u201d<\/span><\/p>\n<h2 style=\"text-align: justify;\"><span class=\"ez-toc-section\" id=\"Bat_dau_su_dung_Datastream_ngay_hom_nay\"><\/span><b>Start using Datastream today<\/b><span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p style=\"text-align: justify;\"><span style=\"font-weight: 400;\">You can get started today with Datastream, available to all customers in all regions of Google Cloud. For more information about <strong>Datastream for BigQuery<\/strong>, please see it <\/span><a href=\"http:\/\/cloud.google.com\/datastream-for-bigquery\" rel=\"nofollow noopener\" target=\"_blank\"><span style=\"font-weight: 400;\">product page.<\/span><\/a><span style=\"font-weight: 400;\">.<\/span><\/p>\n<p style=\"text-align: justify;\">If your business is interested in the <a href=\"https:\/\/gcloudvn.com\/en\/google-cloud-platform\/\">Cloud Platform Google<\/a>\u00a0Platform then you can connect to Gimasys - Google Premier Partner - for consulting solutions according to the unique needs of your business. Contact now:<\/p>\n<ul style=\"text-align: justify;\">\n<li aria-level=\"1\"><b>Gimasys \u2013 Google Cloud Premier Partner<\/b><\/li>\n<li aria-level=\"1\"><b>Hotline:\u00a0<\/b>Ha Noi:\u00a00987 682 505\u00a0\u2013 Ho Chi Minh:\u00a00974 417 099<\/li>\n<li aria-level=\"1\"><b>Email:\u00a0<\/b>gcp@gimasys.com<\/li>\n<\/ul>\n<p style=\"text-align: right;\"><em><strong>Source: <a href=\"https:\/\/gcloudvn.com\/en\/\">gcloudvn.com<\/a><\/strong><\/em><\/p>","protected":false},"excerpt":{"rendered":"<p>In today&#039;s competitive environment, organizations need to quickly and easily make decisions based on real-time data. That&#039;s why Google announced Datastream for BigQuery, now\u2026<\/p>","protected":false},"author":2,"featured_media":9545,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"inline_featured_image":false,"footnotes":""},"categories":[1,135],"tags":[],"class_list":["post-9551","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-kienthuc","category-google-cloud-platform","entry","has-media"],"_links":{"self":[{"href":"https:\/\/gcloudvn.com\/en\/wp-json\/wp\/v2\/posts\/9551","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/gcloudvn.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/gcloudvn.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/gcloudvn.com\/en\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/gcloudvn.com\/en\/wp-json\/wp\/v2\/comments?post=9551"}],"version-history":[{"count":0,"href":"https:\/\/gcloudvn.com\/en\/wp-json\/wp\/v2\/posts\/9551\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/gcloudvn.com\/en\/wp-json\/wp\/v2\/media\/9545"}],"wp:attachment":[{"href":"https:\/\/gcloudvn.com\/en\/wp-json\/wp\/v2\/media?parent=9551"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/gcloudvn.com\/en\/wp-json\/wp\/v2\/categories?post=9551"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/gcloudvn.com\/en\/wp-json\/wp\/v2\/tags?post=9551"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}