What is Cloud Pub/Sub? Compare Cloud Pub/sub and Kafka

In order for the business operation process to be smooth and uninterrupted, application software will regularly send information (messages) to each other, however, during the process of sharing information, there may be situations where it is not received. receive full information from the sender, to resolve this issue Cloud Pub/Sub was born, so what is Cloud pub/sub? Let's find out details through the article below.  

What is cloud pub/sub?

Cloud Pub/Sub is an asynchronous and scalable Messaging service. Pub/Sub can be imagined as a transfer station between the sender of information called Publisher and the receiving party Subscriber.

Pub/Sub allows you to create a system of exchange events between publishers and subscribers. The Publisher will communicate asynchronously with the Subscriber by sending events with a latency of about 100 million seconds instead of synchronous remote procedure calls (RPC).

Publisher send events to the Pub/Sub service portal without regard to how and when these events will be handled. Pub/Sub will then distribute the events to all services that need to receive the information. Asynchronous integration in Pub/Sub increases the flexibility and robustness of the entire system.

How Cloud Pub/Sub works

Some important concepts in Cloud Pub/Sub

• Publisher: create messages and send (publish) them to Messaging service according to a specific topic.

• Message: data moving through the service.
• Topic: entity representing a message feed.
• Schema: entity that governs the data format of Pub/Sub messages.
• Subscription: entity expressing interest in receiving messages about a specific topic.
• Subscriber : receive messages on a specified subscriber.

The model illustrates how cloud Pub/Sub works as follows:

• Two Publisher send two different messages to a common Pub/Sub topic.

• Lifecycle of a message in Pub/Sub: Suppose a publisher client is connected to a specific topic. Topic has an attached subscription and a subscriber connected to the subscription.

• Select the Publish template and Subscribe Pub/Sub

When there are multiple Publisher and Subscriber customers, it is necessary to choose the type of Publish and Subscribe architecture you want to set up.

How does Cloud Pub/Sub apply?

• Collect user and server interaction events: To use user interaction events collected from your application, you can forward them to the Pub/Sub service, which in turn uses a stream processing such as ., this tool will support transferring collected events to the database, Pub/Sub allows you to collect events from multiple customers at the same time. Examples of databases include BigQuery, Cloud Bigtable, and Cloud Storage. 
• Real-time event distribution: Events can be distributed to multiple applications within the organization for real-time processing. 

• Copy data between databases. Pub/Sub is commonly used to distribute change events from the database. Administrators can use these events to build historical and database status views in BigQuery and other data storage systems.

• Parallel processing and workflows. You can efficiently distribute multiple tasks among multiple workers by using Pub/Sub notifications to connect to Cloud Functions. Examples of such tasks are compressing text files, sending email notifications, evaluating models, and reformatting images.

• Transmit data from applications, services, or IoT devices. For example, sensors can stream data to Pub/Sub for use in other Google Cloud products via a Dataflow pipeline.

• Refresh distributed cachesi. For example, an application can publish invalidation events to update the IDs of objects that have changed.

• Ensure load balancing (Lead balancing). 

How to install Cloud pub/sub

B1. Create or select a Google Cloud project where you can set up Pub/Sub.

B2. Enable Pub/Sub API.

B3. Get the roles and permissions needed to run Pub/Sub.

B4. Create a topic (Topic)

B5. Determine the schema for your messages.

B6. Attach schema to topic.

B7. Configure the publisher client to be able to publish messages to the topic.

B8. If required, configure advanced publishing options such as flow control, bulk messaging, and concurrency control.

B9. Choose your subscription type based on how you want to receive your messages.

B10. Create a subscription for the selected topic.

B11. Configure the subscriber client to be able to receive messages from the subscription.

B12. If required, configure advanced messaging options such as exactly once delivery, lease management, on-demand delivery, and flow control.

B13. Start publishing messages from the publisher client to the topic.

B14. At the same time, set up a subscription client to receive and process these messages.

Compare cloud pub/sub vs kafka

Kafka: 

• Kafka is an open-source platform for building real-time streaming and data pipeline applications, designed to handle large data streams with the ability to handle and handle large data streams. Error handling with low latency. Kafka is widely used in industries such as finance, e-commerce, and social media
• Kafka provides official client libraries for Java, Python, and Go. Kafka is highly scalable but requires manual intervention.
• Kafka is open source with no licensing fees but requires significant infrastructure installation and maintenance.

Cloud Pub/Sub: 

• GCP Pub/Sub is a fully managed messaging service powered by Google Cloud Platform that enables message exchange between applications with persistent message archiving, real-time delivery, and auto-opening wide scale. Cloud Pub/Sub has the ability to integrate with other GCP services such as Dataflow, Cloud Functions, and BigQuery. 
• Pub/Sub has client libraries for Java, Python, Node.js, and C#. Cloud Pub/Sub has the ability to scale automatically without manual intervention.
• Pub/Sub is a paid service with costs that will vary based on actual usage.

Above is a summary of all information about Cloud Pub/Sub – messaging service from Google. With powerful fast processing advantages to ensure the process of sending information between business applications combined with extensive integration capabilities with other Google Cloud solutions, Cloud Pub/Sub is the optimal choice for businesses. during the transition to using Google Cloud solutions.