{"id":20175,"date":"2024-09-26T10:21:48","date_gmt":"2024-09-26T03:21:48","guid":{"rendered":"https:\/\/gcloudvn.com\/?p=20175"},"modified":"2024-10-01T09:44:09","modified_gmt":"2024-10-01T02:44:09","slug":"accelerate-ray-in-production-with-new-ray-operator-on-gke","status":"publish","type":"post","link":"https:\/\/gcloudvn.com\/en\/kienthuc\/accelerate-ray-in-production-with-new-ray-operator-on-gke\/","title":{"rendered":"Accelerate Ray in production with new Ray Operator on GKE"},"content":{"rendered":"

L\u0129nh v\u1ef1c AI li\u00ean t\u1ee5c ph\u00e1t tri\u1ec3n,<\/b> D\u1eb7c bi\u1ec7t v\u1edbi nh\u1eefng ti\u1ebfn b\u1ed9 g\u1ea7n \u0111\u00e2y trong tr\u00ed tu\u1ec7 nh\u00e2n t\u1ea1o sinh (generative AI), c\u00e1c m\u00f4 h\u00ecnh ng\u00e0y c\u00e0ng l\u1edbn v\u00e0 ph\u1ee9c t\u1ea1p h\u01a1n, bu\u1ed9c c\u00e1c t\u1ed5 ch\u1ee9c ph\u1ea3i ph\u00e2n ph\u1ed1i hi\u1ec7u qu\u1ea3 c\u00e1c t\u00e1c v\u1ee5 tr\u00ean nhi\u1ec1u m\u00e1y h\u01a1n. M\u1ed9t c\u00e1ch ti\u1ebfp c\u1eadn m\u1ea1nh m\u1ebd l\u00e0 ch\u1ea1y ray.io, m\u1ed9t khung ph\u00e2n t\u00e1n m\u00e3 ngu\u1ed3n m\u1edf cho kh\u1ed1i l\u01b0\u1ee3ng c\u00f4ng vi\u1ec7c AI\/ML ph\u00e2n t\u00e1n, tr\u00ean Google Kubernetes Engine (GKE), d\u1ecbch v\u1ee5 orchestration container \u0111\u01b0\u1ee3c qu\u1ea3n l\u00fd c\u1ee7a Google Cloud. \u0110\u1ec3 bi\u1ebfn m\u00f4 h\u00ecnh n\u00e0y tr\u1edf n\u00ean si\u00eau d\u1ec5 d\u00e0ng \u0111\u1ec3 tri\u1ec3n khai, gi\u1edd \u0111\u00e2y b\u1ea1n c\u00f3 th\u1ec3 b\u1eadt c\u00e1c API khai b\u00e1o \u0111\u1ec3 qu\u1ea3n l\u00fd c\u00e1c c\u1ee5m Ray tr\u00ean GKE ch\u1ec9 v\u1edbi m\u1ed9t t\u00f9y ch\u1ecdn c\u1ea5u h\u00ecnh!<\/span><\/p>\n

Ray cung c\u1ea5p m\u1ed9t API \u0111\u01a1n gi\u1ea3n \u0111\u1ec3 ph\u00e2n ph\u1ed1i v\u00e0 song song h\u00f3a c\u00e1c t\u00e1c v\u1ee5 h\u1ecdc m\u00e1y m\u1ed9t c\u00e1ch li\u1ec1n m\u1ea1ch, trong khi GKE cung c\u1ea5p m\u1ed9t n\u1ec1n t\u1ea3ng c\u01a1 s\u1edf h\u1ea1 t\u1ea7ng c\u00f3 th\u1ec3 m\u1edf r\u1ed9ng v\u00e0 linh ho\u1ea1t gi\u00fap \u0111\u01a1n gi\u1ea3n h\u00f3a qu\u1ea3n l\u00fd \u1ee9ng d\u1ee5ng v\u00e0 c\u1ea3i thi\u1ec7n vi\u1ec7c s\u1eed d\u1ee5ng t\u00e0i nguy\u00ean. C\u00f9ng nhau, GKE v\u00e0 Ray cung c\u1ea5p kh\u1ea3 n\u0103ng m\u1edf r\u1ed9ng, ch\u1ecbu l\u1ed7i v\u00e0 d\u1ec5 s\u1eed d\u1ee5ng \u0111\u1ec3 x\u00e2y d\u1ef1ng, tri\u1ec3n khai v\u00e0 qu\u1ea3n l\u00fd c\u00e1c \u1ee9ng d\u1ee5ng Ray. H\u01a1n n\u1eefa, Ray Operator t\u00edch h\u1ee3p s\u1eb5n tr\u00ean GKE gi\u00fap \u0111\u01a1n gi\u1ea3n h\u00f3a thi\u1ebft l\u1eadp ban \u0111\u1ea7u v\u00e0 h\u01b0\u1edbng d\u1eabn ng\u01b0\u1eddi d\u00f9ng t\u1edbi c\u00e1c th\u1ef1c ti\u1ec5n t\u1ed1t nh\u1ea5t \u0111\u1ec3 ch\u1ea1y Ray trong m\u00f4i tr\u01b0\u1eddng s\u1ea3n xu\u1ea5t. N\u00f3 \u0111\u01b0\u1ee3c x\u00e2y d\u1ef1ng v\u1edbi c\u00e1c ho\u1ea1t \u0111\u1ed9ng ng\u00e0y th\u1ee9 2, v\u1edbi h\u1ed7 tr\u1ee3 t\u00edch h\u1ee3p cho Cloud Logging v\u00e0 Cloud Monitoring \u0111\u1ec3 n\u00e2ng cao kh\u1ea3 n\u0103ng quan s\u00e1t c\u00e1c \u1ee9ng d\u1ee5ng Ray c\u1ee7a b\u1ea1n tr\u00ean GKE.<\/span><\/p>\n

\"Ray Operator on GKE has transformed our workflow. We've slashed maintenance time and now spin up Ray clusters in 30 minutes \u2013 a task that used to take days. It's a game-changer.\" - Mengliao (Mike) Wang, Geotab<\/span><\/p>\n

Getting started<\/b><\/p>\n

In the Google Cloud console, when creating a new GKE Cluster, select the feature checkbox to \u2018Enable Ray Operator.\u2019 With a GKE Autopilot Cluster, this can be found in \u2018Advanced Settings\u2019 under \u2018AI and Machine Learning.\u2019<\/span><\/p>\n

\"\"With a Standard Cluster, you can find the Enable Ray Operator feature checkbox in the \u2018Features\u2019 Menu under \u2018AI and Machine Learning.\u2019<\/span><\/p>\n

To use the gcloud CLI, you can set an addons flag as following:<\/span><\/p>\n

gcloud container clusters create CLUSTER_NAME \\<\/span><\/p>\n

— cluster-version=VERSION \\<\/span><\/p>\n

— addons=RayOperator<\/span><\/p>\n

\u0110\u1ec3 s\u1eed d\u1ee5ng Terraform, b\u1ea1n c\u00f3 th\u1ec3 k\u00edch ho\u1ea1t addon nh\u01b0 sau:<\/span><\/p>\n

resource “google_container_cluster” “ray-cluster” {<\/span><\/p>\n

\u00a0name \u00a0 \u00a0 = “gke-standard-regional-ray-operator”<\/span><\/p>\n

\u00a0location = “us-west1”<\/span><\/p>\n

\u200b<\/span><\/p>\n

\u00a0initial_node_count = 1<\/span><\/p>\n

\u200b<\/span><\/p>\n

\u00a0release_channel {<\/span><\/p>\n

\u00a0\u00a0\u00a0channel = “RAPID”<\/span><\/p>\n

\u00a0}<\/span><\/p>\n

\u200b<\/span><\/p>\n

\u00a0addons_config {<\/span><\/p>\n

\u00a0\u00a0\u00a0ray_operator_config {<\/span><\/p>\n

\u00a0\u00a0\u00a0\u00a0\u00a0enabled = true<\/span><\/p>\n

\u00a0\u00a0\u00a0\u00a0\u00a0ray_cluster_logging_config {<\/span><\/p>\n

\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0enabled = true<\/span><\/p>\n

\u00a0\u00a0\u00a0\u00a0\u00a0}<\/span><\/p>\n

\u00a0\u00a0\u00a0\u00a0\u00a0ray_cluster_monitoring_config {<\/span><\/p>\n

\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0enabled = true<\/span><\/p>\n

\u00a0\u00a0\u00a0\u00a0\u00a0}<\/span><\/p>\n

\u00a0\u00a0\u00a0}<\/span><\/p>\n

\u00a0}<\/span><\/p>\n

}<\/span><\/p>\n

 <\/p>\n

Once enabled, GKE hosts and manages the Ray Operator on your behalf. Your cluster will be ready to create Ray Clusters and run Ray applications after cluster creation.<\/span>.<\/b><\/p>\n

You can find examples for using Ray to serve large language models in our documentation. Or see how to train a model with Ray and Pytorch.\u00a0<\/span><\/p>\n

\n
\n

Table of contents<\/p>\nToggle<\/span><\/path><\/svg><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\n