Using the CDSW to Cloudera AI Migration tool
A step-by-step procedure covers how to migrate Cloudera Data Science Workbench (CDSW) 1.10.0 and higher version Private Cloud to Cloudera AI. You use the UI-driven migration tool to migrate your workload automatically from your deployed CDSW, which you installed on the same cluster as Cloudera AI.
In this task, you automatically migrate CDSW 1.10.0 or later cluster to Cloudera AI in Private Cloud1.5.3 or later. You can expect some downtime, which is proportional to the volume the workloads you have to migrate.
-
Log into Cloudera Private Cloud, and navigate to Cloudera AI > Workbenches.
The system detects the presence of your legacy CDSW installation and provides a button to migrate CDSW to Cloudera AI.
-
Click Migrate CDSW to Cloudera AI.
The Migration tool provides an option to check Cloudera AI for CDSW migration readiness or just continue with the migration.
-
Click the option for how you would like to proceed.
- Check Cloudera AI for CDSW migration readiness - The readiness or preflight check creates a new workbench and runs readiness checks on the workbench prior to performing the migration.
- Migrate CDSW to Cloudera AI - The Migration tool provides a Cloudera AI Workbench provision window with additional options.
-
If you choose the Check Cloudera AI for CDSW migration readiness
option, the Migration tool displays the validation page.
-
If you would like to provide a Kubeconfig for the migration check,
click File Upload, then Choose
File, and select the Kubeconfig file.
The Kubeconfig file can be found at /etc/kubernetes/admin.conf on the CDSW cluster.If you cannot access /etc/kubernetes/admin.conf from the UI as instructed in the previous step, download the file from your CDSW cluster to your local machine, and then try to select the Kubeconfig file from the UI again.
-
In the Migration timeout section, accept the
default 24 hours timeout, or if your CDSW workload is hundreds of
gigabytes, increase the migration time up to 336 hours (14 days).
Increasing the migration timeout value does not cause a delay in the migration of a small workload.
- In the Workbench Name field, type an arbitrary name.
-
In the Select Environment field, select your Cloudera
environment.
After the readiness check has completed, the Migration tool displays a status. You can see the readiness check summary in the Workbench Details page.
- If the readiness check fails, you can obtain additional
information about the failure on the Workbench Details page.
- After you've addressed the issue resulting in the failed
readiness check, you can retry the readiness check by choosing
Retry Migration Readiness check from
the Actions menu.
- If the readiness check fails, you can obtain additional
information about the failure on the Workbench Details page.
- After the readiness check, incremental migration can be performed to continue the migration operation Alternatively, you can choose to create a new workbench by clicking the Migrate CDSW to Cloudera AI button.
-
If you would like to provide a Kubeconfig for the migration check,
click File Upload, then Choose
File, and select the Kubeconfig file.
-
When you proceed with the CDSW to Cloudera AI migration, the Migration tool displays
the Migrations Settings window.
-
If you would like to provide a Kubeconfig for the migration, click
File Upload, then Choose File,
and select the Kubeconfig file.
The Kubeconfig file can be found at /etc/kubernetes/admin.conf on the CDSW cluster.If you cannot access /etc/kubernetes/admin.conf from the UI as instructed in the previous step, download the file from your CDSW cluster to your local machine, and then try to select the Kubeconfig file from the UI again.
-
In the Migration timeout section, accept the default 24
hours timeout, or if your CDSW workload is hundreds of gigabytes, increase the
migration time up to 48 hours.
Increasing the migration timeout value does not cause a delay in the migration of a small workload.
- In the Workbench Name field, type an arbitrary name.
- In the Select Environment field, select your Cloudera environment.
-
Accept default values for other options, and click Provision
Workbench.
After the Cloudera AI installation, the migration readiness checks and the migration follow automatically. Status indicators show the progress of the installation and migration. During the migration, you can access the CDSW cluster. The migration process does not stop CDSW pods. The Cloudera AI Workbench is stopped.
-
To display the progress of the migration including events and logs while the
workbench is in migration mode, navigate to the Workbench
Details page and click the Migration
Progress tab.
You can also view details of the migration and events and logs by clicking the appropriate tabs.
-
When the initial migration is complete, the state changes to
Validate Migration Started.
-
Open the workbench by clicking the workbench name and validate the
workloads.
Any changes made to the workbench while you are validating the workbench will be overwritten during the incremental migration.
-
At this point, you can choose to do multiple incremental migrations or a
single, longer migration.
- To perform incremental migrations, select the Incremental
CDSW Migration option from the
Actions menu.
CDSW is not stopped during incremental migrations.
- To perform a single, longer migration, select the Incremental
CDSW Migration option from the
Actions menu, click the Perform
final migration checkbox, then click
OK.
During the final migration, CDSW will be stopped and will not be restarted. After the final migration, only the Cloudera AI Workbench will be active.
The Migration tool displays the Incremental CDSW migration dialog box. - To perform incremental migrations, select the Incremental
CDSW Migration option from the
Actions menu.
-
If you choose to perform incremental migrations, choose your parameters for the
migrations.
- CDSW Kubeconfig
- This is an optional parameter. If there is no change to the CDSW Kubeconfig, then you do not need to select this option. The system will use the kubeconfig that was provided during the initial migration. This option can be helpful if CDSW is restarted during the migration and there are changes to the CDSW kubeconfig.
- Migration timeout
- You can specify the amount of time allowed for the migration before it timeouts. This timeout value is specified in hours and can range from 0 to 48 hours.
-
If you choose to perform incremental migrations, when the Cloudera AI validation is
complete and all workloads work as expected, select Perform final
migration and choose the appropriate option.
After you perform the final migration, CDSW will be in a stopped state.Perform final migration provides three options:
-
- Stop applications
- If you select the Stop applications option, the applications in the Cloudera AI will be in the stopped state after the migration. You must manually start each application after the final migration.
- Stop jobs
- If you select the Stop jobs option, the recurring jobs in the Cloudera AI will be in the paused state after the migration. You must manually start each job after the final migration.
- Stop models
- If you select the Stop models option, the models in the Cloudera AI will be in the stopped state after the migration. You must manually start each model after the final migration.
-
- Now that the migration is complete, you can use Cloudera AI.